Skip to main content

Collector Configuration

Collector Capacity Planning​

The recommended configuration for the host deploying Collector is as follows:

  • CPU: 4 Cores and above
  • Physical Memory: 8GB and above

The number of traces that a single Collector can process per second is related to the number of spans in the trace. The more spans in a single trace, the more computing resources are needed. A Collector deployed on a 4 Core, 8GB machine can typically handle 4000 TPS of trace data.

CPU usage is also related to the number of application instances related to Collector. The more application instances, the more computing resources are needed.

In practice, we recommend calculating the number of Collectors that need to be deployed according to the following formula:

Number of Collectors = Max(Total number of application instances/100, Peak TPS in a day/4000) + 1

Main Configuration File Path: /opt/tingyun-collector/conf/Collector.yaml​

Collector contains three major components: APM Collector, Infra Collector, and Guanyun Collector. In special scenarios, you may not need to use some functions, and you can disable them at this time.

## Whether to enable INFRA
infra_enabled

## Whether to enable APM
apm_enabled

## Whether to enable Guanyun to integrate Zabbix, Prometheus, Opentelemetry, cloud vendor monitoring, etc.
guanyun_enabled

## JVM parameters of APM Collector
apm_jvm_args

After modification, you need to execute sudo systemctl restart tingyun-collector to restart Collector.

Note:

  • Disabling Infra Collector will cause APM Collector to fail to automatically upgrade.

Precautions When Enabling APM Trace Tail-based Sampling​

  • All Collectors in the same deployment environment should be upgraded to version 4.2 and above.

  • Applies only to Collectors within the same deployment environment; scenarios where a single request trace spans across different deployment environments are not supported.

  • All Collectors need to transmit tail-based sampling information to each other, and you need to confirm that the firewall policies between Collectors are opened.

  • Traces will be temporarily stored in the local file /opt/tingyun-collector/tail, which requires relatively high disk, CPU, and memory resources.

    Try to use SSD disk and reserve more than 10G of disk space. Or map the file to memory to improve performance: sudo mount -t tmpfs -o size=4G tmpfs /opt/tingyun-collector/tail

  • As long as any trace within the tail-based sampling window is retained by the sampling strategy, all traces on the link related to this trace are retained.

  • The tail-based sampling window is 1 minute. Traces with a link request time difference exceeding 1 minute no longer follow the tail-based sampling strategy.

  • Trace reporting will increase by 3 minutes of latency.

Other Configurations of APM Collector​

The configuration of APM Collector is in the ./conf/apm/collector.properties file in the Collector root directory.

When the available CPU of Collector exceeds 4 Cores, it is recommended to set trace.consumer_worker to the number of CPU cores.

Other Configurations of Infra Collector​

The configuration of Infra Collector is in the ./conf/infra/ directory in the Collector root directory, and usually does not need to be modified.

Other Configurations of Guanyun Collector​

The main configuration of Guanyun Collector is in the ./conf/guanyun/collector.yaml file in the Collector root directory, and usually does not need to be modified.

The integration data configuration of Guanyun Collector is in ./conf.d/ in the Collector root directory. You need to configure different configuration files according to different integration requirements.

Viewing Logs​

  • Collector service logs

    Path /opt/tingyun-collector/logs/watchdog.log

    Normally, heartbeat logs should appear

    Sending heartbeat       {"url": "http://10.128.8.61:80/apm/data/collectorHeartbeat", "data": "{\"CollectorId\":\"0fc908dfe7274aeab40aa0686ad3e511\",\"ZoneName\":\"zhenzh-vm\",\"CollectorName\":\"10.0.2.15\",\"Ip\":\"10.0.2.15\",\"Version\":\"4.2.0-3\""}

  • APM Collector logs

    Path /opt/tingyun-collector/logs/apm/collector.log

    Normally, heartbeat logs should appear

    Send 1483 bytes to http://10.128.8.61:80/apm/data/heartbeat  {"disabled":false,"name":"10.0.2.15","version":"20251010",

  • Infra Collector logs

    Path /opt/tingyun-collector/logs/infra/infra_collector.log

    Normally, heartbeat logs should appear

    POST request to http://10.128.8.61:80/infra/data/service/sync/heartBeat with params {\"address\":\"10.0.2.15:27666\",\"apmPort\":\"27665\",\"collectorName\"

  • Guanyun Collector logs

    Path /opt/tingyun-collector/logs/guanyun/collector.log

    You only need to view the logs when a data source is configured for integration

    After integrating the data source, logs of sending Metric data should appear

    info    Metrics {"resource": {"service.instance.id": "438fbe51-4f17-43a0-b8c4-eb50b91589d3", "service.name": "guanyun-collector", "service.version": "4.2.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "resource metrics": 1, "metrics": 24, "data points": 43}