Nginx Agent Deployment

The Nginx agent is integrated into UniAgent and supports monitoring Nginx applications.

Compatibility Requirements

Before installing the agent, please ensure your system meets the following requirements:

• 64-bit Linux operating system, meeting one of the following:

  • RedHat Enterprise Linux (RHEL) 6.0 or higher
  • CentOS 6.0 or higher
  • SuSE Linux 10.0 or higher
  • Debian 6.0 ("lenny") or higher
  • Ubuntu 9.10 ("Karmic Koala") or higher
  • Any other Linux distribution with kernel version above 2.6.32 and GLIBC version above 2.12.

• Nginx must meet one of the following:

  • Nginx: 1.11.5 - 1.29.4
  • Openresty: 1.13.6 - 1.25.3
  • Tengine: 2.3.0 - 3.1.0
  • Any product based on Nginx or Openresty, such as apisix.

  The above Nginx versions must be official releases. If the source code has been modified, the agent may not work properly.

  For custom-developed Nginx, you must provide the modified Nginx source code, compilation platform type (currently only x86_64 and aarch64 are supported), compilation options /path/to/nginx -V, and the compiled Nginx signature info strings /path/to/nginx | grep 8,4,8.

Installation

The Nginx agent is included in the UniAgent package and must be installed with root privileges.

1. Download the UniAgent installation script and execute it as root:

   sudo sh tingyun-uniagent-<version>.sh

2. Manually enable the Nginx switch:

   sudo sed -i -e "s/nginx_enabled.*/nginx_enabled=true/" /opt/tingyun-oneagent/conf/interceptor.conf

3. Restart the Nginx service:

   sudo systemctl restart nginx

Configuration

The agent configuration file path is /opt/tingyun-oneagent/conf/nginx.conf.

1. Change the agent log level: modify the agent_log_level field. No need to restart Nginx for the change to take effect.

2. Capture HTTP request headers: modify the capture_header field. Currently, only one HTTP header can be configured. No need to restart Nginx for the change to take effect.

3. Change Nginx application naming mode: modify the naming_mode field. No need to restart Nginx for the change to take effect.

   Supported naming modes:

   0 - server_name:port
   1 - server_name
   2 - http_host:port
   3 - http_host
   4 - host IP:port
   5 - host IP

   Where:

   • server_name refers to the server name specified in the Nginx configuration file.
   • http_host refers to the domain in the HTTP request; if not present, the client IP is used.
   • host IP refers to the value of the host_ip field in the current configuration file, which defaults to the current host or Pod IP, but can be manually set.

   ⚠️ If a server block in the Nginx configuration specifies multiple server_names, e.g.:

   server {
       listen       80;
       server_name  www.test.com www1.test.com www2.test.com;
       root         /usr/share/nginx/html;
   }

   The agent can only obtain the first name. For the above, only www.test.com will be used. If naming_mode is set to 0, requests for all three domains will be aggregated under www.test.com. To split different domains, set naming_mode to 2, so requests are named by domain:port.

Function Scope

• Only supports collecting performance data for HTTP/HTTPS requests.
• Does not support collecting performance data for gRPC, WebSocket, SMTP, IMAP, or POP3 requests.
• Does not support collecting performance data for TCP/UDP-layer proxying, such as MySQL, Redis, or SMTP.
• Unsupported features: manual script injection, user tracking, and custom transaction naming.

Enable Web Front-end Performance Analysis

UniAgent supports automatically injecting the JS agent into HTML pages through the Nginx agent to collect web front-end performance data.

To enable Web front-end performance analysis, please take the following steps:

1. In the left navigation bar, click All Apps， then click Real User Monitoring in the lite app list.
2. On the real user monitoring page, click Create new application in the top-right corner.
3. On the Data Integration page, find the UniAgent section and click Web.
4. Enter the application name, and then click Next.
5. In the Embed code section, select APM Injection, and then select the back-end application that corresponds to Nginx.
6. Click Next to enable web front-end performance analysis.

In the following scenarios, JS agent injection is not possible:

• Requests not initiated by browsers, such as those from Curl, Wget, or HttpClient.
• Backend application responses to Nginx are compressed.
• Content-Type is not text/html.
• Response status code is not 200.
• JS injection position exceeds 64K.

FAQ

How to view Nginx agent logs

The Nginx agent log file is at /opt/tingyun-oneagent/logs/agent/nginx.log.

Set agent_log_level in /opt/tingyun-oneagent/conf/nginx.conf to DEBUG to output debug-level logs. Changing the log level does not require restarting Nginx; it takes effect within about 1 minute.

To search Nginx logs:

tail -f /path/to/nginx/logs/error.log | grep tingyun

How to confirm if the Nginx agent is loaded successfully?

1. Check agent version and support list:

   • Check UniAgent version:

   cat /opt/tingyun-oneagent/Version
   • Check supported Nginx versions:

   grep nginx /opt/tingyun-oneagent/conf/interceptor.conf

2. Check if UniAgent service is running:

   sudo systemctl status tingyun-oneagent

   The service status should be Active: active (running).

   sudo cat /etc/ld.so.preload

   The content should be /lib64/libinterceptor.so.

   If the output is empty, UniAgent cannot communicate with the backend. Check /opt/tingyun-oneagent/logs/oneagent/oneagent.log to identify the issue, and then adjust settings such as network policies and the Collector address based on the logs.

3. Check if the Nginx agent switch is enabled:

   grep nginx_enabled /opt/tingyun-oneagent/conf/interceptor.conf

   Ensure nginx_enabled is true. If not, modify and restart Nginx.

4. Check if the Nginx agent is loaded:

   • View Nginx process list:

   ps -ef | grep nginx
   • Find a Nginx process PID and run:

   sudo grep "tingyun" /proc/<nginx_pid>/maps

   If you see entries like agent/nginx_xxx/lib/tingyun_nginx_module, the agent is loaded successfully.

   Example:

   [tingyun@bogon ~]$ sudo grep "tingyun" /proc/23241/maps
   7f0467f02000-7f0467f05000 r-xp 00000000 fd:01 34581021  /opt/tingyun-oneagent/agent/nginx_2.0.0/lib/tingyun_nginx_module-1.13.10-1111111111.so
   7f0467f05000-7f0468104000 ---p 00003000 fd:01 34581021  /opt/tingyun-oneagent/agent/nginx_2.0.0/lib/tingyun_nginx_module-1.13.10-1111111111.so
   7f0468104000-7f0468105000 rw-p 00002000 fd:01 34581021  /opt/tingyun-oneagent/agent/nginx_2.0.0/lib/tingyun_nginx_module-1.13.10-1111111111.so

   If not, check if nginx_enabled=true is set in interceptor.conf. If enabled but still not loaded, the Nginx version or compilation options may not be supported. Check the load log:

   grep nginx /opt/tingyun-oneagent/logs/oneagent/preload.log

   If you do not know the actual Nginx path, run:

   sudo readlink -f /proc/<nginx_pid>/exe
   • Check Nginx version:

   <nginx_path> -v

   Output should look like:

   [tingyun@bogon ~]$ /usr/sbin/nginx -v
   nginx version: nginx/1.14.1
   • Check Nginx compilation options:

   <nginx_path> -V

   Output should look like:

   [tingyun@bogon ~]$ /usr/sbin/nginx -V
   nginx version: nginx/1.14.1
   built by gcc 8.2.1 20180905 (Red Hat 8.2.1-3) (GCC)
   built with OpenSSL 1.1.1 FIPS  11 Sep 2018 (running with OpenSSL 1.1.1k  FIPS 25 Mar 2021)
   TLS SNI support enabled
   configure arguments: --prefix=/usr/share/nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_dav_module --with-http_gzip_static_module
   • Check Nginx signature info:

   Output should look like:

   [tingyun@bogon ~]$ strings /usr/sbin/nginx | grep 8,4,8
   8,4,8,0010111111010111001110111111111110
   • Check whether the corresponding .so file exists. If it does not exist, the current Nginx build is not supported:

   sign=$(strings <nginx_path> | grep 8,4,8, | cut -c 29-38)
   echo "signature: ${sign}"
   ls /opt/tingyun-oneagent/agent/nginx_$(cat /opt/tingyun-oneagent/agent/nginx_version)/lib/tingyun_nginx_module-*-${sign}.so

5. Check for crash information:

   cat /opt/tingyun-oneagent/logs/agent/nginx.crash

   If the file is not empty, the agent has crashed. Please collect and provide the file content.

6. Verify that the Nginx agent is working:

   Set agent_log_level to DEBUG in /opt/tingyun-oneagent/conf/nginx.conf. No restart is required; the change takes effect in about 1 minute.

   tail -f /opt/tingyun-oneagent/logs/agent/nginx.log

   Common log messages and what they mean:

   • If you see app name is null, the agent failed to determine the application name. By default, the agent uses server_name. If server_name is not configured in Nginx, update naming_mode in /opt/tingyun-oneagent/conf/nginx.conf to 2 or 4.
   • If you see create io thread fail, open socket fail, connect to daemon fail, send to daemon fail, the agent cannot communicate with the Collector.
   • If the log contains a large number of send cmd to daemon entries, the communication between the Collector and the DC (Data Center) is abnormal. Check the Collector configuration and review /opt/tingyun-oneagent/logs/agent/daemon.log.
   • If you see recv config, the Collector and DC are communicating normally.
   • If you see recv config: enabled=true, the agent switch is enabled.
   • When an HTTP request is received, the agent logs upstream header: X-Tingyun.
   • When the upstream response includes agent headers, the agent logs upstream response: X-Tingyun-Data.
   • When returning the HTTP response, if cross-application tracing headers are present, the agent logs outbound header: X-Tingyun-Data.
   • If the request is served locally by Nginx and not forwarded upstream (for example, static image files), the agent does not collect traces and logs request is static.
   • When trace data is sent, the agent logs send trace to daemon.
   • If sending trace data fails, the agent logs send trace to daemon fail.