Slyber

Monitoring

Updated:
Author: Erkin Khidirov

Slyber Turn Server includes a powerful built-in telemetry system that tracks active sessions in real time, maintains 24-hour hourly statistics, identifies attacking IP addresses, and persists data to disk.

Access to internal endpoints (/metrics, /events, /api/node-stats): These routes are restricted from unauthorized access (isInternalOrAuthorized). Access is granted only if:
  1. The X-Turn-Secret (or Authorization: Bearer) header is provided;
  2. OR the request originates directly from the local machine (127.0.0.1 / ::1);
  3. OR the client IP is explicitly included in the ALLOWED_IPS allowlist.

1. Health Check: GET /healthz

A lightweight public endpoint for server monitoring:

Bash 
curl http://127.0.0.1:8080/healthz
# Response (200 OK): {"status":"ok"}

2. Metrics: GET /metrics

Returns metrics in the standard Prometheus text format (text/plain; version=0.0.4):

  • turn_auth_requests_total{status="success"} (counter) - total number of unique successful user connections to TURN.
  • turn_auth_requests_total{status="failed"} (counter) - total number of rejected authentication attempts (accounting for 10-second deduplication of retries from the same IP).
  • turn_active_allocations (gauge) - current number of active user TURN sessions.
  • turn_relayed_bytes_total (counter) - total volume of media traffic in bytes relayed through the TURN server.
  • turn_ssrf_blocked_total (counter) - number of blocked SSRF attack attempts targeting private and system addresses.
  • turn_udp_listeners_active (gauge) - number of active server UDP sockets (TURN_THREADS).

3. Full Node Analytics: GET /api/node-stats

Returns a comprehensive JSON snapshot of the server state for building dashboards and control panels:

JSON 
{
  "status": "online",
  "uptime_sec": 86400,
  "active_sessions": 2,
  "allocated_ports": 3,
  "active_sessions_list": [
    {
      "id": 14,
      "user_id": "user_777",
      "client_ip": "198.51.100.10",
      "relay_port": 49200,
      "transport": "udp (+1 standby)",
      "started_at": "2026-10-27T08:10:00Z",
      "duration_sec": 345,
      "relayed_bytes": 48250112
    }
  ],
  "port_pool_capacity": 16384,
  "port_pool_used_percent": 0.0183,
  "auth_success": 1250,
  "auth_failed": 4,
  "relayed_bytes": 10737418240,
  "ssrf_blocked": 1,
  "lan_skipped": 420,
  "udp_listeners": 8,
  "hourly_24h": [
    {
      "hour_unix": 1761552000,
      "peak_active": 18,
      "connections": 95,
      "relayed_bytes": 524288000,
      "auth_failed": 0,
      "ssrf_blocked": 0
    }
  ],
  "top_offenders": [
    {
      "ip": "203.0.113.199",
      "last_user_id": "attacker",
      "auth_fails": 12,
      "ssrf_blocks": 2,
      "last_seen": "2026-10-27T08:05:12Z"
    }
  ]
}

Active Session Counting and Network Handover (Wi-Fi ↔ 4G) Behavior:

  • Inactivity Timeout (maxSessionIdle = 30s): If an allocated socket sees no traffic for more than 30 seconds, it is no longer shown in the active sessions list.
  • Smart Network Handover Handling (supersededSocketIdle = 5s): When a user leaves home during a call and their phone switches from Wi-Fi to 4G, a new TURN allocation is created while the old one goes silent. The server detects that the same user_id has a fresh active socket (with traffic within the last 3 seconds) and hides the abandoned old socket after just 5 seconds of silence instead of 30.
  • Socket Grouping (+N standby): If a client opens multiple parallel allocations (e.g., for audio and video, or a fallback socket), they are grouped into a single session entry with aggregated traffic and labeled udp (+1 standby).
  • Offender Ranking (top_offenders): Sorts IP addresses by threat index using the formula Score = AuthFails + (SSRFBlocks * 5) and returns the top 10 most dangerous IPs.

4. Event Log: GET /events

The server maintains an in-memory ring buffer of the last 2,000 events (SmartEvent). If the exact same event (matching type, user_id, client_ip, and target_ip) repeats within 30 seconds, the server avoids creating duplicates by incrementing the "count" field of the existing entry and updating its timestamp.

GET /events Filter Parameters:

  • type - filter by event type:
  • ALL (or empty) - all events.
  • ALERTS - security incidents only (AUTH_FAIL and SSRF_BLOCK).
  • CREDS_ISSUED - credentials issued via the HTTP API.
  • AUTH_OK - successful establishment of a new user TURN session.
  • SESSION_CLOSED - session termination with a final summary (port, duration, and relayed traffic volume in B/KB/MB).
  • AUTH_FAIL - authentication error (invalid format, expired TTL, unauthorized Origin/IP, missing secret).
  • SSRF_BLOCK - blocked SSRF attack attempt.
  • user_id - case-insensitive substring search by user_id.
  • ip - substring search across client_ip or target_ip.
  • limit - maximum number of returned entries (ordered from newest to oldest).

Example request:

Bash 
curl "http://127.0.0.1:8080/events?type=ALERTS&limit=20"

5. Disk Persistence

To prevent statistics from being lost during server restarts or version upgrades, NodeTelemetry atomically flushes data to the DATA_DIR directory every 10 seconds (only if changes occurred - indicated by the dirty flag), as well as upon server shutdown (SIGTERM/SIGINT):

  • stats.json - global counters, 24-hour hourly buckets, and the offenders database.
  • events.jsonl - event log in JSON Lines format.

Data is first written to a temporary .tmp file with 0600 permissions, followed by an atomic os.Rename operation, which prevents file corruption in the event of a sudden power loss.

← Previous HTTP API Next → Internal Architecture