WHIP and WHEP Gateways, AI Voice Pipeline, Node Cascade, and Monitoring

Single-Request Broadcasting and Playback via WHIP and WHEP

• Stream Publishing (POST /whip/{room_id}/{track_id}). The client sends an SDP Offer in the request body with Content-Type set to application/sdp and an authorization token in the Authorization: Bearer <TOKEN> header. The server creates a session with the primary video track {track_id} and companion audio track {track_id}-audio, and returns 201 Created, a complete SDP Answer, a Location header, a unique ETag resource identifier, and Link headers containing external TURN server credentials.
• Viewer Subscription (POST /whep/{room_id}/{track_id}). The viewer sends a recvonly SDP Offer and immediately receives 201 Created with an SDP Answer already bound to both the broadcast video and audio streams without requiring extra renegotiation. The Location header returns the dedicated subscription resource URL /whep/{room_id}/{track_id}/{sub_id}.
• Trickle ICE and Owner Protection (PATCH and DELETE). To send Trickle ICE candidates, the client submits a PATCH request with content type application/trickle-ice-sdpfrag. To end the stream, a DELETE request is sent. The server strictly verifies the resource owner ownerID and the If-Match (ETag) header. Any attempt by a third party to overwrite or delete another user's stream is rejected with 403 Forbidden or 409 Conflict.

Full-Duplex Audio Gateway for AI Voice (/ai/ws)

GET /ai/ws?room=room-dialog&mode=mux&pacing=true&auto_flush=true HTTP/1.1
Host: 127.0.0.1:8090
Upgrade: websocket
Connection: Upgrade
X-Slyber-Node-Secret: your-24-char-production-node-secret

Operating Modes and Managing Multiple Bots in One Room

• Single Mode (default). When connecting to /ai/ws?room=room-dialog, the server automatically creates the ai-voice-bot audio track in the room and sends an ai_ws_ready text frame. All binary frames (Opcode 0x02) your service sends into the socket carry raw Opus packets of the bot's voice, while the server streams binary frames containing raw Opus audio from human microphones in the room back to your service.
• Multiplexed Mode (mode=mux). Allows a single backend process to control multiple independent AI agents in the same room simultaneously. Inbound binary frames from bots carry a 2-byte prefix [slot, audio_level] ahead of the Opus payload, while outbound binary frames carrying human speech include a prefix [len(speaker_id), audio_level, speaker_id...] so the AI knows exactly which room participant is speaking.
• Text Control Commands. Over the same socket, you can send JSON commands such as {"type": "spawn_agent", "slot": 1, "agent_id": "bot-alina", "track_id": "ai-bot-alina"} to dynamically add new bots (the server replies with ai_agent_ready and notifies the room), {"type": "remove_agent", "slot": 1} to remove a bot, and {"type": "clear_buffer"} to immediately flush the audio queue.
• Built-In 20ms Pacer and Barge-In Detector (VAD Barge-In). Neural models often generate audio in uneven bursts. The server's built-in pacer buffers up to 600 frames (12 seconds of speech) per agent and releases them smoothly into the room every 20 milliseconds with accurate RTP Timestamp increments (+960 samples). Simultaneously, the server monitors human microphone levels via RFC 6464 headers. As soon as a person produces 3 consecutive voiced frames, the server detects an interruption, immediately flushes the bot's unspoken audio queue when auto_flush=true is enabled, sends an ai_vad_barge_in event over the socket, and emits ai_vad_speech_end after 360 milliseconds of silence.
• Single-Frame Injection (POST /ai?room=room-dialog). For simple voice notifications, you can push a raw Opus frame via a standard HTTP POST request without opening a WebSocket.

Inter-Server Cascading (/cascade/) and MoQ Relay (/moq/)

• On the receiving Node B, call POST /cascade/ with "direction": "ingress", along with the room name, track name, codec, and target SSRC.
• On the sending Node A, call POST /cascade/ with "direction": "egress" and the UDP address of Node B.
• Node A forwards packets over UDP, prepending a 28-byte header to each packet (magic bytes SC, protocol version, keyframe and spatial layer flags, 64-bit packet sequence number, and a 16-byte HMAC-SHA256-128 cryptographic signature derived from NODE_SECRET). When ALLOW_INTERNAL_CASCADE is disabled, the built-in SSRF filter strictly blocks any attempt to target loopback, private (RFC 1918), or CGNAT (100.64.0.0/10) addresses.

Monitoring, Inspector, and Prometheus Metrics

• GET /inspector/stats returns detailed JSON with Go runtime metrics (uptime, goroutine count, live heap size in megabytes read via the lock-free runtime/metrics interface), Pion WebRTC stack statistics (the webrtc_pion section with active sessions, tracks, and received/sent bytes), QoS metrics (the webrtc_qos section with nacks_retransmitted, plis_forwarded, plis_sent_to_pub, and ratelimit_drops counters), and counters for all gateways (the gateways section). This endpoint is consumed directly by the Slyber Inspector visual test bench.
• GET /metrics exports all node indicators in standard Prometheus text exposition format (metrics including slyber_sfu_uptime_seconds, slyber_sfu_active_rooms, slyber_sfu_webrtc_sessions, slyber_sfu_rtp_rx_bytes_total, slyber_sfu_rtp_tx_bytes_total, slyber_sfu_nacks_retransmitted_total, and slyber_sfu_ai_active_agents) labeled with node for building Grafana dashboards.