WebSocket Signaling, Simulcast and SVC Routing, QoS, and Fast Swap

Updated: Nov 15, 2026

Author: Erkin Khidirov

For room management, SDP exchange, and video quality switching, Slyber SFU Server provides a JSON protocol over a /ws WebSocket connection and a synchronous POST /signaling HTTP endpoint. This article details all signaling message types, the publish and subscribe lifecycle in the Pion WebRTC stack, and the video quality control algorithms.

Connecting to the Signaling Gateway and Session Binding

The client application establishes a persistent WebSocket connection at ws://<HOST>:8090/ws. The server supports up to 16,384 concurrent signaling connections, automatically sends keep-alive Ping frames every 30 seconds, and requires all client WebSocket frames to be masked according to RFC 6455.

The first message from the client must include a valid authorization token in the token field (token generation rules are covered in Authentication and End-to-End Encryption). Once the token is verified, the WebSocket connection is strictly bound to the participant_id and room_id pair. Any attempt to send a token for a different user or another room over the same socket is rejected with a security error. If other participants are already in the room with active streams, the server immediately sends the list of existing tracks to the new client via publish_track messages upon entry.

SignalEnvelope Message Format

All inbound and outbound messages are transmitted in a unified SignalEnvelope JSON format. Below are the primary fields of the structure.

JSON

{
  "type": "join | joined | offer | answer | trickle_ice | publish_track | subscribe_track | set_layer | roulette_swap | active_speakers | e2ee_key_ratchet | leave | error",
  "room_id": "conf-room-01",
  "participant_id": "user-42",
  "token": "eyJhbGciOiJIUzI1NiIs...",
  "sdp": "v=0\r\no=- ...",
  "candidate": "candidate:1 1 udp ...",
  "track_id": "cam-feed",
  "target_track_id": "partner-2-cam",
  "kind": "video",
  "codec": "vp8",
  "ssrcs": [1001, 1002, 1003],
  "spatial_layer": 2,
  "temporal_layer": 2,
  "e2ee_key_id": 1,
  "mls_key_package": "..."
}

Publishing a Media Stream (Join, Offer, and PublishHints)

In the Pion WebRTC-based architecture, publishing a stream from a client takes three steps.

Pre-announcement (optional). The client can send a publish_track message specifying the desired stream name in track_id, the media kind (audio or video), and the encryption key identifier e2ee_key_id. The server stores this metadata as PublishHints for upcoming track binding.
SDP Exchange (Join or Offer). The client adds local media tracks to its RTCPeerConnection, generates an SDP Offer, and sends it to the server inside a join message (or as a separate offer message). The server processes the offer in AcceptOffer (automatically performing an SDPTypeRollback if a signaling glare collision occurs), gathers local ICE candidates, and returns the completed SDP Answer in a joined (or answer) response along with external TURN server configurations in the ice_servers field.
Automatic Track Registration in the Room. As soon as ICE connectivity and the DTLS handshake complete, the first encrypted SRTP packets arrive from the client. Inside Pion, the pc.OnTrack handler fires, creates an UpTrack object in the room, requests a keyframe from the publisher via RTCP PLI, and broadcasts a publish_track notification with the track name, codec, and SSRC to all room participants.

Subscribing to a Stream and Server-Initiated Unified Plan Renegotiation

When a client receives a publish_track notification (or wants to subscribe to a known stream in the room), it sends a subscribe_track request specifying track_id and the desired spatial_layer (from 0 to 2).

Upon receiving this request, Slyber SFU Server performs the following steps:

Checks the actually published layers of the target UpTrack using ClampSpatialLayer. For example, if the publisher only transmits two Simulcast layers (0 and 1) while the viewer requests layer 2, the server automatically clamps the subscription to the highest available layer 1 so the viewer never stalls waiting for a non-existent layer.
Creates a local outbound TrackLocalStaticRTP track and adds it to the viewer's PeerConnection.
Triggers server-side Unified Plan renegotiation (via triggerNegotiation) and sends an offer message containing the new server SDP to the viewer over WebSocket.
The client application must apply this server offer via setRemoteDescription, generate an answer with createAnswer, and send it back to the server in an answer message. Immediately afterward, the server requests a fresh keyframe (TriggerPLI) and begins delivering video to the viewer.

Simulcast, SVC Filtering, and Automatic Quality Switching (BWE)

The server supports both multi-stream Simulcast (where layers are identified via sdes:rtp-stream-id RTP header extensions with values q, h, f or 0, 1, 2) and single-stream SVC for the VP9 codec (where spatial and temporal indices are parsed directly from the payload descriptor header).

Three Quality of Service (QoS) mechanisms operate on every video stream:

Per-Layer NACK and PLI Buffers. Each UpTrack maintains three independent PacketRingBuffer instances and three independent PLICoalescer filters with a 500-millisecond cooldown. When a viewer loses a packet, the server immediately looks up the original sequence number via ReverseTranslateNACK history and retransmits it from memory without bothering the sender.
Automatic Bandwidth Controller (SubscriberBWE). The server continuously reads RTCP TransportLayerCC (TWCC), ReceiverEstimatedMaximumBitrate (REMB), and ReceiverReport feedback from each viewer. When packet loss exceeds 10%, the bandwidth estimate is forcibly reduced by 25%. Depending on available throughput in kbps, the controller automatically selects the optimal spatial and temporal layer (above 1800 kbps enables layer 2/2, above 1200 selects 2/1, above 600 selects 1/2, above 350 selects 1/1, above 150 selects 0/1, and below 150 kbps falls back to emergency layer 0/0).
Manual Layer Selection (set_layer). If a client sends a set_layer command with explicit spatial_layer and temporal_layer fields, the server disables automatic BWE switching for that track (DisableAutoLayerForTrack) and locks the user-selected quality. When upgrading to a higher Simulcast layer, the switch occurs seamlessly at the exact arrival of the new layer's IsKeyFrame.

Active Speaker Detector, Last-N Policy, and Roulette Swap

On audio packets, the server inspects the urn:ietf:params:rtp-hdrext:ssrc-audio-level (RFC 6464) header extension without decrypting the audio payload. The ActiveSpeakerDetector computes a smoothed voice energy score for each participant over the past 1.2 seconds. Once per second, a background room ticker broadcasts an active_speakers message listing up to three loudest speakers and invokes EnforceLastNVideo. If the number of active cameras in the room exceeds LAST_N_VIDEO_LIMIT, video streams from silent participants are paused and seamlessly resumed on a keyframe as soon as they speak again.

For one-on-one video roulettes, the protocol provides the roulette_swap message carrying the current track_id and the new target_track_id. The FastSwapRouletteTrack method rebinds the viewer's existing DownTrack to the new UpTrack without closing the connection or exchanging new SDP offers. The SeqTimestampRewriter module automatically recalculates seqOffset and tsOffset relative to the last transmitted packet, allowing the browser's decoder to continue playing the new partner's video without a single sequence gap.