Capture
ScreenCaptureKit captures the selected physical or Virtual Display and system audio. The Host runs at a configured 60 fps cap and begins with GPU-friendly pixel buffers.
Technology
A native Mac pipeline, end to end.
Screen Ferry combines Apple-native capture, codecs, graphics, audio, input, display, networking, and cryptography with a session protocol designed for freshness, priority, and route migration.
From remote display to local pixels
flowchart TB
SCK["ScreenCaptureKit capture"] --> VT1["VideoToolbox low-delay H.264"]
VT1 --> SEAL["Screen Ferry sealed datagrams"]
SEAL --> CARRIER{"Active carrier"}
CARRIER -->|"Relay"| QUIC["QUIC datagrams · rdk1"]
CARRIER -->|"P2P"| UDP["Qualified direct UDP"]
QUIC --> VT2["VideoToolbox decode"]
UDP --> VT2
VT2 --> METAL["Metal presentation"]
subgraph REALTIME["Independent real-time paths"]
direction LR
CURSOR["Host cursor state"] --> LOCAL["Client local cursor"]
AUDIO["AAC-ELD audio"] --> CORE["Adaptive jitter → Core Audio"]
end
Low-delay encode
VideoToolbox encodes H.264 in real-time mode with frame reordering disabled. The running session can adjust average bitrate and its one-second data-rate cap without restarting the stream.
Session transport
Compressed frames use 256 KiB default application chunks, then fragment into approximately 1,200-byte authenticated datagrams. Reassembly is bounded by time, memory, display, and whole-frame rules.
Decode and present
VideoToolbox decodes into NV12 pixel buffers and Metal presents them through CAMetalLayer. The path avoids a CPU color-conversion round trip between decode and display.
Four traffic classes
A click does not wait behind a desktop frame.
Input, cursor, and audio use the real-time class. Video uses high priority. Control, reports, and clipboard use normal priority, while bulk work remains background. Bounded queues and aging prevent a busy stream from turning interactivity into a backlog.
Reliable classes use selective acknowledgements, retransmission-timeout backoff, fast retransmit, and flow control. Video and audio use freshness-oriented delivery because retransmitting expired media would only manufacture latency.
Carrier-independent session
The connection can move without rebuilding your work.
flowchart TB START(["Session established"]) --> RELAY["Relay active"] RELAY -->|"Network changes or P2P is enabled"| GATHER["Gather and validate candidates"] GATHER -->|"No better path qualifies"| RELAY GATHER -->|"Direct path wins RTT comparison"| DIRECT["Direct active"] DIRECT -->|"Direct path degrades"| RELAY DIRECT -->|"Network changes"| GATHER
The logical Screen Ferry session lives above its carrier. It can begin over Network.framework QUIC datagrams using the rdk1 ALPN, move to a qualified direct UDP path, and return to a validated relay standby.
Network viability changes trigger make-before-break candidate gathering. After migration, reliable unacknowledged control data becomes retransmittable, congestion state resets, and measured route RTT seeds recovery timing.