<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Performance on Rason Yang</title><link>https://rasonyang.dev/tags/performance/</link><description>Recent content in Performance on Rason Yang</description><generator>Hugo -- gohugo.io</generator><language>en-US</language><copyright>© 2026 Rason Yang</copyright><atom:link href="https://rasonyang.dev/tags/performance/index.xml" rel="self" type="application/rss+xml"/><item><title>A Realtime Voice Bot on One Mac, and Why the Third Call Breaks It</title><link>https://rasonyang.dev/posts/realtime-voice-on-one-mac/</link><pubDate>Mon, 26 Oct 2026 15:03:35 +0800</pubDate><guid>https://rasonyang.dev/posts/realtime-voice-on-one-mac/</guid><description>&lt;p&gt;AICC answers calls with anything that speaks the OpenAI Realtime protocol.&#10;In &lt;a href="https://rasonyang.dev/posts/ai-native-call-center-in-four-projects/#the-model-cascade-realtime-gateway" &gt;the first post in this series&lt;/a&gt; that meant a hosted model, or&#10;Cascade Realtime Gateway, which runs ASR → LLM → TTS behind the protocol but&#10;still calls hosted providers for each stage. The question this post answers:&#10;can one Mac run the whole cascade itself, every model local, and still answer&#10;inside the deadlines AICC&amp;rsquo;s client enforces?&lt;/p&gt;&#10;&lt;p&gt;For one call, yes. On an M3 Max, from the end of the caller&amp;rsquo;s speech to the&#10;first audio sent, &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L21-L25" target="_blank" rel="noreferrer"&gt;P50 is 1.37–1.39 s and P95 1.52–1.54 s&lt;/a&gt;&#10;on turns the turn detector judged complete, and&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L190-L197" target="_blank" rel="noreferrer"&gt;every one of AICC&amp;rsquo;s windows had more than 2 s of margin&lt;/a&gt;. For&#10;three calls, not reliably. &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L172-L177" target="_blank" rel="noreferrer"&gt;The TTS synthesizes one stream at a time&lt;/a&gt;,&#10;and at three sessions one answer&amp;rsquo;s first audio came&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L204-L207" target="_blank" rel="noreferrer"&gt;3.39 s after &lt;code&gt;response.created&lt;/code&gt;&lt;/a&gt;, past AICC&amp;rsquo;s 3 s limit.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;What runs where&#10; &lt;div id="what-runs-where" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-runs-where" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;mlx-realtime &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L3-L7" target="_blank" rel="noreferrer"&gt;speaks a documented subset of the OpenAI Realtime API (GA) over&#10;WebSocket at &lt;code&gt;/v1/realtime&lt;/code&gt;&lt;/a&gt;, and the official &lt;code&gt;openai&lt;/code&gt; Python&#10;SDK connects by changing only &lt;code&gt;base_url&lt;/code&gt;. AICC&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L19-L22" target="_blank" rel="noreferrer"&gt;attaches through its existing &lt;code&gt;gateway&lt;/code&gt; profile, with no AICC change&lt;/a&gt;.&#10;Inside, it is &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L45-L66" target="_blank" rel="noreferrer"&gt;four processes&lt;/a&gt;:&lt;/p&gt;&#10;&lt;p&gt;&lt;figure&gt;&lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="low"&#10; alt="Architecture: AICC, the openai SDK and the benchmark connect over WebSocket to a Python gateway that runs Silero VAD, Smart Turn, two echo cancellers and G.711 on the CPU; the gateway calls three MLX processes over HTTP on localhost: Qwen3-ASR-1.7B 8-bit (2.5 GB), Qwen3.6-35B-A3B 4-bit under mlx_lm.server (22.9 GB), and Qwen3-TTS-0.6B 8-bit with a 4-bit fallback (3.8 GB), which synthesizes one stream at a time"&#10; src="https://rasonyang.dev/posts/realtime-voice-on-one-mac/architecture.svg"&#10; &gt;&lt;/figure&gt;&#10;&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;The gateway&lt;/strong&gt;, Python 3.12 and asyncio, owns the protocol, sessions,&#10;echo cancellation, Silero VAD and Smart Turn, segmentation and barge-in.&#10;Its environment &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L68-L73" target="_blank" rel="noreferrer"&gt;has no &lt;code&gt;mlx&lt;/code&gt; packages installed&lt;/a&gt;, so it&#10;cannot load an MLX model by construction; Silero and Smart Turn run on&#10;the CPU under onnxruntime.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Three MLX backends&lt;/strong&gt; on localhost, each its own uv project:&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L620-L627" target="_blank" rel="noreferrer"&gt;Qwen3-ASR-1.7B 8-bit, Qwen3.6-35B-A3B 4-bit under &lt;code&gt;mlx_lm.server&lt;/code&gt;, and&#10;Qwen3-TTS-0.6B CustomVoice 8-bit with its 4-bit build as the fallback&lt;/a&gt;,&#10;every one pinned by revision.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;Resident after warmup on the real stack:&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L628-L629" target="_blank" rel="noreferrer"&gt;LLM 22.9 GB with 64 cached prompts, TTS 3.8 GB, ASR 2.5 GB, gateway 0.25 GB&lt;/a&gt;,&#10;about 29.5 GB in all, more than the README&amp;rsquo;s &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L39-L48" target="_blank" rel="noreferrer"&gt;&amp;ldquo;about 25 GB&amp;rdquo;&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;Cascade Realtime Gateway is&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L26-L27" target="_blank" rel="noreferrer"&gt;&amp;ldquo;a design reference only, not a client of this service&amp;rdquo;&lt;/a&gt;:&#10;mlx-realtime reuses its protocol decisions and lessons where they are&#10;GA-faithful. The service is&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L39-L48" target="_blank" rel="noreferrer"&gt;sized for development and testing&lt;/a&gt;: at most four sessions,&#10;because a human call AICC transcribes&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L80-L90" target="_blank" rel="noreferrer"&gt;opens two transcription sessions, one per leg&lt;/a&gt;.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;The deadlines&#10; &lt;div id="the-deadlines" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-deadlines" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;AICC&amp;rsquo;s client &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L63-L69" target="_blank" rel="noreferrer"&gt;enforces four deadlines&lt;/a&gt;:&lt;/p&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;From&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;To&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Limit&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;response.create&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;response.created&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;3 s&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;response.created&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;first audio delta&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;3 s&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;one audio delta&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;the next&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;under 2 s&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;response.created&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;response.done&lt;/code&gt;, for a tool-only reply&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;3 s&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;What happens when one is missed:&#10;&amp;ldquo;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L149" target="_blank" rel="noreferrer"&gt;A missed deadline silently abandons the turn.&lt;/a&gt;&amp;rdquo; No error,&#10;no retry. AICC also &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L156" target="_blank" rel="noreferrer"&gt;streams caller audio continuously, including while the&#10;bot speaks, and has no echo cancellation&lt;/a&gt;. Both facts shape&#10;everything below.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Where 1.4 seconds goes&#10; &lt;div id="where-14-seconds-goes" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#where-14-seconds-goes" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The benchmark &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L15-L23" target="_blank" rel="noreferrer"&gt;replays 126 public utterances&lt;/a&gt; (FLEURS Chinese and&#10;English, ASCEND code-switched conversation, 1.6 to 9.9 s each) in 20 ms&#10;frames at real time, with &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L29-L40" target="_blank" rel="noreferrer"&gt;AICC&amp;rsquo;s own session settings&lt;/a&gt;:&#10;&lt;code&gt;server_vad&lt;/code&gt; with 500 ms of silence. The machine was&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L110-L112" target="_blank" rel="noreferrer"&gt;an M3 Max with 128 GB on AC power, High Power mode, nothing else on the GPU&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;&lt;figure&gt;&lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="low"&#10; alt="Stacked bar of the median time per stage for one call on the telephone channel, G.711 μ-law: end of speech detected 525 ms, of which 500 ms is AICC&amp;rsquo;s silence setting; Smart Turn 68 ms; ASR 220 ms; LLM first token 318 ms; first sentence 96 ms; TTS first chunk 76 ms; first audio sent 102 ms; about 1.4 s in all"&#10; src="https://rasonyang.dev/posts/realtime-voice-on-one-mac/stages.svg"&#10; &gt;&lt;/figure&gt;&#10;&lt;/p&gt;&#10;&lt;p&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L130-L141" target="_blank" rel="noreferrer"&gt;Per stage, one session&lt;/a&gt;, on the telephone variant (8 kHz G.711&#10;μ-law in):&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;522–525 ms is the caller&amp;rsquo;s own silence.&lt;/strong&gt; The VAD&amp;rsquo;s end-of-speech&#10;decision is &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L59-L67" target="_blank" rel="noreferrer"&gt;&lt;code&gt;silence_duration_ms&lt;/code&gt; plus detection&lt;/a&gt;, and AICC&#10;asks for 500 ms. More than a third of the wait is a setting the client&#10;chose.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;The models take about 610 ms&lt;/strong&gt;: ASR 220 ms, LLM first token 318 ms,&#10;TTS first chunk 76 ms.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;The ~100 ms of &lt;code&gt;first_audio_sent&lt;/code&gt; is silence.&lt;/strong&gt; Qwen3-TTS&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L149-L150" target="_blank" rel="noreferrer"&gt;puts about 0.45 s of silence before every segment&lt;/a&gt;; the TTS&#10;synthesizes it and the trimmer drops it.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L143-L147" target="_blank" rel="noreferrer"&gt;The sum of the stage medians matches the end-to-end median&lt;/a&gt;. The&#10;P95 is another story. Over all turns it is 3.78 to 3.87 s, because&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L143-L147" target="_blank" rel="noreferrer"&gt;Smart Turn held 12–13 turns per run as incomplete&lt;/a&gt; until its 3 s&#10;ceiling. Ten of those are ASCEND excerpts that really do stop mid-sentence&#10;(&amp;ldquo;我认为现在的AI，比如说在。&amp;rdquo;). That is why the headline figures are&#10;quoted on turns that ended on a complete verdict, n = 113–114 per run.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Fitting the LLM into 3 seconds&#10; &lt;div id="fitting-the-llm-into-3-seconds" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#fitting-the-llm-into-3-seconds" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The median turn is the easy case. The hard ones are AICC&amp;rsquo;s phase changes:&#10;the flow swaps the instructions, then the caller asks to be transferred, and&#10;the model has to emit a &lt;code&gt;transfer_to_agent&lt;/code&gt; call inside the tool-only&#10;window.&lt;/p&gt;&#10;&lt;p&gt;Qwen3.6 is &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L604-L609" target="_blank" rel="noreferrer"&gt;hybrid, Gated DeltaNet plus attention, so its cache cannot be&#10;trimmed back&lt;/a&gt;. &lt;code&gt;mlx_lm.server&lt;/code&gt; can reuse only whole prefixes it&#10;saved, so each of AICC&amp;rsquo;s phase changes re-prefilled about 1650 tokens.&#10;The LLM backend now saves checkpoints where the tool definitions end&#10;and where the system message stops matching recent ones, so a phase change&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L604-L609" target="_blank" rel="noreferrer"&gt;prefills ~190 tokens, not ~1650&lt;/a&gt;. Together with warming the&#10;prefix before the caller&amp;rsquo;s next turn,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1503-L1507" target="_blank" rel="noreferrer"&gt;median prefill fell from 1625 tokens to 47 on a cold call, and from 1711 to 123 on AICC&amp;rsquo;s turn after a tool result&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1476-L1483" target="_blank" rel="noreferrer"&gt;Measured on the LLM alone&lt;/a&gt; (mock ASR and TTS, one call at a time,&#10;20 runs per case, so p95 is the maximum), from &lt;code&gt;response.created&lt;/code&gt; to the&#10;first sentence boundary or, for a tool-only reply, to &lt;code&gt;response.done&lt;/code&gt;:&lt;/p&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Case&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Before: no warming, stock cache (p50 / p95 s)&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Checkpoints + warming&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;A cold call, new instructions&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1509-L1521" target="_blank" rel="noreferrer"&gt;1.32 / 1.38&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;0.41 / 0.49&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Transfer after an instructions change (tool-only)&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1509-L1521" target="_blank" rel="noreferrer"&gt;2.89 / 3.46&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;1.50 / 1.88&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;AICC&amp;rsquo;s reply to a tool result (spoken)&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1509-L1521" target="_blank" rel="noreferrer"&gt;1.48 / 1.80&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;0.52 / 0.64&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;AICC&amp;rsquo;s next turn is a transfer (tool-only)&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1509-L1521" target="_blank" rel="noreferrer"&gt;3.02 / 4.29&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;1.53 / 2.08&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;Without them, transfers missed AICC&amp;rsquo;s 3 s window at p95 even before the ASR&#10;was counted. With both changes,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1569-L1582" target="_blank" rel="noreferrer"&gt;every case is inside it at p95, the tightest at 2.17–2.58 s with ASR&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;The other LLM problem was not latency. In M0,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L742-L745" target="_blank" rel="noreferrer"&gt;11% of realistic transfer turns ended with &lt;code&gt;finish_reason: tool_calls&lt;/code&gt; and no parsed call&lt;/a&gt;:&#10;Qwen3.6 writes &lt;code&gt;&amp;lt;transfer_to_agent&amp;gt;&lt;/code&gt; instead of &lt;code&gt;&amp;lt;function=transfer_to_agent&amp;gt;&lt;/code&gt;.&#10;The LLM backend now&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L600-L603" target="_blank" rel="noreferrer"&gt;rewrites that form for tools the request declared, and never for undeclared names&lt;/a&gt;.&#10;Live, &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1560-L1563" target="_blank" rel="noreferrer"&gt;60 of 60 transfers were delivered, 4 of them recovered that way&lt;/a&gt;.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;The third call&#10; &lt;div id="the-third-call" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-third-call" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The TTS server &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L632-L633" target="_blank" rel="noreferrer"&gt;serves streams strictly one at a time&lt;/a&gt;. In M0,&#10;with two concurrent requests the second waited 2.9 s for its first chunk;&#10;with four, the last waited 8.4 s. The LLM batches and the ASR takes four at&#10;once, so &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L80-L90" target="_blank" rel="noreferrer"&gt;the TTS queue is the capacity limit of bot calls&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;The gateway shares it as well as it can. The queue is&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L416" target="_blank" rel="noreferrer"&gt;earliest-deadline first&lt;/a&gt;: the session that will run out of audio&#10;soonest goes next. An &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L1074-L1080" target="_blank" rel="noreferrer"&gt;audio reserve&lt;/a&gt; holds back up to 1.5 s&#10;of synthesized audio and releases it a delta at a time, to keep AICC&amp;rsquo;s 2 s&#10;gap watchdog fed. That was not enough. In the two-session&#10;run &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L198-L203" target="_blank" rel="noreferrer"&gt;one answer&amp;rsquo;s second segment waited 2.3 s behind another session&amp;rsquo;s long&#10;segment, and the wire went quiet for 2092 ms&lt;/a&gt;: AICC would have&#10;abandoned the turn. Now, when the reserve runs dry and more audio is&#10;coming, the gateway &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/DECISIONS.md#L44" target="_blank" rel="noreferrer"&gt;sends 20 ms of silence every second&lt;/a&gt;. In&#10;the three-session soak, 23 responses were bridged that way and no gap&#10;exceeded 1.0 s.&lt;/p&gt;&#10;&lt;p&gt;Then the first-audio window, the one that does not bend:&lt;/p&gt;&#10;&lt;p&gt;&lt;figure&gt;&lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="low"&#10; alt="Horizontal bars of AICC&amp;rsquo;s first-audio window, response.created to the first audio delta at the client, for one, two and three sessions, against AICC&amp;rsquo;s 3 s limit: P95 0.94–0.97 s and maximum 1.1 s with one session; P95 1.27–1.96 s and maximum 2.65 s with two; P95 1.94–2.23 s and maximum 3.39 s with three, past the limit. Beside each row, the TTS first chunk P95 (79–80 ms, 0.6–1.0 s, 1.1–1.4 s) and backend_busy failures (0, 1, 7)"&#10; src="https://rasonyang.dev/posts/realtime-voice-on-one-mac/sessions.svg"&#10; &gt;&lt;/figure&gt;&#10;&lt;/p&gt;&#10;&lt;p&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L158-L170" target="_blank" rel="noreferrer"&gt;Across three runs per configuration&lt;/a&gt;, the medians barely move.&#10;The tails are the TTS queue:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;TTS first chunk, P95&lt;/strong&gt;: 79–80 ms at one session, 645–1036 ms at two,&#10;1135–1420 ms at three. A segment of N seconds of speech&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L172-L177" target="_blank" rel="noreferrer"&gt;holds the TTS for about N/4 s&lt;/a&gt;, and the second segment of a&#10;one-sentence answer is often 6–9 s of speech.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;End to end, complete verdicts, P95&lt;/strong&gt;: 1.52–1.54 s at one,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L21-L25" target="_blank" rel="noreferrer"&gt;2.53–2.78 s at three&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Failures&lt;/strong&gt;: &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L179-L183" target="_blank" rel="noreferrer"&gt;7 responses at three sessions failed with &lt;code&gt;backend_busy&lt;/code&gt;, 1.8%&lt;/a&gt;:&#10;their segment waited in the queue more than 2 s past its deadline.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;AICC&amp;rsquo;s window&lt;/strong&gt;: &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L204-L207" target="_blank" rel="noreferrer"&gt;one first audio at 3.39 s&lt;/a&gt;, of which 1.92 s&#10;was the TTS queue, and one failed response at 3.22 s. AICC would have&#10;abandoned both.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;The bench is also busier than a call:&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L154-L156" target="_blank" rel="noreferrer"&gt;each session talks without a pause&lt;/a&gt;. That makes the three-call&#10;numbers pessimistic, not wrong.&lt;/p&gt;&#10;&lt;p&gt;The shared GPU makes it worse, and merging processes would not help.&#10;While the LLM decodes, the TTS&amp;rsquo;s real-time factor rises&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L264-L275" target="_blank" rel="noreferrer"&gt;from 0.25 to 0.33–0.38&lt;/a&gt;. While the LLM prefilled ~16k&#10;tokens it reached 2.08, half real time, and any session speaking then would&#10;starve. AICC&amp;rsquo;s prompts are ~1.6k tokens, a tenth of that. And merging&#10;the three model processes into one&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/DECISIONS.md#L42" target="_blank" rel="noreferrer"&gt;would fix nothing&lt;/a&gt;: &amp;ldquo;the capacity limit is the TTS stream itself&amp;rdquo;.&lt;/p&gt;&#10;&lt;p&gt;What would help is&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/DECISIONS.md#L47" target="_blank" rel="noreferrer"&gt;more TTS capacity or shorter waits&lt;/a&gt;: a second TTS process (a code&#10;change, since the provider takes one &lt;code&gt;base_url&lt;/code&gt;), mlx-audio&amp;rsquo;s batched&#10;generation, capping continuation segments so none holds the TTS for more&#10;than about a second, or a longer queue timeout now that the keepalive feeds&#10;the watchdog. None is done yet.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;The service hears itself&#10; &lt;div id="the-service-hears-itself" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-service-hears-itself" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;Echo on these calls is mlx-realtime&amp;rsquo;s problem.&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L822-L829" target="_blank" rel="noreferrer"&gt;Neither AICC nor FreeSWITCH cancels it&lt;/a&gt;, and the server cancels&#10;its own response when it hears the caller, so it &amp;ldquo;would therefore interrupt&#10;itself on its own echo, and AICC&amp;rsquo;s 800 ms barge guard cannot prevent that&amp;rdquo;.&#10;On simulated telephone echo with no canceller,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L514-L515" target="_blank" rel="noreferrer"&gt;Silero fired on 75% of echo-only windows: 1005 false &lt;code&gt;speech_started&lt;/code&gt; in 96 calls&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;One canceller could not serve both consumers. On simulated G.168 echo paths:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;WebRTC AEC3&lt;/strong&gt; keeps the VAD quiet, but its suppressor removes quiet&#10;caller speech. Offline, in continuous double talk at −15 dB,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L704-L717" target="_blank" rel="noreferrer"&gt;it cost the English ASR +28.4 points of WER&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;SpeexDSP&amp;rsquo;s linear canceller&lt;/strong&gt; keeps the caller intelligible, but&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L704-L717" target="_blank" rel="noreferrer"&gt;its residual triggers the VAD&lt;/a&gt;: &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/SPIKE.md#L517-L526" target="_blank" rel="noreferrer"&gt;0 of 96 calls&lt;/a&gt;&#10;kept the VAD quiet on echo alone.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;So each session &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L831-L836" target="_blank" rel="noreferrer"&gt;runs both&lt;/a&gt;: AEC3 in front of Silero and Smart&#10;Turn, Speex with its preprocessor in front of the ASR. The reference for&#10;both is the bot&amp;rsquo;s own output,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L838-L849" target="_blank" rel="noreferrer"&gt;placed on a model of AICC&amp;rsquo;s playout&lt;/a&gt;. Through the gateway&amp;rsquo;s own&#10;path, still on simulated echo,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L1797-L1816" target="_blank" rel="noreferrer"&gt;the dual path kept double-talk error within its limits in all 8 cells&lt;/a&gt;:&#10;the worst was continuous double talk at −15 dB, +1.8 points in Chinese and&#10;+1.6 in English, against +13.7 and +33.2 for AEC3 alone.&lt;/p&gt;&#10;&lt;p&gt;A line without echo is its own failure. AEC3&amp;rsquo;s suppressor then takes a&#10;caller who talks over the bot for echo, and&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L858-L866" target="_blank" rel="noreferrer"&gt;silenced them by up to 44 dB, so barge-in came 2.5–3.4 s late or not at all&lt;/a&gt;.&#10;AEC3 is now bypassed after 1.5 s of playing with no echo seen. The first&#10;version of that bypass let chance correlation peaks in the caller&amp;rsquo;s own&#10;speech turn it back on; requiring three agreeing windows cut&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/DECISIONS.md#L45" target="_blank" rel="noreferrer"&gt;false resumes from 29 to 2, and the SDK barge-in on the real stack from 1503 ms to 539 ms&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;I have since run real calls through AICC and FreeSWITCH: the bot did not&#10;cut itself off on its own echo, and the ASR kept up. That is an&#10;observation, not a measurement.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;What I am not claiming&#10; &lt;div id="what-i-am-not-claiming" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-i-am-not-claiming" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Live echo numbers.&lt;/strong&gt; Every echo figure above is on&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L869-L870" target="_blank" rel="noreferrer"&gt;simulated G.168 echo paths&lt;/a&gt;. The real calls were observed,&#10;not measured: there is no echo path, ERL, delay or double-talk number&#10;from a real line, and&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/PROGRESS.md#L2303" target="_blank" rel="noreferrer"&gt;the repository still lists that measurement as deferred&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Recorded phone calls.&lt;/strong&gt; The latency corpus is public read speech and&#10;conversation excerpts &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L15-L23" target="_blank" rel="noreferrer"&gt;sent through a telephone codec&lt;/a&gt;, and&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L29-L40" target="_blank" rel="noreferrer"&gt;the bench plays nothing back, so it has no echo&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;One code version.&lt;/strong&gt; &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L117-L121" target="_blank" rel="noreferrer"&gt;The one-, two- and three-session runs ran on&#10;&lt;code&gt;bb75826&lt;/code&gt; plus a working tree that gained fixes between runs&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Statistics.&lt;/strong&gt; One machine, one run per audio variant, three variants&#10;per session count, no confidence intervals on latency.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Approved targets.&lt;/strong&gt; The per-stage budget and the latency targets are&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/DECISIONS.md#L41" target="_blank" rel="noreferrer"&gt;proposed, not approved&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Capacity.&lt;/strong&gt; One call meets AICC&amp;rsquo;s deadlines;&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md#L329-L334" target="_blank" rel="noreferrer"&gt;three do not reliably&lt;/a&gt;. Two calls sit in between: one&#10;&lt;code&gt;backend_busy&lt;/code&gt; and the one 2 s gap that led to the keepalive silence.&#10;Two calls have not been rerun since.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;A comparison with Cascade or a hosted model.&lt;/strong&gt; Different datasets,&#10;different networks, different hardware.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Production.&lt;/strong&gt; The service is&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L39-L48" target="_blank" rel="noreferrer"&gt;sized for development and testing&lt;/a&gt;. WebRTC and SIP&#10;transports, Docker, multiple machines and persistence are&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md#L40-L41" target="_blank" rel="noreferrer"&gt;out of scope&lt;/a&gt;.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Run it&#10; &lt;div id="run-it" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#run-it" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;On an Apple Silicon Mac with roughly 30 GB of memory to spare,&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L50-L59" target="_blank" rel="noreferrer"&gt;from the README&lt;/a&gt;:&lt;/p&gt;&#10;&lt;div class="highlight-wrapper"&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;make models-mlx &lt;span class="c1"&gt;# once: the pinned MLX models, ~25 GB of downloads&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;make dev &lt;span class="c1"&gt;# the backends and the gateway, on 127.0.0.1:8000&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;make bench &lt;span class="c1"&gt;# the latency and ASR report against the running gateway&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&#10;&lt;p&gt;Then point any Realtime client at it, &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L74-L94" target="_blank" rel="noreferrer"&gt;as the README&amp;rsquo;s SDK example does&lt;/a&gt;:&lt;/p&gt;&#10;&lt;div class="highlight-wrapper"&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;AsyncOpenAI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;base_url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;http://127.0.0.1:8000/v1&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;api_key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;local&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&#10;&lt;p&gt;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/README.md#L133" target="_blank" rel="noreferrer"&gt;&lt;code&gt;make bench&lt;/code&gt;&lt;/a&gt; writes the report whose numbers are above;&#10;&lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md" target="_blank" rel="noreferrer"&gt;LATENCY.md&lt;/a&gt; explains how each one is produced.&lt;/p&gt;&#10;&lt;p&gt;Code: &lt;a href="https://github.com/rasonyang/mlx-realtime" target="_blank" rel="noreferrer"&gt;mlx-realtime&lt;/a&gt; · &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/LATENCY.md" target="_blank" rel="noreferrer"&gt;docs/LATENCY.md&lt;/a&gt; · &lt;a href="https://github.com/rasonyang/mlx-realtime/blob/8234db193cc3e007c0408f29c4b1e1e508f98772/docs/ARCHITECTURE.md" target="_blank" rel="noreferrer"&gt;docs/ARCHITECTURE.md&lt;/a&gt;&lt;/p&gt;&#10;</description><media:content xmlns:media="http://search.yahoo.com/mrss/" url="https://rasonyang.dev/posts/realtime-voice-on-one-mac/featured.jpg"/></item><item><title>What One 4-vCPU FreeSBC Carries</title><link>https://rasonyang.dev/posts/freesbc-load-test-results/</link><pubDate>Tue, 20 Oct 2026 08:51:38 +0800</pubDate><guid>https://rasonyang.dev/posts/freesbc-load-test-results/</guid><description>&lt;p&gt;The &lt;a href="https://rasonyang.dev/posts/ai-native-call-center-in-four-projects/#what-i-am-not-claiming-yet" &gt;first post in this series&lt;/a&gt; ended with a promise: FreeSBC&amp;rsquo;s&#10;capacity numbers would get their own post, with the setup next to them.&#10;Here they are. On one 4 vCPU cloud VM, FreeSBC v0.5.0 relayed&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L13" target="_blank" rel="noreferrer"&gt;5000 concurrent G.711 calls with both media legs anchored&lt;/a&gt;,&#10;set up &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L15-L17" target="_blank" rel="noreferrer"&gt;3500 calls per second from a carrier without loss&lt;/a&gt;, added&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L21" target="_blank" rel="noreferrer"&gt;less than 1 ms&lt;/a&gt; to call setup, and ran&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L14" target="_blank" rel="noreferrer"&gt;8 hours at 3500 concurrent calls with no growth&lt;/a&gt; in goroutines,&#10;memory, file descriptors or ports.&lt;/p&gt;&#10;&lt;p&gt;FreeSBC is a &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/README.md#L3-L7" target="_blank" rel="noreferrer"&gt;pure-Go SIP/WebRTC edge proxy: one binary, one YAML file&lt;/a&gt;.&#10;It anchors RTP, &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/README.md#L114" target="_blank" rel="noreferrer"&gt;does no transcoding&lt;/a&gt;, and&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/README.md#L115" target="_blank" rel="noreferrer"&gt;terminates ICE-Lite and DTLS-SRTP&lt;/a&gt; for browsers. Every number&#10;below comes from &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L3" target="_blank" rel="noreferrer"&gt;one formal three-host run&lt;/a&gt; on 2026-10-18 and&#10;2026-10-19, and the harness that produced them is in the repository.&lt;/p&gt;&#10;&lt;p&gt;I have not run Kamailio, OpenSIPS, rtpengine or a commercial SBC on the same&#10;rig, so this post does not say FreeSBC is faster than any of them. It gives&#10;numbers in units you can hold against your own: calls per vCPU, call setups&#10;per second on 4 vCPU, added latency, and drift over 8 hours. It also re-runs&#10;one commercial SBC&amp;rsquo;s own published test.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;The numbers&#10; &lt;div id="the-numbers" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-numbers" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;What&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Result on 4 vCPU&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Kind&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Concurrent G.711 calls, both legs anchored&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L13" target="_blank" rel="noreferrer"&gt;5000, CPU 310 of 400&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;knee at the 80% CPU line&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Call setup, carrier to switch&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L15-L17" target="_blank" rel="noreferrer"&gt;3500 CPS, loss-free&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;knee&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Call setup, switch to carrier&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L15-L17" target="_blank" rel="noreferrer"&gt;3250 CPS&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;knee&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Call setup, client to switch&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L15-L17" target="_blank" rel="noreferrer"&gt;2250 CPS or more&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;lower bound&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Latency added to INVITE → 200&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L21" target="_blank" rel="noreferrer"&gt;below 1 ms&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;measured&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;8 h soak at 3500 concurrent calls&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L14" target="_blank" rel="noreferrer"&gt;432,000 calls, 0 failed, no growth&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;measured&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Registrations held / refreshed&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L18" target="_blank" rel="noreferrer"&gt;19,000 / 20,000 REGISTER/s or more&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;lower bound&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;WebRTC, DTLS-SRTP&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L19" target="_blank" rel="noreferrer"&gt;100 handshakes/s, 600 calls, or more&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;lower bound&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;FreeSWITCH 1.10.0 as the switch&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L20" target="_blank" rel="noreferrer"&gt;3000 calls, 1500 CPS, or more&lt;/a&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;lower bound&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;A knee is the last step before the first one that failed the pass line. A&#10;lower bound means&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L5" target="_blank" rel="noreferrer"&gt;the generator, the switch or the test range ran out before FreeSBC did&lt;/a&gt;.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;The rig&#10; &lt;div id="the-rig" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-rig" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;Three cloud VMs on Alibaba Cloud, &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L70" target="_blank" rel="noreferrer"&gt;all Ubuntu 24.04 on kernel 6.8, with real&#10;NICs on two subnets of one private network&lt;/a&gt;. One subnet plays the&#10;public side, the other the private side. The &lt;a href="https://github.com/rasonyang/freesbc/issues/109#issuecomment-6249040280" target="_blank" rel="noreferrer"&gt;instance types&lt;/a&gt; and&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L72-L76" target="_blank" rel="noreferrer"&gt;CPUs&lt;/a&gt;:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;FreeSBC&lt;/strong&gt;: ecs.c9i.xlarge, Intel Xeon 6982P-C, 4 vCPU (2 physical cores&#10;with hyper-threading), 7.4 GiB usable. &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L78" target="_blank" rel="noreferrer"&gt;v0.5.0, Go 1.27.2, sipgo v1.6.0,&#10;&lt;code&gt;GOMEMLIMIT=6GiB&lt;/code&gt;&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Load generator&lt;/strong&gt;: ecs.c8a.2xlarge, AMD EPYC 9T24, 8 vCPU, running SIPp&#10;3.6.1 and &lt;code&gt;webrtcload&lt;/code&gt;, the repository&amp;rsquo;s WebRTC load client.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Switch&lt;/strong&gt;: ecs.c9i.2xlarge, Xeon 6982P-C, 8 vCPU, running a&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L79" target="_blank" rel="noreferrer"&gt;SIPp UAS that answers at once and echoes RTP&lt;/a&gt;, or FreeSWITCH&#10;1.10.0 for one set of runs.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;&lt;figure&gt;&lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="low"&#10; alt="Topology: a load generator on the public-side subnet sends SIP over UDP and G.711 RTP to FreeSBC (4 vCPU, Xeon 6982P-C), which relays signaling and media to a switch on the private-side subnet; below it, the pass line every figure must meet"&#10; src="https://rasonyang.dev/posts/freesbc-load-test-results/topology.svg"&#10; &gt;&lt;/figure&gt;&#10;&lt;/p&gt;&#10;&lt;p&gt;Signaling is UDP. Media is &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L80" target="_blank" rel="noreferrer"&gt;G.711 at 20 ms, in both directions, with both&#10;legs anchored by FreeSBC&lt;/a&gt;, so every RTP packet enters and leaves&#10;FreeSBC on each leg. FreeSBC was &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L82" target="_blank" rel="noreferrer"&gt;restarted before each ladder value&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;A figure only counts if its run had&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L81" target="_blank" rel="noreferrer"&gt;under 0.1% failed calls, FreeSBC under 80% of its cores, RTP sent over&#10;received of at least 0.999 with zero kernel receive-buffer errors, and the&#10;generator and the switch each under 90% of a core&lt;/a&gt;. CPU is quoted&#10;with 100 meaning one vCPU, so the line on this box is 320 of 400. The last&#10;check matters most. &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L68-L70" target="_blank" rel="noreferrer"&gt;SIPp is mostly single-threaded per instance&lt;/a&gt;,&#10;and a fast proxy measured with a saturated generator looks like a slow proxy.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Media: about 1600 calls per vCPU&#10; &lt;div id="media-about-1600-calls-per-vcpu" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#media-about-1600-calls-per-vcpu" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The concurrent-call ladder &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L125-L137" target="_blank" rel="noreferrer"&gt;stepped from 250 to 5500 calls&lt;/a&gt;,&#10;each with a 60 s ramp and two 60 s holds. CPU grew in a straight line: 20 at&#10;250 calls, 130 at 2000, 251 at 4000. At 5000 calls it was&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L127-L136" target="_blank" rel="noreferrer"&gt;310 average and 338 peak&lt;/a&gt;, with 500,588 RTP packets per second&#10;each way, 373 MB of RSS and 0 failed calls. That is&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L140" target="_blank" rel="noreferrer"&gt;668 Mbit/s each way&lt;/a&gt;. At 5500 the CPU crossed the line and the&#10;generator&amp;rsquo;s SIPp was at 94% of a core, so the figure is&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L139" target="_blank" rel="noreferrer"&gt;5000, or slightly more, generator-bound above&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;Divided out, that is &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L33" target="_blank" rel="noreferrer"&gt;about 1600 concurrent calls per vCPU&lt;/a&gt;,&#10;and &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L36" target="_blank" rel="noreferrer"&gt;about 0.07 MB of RSS per call&lt;/a&gt;, baseline included.&#10;The per-vCPU figure is arithmetic: only the 4 vCPU size ran this ladder.&lt;/p&gt;&#10;&lt;p&gt;The profile at 5000 calls shows where the CPU goes:&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L178" target="_blank" rel="noreferrer"&gt;85% cumulative in the forward loop, and about 66 to 70% flat in the&#10;&lt;code&gt;sendto&lt;/code&gt; and &lt;code&gt;recvfrom&lt;/code&gt; syscalls&lt;/a&gt;. One packet in, one packet&#10;out, one syscall each. GC is not among the top entries. Batching the&#10;syscalls per socket &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L25" target="_blank" rel="noreferrer"&gt;would save few of them&lt;/a&gt;, so that issue&#10;was closed as not worth it at this architecture.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Signaling: thousands of setups per second, under 5 ms&#10; &lt;div id="signaling-thousands-of-setups-per-second-under-5-ms" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#signaling-thousands-of-setups-per-second-under-5-ms" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The call-setup ladders &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L100-L102" target="_blank" rel="noreferrer"&gt;run without media, with 2 s calls, in 40 s steps&#10;from 250 CPS&lt;/a&gt;, one per direction. Each knee then got a confirm&#10;run at 90% of it for 120 s.&lt;/p&gt;&#10;&lt;p&gt;&lt;figure&gt;&lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="low"&#10; alt="Bar chart of the highest passing call setup rate on 4 vCPU: carrier to switch 3500 CPS loss-free, first failing step 5000; switch to carrier 3250 CPS, first failing step 3500; client to switch at least 2250 CPS, a lower bound, first failing step 2500 on the per-source cap"&#10; src="https://rasonyang.dev/posts/freesbc-load-test-results/cps.svg"&#10; &gt;&lt;/figure&gt;&#10;&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Carrier to switch: 3500 CPS.&lt;/strong&gt; CPU averaged&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L104-L108" target="_blank" rel="noreferrer"&gt;190 of 400 at the knee, with 4296 MB of RSS&lt;/a&gt;. Calls kept&#10;succeeding past it: &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L111" target="_blank" rel="noreferrer"&gt;0.047% failed at 4750 CPS, and the first failing step&#10;was 5000&lt;/a&gt;. What ends the loss-free range is the kernel&#10;dropping datagrams on FreeSBC&amp;rsquo;s UDP receive buffers from 3750 CPS, while&#10;the CPU was at 209 to 265 of 400. That is a receive-path limit, open as&#10;&lt;a href="https://github.com/rasonyang/freesbc/issues/235" target="_blank" rel="noreferrer"&gt;#235&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Switch to carrier: 3250 CPS.&lt;/strong&gt; At 3500 the carrier leg&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L104-L108" target="_blank" rel="noreferrer"&gt;timed out on 0.204% of calls&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Client to switch: 2250 CPS or more.&lt;/strong&gt; This one is a floor. FreeSBC&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/internal/edge/invite.go#L35" target="_blank" rel="noreferrer"&gt;allows 64 unanswered INVITEs per public source IP&lt;/a&gt;, and the&#10;generator has one IP. At 2500 CPS &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L110" target="_blank" rel="noreferrer"&gt;all 248 failures were that cap, with&#10;FreeSBC at 37% CPU&lt;/a&gt;.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;None of the three knees is a CPU limit:&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L24" target="_blank" rel="noreferrer"&gt;FreeSBC averaged 33 to 48% of its CPU at all of them&lt;/a&gt;.&#10;The confirm runs passed every call:&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L113" target="_blank" rel="noreferrer"&gt;243,000 at 2025 CPS, 378,000 at 3150 and 351,000 at 2925&lt;/a&gt;,&#10;with INVITE-to-200 p95 under 5 ms at the knees.&lt;/p&gt;&#10;&lt;p&gt;FreeSBC&amp;rsquo;s own share of that latency was measured separately at 1000&#10;concurrent calls. INVITE to 200&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L141" target="_blank" rel="noreferrer"&gt;averaged 3.0 ms through FreeSBC and 3.0 ms straight to the switch&lt;/a&gt;.&#10;The difference is below SIPp&amp;rsquo;s 1 ms resolution.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;It doesn&amp;rsquo;t drift&#10; &lt;div id="it-doesnt-drift" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#it-doesnt-drift" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;A proxy that leaks a little per call passes every ladder and fails a week&#10;later. The soak held &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L185" target="_blank" rel="noreferrer"&gt;3500 concurrent calls, 70% of the media maximum, for&#10;8 hours&lt;/a&gt;: 15 new calls per second, each held 240 s.&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L187-L197" target="_blank" rel="noreferrer"&gt;The results&lt;/a&gt;:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;432,000 calls, 0 failed, 0 retransmissions.&lt;/li&gt;&#10;&lt;li&gt;CPU 235 average, 245 peak, out of 400.&lt;/li&gt;&#10;&lt;li&gt;From the first quarter of the run to the last: goroutines +0.0%, RSS&#10;−0.3%, open file descriptors +0.0%, ports in use +0.0%. RSS peaked at&#10;240 MB.&lt;/li&gt;&#10;&lt;li&gt;10.37 billion RTP packets each way, sent over received 1.0000.&lt;/li&gt;&#10;&lt;li&gt;After the load stopped: 33 goroutines, 13 file descriptors, 0 media&#10;ports.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;The &lt;a href="https://github.com/rasonyang/freesbc/issues/109#issuecomment-6256238968" target="_blank" rel="noreferrer"&gt;soak report&lt;/a&gt; has the raw medians: 22,113 goroutines in the first&#10;quarter and 22,113 in the last. No host was reclaimed or rebooted during&#10;the 8 hours.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;A commercial SBC&amp;rsquo;s own test, on one vCPU&#10; &lt;div id="a-commercial-sbcs-own-test-on-one-vcpu" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#a-commercial-sbcs-own-test-on-one-vcpu" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;One commercial SBC vendor publishes a load test of its own product:&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L135-L137" target="_blank" rel="noreferrer"&gt;50 calls per second up to 1000 concurrent calls, each held 600 s&lt;/a&gt;,&#10;on a &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L18-L22" target="_blank" rel="noreferrer"&gt;Xeon X5670 with 6 cores and 12 threads, 8 GB of RAM and Debian 10&lt;/a&gt;.&#10;The published result is &amp;ldquo;Used Memory 1730M, CPU Load 26%&amp;rdquo;. I am not naming&#10;the vendor. The test is the useful part.&lt;/p&gt;&#10;&lt;p&gt;The harness &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L153-L160" target="_blank" rel="noreferrer"&gt;reproduces that test&lt;/a&gt;: 2000 calls, G.711 both ways on&#10;both legs, a plateau of about 1180 s. It ran on all four vCPU, then with&#10;FreeSBC limited to two, then to one. &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L88-L94" target="_blank" rel="noreferrer"&gt;The results&lt;/a&gt;:&lt;/p&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;FreeSBC cores&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Failed&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;CPU avg / peak (100 = one vCPU)&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;RSS max&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Verdict&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;4 vCPU&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;0 of 2000&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;64 / 70&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;120 MB&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;PASS&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;2&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;0 of 2000&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;52 / 57&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;117 MB&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;PASS&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;1&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;0 of 2000&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;40 / 44&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;115 MB&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;PASS&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;Probe calls inside the plateau measured&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L96" target="_blank" rel="noreferrer"&gt;0.000% loss and 0.47 ms mean jitter&lt;/a&gt;. So one vCPU of this box&#10;carries the vendor&amp;rsquo;s test at 40% of that vCPU. CPU is lower on one core&#10;than on four because &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L97" target="_blank" rel="noreferrer"&gt;with one core the Go runtime has no idle processors&#10;spinning&lt;/a&gt;.&lt;/p&gt;&#10;&lt;p&gt;Four caveats before reading more into it:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;I ran FreeSBC. I did not run the vendor&amp;rsquo;s product; its numbers are as&#10;published.&lt;/li&gt;&#10;&lt;li&gt;The vendor &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L98" target="_blank" rel="noreferrer"&gt;does not define &amp;ldquo;CPU Load 26%&amp;rdquo;&lt;/a&gt;: it could be one&#10;core, all twelve threads, or a load average.&lt;/li&gt;&#10;&lt;li&gt;&amp;ldquo;Used Memory 1730M&amp;rdquo; is host memory. The comparable figure is&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L98" target="_blank" rel="noreferrer"&gt;the whole FreeSBC host at 855 MB used&lt;/a&gt;, not FreeSBC&amp;rsquo;s 120 MB&#10;RSS. The whole host&amp;rsquo;s CPU averaged 15.9% of its vCPUs.&lt;/li&gt;&#10;&lt;li&gt;The hardware is from different generations. The test plan&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L18-L22" target="_blank" rel="noreferrer"&gt;estimates the two boxes as roughly comparable in total compute, or this&#10;one somewhat lower&lt;/a&gt;: fewer cores here, each much faster than a&#10;2010-era Westmere core.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&#10;&lt;h2 class="relative group"&gt;The 32-second goroutine&#10; &lt;div id="the-32-second-goroutine" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-32-second-goroutine" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The earliest numbers had a problem the three-host run no longer shows. The&#10;first run was a single host, Docker on a laptop, with sipgo v1.4.3. At 2500&#10;CPS &lt;a href="https://github.com/rasonyang/freesbc/issues/170" target="_blank" rel="noreferrer"&gt;FreeSBC had 189,159 goroutines, and 159,195 of them (84%) sat in one&#10;stack&lt;/a&gt;: sipgo&amp;rsquo;s &lt;code&gt;TerminateGracefully&lt;/code&gt;, waiting about 32 s for each&#10;finished transaction&amp;rsquo;s timers. That cost about 76 goroutines and 2.3 MB of&#10;RSS per sustained call per second, and at 3500 CPS the process was killed&#10;at its 8 GiB limit.&lt;/p&gt;&#10;&lt;p&gt;Two changes removed it. &lt;a href="https://github.com/rasonyang/freesbc/pull/231" target="_blank" rel="noreferrer"&gt;sipgo v1.6.0&lt;/a&gt; returns from&#10;&lt;code&gt;TerminateGracefully&lt;/code&gt; at once for a finished UDP transaction and leaves the&#10;rest to its timers. &lt;a href="https://github.com/rasonyang/freesbc/pull/232" target="_blank" rel="noreferrer"&gt;PR #232&lt;/a&gt; stopped ended dialogs from staying&#10;reachable through &lt;code&gt;OnCancel&lt;/code&gt;, &lt;code&gt;watch2xx&lt;/code&gt; and backstop timers&#10;(&lt;a href="https://github.com/rasonyang/freesbc/issues/228" target="_blank" rel="noreferrer"&gt;#228&lt;/a&gt;).&lt;/p&gt;&#10;&lt;p&gt;On the three-host rig, &lt;a href="https://github.com/rasonyang/freesbc/issues/170#issuecomment-6249485186" target="_blank" rel="noreferrer"&gt;RSS came out at 1.22 to 1.31 MB per sustained CPS,&#10;and goroutines at 43.7 to 46.5&lt;/a&gt;, flat from 1000 to 3500 CPS. Flat is&#10;the point: the cost is per call, held for a fixed time, and goroutines fall&#10;back to about 33 after each run. It is not a leak. The before and after&#10;come from different rigs, one laptop against three VMs, so this is not a&#10;controlled comparison.&lt;/p&gt;&#10;&lt;p&gt;&lt;figure&gt;&lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="low"&#10; alt="Two bar pairs: RSS per sustained call per second fell from about 2.3 MB on the single-host rig with sipgo v1.4.3 to about 1.25 MB on the three-host rig with sipgo v1.6.0 and PR 232; goroutines per sustained call per second fell from about 76 to about 44"&#10; src="https://rasonyang.dev/posts/freesbc-load-test-results/rss-per-cps.svg"&#10; &gt;&lt;/figure&gt;&#10;&lt;/p&gt;&#10;&lt;p&gt;What remains is mostly heap. sipgo keeps each UDP transaction, with its&#10;request and response, for 32 s after a 2xx: Timer L, RFC 6026&amp;rsquo;s Accepted&#10;state. &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L58" target="_blank" rel="noreferrer"&gt;I left that alone on purpose&lt;/a&gt;; it is sipgo&amp;rsquo;s to&#10;change. Of the 44 goroutines per CPS, &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L51-L54" target="_blank" rel="noreferrer"&gt;31.7 were FreeSBC&amp;rsquo;s own&#10;&lt;code&gt;drainAcks&lt;/code&gt;&lt;/a&gt;, waiting the same 32 s for an ACK that&#10;follows a 2xx by one round trip. &lt;a href="https://github.com/rasonyang/freesbc/pull/243" target="_blank" rel="noreferrer"&gt;PR #243&lt;/a&gt; bounds that wait at 8 s,&#10;which &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L57" target="_blank" rel="noreferrer"&gt;cut goroutines per CPS from 32.1 to 8.1 in a local in-process&#10;test&lt;/a&gt;. It landed after the run, so every figure in this post&#10;predates it. The next three-host run will measure it.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;What I am not claiming&#10; &lt;div id="what-i-am-not-claiming" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-i-am-not-claiming" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Other sizes.&lt;/strong&gt; &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L279" target="_blank" rel="noreferrer"&gt;Only 4 vCPU was measured&lt;/a&gt;; one and two&#10;vCPU ran only the 1000-call test above. Per-vCPU figures are division.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Statistics.&lt;/strong&gt; &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L81" target="_blank" rel="noreferrer"&gt;Each ladder ran once, with one confirm run per&#10;knee&lt;/a&gt;. There are no confidence intervals.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Client call rate.&lt;/strong&gt; &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L273" target="_blank" rel="noreferrer"&gt;2250 CPS is bounded by one source IP&lt;/a&gt;,&#10;not by FreeSBC.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Carrier-in above 3500 CPS.&lt;/strong&gt; The kernel drops datagrams from 3750 CPS&#10;&lt;a href="https://github.com/rasonyang/freesbc/issues/109#issuecomment-6256238968" target="_blank" rel="noreferrer"&gt;while FreeSBC is at 52 to 66% CPU&lt;/a&gt;. The cause is open in&#10;&lt;a href="https://github.com/rasonyang/freesbc/issues/235" target="_blank" rel="noreferrer"&gt;#235&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Signaling CPU.&lt;/strong&gt; &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L177" target="_blank" rel="noreferrer"&gt;About half is in sipgo&amp;rsquo;s transaction layer, and GC&#10;takes 20 to 23%&lt;/a&gt;. Allocation per request is open in&#10;&lt;a href="https://github.com/rasonyang/freesbc/issues/173" target="_blank" rel="noreferrer"&gt;#173&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Untested paths.&lt;/strong&gt; &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L277" target="_blank" rel="noreferrer"&gt;SIP over TCP and TLS, SDES-SRTP, and switch-to-client&#10;calls were not loaded&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Memory at high call rates.&lt;/strong&gt; Memory follows calls per second, not&#10;concurrent calls. &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L42" target="_blank" rel="noreferrer"&gt;At 5000 CPS carrier-in RSS reached 5.8 GB&lt;/a&gt;&#10;on this 7.4 GiB host.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Other proxies.&lt;/strong&gt; No other open-source SBC or proxy was run on this rig.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Sizing, and how to rerun it&#10; &lt;div id="sizing-and-how-to-rerun-it" class="anchor"&gt;&lt;/div&gt;&#10; &#10; &lt;span&#10; class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"&gt;&#10; &lt;a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#sizing-and-how-to-rerun-it" aria-label="Anchor"&gt;#&lt;/a&gt;&#10; &lt;/span&gt;&#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;A rule of thumb for this hardware class, from the&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L27-L45" target="_blank" rel="noreferrer"&gt;sizing guide&lt;/a&gt;:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Media&lt;/strong&gt;: &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L33" target="_blank" rel="noreferrer"&gt;about 1600 anchored G.711 calls per vCPU&lt;/a&gt;, and&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L36" target="_blank" rel="noreferrer"&gt;about 0.07 MB of RAM per call&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Call rate&lt;/strong&gt;: &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L42" target="_blank" rel="noreferrer"&gt;about 1.2 GB of RAM per 1000 sustained CPS, plus&#10;headroom&lt;/a&gt;. Set &lt;code&gt;GOMEMLIMIT&lt;/code&gt; to&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L62" target="_blank" rel="noreferrer"&gt;80 to 90% of the memory the process may use&lt;/a&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Ports&lt;/strong&gt;: the default RTP range&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L40" target="_blank" rel="noreferrer"&gt;admits about 5000 calls&lt;/a&gt;; widen it above that.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;The kit is in &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md" target="_blank" rel="noreferrer"&gt;&lt;code&gt;test/perf/hosts/&lt;/code&gt;&lt;/a&gt;. &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md" target="_blank" rel="noreferrer"&gt;MEASURE.md&lt;/a&gt; defines&#10;each step, its ladder and its pass line, and the&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/README.md#L313-L340" target="_blank" rel="noreferrer"&gt;harness README&lt;/a&gt; shows how &lt;code&gt;run.sh&lt;/code&gt; drives three real hosts.&#10;From the generator host, &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L153-L160" target="_blank" rel="noreferrer"&gt;the commercial SBC&amp;rsquo;s test&lt;/a&gt;, the&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L210" target="_blank" rel="noreferrer"&gt;carrier-in ladder&lt;/a&gt;, the &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L245-L247" target="_blank" rel="noreferrer"&gt;concurrent-call ladder&lt;/a&gt; and&#10;the &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/test/perf/hosts/MEASURE.md#L423-L424" target="_blank" rel="noreferrer"&gt;soak&lt;/a&gt;:&lt;/p&gt;&#10;&lt;div class="highlight-wrapper"&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;cd&lt;/span&gt; /opt/freesbc &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; . test/perf/hosts/env.sh sipp&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;test/perf/hosts/m03.sh all sipp&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;test/perf/run.sh -n -s 250:8000:250:40 -H &lt;span class="m"&gt;2&lt;/span&gt; carrier-in&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;test/perf/hosts/ladder.sh m02 &lt;span class="s2"&gt;&amp;#34;-c @N@ -H 60 -t 180 -u @N@ call&amp;#34;&lt;/span&gt; &lt;span class="m"&gt;250&lt;/span&gt; &lt;span class="m"&gt;500&lt;/span&gt; &lt;span class="m"&gt;750&lt;/span&gt; &lt;span class="m"&gt;1000&lt;/span&gt; &lt;span class="m"&gt;1250&lt;/span&gt; &lt;span class="m"&gt;1500&lt;/span&gt; &lt;span class="m"&gt;2000&lt;/span&gt; &lt;span class="m"&gt;2500&lt;/span&gt; &lt;span class="m"&gt;3000&lt;/span&gt; &lt;span class="m"&gt;4000&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;test/perf/hosts/m08.sh &lt;span class="m"&gt;3500&lt;/span&gt; &lt;span class="m"&gt;8&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&#10;&lt;p&gt;MEASURE.md runs each one detached with &lt;code&gt;nohup&lt;/code&gt; and judges every run with&#10;&lt;code&gt;runcheck.sh&lt;/code&gt; and &lt;code&gt;rigcheck.sh&lt;/code&gt;. For a quick look on a laptop, the&#10;&lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md#L246-L260" target="_blank" rel="noreferrer"&gt;single-host Docker scenarios&lt;/a&gt; still work. Their numbers do&#10;not compare with these.&lt;/p&gt;&#10;&lt;p&gt;Code: &lt;a href="https://github.com/rasonyang/freesbc" target="_blank" rel="noreferrer"&gt;FreeSBC&lt;/a&gt; · &lt;a href="https://github.com/rasonyang/freesbc/blob/26f1d9e631249b51097cc65fd5ebfb2444057b7f/docs/performance.md" target="_blank" rel="noreferrer"&gt;docs/performance.md&lt;/a&gt; · &lt;a href="https://github.com/rasonyang/freesbc/issues/109" target="_blank" rel="noreferrer"&gt;issue #109&lt;/a&gt;&lt;/p&gt;&#10;</description><media:content xmlns:media="http://search.yahoo.com/mrss/" url="https://rasonyang.dev/posts/freesbc-load-test-results/featured.jpg"/></item></channel></rss>