The problem
HANA Sentinel is an ops-side tool for talking to a SAP HANA database in natural language, describe the tenant you want to check on, ask about a runaway statement, get a summarized answer with the underlying SQL that produced it. Most of it is a text chat surface. But a chunk of the operator workflow happens away from the keyboard, walking between racks, on a call with a customer, in a meeting where opening a laptop and typing feels rude. So we needed a voice mode. Press-and-hold to talk, release to send, hear a spoken answer back.
The whole thing runs inside a corporate network. That is the constraint that shapes every other decision in this post. The operators are on locked-down laptops behind an enterprise proxy that inspects TLS, blocks UDP outbound, and gates microphone access through group policy. Any voice architecture that assumes a consumer network, home Wi-Fi, mobile hotspot, coffee-shop LAN, is going to fail on day one. The interesting part of this build turned out not to be the audio pipeline. It was making WebRTC work at all in a network designed to keep WebRTC out.
Prior art
Before wiring up LiveKit I looked at the four things people usually reach for.
Discord / Google Meet. The gold standard for consumer voice UX, and both are more sophisticated in-browser than anything I could build. But neither works inside our target network without an IT ticket. Google Meet is fine on the corporate laptop because IT already whitelisted it; anything else on the same architecture gets blocked. That's the shape of the problem, SOTA consumer WebRTC assumes you own the network or that IT will punch holes in it for you.
Zoom Web SDK. Zoom has done more real work on enterprise-network traversal than anyone. Their SDK falls back through STUN, TURN/UDP, TURN/TCP, and TURN/TLS-over-443 in a way that just quietly works behind most proxies. If I were building a general-purpose meetings app I would use it in a heartbeat. What it doesn't do well is fit a machine-to-human agent loop where one participant is a program and the other is a human, you can build that, but you're paying for meeting-shaped primitives (host controls, waiting rooms, breakout rooms) that don't apply.
Twilio Voice / Programmable Voice. Rock-solid PSTN and WebRTC voice, mature JS SDK, first-class relay infrastructure. The problem for this workload is that Twilio's mental model is a call, one leg, one participant, one media stream to a server. HANA Sentinel wants a persistent room the operator can enter and leave without teardown and the ability for the server-side agent to push media whenever an answer is ready. All of that is possible on Twilio, but you're rebuilding LiveKit's room abstraction on top of Voice primitives.
Vanilla getUserMedia + a hand-rolled SFU. Absolute maximum flexibility, minimum time-to-value. Even mediasoup and Janus need ICE server configuration, TLS termination, SDP munging for browser quirks, and a signaling protocol you write yourself. On a project that needed to ship in weeks, this was over-budget. The lesson from the DIY route is real though: if you're going to depend on an SFU, you should still understand the ICE/SDP layer well enough to debug it when it breaks. Which it will.
Retell / Vapi. The 2026-shaped answer: hosted voice-agent platforms that take a phone number or a web widget and give you a working conversational agent. Great if your endpoint is public. For an internal ops tool that has to authenticate against SAP identity and hand tenant IDs into the backend, "call our SaaS endpoint" is a non-starter, the corporate boundary matters more than the ergonomics.
The common failure mode across all of them, when you ask "does this work behind a proxy that blocks UDP entirely," is that vendor documentation quietly assumes UDP works. Which for consumer traffic is fine, and for our operators is wrong.
What we did differently
Three decisions fell out of the constraint.
LiveKit as the room abstraction, not a meetings platform. LiveKit gives you rooms, participants, tracks, and server-signed access tokens as first-class primitives. It does not give you host controls, waiting rooms, or any of the meeting-shaped scaffolding I did not need. The mental model is "a WebRTC session with a scoped identity," which is exactly the shape of what a voice-to-HANA agent wants. The server-side agent joins the same room as a participant. Media flows in both directions. No leg management, no dial-in, no call state machine.
Server-signed short-lived tokens, minted per session, scoped to one room and one participant. No long-lived credentials in the browser. The frontend calls a token endpoint on our backend, which mints a JWT signed with the LiveKit API secret, containing the room name, the participant identity, and an expiry a few minutes out. The browser hands that token to LiveKit's connect method, and the trust chain ends there. If a token leaks it's useless within minutes, and it can only join one specific room as one specific participant.
TURN-over-TLS on 443 as the default, not the fallback. The consumer-network default is STUN first, direct peer connection, TURN/UDP as a fallback, TURN/TCP after that, TURN/TLS on 443 last. On our network, every step before the last one fails. So we invert the order. The client is configured to try TURN/TLS on 443 first. This adds a hop and adds latency versus a direct P2P connection, but on the target network there is no such thing as a direct P2P connection, the choice is TURN or nothing.
Microphone-first UX with graceful degradation. The mic permission dialog is a native affordance that group policy can either allow, block, or gate behind an IT ticket. The UI treats each of those states as a real state, not an error. If the mic is denied, the voice button flips into a disabled state with a link to the internal doc explaining how to request the permission. If the mic works, we show a level meter so the operator knows we're hearing them before they commit to speaking.
System design
The system is four moving parts. Nothing exotic, the interesting part is how each one is configured for the target network.
The token service. A small backend endpoint that takes an authenticated ops user, mints a LiveKit JWT with roomName, participantIdentity, and an exp set roughly ten minutes into the future, and hands it back. The JWT is signed with the LiveKit API key and secret held on the server. This fits the workload because the browser never sees a long-lived credential, the entire trust story is "the ops user was already authenticated in the enterprise session, and this token is a scoped, short-lived derivative of that authentication." The exp window is deliberately short: long enough to survive a genuinely slow room connect on a bad day, short enough that a captured token is not useful.
The LiveKit client, configured for enterprise networks. The Room connect call passes an ICE server list containing only the TURN/TLS 443 endpoint, and sets iceTransportPolicy: "relay". That flag is the difference between "try direct first and fall back" and "don't even bother probing UDP." On a network that blocks UDP the probe wastes several seconds before the fallback kicks in, and the user is staring at a spinner in the meantime. Forcing relay from the start turns a five-to-ten second cold connect into something closer to one to two seconds, directional, from watching the connection timeline in the browser devtools with and without the flag. Not a benchmark.
The mic permission gate. A tiny wrapper around navigator.mediaDevices.getUserMedia({ audio: true }) that treats the three outcomes, resolved with a track, rejected with NotAllowedError, rejected with NotFoundError, as three UI states. The rejected states carry an internal-docs link. This fits the workload because on locked-down corporate laptops the permission dialog behavior is not always the browser's, it's whatever the group policy layered on top says it should be, and we don't get to fix that from the app. What we can do is not treat the denial as a bug and not force the user to open devtools to figure out what went wrong. The level meter is the last piece of that story: when the mic is allowed, we surface a live input level so the operator can verify the audio is flowing before they trust the interaction.
The server-side agent. A LiveKit agent process that joins the room as its own participant, subscribes to the operator's audio track, runs the transcription and the HANA reasoning, and publishes an audio response back into the room. The frontend has no idea what the agent is doing, it just sees another participant in the room whose audio track it plays through the browser's audio output. This is the piece that decouples the voice frontend from the HANA backend: swap the agent, swap the model, swap the transcription vendor, and the frontend doesn't change. The frontend's contract is "there's a room, there might be another participant in it, play their audio."
What broke first
The first thing that broke, and the thing that reshaped the whole architecture, was the UDP block.
I had the first version working on my laptop over home Wi-Fi in an afternoon. Standard LiveKit sample code, default ICE server config, mic permission granted, agent joined, audio flowing both ways. Ship it.
I opened it on the corporate laptop the next morning. Connect timeout. No error, no failure state, just a spinner that never resolved and eventually a LiveKit "unable to establish connection" toast about thirty seconds in. The devtools network tab showed the signaling WebSocket connecting fine, the token exchange worked, the room join succeeded at the signaling layer, but no media was flowing. The peer connection stayed in checking state forever.
The chrome://webrtc-internals page told the real story. Every ICE candidate the client was gathering was UDP. Every ICE candidate the server was returning was UDP. Every single candidate pair was timing out in the connectivity check. The corporate proxy dropped every UDP packet outbound. WebRTC's whole discovery dance was happening in a black hole.
There are a few flavors of fix. The first is to enable TURN/TCP and hope. TCP-relayed TURN would in theory work, but a lot of enterprise proxies block outbound TCP to non-standard ports too, TURN typically runs on 3478 or 5349, and 3478 is the first thing a paranoid proxy blocks. The second is to enable TURN/TLS on 443, which looks identical to HTTPS on the wire and rides through the same TLS-inspection path the proxy already allows for regular web traffic. The third is to file an IT ticket for every operator.
We went with option two, and we went further: we made TURN/TLS on 443 the only candidate the client considers. The ICE server list is a single-entry array pointing at our TURN/TLS endpoint, and iceTransportPolicy is set to "relay". The client never tries direct P2P, never tries UDP, never tries any other TURN variant. It goes straight to the one candidate that works on this network, and connects in about a second and a half.
The tradeoff is honest: every byte of audio round-trips through our TURN server, adding a hop and adding some latency versus a direct connection. On a home network that would be a regression. On the target network there is no direct connection, so the comparison isn't fair, it's "does the app work or not."
The second thing that broke, smaller: the mic permission flow behaved differently on the corporate build of Chrome than on my personal build. Group policy had switched the mic to "ask" with a system-level prompt above the browser's own, and if you clicked away from that prompt without answering, getUserMedia never resolved or rejected, it just hung forever. I now race the getUserMedia promise against a ten-second timeout and treat "hung" as a fourth state with its own UI copy. Not the cleanest fix. But it stopped the support tickets.
What I would do differently
Three things I would pick up in v2.
Ship a lightweight preflight check as the first screen. Right now the app tries to connect and fails visibly if the network can't reach the TURN endpoint. A dedicated preflight, hit the TURN endpoint over TLS 443, verify the round trip, verify the mic permission, verify audio output, before we ever ask the operator to press-and-hold would replace three or four minutes of confused debugging with a red banner that says exactly what's wrong. WebRTC has a "trickle ICE" test tool that's essentially this; folding it into the app itself as a first-run diagnostic is the right shape.
Move the token minting into a signed WebSocket subprotocol. Right now the token is a JWT fetched from an HTTP endpoint and passed to LiveKit's connect method. That works but it separates the ops-user authentication from the LiveKit connect into two hops, and it means the token is briefly a bearer credential in JS memory. A tighter design would be a signed subprotocol handshake where the server proves the operator's identity to LiveKit during the WebSocket upgrade, and the JWT never touches JavaScript. Marginal security win, real code-simplicity win.
Push-to-talk as a real state machine, not a mousedown handler. The current PTT is onmousedown/onmouseup on a button. That falls apart under any of the usual edge cases: the operator drags their mouse off the button mid-utterance, the operator switches tabs while holding, the operator's Bluetooth headset button fires a keyboard shortcut instead of a mouse event. A proper state machine, Idle, Requesting-Mic, Listening, Sending, Awaiting-Response, Playing-Response, Error, would make each of those cases a named transition rather than a bug report.
The transferable lesson
If your target network isn't a consumer network, throw away the vendor's happy-path defaults and read the fallback documentation as if it were the main documentation. WebRTC's whole discovery dance assumes you can send UDP. The corporate world often can't. The right architecture doesn't try to detect that at runtime and gracefully fall back, it just starts from the fallback and doesn't waste your users' time on discovery. Pick the primitive that works on the worst network you have to support, and use it as the default on every network, and you'll ship something that works everywhere you care about instead of something that works beautifully in the demo and mysteriously not in the field.
See also
- The <700ms latency budget for a personal AI voicemail line, the same LiveKit primitives on a consumer network, where the constraint is latency instead of proxy traversal.
- An LRU key-rotation state machine for a personal credential vault, how the STT and LLM keys behind the server-side agent get rotated without hardcoding them anywhere in the ops backend.
- Teaching agentic AI at SAP, the broader SAP-side context in which HANA Sentinel lives.