The Challenge
Career advisers at a Singapore public-sector programme spent their sessions half-listening and half-typing case notes into a fixed template. Any tool had to keep personal data inside Singapore and never store session audio.
My Approach
Browser audio streams over WebSocket to a Cloud Run service in asia-southeast1, which feeds Google Speech-to-Text and discards the audio in memory. Every 45 seconds one orchestrated Gemini 2.5 Flash call on Vertex AI tags the conversation phase, fills the case-note fields, suggests courses from the national skills framework, and prompts the adviser with questions for whatever is still missing. A keyword classifier tags each line instantly and Gemini refines the low-confidence ones in the background. Auto-reconnect and 30-second session recovery handle patchy office Wi-Fi.
The Result
Piloted in early 2026, then handed over for production use in March 2026. Advisers finish a session with a draft case note that is already in the template, edit it, and export to Excel with the raw and edited versions kept side by side.