Voice AI Is Easy to Demo, Hard to Productise

AI makes the first voice AI demo faster. It does not make a production-grade voice product appear in a week.

That was the clearest lesson from our internal open-source voice AI challenge. The goal was not to show that a mature meeting assistant can be rebuilt in a short sprint (it cannot), but to find where the real engineering work begins.

Production-grade meeting intelligence is a live audio system, not a transcript with an LLM summary on top. It has to capture the right streams, preserve speaker identity, align timestamps, recover from interruptions, protect sensitive data and keep working when real meetings get messy.

What makes voice AI hard to productise?

Voice AI becomes hard when it moves from a controlled demo to a live meeting.

In a demo, you can record one audio file, run speech-to-text, send the transcript to a large language model and generate a tidy summary. It looks useful because the environment is simple.

In production, the system has to deal with people joining late, leaving early, reconnecting, switching devices, speaking over each other, muting and unmuting, changing names, losing connection and expecting the notes to remain accurate.

That is where AI engineering starts to look less like prompt writing and more like product architecture.

Why is a meeting not just one audio file?

The simplest transcription approach is to record the local microphone or capture mixed browser audio. That can produce a readable transcript, but it destroys context that the product may need later.

Once every voice is mixed into one stream, the system has already lost valuable information:

• who was speaking;
• when a speaker joined or left;
• whether two people overlapped;
• which audio belonged to which participant;
• where a decision happened in the call;
• whether the summary can cite the right moment.

A diarisation model can try to reconstruct that information afterwards, but it is working from damaged input.

This is why clean audio acquisition matters. In production AI, better input often beats a cleverer model.

Why does speaker identity matter in meeting intelligence?

A transcript without speaker identity is useful. A transcript with a reliable speaker identity starts to become a product.

For meeting intelligence, "what was said" is only half the question. The product also needs to know who said it, when they said it and whether the system can prove it.

That is harder than it sounds. Live calls are dynamic systems. People join late, reconnect, change devices, appear under unstable names or speak at the same time. The engineering problem is not simply "detect speakers". It is to preserve the relationship between audio, person, timestamp and meeting context while the call is changing in real time.

Users do not care whether the system had a difficult browser event, a race condition or a partial audio stream. They care whether the notes say the right person made the right decision.

Why is the model only part of the product?

Open-source speech models are already good enough to produce useful transcripts. The risk is assuming the model is the product.

A production-grade voice AI product still needs the surrounding layer: audio capture, silence detection, chunking, queueing, timestamp alignment, speaker diarisation, identity mapping, summary generation, output validation, retries, caching, observability, permissions, export flows and failure recovery.

This is where AI projects are often underestimated: a demo shows that the model can produce something impressive once; a product has to produce something dependable every day.

The strongest work in our challenge treated the voice pipeline as an architecture: modular, observable, with replaceable components, and resilient when the meeting did something inconvenient.

Models change quickly. Product architecture has to survive that change.

What architecture choices matter for voice AI products?

There is no single correct architecture for meeting intelligence.

One approach can prioritise clean per-speaker audio from the meeting source. Another can prioritise operational independence by joining calls as a separate attendee. Another can prioritise broad browser compatibility by capturing audio from any web-based source.

Each path has a trade-off.

Cleaner speaker separation can create tighter dependency on one meeting platform. Broader compatibility can make diarisation harder. A bot-based approach can simplify some workflows and introduce new constraints around access, permissions and user experience.

This is why discovery matters. The right architecture depends on privacy requirements, supported platforms, expected call volume, latency targets, accuracy expectations, deployment environment, integration needs and what users actually need from the notes.

AI does not remove those trade-offs. It makes them arrive earlier.

What does production-grade voice AI need beyond transcription?

Production-grade voice AI needs the unglamorous layer that keeps the product useful after the first demo.

That includes:

• reliable audio capture;
• speaker diarisation and identity mapping;
• timestamp alignment;
• structured summaries with decisions and action points;
• validation for messy model outputs;
• recovery from interrupted calls;
• observability and debugging tools;
• privacy-aware data handling;
• export and integration flows;
• fallback paths when models, browsers or meeting platforms behave unexpectedly.

These are not edge cases. They are normal usage.

That is why so much AI engineering time goes into this middle layer: it is less visible than the model, but it decides how reliable the product is.

What did the challenge confirm?

It confirmed the pattern we see across AI projects: AI accelerates exploration, but it does not delete engineering.

A short prototype sprint can reveal whether an idea is technically plausible. It can expose hidden risks. It can show which architecture has a better chance of becoming a real product.

It cannot replace product discovery, security review, integration design, infrastructure planning, QA, observability, support flows or the work of making a system behave predictably for real users.

The opportunity is real: open-source voice models are improving quickly, local-first deployments are becoming more realistic, and meeting intelligence is a strong use case for AI-assisted workflows.

Companies get value from voice AI when they combine good models with solid system design, not when they treat AI as a shortcut around engineering.

That is where our team is most useful: knowing what has to happen after the demo for the product to work in real meetings.

Key takeaways for AI product teams

• Users do not pay for a transcript; they pay for a workflow that turns it into something they can act on.
• Speaker identity is often the first problem you hit. The hard part is not naming who is talking but keeping that name attached to the right person through reconnects, overlaps and device swaps.
• Clean audio at the start of the pipeline often matters more than a better model.
• Open-source speech models are good enough. The diarisation, recovery, validation and export flows around them decide whether anyone trusts the output.
• AI in the stack does not remove architecture trade-offs. They surface earlier, often during the demo.
• The gap between a working prototype and a working product is filled with recovery, validation, observability, privacy controls and the discipline of producing the same result every day.