Voice AI Is Easy to Demo, Hard to Productise

AI makes the first demo faster. It does not make a production-grade voice product appear in a week.

That was the clearest lesson from our internal open-source voice AI challenge. We were not trying to prove that a mature meeting assistant can be rebuilt in a short sprint. It cannot. We were trying to expose where the real engineering work begins.

The short answer: production-grade meeting intelligence is not a transcript with an LLM summary on top. It is a live audio system that has to capture the right streams, preserve speaker identity, align timestamps, recover from interruptions, protect sensitive data and keep working when real meetings behave like real meetings.

The transcript is the visible part. Trust is the product.

What makes voice AI hard to productise?

Voice AI becomes hard when it moves from a controlled demo to a live meeting.

In a demo, you can record one audio file, run speech-to-text, send the transcript to a large language model and generate a tidy summary. It looks useful because the environment is simple.

In production, the system has to deal with people joining late, leaving early, reconnecting, switching devices, speaking over each other, muting and unmuting, changing names, losing connection and expecting the notes to remain accurate.

That is where AI engineering starts to look less like prompt writing and more like product architecture.

Why is a meeting not just one audio file?

Because a meeting is made of people, not sound waves.

The simplest transcription approach is to record the local microphone or capture mixed browser audio. That can produce a readable transcript, but it destroys context that the product may need later.

Once every voice is mixed into one stream, the system has already lost valuable information:

• who was speaking;
• when a speaker joined or left;
• whether two people overlapped;
• which audio belonged to which participant;
• where a decision happened in the call;
• whether the summary can cite the right moment.

A diarisation model can try to reconstruct that information afterwards, but it is working from damaged input.

This is why clean audio acquisition matters. In production AI, better input often beats a cleverer model.

Why does speaker identity matter in meeting intelligence?

A transcript without speaker identity is useful. A transcript with a reliable speaker identity starts to become a product.

For meeting intelligence, "what was said" is only half the question. The product also needs to know who said it, when they said it and whether the system can prove it.

That is harder than it sounds. Live calls are dynamic systems. People join late, reconnect, change devices, appear under unstable names or speak at the same time. The engineering problem is not simply "detect speakers". It is to preserve the relationship between audio, person, timestamp and meeting context while the call is changing in real time.

Users do not care whether the system had a difficult browser event, a race condition or a partial audio stream. They care whether the notes say the right person made the right decision.

That is the standard a production system has to meet.

Why is the model only part of the product?

Open-source speech models are already strong enough to create useful transcripts. That is good news.

It is also the trap.

The model creates the visible magic. The system around the model creates trust.

A production-grade voice AI product still needs the surrounding layer: audio capture, silence detection, chunking, queueing, timestamp alignment, speaker diarisation, identity mapping, summary generation, output validation, retries, caching, observability, permissions, export flows and failure recovery.

This is where many AI projects get underestimated. The demo is about whether the model can produce something impressive. The product is about whether the system can produce something dependable every day.

The strongest work in our challenge was not just the work that produced text. It was the work that treated the voice pipeline as an architecture: modular, observable, replaceable and resilient when the meeting did something inconvenient.

Models change quickly. Product architecture has to survive that change.

What architecture choices matter for voice AI products?

There is no single correct architecture for meeting intelligence.

One approach can prioritise clean per-speaker audio from the meeting source. Another can prioritise operational independence by joining calls as a separate attendee. Another can prioritise broad browser compatibility by capturing audio from any web-based source.

Each path has a trade-off.

Cleaner speaker separation can create tighter dependency on one meeting platform. Broader compatibility can make diarisation harder. A bot-based approach can simplify some workflows and introduce new constraints around access, permissions and user experience.

This is exactly why discovery matters. The right architecture depends on privacy requirements, supported platforms, expected call volume, latency targets, accuracy expectations, deployment environment, integration needs and what users actually need from the notes.

AI does not remove those trade-offs. It makes them arrive earlier.

What does production-grade voice AI need beyond transcription?

Production-grade voice AI needs the unglamorous layer that keeps the product useful after the first demo.

That includes:

• reliable audio capture;
• speaker diarisation and identity mapping;
• timestamp alignment;
• structured summaries with decisions and action points;
• validation for messy model outputs;
• recovery from interrupted calls;
• observability and debugging tools;
• privacy-aware data handling;
• export and integration flows;
• fallback paths when models, browsers or meeting platforms behave unexpectedly.

These are not edge cases. They are for normal usage.

That is why serious AI engineering spends so much time on the middle layer. It is not the most visible part of the product, but it is where reliability lives.

What did the challenge confirm?

It confirmed the pattern we see across AI projects: AI accelerates exploration, but it does not delete engineering.

A short prototype sprint can reveal whether an idea is technically plausible. It can expose hidden risks. It can show which architecture has a better chance of becoming a real product.

It cannot replace product discovery, security review, integration design, infrastructure planning, QA, observability, support flows or the work of making a system behave predictably for real users.

That is not a conservative view of AI. It is the practical one.

The opportunity is real. Open-source voice models are improving quickly. Local-first deployments are becoming more realistic. Meeting intelligence is a strong use case for AI-assisted workflows.

But the companies that get value from voice AI will not be the ones that treat it as a shortcut around engineering. They will be the ones who combine strong models with strong systems thinking.

That is where our team is most useful: not in making a demo look clever, but in knowing what has to happen after the demo for the product to survive contact with real work.

Key takeaways for AI product teams

Nobody buys a transcript. They buy the workflow that turns it into something they can act on.

Speaker identity will trip you up before anything else. Models can name who is talking. Keeping that name attached to the right person through reconnects, overlaps, and device swaps is the actual problem.

Clean audio at the front of the pipeline punches above its weight.

Open-source speech models are good enough now. What sits around them, the diarisation, the recovery layer, the validation, the export flows, decides whether anyone trusts the output.

Architecture trade-offs do not disappear because AI is in the stack. They surface earlier, often inside the demo that was supposed to feel easy.

Between a working prototype and a working product sits a longer list of unglamorous things. Recovery. Validation. Observability. Privacy controls. The boring discipline of producing the same result on Monday and Friday.