Live transcription on an open microphone
There is no wake word and no button on a panel. Audio streams in, transcription runs against it as people speak, and the decision to answer is made on a transcript that is still being written.
Solution · Events and conferences
A live AI panelist that answers on stage, in real time.
Hephon is an AI agency in Dubai (Hephon L.L.C-FZ) that builds real-time voice AI for stages, calls and customer service. With KalaMena AI, we built Maryam 3.0, a virtual panelist that answered the moderator live for a full hour at ForsaTEK, Emirates Group’s flagship innovation event.
What is a live AI panelist?
A voice AI that takes a real seat on a panel. It listens to an open room, works out who is speaking and whether a question is meant for it, answers out loud through the venue’s sound system at conversational speed, and stays quiet when people are talking to each other. Nothing is pre-recorded.
At a glance
- 100-200 ms
- from end of question to first sound of the answer
- 1 hour
- live on the panel, no recorded clips
- 4
- names on the stage screen, one of them the AI
The problem, as the brief put it
Playing a recorded clip on a screen is the safe way to put AI in front of an audience. It is also obvious, and a room full of senior leaders can tell the difference. The ForsaTEK brief was a real seat on a real panel, Building the New AI Industrial Revolution, with three executives and a moderator.
That meant the moderator asks, the AI answers, and the audience watches it happen at conversational speed, with everything that can go wrong still able to go wrong in front of senior leadership, industry partners and government stakeholders.
Answering was the easy part. Current models answer well. The hard part was everything around the answer: hearing the room, knowing who is speaking, deciding whether anyone is waiting on a reply, and getting sound out of the PA fast enough that the exchange still feels like a conversation.
What we built
A pipeline that runs continuously rather than in request and response turns, so the question is largely understood by the time the speaker finishes.
Endpointing that reads what was said
A fixed silence timer fails on stage: panellists pause mid-sentence to think, and run two questions together with no gap. Endpointing looks at whether the sentence is complete, whether it ended on a question and how long this person usually pauses, tuned against real rehearsal audio.
Knowing when a question is meant for it
The moderator can address Maryam by name, hand the question to a human panellist, or open it to everyone. The system reads a direct address and stays quiet when the room is talking among itself.
Speaker recognition by voice
Every stretch of speech is attributed to a person as it is transcribed. That lets the AI answer the right person, resolve references to earlier points, and treat the moderator differently from a panellist.
Yielding, and not hearing itself
If a person starts speaking while Maryam is answering, she yields. Because her answers come out of the same PA the microphones hear, the system also filters out its own voice so it never answers itself.
A latency budget with overlapping stages
Generation starts on the partial transcript and speech synthesis streams, so the first words play while the rest is still being produced. Every hop, including the one out to the sound desk, counts against the budget.
A labelled transcript of the whole room
Everything said, including the AI’s own answers, is transcribed and attributed by speaker. It keeps an hour of answers consistent, and it gives the organiser a record of what was asked and answered afterwards.
Who approves what
On a stage there is no undo, so the controls are set before anyone walks on.
Guardrails agreed before the event
What the AI may and may not talk about was set before the panel and held for the full hour. It stayed inside its brief, said nothing it could not support and made nothing up.
The moderator stays in charge of the floor
The AI answers when addressed, yields when a person speaks, and treats the moderator’s turns differently from a panellist’s.
Engineers on the console
Hephon engineering was on site and on the console for the length of the panel, with a plan for the moment something breaks in front of a full room.
A transcript for review
The speaker-labelled transcript lets the organisation check afterwards exactly what was asked, by whom, and what the AI said back.
How the work runs
Brief
Agree the panel format, the persona and the guardrails with the organiser and the event partner.
Rehearsals
Tune endpointing, address detection and speaker recognition against real rehearsal audio, not thresholds in a config file.
Event day
Engineers on site and on the console for the full session; live transcript running.
After
The speaker-labelled transcript goes to the organiser for review.
ForsaTEK was delivered with KalaMena AI, who led the engagement with Emirates Group and brought Hephon in as the technology partner. Maryam 3.0 was built and run jointly.
Questions
Was anything pre-recorded?
No. Maryam 3.0 heard each question in the room and answered it live through the venue’s sound system for the full hour.
How fast did it answer?
Between 100 and 200 milliseconds from the end of the question to the first sound of the answer. That is roughly the gap a person leaves before replying in ordinary conversation.
How does it know a question is for the AI and not for a human panellist?
It reads direct address, notices when the moderator has handed a question to someone else, and stays quiet when people are talking among themselves. Speaker recognition tells it who is speaking.
What stops it from saying something wrong on stage?
Guardrails set before the event and held for the whole session, a running transcript that keeps answers consistent with what was already said, and engineers on the console throughout.
Who delivered the ForsaTEK panelist?
KalaMena AI led the engagement with Emirates Group and brought Hephon in as the technology partner. The stage credit read developed by KalaMena AI and Hephon.
Does the same technology work on phone calls?
The same problems decide whether a phone agent feels natural: knowing when the caller has finished, yielding when interrupted, and answering fast. That is the work behind our AI voice agents.
The proof and the service.
Want this for your business?
Tell us what you run today. We will say plainly whether this fits, what we would change for your setup, and what a fixed-price first build looks like.