Research
Voice models can now sound human. In a group, they still interrupt, stay quiet when someone asks them a question, or answer the wrong person. Our work is about the part of conversation that is not words: who has the floor, when it changes hands, and what silence means.
Questions we are working on
Some of these have partial answers in the literature. None of them have answers that hold up in a real meeting.
- When should a machine in a group stay silent? Most of the time, the right move for an agent in a meeting is to say nothing. The hard part is knowing the few moments when it is not.
- Can one model hear six people as six people? Voice agents today receive a meeting as one mixed signal. People do not hear a room that way.
- How fast is fast enough? People in conversation leave about 200 milliseconds between turns, in every language that has been measured. A machine that needs a full second of silence is always late.
- What does an interruption cost, and who pays for it? Talking over someone and waiting too long are both failures. We want to measure them in the same units.
- Does timing learned in simulation survive a real room? Synthetic conversations teach content well. Whether they teach timing is an open question, and we intend to measure it.
Index
Models, benchmarks and environments in progress. Entries become links when they are published.
| Work | Kind | Status |
|---|---|---|
| Floor: a small model for turn-taking in groups | Model | Underway |
| Turn Bench: measuring when agents should speak in multi-party audio | Benchmark | Forthcoming |
| Chorus: full-duplex listening across many speakers | Model | Early research |
| Agora: simulated meetings for training conversational timing | Environment | Building |
| redacted title of a forthcoming paper | Paper | Forthcoming |
A machine has three jobs in a conversation
To listen while others speak, to decide, and to speak. Kempelen's machine of 1791 had three parts too: bellows that take in air, a reed that makes the voice, and a mouth that shapes it.
Work with us
We are open to research collaborations on multi-party turn-taking, full-duplex dialogue and conversational benchmarks. If you run a lab or a challenge in this area, we would like to hear from you.