Voice agent
In agent mode, Vaux listens to remote audio (channel 2), and a language model answers in the session's voice, using only a brief you provide.
const session = await vaux.realtime.connect(null, { voice: 'tessa', mode: 'agent' });
session.brief(`I'm Ada, product lead at Acme. Launch is Nov 3. Never agree to scope changes.`, 'Ada');
await session.attachRemoteAudio(meetingStream); // e.g. a WebRTC remote track
Behaviour:
- It stays silent when it isn't addressed or nothing useful needs saying (
agent.silent). - It stops talking when someone else starts (
bargeIn: true, the default). - With
autoReply: false, replies arrive asagent.suggestionfor you to approve withsay(). whisper(text)adds private guidance to the next reply.- The agent may set its tone per reply; you'll see it in
agent.said.emotion.
| Event | Fields |
|---|---|
agent.thinking | id |
agent.text | id, delta: streaming reply text |
agent.said | id, text, emotion, aborted |
agent.suggestion | id, text, emotion |
agent.silent | id |
Agent mode requires a plan that includes the agent, and a token with allowedModes containing "agent". It is billed per minute of remote audio processed.
You are responsible for disclosing AI participation and synthetic voices where the law or platform rules require it. See the Acceptable Use Policy.