# Table of Contents - [LiveKit Agents](#livekit-agents) - [livekit-compose-components](#livekit-compose-components) - [SIP overview | LiveKit Docs](#sip-overview-livekit-docs) - [Worker lifecycle | LiveKit Docs](#worker-lifecycle-livekit-docs) - [SIP cloud and region pinning | LiveKit Docs](#sip-cloud-and-region-pinning-livekit-docs) - [Accepting inbound calls | LiveKit Docs](#accepting-inbound-calls-livekit-docs) - [Building voice agents | LiveKit Docs](#building-voice-agents-livekit-docs) - [LiveKit Agents integrations | LiveKit Docs](#livekit-agents-integrations-livekit-docs) - [LiveKit Cloud | LiveKit Docs](#livekit-cloud-livekit-docs) - [LiveKit Agents | LiveKit Docs](#livekit-agents-livekit-docs) - [SDK migration from v1 to v2 | LiveKit Docs](#sdk-migration-from-v1-to-v2-livekit-docs) - [SIP inbound trunk | LiveKit Docs](#sip-inbound-trunk-livekit-docs) - [SDK Quickstarts | LiveKit Docs](#sdk-quickstarts-livekit-docs) - [Agents Overview | LiveKit Docs](#agents-overview-livekit-docs) - [LiveKit integration guides | LiveKit Docs](#livekit-integration-guides-livekit-docs) - [Making outbound calls | LiveKit Docs](#making-outbound-calls-livekit-docs) - [Recording and composition | LiveKit Docs](#recording-and-composition-livekit-docs) - [Intro to LiveKit | LiveKit Docs](#intro-to-livekit-livekit-docs) - [Rooms, participants, and tracks | LiveKit Docs](#rooms-participants-and-tracks-livekit-docs) - [Inbound calls with Twilio Voice | LiveKit Docs](#inbound-calls-with-twilio-voice-livekit-docs) - [OpenAI integration guide | LiveKit Docs](#openai-integration-guide-livekit-docs) - [SIP dispatch rule | LiveKit Docs](#sip-dispatch-rule-livekit-docs) - [Authentication | LiveKit Docs](#authentication-livekit-docs) - [Ingress overview | LiveKit Docs](#ingress-overview-livekit-docs) - [Make outbound calls | LiveKit Docs](#make-outbound-calls-livekit-docs) - [Transferring calls | LiveKit Docs](#transferring-calls-livekit-docs) - [Handling DTMF | LiveKit Docs](#handling-dtmf-livekit-docs) - [SIP outbound trunk | LiveKit Docs](#sip-outbound-trunk-livekit-docs) - [HD voice for SIP | LiveKit Docs](#hd-voice-for-sip-livekit-docs) - [livekit_client - Dart API docs](#livekit-client-dart-api-docs) - [SIP participant | LiveKit Docs](#sip-participant-livekit-docs) - [SIP APIs | LiveKit Docs](#sip-apis-livekit-docs) - [Android Components | LiveKit Docs](#android-components-livekit-docs) - [LiveKit JS Client SDK - v2.15.4](#livekit-js-client-sdk-v2-15-4) - [livekit-android-sdk](#livekit-android-sdk) - [LiveKit SFU | LiveKit Docs](#livekit-sfu-livekit-docs) - [server-sdk-js | LiveKit Docs](#server-sdk-js-livekit-docs) - [Client Protocol | LiveKit Docs](#client-protocol-livekit-docs) - [AI voice agents | LiveKit Docs](#ai-voice-agents-livekit-docs) - [Agents playground | LiveKit Docs](#agents-playground-livekit-docs) - [Agents Playground | LiveKit Docs](#agents-playground-livekit-docs) - [Server APIs | LiveKit Docs](#server-apis-livekit-docs) - [Deployment and scaling | LiveKit Docs](#deployment-and-scaling-livekit-docs) - [Voice AI quickstart | LiveKit Docs](#voice-ai-quickstart-livekit-docs) - [Agents v0.x migration guide | LiveKit Docs](#agents-v0-x-migration-guide-livekit-docs) - [Agent dispatch | LiveKit Docs](#agent-dispatch-livekit-docs) - [Web and mobile frontends | LiveKit Docs](#web-and-mobile-frontends-livekit-docs) - [Deploying to production | LiveKit Docs](#deploying-to-production-livekit-docs) - [Logs, metrics, and telemetry | LiveKit Docs](#logs-metrics-and-telemetry-livekit-docs) - [Samples | LiveKit Docs](#samples-livekit-docs) - [Unknown](#unknown) - [Audio rendering with React Components | LiveKit Docs](#audio-rendering-with-react-components-livekit-docs) - [RoomScope](#roomscope) - [ParticipantScope](#participantscope) - [livekit.plugins.nltk API documentation](#livekit-plugins-nltk-api-documentation) - [livekit.plugins.turn_detector API documentation](#livekit-plugins-turn-detector-api-documentation) - [livekit.plugins.silero API documentation](#livekit-plugins-silero-api-documentation) - [livekit.plugins.cartesia API documentation](#livekit-plugins-cartesia-api-documentation) - [livekit.plugins.rag API documentation](#livekit-plugins-rag-api-documentation) - [livekit.plugins.playai API documentation](#livekit-plugins-playai-api-documentation) - [rememberParticipants](#rememberparticipants) - [CameraPreview](#camerapreview) - [rememberTrackMuted](#remembertrackmuted) - [LiveKit | Recipes and examples for voice AI and more. | LiveKit Docs](#livekit-recipes-and-examples-for-voice-ai-and-more-livekit-docs) - [rememberTracks](#remembertracks) - [OpenAI Realtime API and LiveKit | LiveKit Docs](#openai-realtime-api-and-livekit-livekit-docs) - [VAD | LiveKit Agents](#vad-livekit-agents) - [VideoTrackView](#videotrackview) - [Log collection | LiveKit Docs](#log-collection-livekit-docs) - [Secrets management | LiveKit Docs](#secrets-management-livekit-docs) - [EgressClient | LiveKit JS Server SDK - v2.13.1](#egressclient-livekit-js-server-sdk-v2-13-1) - [Web egress | LiveKit Docs](#web-egress-livekit-docs) - [Track composite egress | LiveKit Docs](#track-composite-egress-livekit-docs) - [Room composite egress | LiveKit Docs](#room-composite-egress-livekit-docs) - [STT | LiveKit Agents](#stt-livekit-agents) - [Welcome to LiveKit | LiveKit Docs](#welcome-to-livekit-livekit-docs) - [API reference | LiveKit Docs](#api-reference-livekit-docs) - [livekit.api API documentation](#livekit-api-api-documentation) - [Room](#room) - [useRemoteParticipants | React Components | LiveKit Docs](#useremoteparticipants-react-components-livekit-docs) - [useRoomContext | React Components | LiveKit Docs](#useroomcontext-react-components-livekit-docs) - [useChat | React Components | LiveKit Docs](#usechat-react-components-livekit-docs) - [useChatToggle | React Components | LiveKit Docs](#usechattoggle-react-components-livekit-docs) - [useRoomInfo | React Components | LiveKit Docs](#useroominfo-react-components-livekit-docs) - [RoomAudioRenderer | React Components | LiveKit Docs](#roomaudiorenderer-react-components-livekit-docs) - [useFocusToggle | React Components | LiveKit Docs](#usefocustoggle-react-components-livekit-docs) - [useEnsureTrackRef | React Components | LiveKit Docs](#useensuretrackref-react-components-livekit-docs) - [useEnsureLayoutContext | React Components | LiveKit Docs](#useensurelayoutcontext-react-components-livekit-docs) - [useTrackRefContext | React Components | LiveKit Docs](#usetrackrefcontext-react-components-livekit-docs) - [useLocalParticipant | React Components | LiveKit Docs](#uselocalparticipant-react-components-livekit-docs) - [useTrackMutedIndicator | React Components | LiveKit Docs](#usetrackmutedindicator-react-components-livekit-docs) - [useIsRecording | React Components | LiveKit Docs](#useisrecording-react-components-livekit-docs) - [useLiveKitRoom | React Components | LiveKit Docs](#uselivekitroom-react-components-livekit-docs) - [useMaybeLayoutContext | React Components | LiveKit Docs](#usemaybelayoutcontext-react-components-livekit-docs) - [useLocalParticipantPermissions | React Components | LiveKit Docs](#uselocalparticipantpermissions-react-components-livekit-docs) - [useMediaDeviceSelect | React Components | LiveKit Docs](#usemediadeviceselect-react-components-livekit-docs) - [useTrackToggle | React Components | LiveKit Docs](#usetracktoggle-react-components-livekit-docs) - [useMaybeTrackRefContext | React Components | LiveKit Docs](#usemaybetrackrefcontext-react-components-livekit-docs) - [useMaybeRoomContext | React Components | LiveKit Docs](#usemayberoomcontext-react-components-livekit-docs) - [useMediaDevices | React Components | LiveKit Docs](#usemediadevices-react-components-livekit-docs) - [useTrackTranscription | React Components | LiveKit Docs](#usetracktranscription-react-components-livekit-docs) - [useMaybeParticipantContext | React Components | LiveKit Docs](#usemaybeparticipantcontext-react-components-livekit-docs) - [useMultibandTrackVolume | React Components | LiveKit Docs](#usemultibandtrackvolume-react-components-livekit-docs) - [useLayoutContext | React Components | LiveKit Docs](#uselayoutcontext-react-components-livekit-docs) - [usePagination | React Components | LiveKit Docs](#usepagination-react-components-livekit-docs) - [useParticipantAttributes | React Components | LiveKit Docs](#useparticipantattributes-react-components-livekit-docs) - [useKrispNoiseFilter | React Components | LiveKit Docs](#usekrispnoisefilter-react-components-livekit-docs) - [useTrackVolume | React Components | LiveKit Docs](#usetrackvolume-react-components-livekit-docs) - [useParticipantAttribute | React Components | LiveKit Docs](#useparticipantattribute-react-components-livekit-docs) - [useTrackByName | React Components | LiveKit Docs](#usetrackbyname-react-components-livekit-docs) - [useParticipantInfo | React Components | LiveKit Docs](#useparticipantinfo-react-components-livekit-docs) - [useParticipantContext | React Components | LiveKit Docs](#useparticipantcontext-react-components-livekit-docs) - [useTranscriptions | React Components | LiveKit Docs](#usetranscriptions-react-components-livekit-docs) - [useParticipantTile | React Components | LiveKit Docs](#useparticipanttile-react-components-livekit-docs) - [usePersistentUserChoices | React Components | LiveKit Docs](#usepersistentuserchoices-react-components-livekit-docs) - [useParticipantTracks | React Components | LiveKit Docs](#useparticipanttracks-react-components-livekit-docs) - [useVisualStableUpdate | React Components | LiveKit Docs](#usevisualstableupdate-react-components-livekit-docs) - [AudioTrack | React Components | LiveKit Docs](#audiotrack-react-components-livekit-docs) - [AudioVisualizer | React Components | LiveKit Docs](#audiovisualizer-react-components-livekit-docs) - [AudioConference | React Components | LiveKit Docs](#audioconference-react-components-livekit-docs) - [usePinnedTracks | React Components | LiveKit Docs](#usepinnedtracks-react-components-livekit-docs) - [ClearPinButton | React Components | LiveKit Docs](#clearpinbutton-react-components-livekit-docs) - [Chat | React Components | LiveKit Docs](#chat-react-components-livekit-docs) - [usePreviewTracks | React Components | LiveKit Docs](#usepreviewtracks-react-components-livekit-docs) - [ChatToggle | React Components | LiveKit Docs](#chattoggle-react-components-livekit-docs) - [useParticipantPermissions | React Components | LiveKit Docs](#useparticipantpermissions-react-components-livekit-docs) - [CarouselLayout | React Components | LiveKit Docs](#carousellayout-react-components-livekit-docs) - [ChatEntry | React Components | LiveKit Docs](#chatentry-react-components-livekit-docs) - [ConnectionQualityIndicator | React Components | LiveKit Docs](#connectionqualityindicator-react-components-livekit-docs) - [ConnectionStateToast | React Components | LiveKit Docs](#connectionstatetoast-react-components-livekit-docs) - [ConnectionState | React Components | LiveKit Docs](#connectionstate-react-components-livekit-docs) - [FocusLayoutContainer | React Components | LiveKit Docs](#focuslayoutcontainer-react-components-livekit-docs) - [ControlBar | React Components | LiveKit Docs](#controlbar-react-components-livekit-docs) - [MediaDeviceMenu | React Components | LiveKit Docs](#mediadevicemenu-react-components-livekit-docs) - [DisconnectButton | React Components | LiveKit Docs](#disconnectbutton-react-components-livekit-docs) - [FocusToggle | React Components | LiveKit Docs](#focustoggle-react-components-livekit-docs) - [ParticipantName | React Components | LiveKit Docs](#participantname-react-components-livekit-docs) - [FocusLayout | React Components | LiveKit Docs](#focuslayout-react-components-livekit-docs) - [ParticipantContext | React Components | LiveKit Docs](#participantcontext-react-components-livekit-docs) - [LayoutContext | React Components | LiveKit Docs](#layoutcontext-react-components-livekit-docs) - [LayoutContextProvider | React Components | LiveKit Docs](#layoutcontextprovider-react-components-livekit-docs) - [ParticipantLoop | React Components | LiveKit Docs](#participantloop-react-components-livekit-docs) - [ParticipantContextIfNeeded | React Components | LiveKit Docs](#participantcontextifneeded-react-components-livekit-docs) - [MediaDeviceSelect | React Components | LiveKit Docs](#mediadeviceselect-react-components-livekit-docs) - [RoomContext | React Components | LiveKit Docs](#roomcontext-react-components-livekit-docs) - [RoomName | React Components | LiveKit Docs](#roomname-react-components-livekit-docs) - [LiveKitRoom | React Components | LiveKit Docs](#livekitroom-react-components-livekit-docs) - [StartMediaButton | React Components | LiveKit Docs](#startmediabutton-react-components-livekit-docs) - [StartAudio | React Components | LiveKit Docs](#startaudio-react-components-livekit-docs) - [ParticipantAudioTile | React Components | LiveKit Docs](#participantaudiotile-react-components-livekit-docs) - [usePreviewDevice | React Components | LiveKit Docs](#usepreviewdevice-react-components-livekit-docs) - [GridLayout | React Components | LiveKit Docs](#gridlayout-react-components-livekit-docs) - [TrackLoop | React Components | LiveKit Docs](#trackloop-react-components-livekit-docs) - [Toast | React Components | LiveKit Docs](#toast-react-components-livekit-docs) - [VideoConference | React Components | LiveKit Docs](#videoconference-react-components-livekit-docs) - [ParticipantTile | React Components | LiveKit Docs](#participanttile-react-components-livekit-docs) - [useToken | React Components | LiveKit Docs](#usetoken-react-components-livekit-docs) - [TrackRefContext | React Components | LiveKit Docs](#trackrefcontext-react-components-livekit-docs) - [TrackToggle | React Components | LiveKit Docs](#tracktoggle-react-components-livekit-docs) - [PreJoin | React Components | LiveKit Docs](#prejoin-react-components-livekit-docs) - [useGridLayout | React Components | LiveKit Docs](#usegridlayout-react-components-livekit-docs) - [useFacingMode | React Components | LiveKit Docs](#usefacingmode-react-components-livekit-docs) - [useIsEncrypted | React Components | LiveKit Docs](#useisencrypted-react-components-livekit-docs) - [TrackMutedIndicator | React Components | LiveKit Docs](#trackmutedindicator-react-components-livekit-docs) - [useTextStream | React Components | LiveKit Docs](#usetextstream-react-components-livekit-docs) - [useIsSpeaking | React Components | LiveKit Docs](#useisspeaking-react-components-livekit-docs) - [useIsMuted | React Components | LiveKit Docs](#useismuted-react-components-livekit-docs) - [useEnsureParticipant | React Components | LiveKit Docs](#useensureparticipant-react-components-livekit-docs) - [VoiceAssistantControlBar | React Components | LiveKit Docs](#voiceassistantcontrolbar-react-components-livekit-docs) - [useEnsureRoom | React Components | LiveKit Docs](#useensureroom-react-components-livekit-docs) - [useStartVideo | React Components | LiveKit Docs](#usestartvideo-react-components-livekit-docs) - [useSwipe | React Components | LiveKit Docs](#useswipe-react-components-livekit-docs) - [useDisconnectButton | React Components | LiveKit Docs](#usedisconnectbutton-react-components-livekit-docs) - [useStartAudio | React Components | LiveKit Docs](#usestartaudio-react-components-livekit-docs) - [VideoTrack | React Components | LiveKit Docs](#videotrack-react-components-livekit-docs) - [useClearPinButton | React Components | LiveKit Docs](#useclearpinbutton-react-components-livekit-docs) - [useConnectionQualityIndicator | React Components | LiveKit Docs](#useconnectionqualityindicator-react-components-livekit-docs) - [useParticipants | React Components | LiveKit Docs](#useparticipants-react-components-livekit-docs) - [useConnectionState | React Components | LiveKit Docs](#useconnectionstate-react-components-livekit-docs) - [useEnsureCreateLayoutContext | React Components | LiveKit Docs](#useensurecreatelayoutcontext-react-components-livekit-docs) - [useSortedParticipants | React Components | LiveKit Docs](#usesortedparticipants-react-components-livekit-docs) - [useCreateLayoutContext | React Components | LiveKit Docs](#usecreatelayoutcontext-react-components-livekit-docs) - [useAudioWaveform | React Components | LiveKit Docs](#useaudiowaveform-react-components-livekit-docs) - [useDataChannel | React Components | LiveKit Docs](#usedatachannel-react-components-livekit-docs) - [useVoiceAssistant | React Components | LiveKit Docs](#usevoiceassistant-react-components-livekit-docs) - [Rendering a single track | LiveKit Docs](#rendering-a-single-track-livekit-docs) - [useSpeakingParticipants | React Components | LiveKit Docs](#usespeakingparticipants-react-components-livekit-docs) - [useTracks | React Components | LiveKit Docs](#usetracks-react-components-livekit-docs) - [useRemoteParticipant | React Components | LiveKit Docs](#useremoteparticipant-react-components-livekit-docs) - [useAudioPlayback | React Components | LiveKit Docs](#useaudioplayback-react-components-livekit-docs) - [BarVisualizer | React Components | LiveKit Docs](#barvisualizer-react-components-livekit-docs) - [RoomLocal](#roomlocal) - [io.livekit.android.compose.local](#io-livekit-android-compose-local) - [io.livekit.android.room](#io-livekit-android-room) - [AudioHandler](#audiohandler) - [livekit.plugins API documentation](#livekit-plugins-api-documentation) - [CommunicationWorkaround](#communicationworkaround) - [AudioProcessingController](#audioprocessingcontroller) - [LKObjects](#lkobjects) - [NetworkCallbackManagerFactory](#networkcallbackmanagerfactory) - [IncomingDataStreamManager](#incomingdatastreammanager) - [RpcManager](#rpcmanager) - [Participant](#participant) - [plugins/agents-plugin-silero - v0.1.3 | LiveKit Agents](#plugins-agents-plugin-silero-v0-1-3-livekit-agents) - [LiveKit JS Server SDK - v2.13.1](#livekit-js-server-sdk-v2-13-1) - [io.livekit.android.compose.state](#io-livekit-android-compose-state) - [AudioSwitchHandler](#audioswitchhandler) - [io.livekit.android.compose.ui](#io-livekit-android-compose-ui) - [plugins/agents-plugin-deepgram - v0.5.7 | LiveKit Agents](#plugins-agents-plugin-deepgram-v0-5-7-livekit-agents) - [SIPGrant | LiveKit JS Server SDK - v2.13.1](#sipgrant-livekit-js-server-sdk-v2-13-1) - [ScaleType](#scaletype) - [AudioTrackPublishDefaults](#audiotrackpublishdefaults) - [RendererType](#renderertype) --- # LiveKit Agents LiveKit Agents -------------- ![The LiveKit icon, the name of the repository and some sample code in the background.](https://raw.githubusercontent.com/livekit/agents-js/main/.github/banner_light.png) [LiveKit Agents for Node.js](https://docs.livekit.io/reference/agents-js/#md:livekit-agents-for-nodejs) ======================================================================================================== The Agent Framework is designed for building realtime, programmable participants that run on servers. Use it to create conversational, multi-modal voice agents that can see, hear, and understand. This is a Node.js distribution of the [LiveKit Agents framework](https://livekit.io/agents) , originally written in Python. [✨ \[NEW\] In-house phrase endpointing model](https://docs.livekit.io/reference/agents-js/#md:%E2%9C%A8-new-in-house-phrase-endpointing-model) ----------------------------------------------------------------------------------------------------------------------------------------------- We’ve trained a new, open weights phrase endpointing model that significantly improves end-of-turn detection and conversational flow between voice agents and users by reducing agent interruptions. Optimized to run on CPUs, it’s available via [`@livekit/agents-plugin-livekit`](https://docs.livekit.io/reference/agents-js/plugins/livekit) package. > \[!WARNING\] This SDK is in beta. During this period, you may encounter bugs, and the APIs may change. > > For production, we recommend using the [more mature version](https://github.com/livekit/agents) > of this framework, built with Python, which supports a larger number of integrations. > > We welcome and appreciate any feedback or contributions. You can create issues here or chat live with us in the [LiveKit Community Slack](https://livekit.io/join-slack) > . [Installation](https://docs.livekit.io/reference/agents-js/#md:installation) ----------------------------------------------------------------------------- To install the core Agents library: pnpm install @livekit/agents The framework includes a variety of plugins that make it easy to process streaming input or generate output. For example, there are plugins for converting text-to-speech or running inference with popular LLMs. To install a plugin: pnpm install @livekit/agents-plugin-openai The following plugins are available today: | Plugin | Features | | --- | --- | | [@livekit/agents-plugin-openai](https://www.npmjs.com/package/@livekit/agents-plugin-openai) | STT, LLM, TTS, Realtime API | | [@livekit/agents-plugin-deepgram](https://www.npmjs.com/package/@livekit/agents-plugin-deepgram) | STT | | [@livekit/agents-plugin-elevenlabs](https://www.npmjs.com/package/@livekit/agents-plugin-elevenlabs) | TTS | | [@livekit/agents-plugin-cartesia](https://www.npmjs.com/package/@livekit/agents-plugin-cartesia) | TTS | | [@livekit/agents-plugin-resemble](https://www.npmjs.com/package/@livekit/agents-plugin-resemble) | TTS | | [@livekit/agents-plugin-neuphonic](https://www.npmjs.com/package/@livekit/agents-plugin-neuphonic) | TTS | | [@livekit/agents-plugin-silero](https://www.npmjs.com/package/@livekit/agents-plugin-silero) | VAD | | [@livekit/agents-plugin-livekit](https://www.npmjs.com/package/@livekit/agents-plugin-livekit) | End-of-turn detection | [Usage](https://docs.livekit.io/reference/agents-js/#md:usage) --------------------------------------------------------------- First, a few concepts: * **Agent**: A function that defines the workflow of a programmable, server-side participant. This is your application code. * **Worker**: A container process responsible for managing job queuing with LiveKit server. Each worker is capable of running multiple agents simultaneously. * **Plugin**: A library class that performs a specific task, _e.g._ speech-to-text, from a specific provider. An agent can compose multiple plugins together to perform more complex tasks. Your main file for an agent is built of two parts: * The boilerplate code that runs when you run this file, creating a new worker to orchestrate jobs * The code that is exported when this file is imported into Agents, to be ran on all jobs (which includes your entrypoint function, and an optional prewarm function) Refer to the [minimal voice assistant](https://docs.livekit.io/examples/src/multimodal_agent.ts) example to understand how to build a simple voice assistant with function calling using OpenAI's model. [Running](https://docs.livekit.io/reference/agents-js/#md:running) ------------------------------------------------------------------- The framework exposes a CLI interface to run your agent. To get started, you'll need the following environment variables set: * `LIVEKIT_URL` * `LIVEKIT_API_KEY` * `LIVEKIT_API_SECRET` * any additional provider API keys (e.g. `OPENAI_API_KEY`) The following command will start the worker and wait for users to connect to your LiveKit server: node my_agent.js start To run the worker in dev mode (outputting colourful pretty-printed debug logs), run it using `dev`: node my_agent.js dev ### [Using playground for your agent UI](https://docs.livekit.io/reference/agents-js/#md:using-playground-for-your-agent-ui) To ease the process of building and testing an agent, we've developed a versatile web frontend called "playground". You can use or modify this app to suit your specific requirements. It can also serve as a starting point for a completely custom agent application. * [Hosted playground](https://agents-playground.livekit.io/) * [Source code](https://github.com/livekit/agents-playground) * [Playground docs](https://docs.livekit.io/agents/playground) ### [Joining a specific room](https://docs.livekit.io/reference/agents-js/#md:joining-a-specific-room) To join a LiveKit room that's already active, you can use the `connect` command: node my_agent.ts connect --room ### [FAQ](https://docs.livekit.io/reference/agents-js/#md:faq) #### [What happens when I run my agent?](https://docs.livekit.io/reference/agents-js/#md:what-happens-when-i-run-my-agent) When you follow the steps above to run your agent, a worker is started that opens an authenticated WebSocket connection to a LiveKit server instance(defined by your `LIVEKIT_URL` and authenticated with an access token). No agents are actually running at this point. Instead, the worker is waiting for LiveKit server to give it a job. When a room is created, the server notifies one of the registered workers about a new job. The notified worker can decide whether or not to accept it. If the worker accepts the job, the worker will instantiate your agent as a participant and have it join the room where it can start subscribing to tracks. A worker can manage multiple agent instances simultaneously. If a notified worker rejects the job or does not accept within a predetermined timeout period, the server will route the job request to another available worker. #### [What happens when I SIGTERM a worker?](https://docs.livekit.io/reference/agents-js/#md:what-happens-when-i-sigterm-a-worker) The orchestration system was designed for production use cases. Unlike the typical web server, an agent is a stateful program, so it's important that a worker isn't terminated while active sessions are ongoing. When calling SIGTERM on a worker, the worker will signal to LiveKit server that it no longer wants additional jobs. It will also auto-reject any new job requests that get through before the server signal is received. The worker will remain alive while it manages any agents connected to rooms. [License](https://docs.livekit.io/reference/agents-js/#md:license) ------------------------------------------------------------------- This project is licensed under `Apache-2.0`, and is [REUSE-3.2](https://reuse.software/) compliant. Refer to [the license](https://docs.livekit.io/reference/agents-js/LICENSES/Apache-2.0.txt) for details. | LiveKit Ecosystem | | | --- | --- | | LiveKit SDKs | [Browser](https://github.com/livekit/client-sdk-js)
· [iOS/macOS/visionOS](https://github.com/livekit/client-sdk-swift)
· [Android](https://github.com/livekit/client-sdk-android)
· [Flutter](https://github.com/livekit/client-sdk-flutter)
· [React Native](https://github.com/livekit/client-sdk-react-native)
· [Rust](https://github.com/livekit/rust-sdks)
· [Node.js](https://github.com/livekit/node-sdks)
· [Python](https://github.com/livekit/python-sdks)
· [Unity](https://github.com/livekit/client-sdk-unity)
· [Unity (WebGL)](https://github.com/livekit/client-sdk-unity-web) | | Server APIs | [Node.js](https://github.com/livekit/node-sdks)
· [Golang](https://github.com/livekit/server-sdk-go)
· [Ruby](https://github.com/livekit/server-sdk-ruby)
· [Java/Kotlin](https://github.com/livekit/server-sdk-kotlin)
· [Python](https://github.com/livekit/python-sdks)
· [Rust](https://github.com/livekit/rust-sdks)
· [PHP (community)](https://github.com/agence104/livekit-server-sdk-php)
· [.NET (community)](https://github.com/pabloFuente/livekit-server-sdk-dotnet) | | UI Components | [React](https://github.com/livekit/components-js)
· [Android Compose](https://github.com/livekit/components-android)
· [SwiftUI](https://github.com/livekit/components-swift) | | Agents Frameworks | [Python](https://github.com/livekit/agents)
· **Node.js** · [Playground](https://github.com/livekit/agent-playground) | | Services | [LiveKit server](https://github.com/livekit/livekit)
· [Egress](https://github.com/livekit/egress)
· [Ingress](https://github.com/livekit/ingress)
· [SIP](https://github.com/livekit/sip) | | Resources | [Docs](https://docs.livekit.io/)
· [Example apps](https://github.com/livekit-examples)
· [Cloud](https://livekit.io/cloud)
· [Self-hosting](https://docs.livekit.io/home/self-hosting/deployment)
· [CLI](https://github.com/livekit/livekit-cli) | ### Settings #### Member Visibility * Inherited #### Theme OSLightDark ### On This Page [LiveKit Agents for Node.js](https://docs.livekit.io/reference/agents-js/#md:livekit-agents-for-nodejs) * [✨ \[NEW\] In-house phrase endpointing model](https://docs.livekit.io/reference/agents-js/#md:%E2%9C%A8-new-in-house-phrase-endpointing-model) * [Installation](https://docs.livekit.io/reference/agents-js/#md:installation) * [Usage](https://docs.livekit.io/reference/agents-js/#md:usage) * [Running](https://docs.livekit.io/reference/agents-js/#md:running) * * [Using playground for your agent UI](https://docs.livekit.io/reference/agents-js/#md:using-playground-for-your-agent-ui) * [Joining a specific room](https://docs.livekit.io/reference/agents-js/#md:joining-a-specific-room) * [FAQ](https://docs.livekit.io/reference/agents-js/#md:faq) * * [What happens when I run my agent?](https://docs.livekit.io/reference/agents-js/#md:what-happens-when-i-run-my-agent) * [What happens when I SIGTERM a worker?](https://docs.livekit.io/reference/agents-js/#md:what-happens-when-i-sigterm-a-worker) * [License](https://docs.livekit.io/reference/agents-js/#md:license) --- # livekit-compose-components livekit-compose-components ========================== Compose Components for the [Livekit Android Client SDK](https://github.com/livekit/client-sdk-android) . Packages -------- [io.livekit.android.compose.chat](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.chat/index.html) Link copied to clipboard SDK Basic chat functionality provided through the `rememberChat` composable function. [io.livekit.android.compose.flow](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.flow/index.html) Link copied to clipboard SDK Utilities to observe Room events and data flows. [io.livekit.android.compose.local](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.local/index.html) Link copied to clipboard SDK Composition locals to aid in development of Livekit features. [io.livekit.android.compose.sorting](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.sorting/index.html) Link copied to clipboard SDK Sorting functions for room participants. [io.livekit.android.compose.state](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.state/index.html) Link copied to clipboard SDK State functions for room/participants to use within composables. [io.livekit.android.compose.state.transcriptions](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.state.transcriptions/index.html) Link copied to clipboard SDK [io.livekit.android.compose.types](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.types/index.html) Link copied to clipboard SDK [io.livekit.android.compose.ui](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/index.html) Link copied to clipboard SDK UI components for use within Jetpack Compose. [io.livekit.android.compose.ui.audio](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui.audio/index.html) Link copied to clipboard SDK --- # SIP overview | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Introduction](https://docs.livekit.io/sip/#introduction) [Concepts](https://docs.livekit.io/sip/#concepts) [SIP participant](https://docs.livekit.io/sip/#sip-participant) [Trunks](https://docs.livekit.io/sip/#trunks) [Dispatch rules](https://docs.livekit.io/sip/#dispatch-rules) [Service architecture](https://docs.livekit.io/sip/#service-architecture) [Using LiveKit SIP](https://docs.livekit.io/sip/#using-livekit-sip) [SIP features](https://docs.livekit.io/sip/#features) [Noise cancellation for calls](https://docs.livekit.io/sip/#noise-cancellation-for-calls) [Next steps](https://docs.livekit.io/sip/#next-steps) Copy pageSee more page options Introduction[](https://docs.livekit.io/sip/#introduction) ---------------------------------------------------------- LiveKit SIP bridges the gap between traditional telephony and modern digital communication. It enables seamless interaction between traditional phone systems and LiveKit rooms. You can use LiveKit SIP to accept calls and make calls. When you add LiveKit Agents, you can use an AI voice agent to handle your inbound and outbound calls. Concepts[](https://docs.livekit.io/sip/#concepts) -------------------------------------------------- LiveKit SIP extends the [core primitives](https://docs.livekit.io/home/get-started/api-primitives/) —participant, room, and track—to include two additional concepts specific to SIP: trunks and dispatch rules. These concepts are represented by objects created through the [API](https://docs.livekit.io/sip/api/) and control how calls are handled. ### SIP participant[](https://docs.livekit.io/sip/#sip-participant) Each caller, callee, and AI voice agent that participates in a call is a LiveKit participant. A SIP participant is like any other participant and can be managed using the [participant APIs](https://docs.livekit.io/home/server/managing-participants/) . They have the same [attributes and metadata](https://docs.livekit.io/home/client/data/participant-attributes/) as any other participant, and have additional [SIP specific attributes](https://docs.livekit.io/sip/sip-participant/) . For inbound calls, a SIP participant is automatically created for each caller. To make an outbound call, you create a SIP participant using the [`CreateSIPParticipant`](https://docs.livekit.io/sip/api/#createsipparticipant) API to make the call. ### Trunks[](https://docs.livekit.io/sip/#trunks) LiveKit SIP trunks bridge your SIP provider and LiveKit. To use LiveKit, you must configure a SIP trunk with your telephony provider. The setup depends on your use case—whether you're handling incoming calls, making outgoing calls, or both. * [Inbound trunks](https://docs.livekit.io/sip/trunk-inbound/) handle incoming calls and can be restricted to specific IP addresses or phone numbers. * [Outbound trunks](https://docs.livekit.io/sip/trunk-outbound/) are used to place outgoing calls. Trunks can be region restricted to meet local telephony regulations. **Note** The same SIP provider trunk can be associated with both an inbound and an outbound trunk in LiveKit. You only need to create an inbound or outbound trunk _once_. ### Dispatch rules[](https://docs.livekit.io/sip/#dispatch-rules) [Dispatch Rules](https://docs.livekit.io/sip/dispatch-rule/) are associated with a specific trunk and control how inbound calls are dispatched to LiveKit rooms. All callers can be placed in the same room or different rooms based on the dispatch rules. Multiple dispatch rules can be associated with the same trunk as long as each rule has a different pin. Dispatch rules can also be used to add custom participant attributes to [SIP participants](https://docs.livekit.io/sip/sip-participant/) . Service architecture[](https://docs.livekit.io/sip/#service-architecture) -------------------------------------------------------------------------- LiveKit SIP relies on the following services: * SIP trunking provider for your phone number. LiveKit SIP supports most SIP providers out of the box. * LiveKit server (part of LiveKit Cloud) for API requests, managing and verifying SIP trunks and dispatch rules, and creating participants and rooms for calls. * LiveKit SIP (part of LiveKit Cloud) to respond to SIP requests, mediate trunk authentication, and match dispatch rules. If you use LiveKit Cloud, LiveKit SIP is ready to use with your project without any additional configuration. If you're self hosting LiveKit, the SIP service needs to be deployed separately. To learn more about self hosting, see [SIP server](https://docs.livekit.io/home/self-hosting/sip-server/) . ![LiveKit SIP service architecture](https://docs.livekit.io/images/sip/architecture.svg)![LiveKit SIP service architecture](https://docs.livekit.io/images/sip/architecture.svg) Using LiveKit SIP[](https://docs.livekit.io/sip/#using-livekit-sip) -------------------------------------------------------------------- The LiveKit SIP SDK is available in multiple languages. To learn more, see [SIP API](https://docs.livekit.io/sip/api/) . LiveKit SIP has been tested with the following SIP providers: **Note** LiveKit SIP is designed to work with all SIP providers. However, compatibility testing is limited to the providers below. | [Twilio](https://www.twilio.com/) | [Telnyx](https://telnyx.com/) | [Exotel](https://exotel.com/) | [Plivo](https://www.plivo.com/) | | --- | --- | --- | --- | SIP features[](https://docs.livekit.io/sip/#features) ------------------------------------------------------ LiveKit SIP supports the following functionality. | Feature | Description | | --- | --- | | DTMF | You can configure DTMF when making outbound calls by adding them to the `CreateSIPParticipant` request. To learn more, see [Making a call with extension codes (DTMF)](https://docs.livekit.io/sip/outbound-calls/#dtmf)
. | | SIP REFER | You can transfer calls using the `TransferSIPParticipant` API. Calls can be transferred to any valid telephone number or SIP URI. To learn more, see [Cold transfer](https://docs.livekit.io/sip/transfer-cold/)
. | | SIP headers | You can map custom `X-*` SIP headers to participant attributes. For example, custom headers can be used to route calls to different workflows. To learn more, see [Custom attributes](https://docs.livekit.io/sip/sip-participant/#custom-attributes)
. | | Noise cancellation | You can enable noise cancellation for callers and callees using Krisp. To learn more, see [Noise cancellation for calls](https://docs.livekit.io/sip/#noise-cancellation-for-calls)
. | | Region pinning | You can restrict incoming and outgoing calls to a specific region to comply with local telephony regulations. To learn more, see [Region pinning for SIP](https://docs.livekit.io/sip/cloud/#region-pinning)
. | ### Noise cancellation for calls[](https://docs.livekit.io/sip/#noise-cancellation-for-calls) [Krisp](https://krisp.ai/) noise cancellation uses AI models to identify and remove background noise in realtime. This improves the quality of calls that occur in noisy environments. For LiveKit SIP applications that use agents, noise cancellation improves the quality and clarity of user speech for turn detection, transcriptions, and recordings. For incoming calls, see the [inbound trunks documentation](https://docs.livekit.io/sip/trunk-inbound/) for the `krisp_enabled` attribute. For outgoing calls, see the [`CreateSIPParticipant`](https://docs.livekit.io/sip/api/#createsipparticipant) documentation for the `krisp_enabled` attribute used during [outbound call creation](https://docs.livekit.io/sip/outbound-calls/) . Next steps[](https://docs.livekit.io/sip/#next-steps) ------------------------------------------------------ See the following guides to get started with LiveKit SIP: [SIP trunk setup\ ---------------\ \ Purchase a phone number and configure your SIP trunking provider for LiveKit SIP.](https://docs.livekit.io/sip/quickstarts/configuring-sip-trunk/) [Accepting inbound calls\ -----------------------\ \ Learn how to accept inbound calls with LiveKit SIP.](https://docs.livekit.io/sip/accepting-calls/) [Making outbound calls\ ---------------------\ \ Learn how to make outbound calls with LiveKit SIP.](https://docs.livekit.io/sip/making-calls/) [Voice AI telephony guide\ ------------------------\ \ Create an AI agent integrated with telephony.](https://docs.livekit.io/agents/start/telephony/) On this page [Introduction](https://docs.livekit.io/sip/#introduction) [Concepts](https://docs.livekit.io/sip/#concepts) [SIP participant](https://docs.livekit.io/sip/#sip-participant) [Trunks](https://docs.livekit.io/sip/#trunks) [Dispatch rules](https://docs.livekit.io/sip/#dispatch-rules) [Service architecture](https://docs.livekit.io/sip/#service-architecture) [Using LiveKit SIP](https://docs.livekit.io/sip/#using-livekit-sip) [SIP features](https://docs.livekit.io/sip/#features) [Noise cancellation for calls](https://docs.livekit.io/sip/#noise-cancellation-for-calls) [Next steps](https://docs.livekit.io/sip/#next-steps) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/) Search --- # Worker lifecycle | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/worker/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/worker/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/worker/#overview) [Worker options](https://docs.livekit.io/agents/worker/#worker-options) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/worker/#overview) ------------------------------------------------------------ When you start your app with `python agent.py dev`, it registers itself as a **worker** with LiveKit server. LiveKit server manages dispatching your agents to rooms with users by sending requests to available workers. A **LiveKit session** is one or more participants in a [room](https://docs.livekit.io/home/get-started/api-primitives/#room) . A LiveKit session is often referred to simply as a "room." When a user connects to a room, a worker fulfills the request to dispatch an agent to the room. An overview of the worker lifecycle is as follows: 1. **Worker registration**: Your agent code registers itself as a "worker" with LiveKit server, then waits on standby for requests. 2. **Job request**: When a user connects to a room, LiveKit server sends a request to an available worker. A worker accepts and starts a new process to handle the job. This is also known as [agent dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/) . 3. **Job**: The job initiated by your`entrypoint` function. This is the bulk of the code and logic you write. To learn more, see [Job lifecycle](https://docs.livekit.io/agents/worker/job/) . 4. **LiveKit session close**: By default, a room is automatically closed when the last non-agent participant leaves. Any remaining agents disconnect. You can also [end the session](https://docs.livekit.io/agents/worker/job/#ending-the-session) manually. The following diagram shows the worker lifecycle: ![Diagram describing the functionality of agent workers](https://docs.livekit.io/images/agents/agents-jobs-overview.svg)![Diagram describing the functionality of agent workers](https://docs.livekit.io/images/agents/agents-jobs-overview.svg) Some additional features of workers include the following: * Workers automatically exchange availability and capacity information with the LiveKit server, enabling load balancing of incoming requests. * Each worker can run multiple jobs simultaneously, running each in its own process for isolation. If one crashes, it won’t affect others running on the same worker. * When you deploy updates, workers gracefully drain active LiveKit sessions before shutting down, ensuring no sessions are interrupted mid-call. Worker options[](https://docs.livekit.io/agents/worker/#worker-options) ------------------------------------------------------------------------ You can change the permissions, dispatch rules, add prewarm functions, and more through [WorkerOptions](https://docs.livekit.io/agents/worker/options/) . On this page [Overview](https://docs.livekit.io/agents/worker/#overview) [Worker options](https://docs.livekit.io/agents/worker/#worker-options) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/worker/) Search --- # SIP cloud and region pinning | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/cloud/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/cloud/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/sip/cloud/#overview) [Region pinning](https://docs.livekit.io/sip/cloud/#region-pinning) [Inbound calls](https://docs.livekit.io/sip/cloud/#inbound-calls) [Outbound calls](https://docs.livekit.io/sip/cloud/#outbound-calls) [Considerations](https://docs.livekit.io/sip/cloud/#considerations) [Available regions](https://docs.livekit.io/sip/cloud/#available-regions) Copy pageSee more page options Overview[](https://docs.livekit.io/sip/cloud/#overview) -------------------------------------------------------- LiveKit SIP is part of [LiveKit Cloud](https://docs.livekit.io/home/cloud/overview/) and runs as a globally distributed service, providing redundancy and high availability. By default, SIP endpoints are global, and calls are routed through the region closest to the origination point. Incoming calls are routed to the region closest to the SIP trunking provider's endpoint. Outgoing calls originate from the same region where the `CreateSIPParticipant` API call is made. In most cases, using the global endpoint is the recommended approach. However, if you need to exercise more control over call routing—for example, to comply with local telephony regulations—LiveKit SIP supports region pinning. This allows you to restrict both incoming and outgoing calls to a specific region. Region pinning[](https://docs.livekit.io/sip/cloud/#region-pinning) -------------------------------------------------------------------- Region pinning allows you to restrict incoming and outgoing calls to a specific region to comply with local telephony regulations. The following sections describe how to enable region pinning. ### Inbound calls[](https://docs.livekit.io/sip/cloud/#inbound-calls) To enable region pinning for incoming calls, configure your SIP trunking provider to use a region-based endpoint. A region-based endpoint is configured to direct traffic only to nodes within a specific region. #### Region-based endpoint format[](https://docs.livekit.io/sip/cloud/#region-based-endpoint) The endpoint format is as follows: {sip\_subdomain}.{region\_name}.sip.livekit.cloud Where: * `{sip_subdomain}` is your LiveKit SIP URI subdomain. This is also your project ID without the `p_` prefix. You can find your SIP URI on the [Project settings](https://cloud.livekit.io/projects/p_/settings/project) page. For example, if your SIP URI is `sip:bwwn08a2m4o.sip.livekit.cloud`, your SIP subdomain is `bwwn08a2m4o`. * `{region_name}` is one of the following [regions](https://docs.livekit.io/sip/cloud/#available-regions) : `eu`, `india`, `sa`, `us` For example to create a SIP endpoint for India, see the following: **Tip** Sign in to LiveKit Cloud to automatically include the subdomain for your project in the example. .india.sip.livekit.cloud Use the region-based endpoint to configure your SIP trunking provider. Follow the instructions for external provider setup in [SIP trunk setup](https://docs.livekit.io/sip/quickstarts/configuring-sip-trunk/) . ### Outbound calls[](https://docs.livekit.io/sip/cloud/#outbound-calls) To originate calls from the same region as the destination phone number, set the `destination_country` parameter for an outbound trunk. This applies region pinning to all calls made through the trunk. When `destination_country` is enabled, outbound calls are routed based on location: * For countries that LiveKit operates data centers in, calls originate from a server within the country. * For other countries, calls originate from a server that is closest to that country. In the unlikely event that the preferred region is non-operational or offline, calls originate from another region nearby. For a full list of supported regions, see [Available regions](https://docs.livekit.io/sip/cloud/#available-regions) . The `destination_country` parameter accepts a two-letter [country code](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) . To learn more, see [CreateSIPOutboundTrunk](https://docs.livekit.io/sip/api/#createsipoutboundtrunk) . #### Example outbound trunk[](https://docs.livekit.io/sip/cloud/#example-outbound-trunk) Create an outbound trunk with the `destination_country` parameter set to India, `india`. 1. Create a file named `outbound-trunk.json`, replacing the phone number with your SIP provider phone number and username and password: { "trunk": { "name": "My outbound trunk", "phone\_number": "+15105550100", "username": "myusername", "password": "mypassword", "destination\_country": "in" } } 2. Create the outbound trunk using the CLI: lk sip outbound create outbound-trunk.json To learn more, see [Outbound trunks](https://docs.livekit.io/sip/trunk-outbound/) . ### Considerations[](https://docs.livekit.io/sip/cloud/#considerations) When you enable region pinning, you turn off automatic failover to the nearest region in the case of an outage. ### Available regions[](https://docs.livekit.io/sip/cloud/#available-regions) The following regions are available for region pinning for SIP: | Region name | Region locations | | --- | --- | | `eu` | France, Germany, Zurich | | `india` | India | | `sa` | Saudi Arabia | | `us` | US Central, US East B, US West B | **Note** This list of regions is subject to change. Last updated 2025-07-23. On this page [Overview](https://docs.livekit.io/sip/cloud/#overview) [Region pinning](https://docs.livekit.io/sip/cloud/#region-pinning) [Inbound calls](https://docs.livekit.io/sip/cloud/#inbound-calls) [Outbound calls](https://docs.livekit.io/sip/cloud/#outbound-calls) [Considerations](https://docs.livekit.io/sip/cloud/#considerations) [Available regions](https://docs.livekit.io/sip/cloud/#available-regions) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/cloud/) Search --- # Accepting inbound calls | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/accepting-calls/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/accepting-calls/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Inbound call workflow](https://docs.livekit.io/sip/accepting-calls/#inbound-call-workflow) [Setup for accepting calls](https://docs.livekit.io/sip/accepting-calls/#setup-for-accepting-calls) [SIP trunking provider setup](https://docs.livekit.io/sip/accepting-calls/#sip-trunking-provider-setup) [LiveKit SIP configuration](https://docs.livekit.io/sip/accepting-calls/#livekit-sip-configuration) [Next steps](https://docs.livekit.io/sip/accepting-calls/#next-steps) Copy pageSee more page options Inbound call workflow[](https://docs.livekit.io/sip/accepting-calls/#inbound-call-workflow) -------------------------------------------------------------------------------------------- When an inbound call is received, your SIP trunking provider sends a text-based INVITE request to LiveKit SIP. The SIP service checks authorization credentials configured for the LiveKit trunk with the credentials configured on your provider's SIP trunk and looks for a matching dispatch rule. If there's a matching dispatch rule, a SIP participant is created for the caller and put into a LiveKit room. Depending on the dispatch rule, other participants (for example, a voice agent or other users) might join the room. ![Inbound SIP workflow](https://docs.livekit.io/images/sip/inbound-sip-workflow.svg)![Inbound SIP workflow](https://docs.livekit.io/images/sip/inbound-sip-workflow.svg) 1. User dials the SIP trunking provider phone number. 2. SIP trunking provider connects caller to LiveKit SIP. 3. LiveKit SIP authenticates the trunk credentials and finds a matching dispatch rule. 4. LiveKit server creates a SIP participant for the caller and places them in a LiveKit room (per the dispatch rule). 5. User hears dial tone until LiveKit SIP responds to the call: 1. If the dispatch rule has a pin, prompts the user with "Please enter room pin and press hash to confirm." * Incorrect pin: "No room matched the pin you entered." Call is disconnected with a tone. * Correct pin: "Entering room now." User continues to hear a dial tone until another participant publishes tracks to the room. Setup for accepting calls[](https://docs.livekit.io/sip/accepting-calls/#setup-for-accepting-calls) ---------------------------------------------------------------------------------------------------- The following are required to accept an inbound SIP call. ### SIP trunking provider setup[](https://docs.livekit.io/sip/accepting-calls/#sip-trunking-provider-setup) 1. Purchase a phone number from a SIP provider. For a list of tested providers, see the table in [Using LiveKit SIP](https://docs.livekit.io/sip/#using-livekit-sip) . 2. Configure SIP trunking with the provider to send SIP traffic to your LiveKit SIP instance. For instructions for setting up a SIP trunk, see [Configuring a SIP provider trunk](https://docs.livekit.io/sip/quickstarts/configuring-sip-trunk/) . ### LiveKit SIP configuration[](https://docs.livekit.io/sip/accepting-calls/#livekit-sip-configuration) 1. Create an [inbound trunk](https://docs.livekit.io/sip/trunk-inbound/) associated with your SIP provider phone number. You only need to create one inbound trunk for each SIP provider phone number. 2. Create a [dispatch rule](https://docs.livekit.io/sip/dispatch-rule/) . The dispatch rules dictate how SIP participants and LiveKit rooms are created for incoming calls. The rules can include whether a caller needs to enter a pin code to join a room and any custom metadata or attributes to be added to SIP participants. Next steps[](https://docs.livekit.io/sip/accepting-calls/#next-steps) ---------------------------------------------------------------------- See the following guide to create an AI agent to receive inbound calls. [Voice AI telephony guide\ ------------------------\ \ Create an AI agent to receive inbound calls.](https://docs.livekit.io/agents/start/telephony/) On this page [Inbound call workflow](https://docs.livekit.io/sip/accepting-calls/#inbound-call-workflow) [Setup for accepting calls](https://docs.livekit.io/sip/accepting-calls/#setup-for-accepting-calls) [SIP trunking provider setup](https://docs.livekit.io/sip/accepting-calls/#sip-trunking-provider-setup) [LiveKit SIP configuration](https://docs.livekit.io/sip/accepting-calls/#livekit-sip-configuration) [Next steps](https://docs.livekit.io/sip/accepting-calls/#next-steps) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/accepting-calls/) Search --- # Building voice agents | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/build/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/build/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/build/#overview) [Agent sessions](https://docs.livekit.io/agents/build/#agent-sessions) [RoomIO](https://docs.livekit.io/agents/build/#roomio) [Voice AI providers](https://docs.livekit.io/agents/build/#voice-ai-providers) [Capabilities](https://docs.livekit.io/agents/build/#capabilities) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/build/#overview) ----------------------------------------------------------- Building a great voice AI app requires careful orchestration of multiple components. LiveKit Agents is built on top of the [Realtime SDK](https://github.com/livekit/python-sdks) to provide dedicated abstractions that simplify development while giving you full control over the underlying code. Agent sessions[](https://docs.livekit.io/agents/build/#agent-sessions) ----------------------------------------------------------------------- The `AgentSession` is the main orchestrator for your voice AI app. The session is responsible for collecting user input, managing the voice pipeline, invoking the LLM, and sending the output back to the user. Each session requires at least one `Agent` to orchestrate. The agent is responsible for defining the core AI logic - instructions, tools, etc - of your app. The framework supports the design of custom [workflows](https://docs.livekit.io/agents/build/workflows/) to orchestrate handoff and delegation between multiple agents. The following example shows how to begin a simple single-agent session: from livekit.agents import AgentSession, Agent, RoomInputOptions from livekit.plugins import openai, cartesia, deepgram, noise\_cancellation, silero from livekit.plugins.turn\_detector.multilingual import MultilingualModel session \= AgentSession( stt\=deepgram.STT(), llm\=openai.LLM(), tts\=cartesia.TTS(), vad\=silero.VAD.load(), turn\_detection\=turn\_detector.MultilingualModel(), ) await session.start( room\=ctx.room, agent\=Agent(instructions\="You are a helpful voice AI assistant."), room\_input\_options\=RoomInputOptions( noise\_cancellation\=noise\_cancellation.BVC(), ), ) ### RoomIO[](https://docs.livekit.io/agents/build/#roomio) Communication between agent and user participants happens using media streams, also known as tracks. For voice AI apps, this is primarily audio, but can include vision. By default, track management is handled by `RoomIO`, a utility class that serves as a bridge between the agent session and the LiveKit room. When an AgentSession is initiated, it automatically creates a `RoomIO` object that enables all room participants to subscribe to available audio tracks. To learn more about publishing audio and video, see the following topics: [Agent speech and audio\ ----------------------\ \ Add speech, audio, and background audio to your agent.](https://docs.livekit.io/agents/build/audio/) [Vision\ ------\ \ Give your agent the ability to see images and live video.](https://docs.livekit.io/agents/build/vision/) [Text and transcription\ ----------------------\ \ Send and receive text messages and transcription to and from your agent.](https://docs.livekit.io/agents/build/text/) [Realtime media\ --------------\ \ Tracks are a core LiveKit concept. Learn more about publishing and subscribing to media.](https://docs.livekit.io/home/client/tracks/) [Camera and microphone\ ---------------------\ \ Use the LiveKit SDKs to publish audio and video tracks from your user's device.](https://docs.livekit.io/home/client/tracks/publish/) #### Custom RoomIO[](https://docs.livekit.io/agents/build/#custom-roomio) For greater control over media sharing in a room, you can create a custom `RoomIO` object. For example, you might want to manually control which input and output devices are used, or to control which participants an agent listens to or responds to. To replace the default one created in `AgentSession`, create a `RoomIO` object in your entrypoint function and pass it an instance of the `AgentSession` in the constructor. For examples, see the following in the GitHub repository: [Toggling audio\ --------------\ \ Create a push-to-talk interface to toggle audio input and output.](https://github.com/livekit/agents/blob/main/examples/voice_agents/push_to_talk.py) [Toggling input and output\ -------------------------\ \ Toggle both audio and text input and output.](https://github.com/livekit/agents/blob/main/examples/voice_agents/toggle_io.py) Voice AI providers[](https://docs.livekit.io/agents/build/#voice-ai-providers) ------------------------------------------------------------------------------- You can choose from a variety of providers for each part of the voice pipeline to fit your needs. The framework supports both high-performance STT-LLM-TTS pipelines and speech-to-speech models. In either case, it automatically manages interruptions, transcription forwarding, turn detection, and more. You may add these components to the `AgentSession`, where they act as global defaults within the app, or to each individual `Agent` if needed. [TTS\ ---\ \ Text-to-speech integrations](https://docs.livekit.io/agents/integrations/tts/) [STT\ ---\ \ Speech-to-text integrations](https://docs.livekit.io/agents/integrations/stt/) [LLM\ ---\ \ Language model integrations](https://docs.livekit.io/agents/integrations/llm/) [Multimodal\ ----------\ \ Realtime multimodal APIs](https://docs.livekit.io/agents/integrations/realtime/) Capabilities[](https://docs.livekit.io/agents/build/#capabilities) ------------------------------------------------------------------- The following guides, in addition to others in this section, cover the core capabilities of the `AgentSession` and how to leverage them in your app. [Workflows\ ---------\ \ Orchestrate complex tasks among multiple agents.](https://docs.livekit.io/agents/build/workflows/) [Tool definition & use\ ---------------------\ \ Use tools to call external services, inject custom logic, and more.](https://docs.livekit.io/agents/build/tools/) [Pipeline nodes\ --------------\ \ Add custom behavior to any component of the voice pipeline.](https://docs.livekit.io/agents/build/nodes/) On this page [Overview](https://docs.livekit.io/agents/build/#overview) [Agent sessions](https://docs.livekit.io/agents/build/#agent-sessions) [RoomIO](https://docs.livekit.io/agents/build/#roomio) [Voice AI providers](https://docs.livekit.io/agents/build/#voice-ai-providers) [Capabilities](https://docs.livekit.io/agents/build/#capabilities) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/build/) Search --- # LiveKit Agents integrations | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/integrations/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/integrations/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/integrations/#overview) [Installing plugins](https://docs.livekit.io/agents/integrations/#install) [Using plugins](https://docs.livekit.io/agents/integrations/#using-plugins) [OpenAI API compatibility](https://docs.livekit.io/agents/integrations/#openai-api-compatibility) [Core plugins](https://docs.livekit.io/agents/integrations/#core-plugins) [Additional plugins](https://docs.livekit.io/agents/integrations/#additional-plugins) [Build your own plugin](https://docs.livekit.io/agents/integrations/#contribute) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/integrations/#overview) ------------------------------------------------------------------ LiveKit Agents includes support for a wide variety of AI providers, from the largest research companies to emerging startups. The open source plugin interface makes it easy to adopt the best AI providers for your app, without needing customized code for each. Installing plugins[](https://docs.livekit.io/agents/integrations/#install) --------------------------------------------------------------------------- Each provider is available as a separate plugin package, included as an optional dependency on the base SDK for Python. For example, to install the SDK with the Cartesia, Deepgram, and OpenAI plugins, run the following command: pip install "livekit-agents\[cartesia,deepgram,openai\]~=1.0" You may also install plugins as individual packages. For example, this is equivalent to the previous command: pip install \\ "livekit-agents~=1.0" \\ "livekit-plugins-cartesia~=1.0" \\ "livekit-plugins-deepgram~=1.0" \\ "livekit-plugins-openai~=1.0" Using plugins[](https://docs.livekit.io/agents/integrations/#using-plugins) ---------------------------------------------------------------------------- The AgentSession class accepts plugins as arguments using a standard interface. Each plugin loads its own associated API key from environment variables. For instance, the following code creates an AgentSession that uses the OpenAI, Cartesia, and Deepgram plugins installed in the preceding section: agent.py .env from livekit.plugins import openai, cartesia, deepgram session \= AgentSession( llm\=openai.LLM(model\="gpt-4o"), tts\=cartesia.TTS(model\="sonic-english"), stt\=deepgram.STT(model\="nova-2"), ) OpenAI API compatibility[](https://docs.livekit.io/agents/integrations/#openai-api-compatibility) -------------------------------------------------------------------------------------------------- Many providers have standardized around the OpenAI API format for chat completions and more. The LiveKit Agents OpenAI plugin provides easy compatibility with many of these providers through special methods which load the correct API key from environment variables. For instance, to use Cerebras instead of OpenAI, you can use the following code: agent.py .env from livekit.plugins import openai session \= AgentSession( llm\=openai.LLM.with\_cerebras(model\="llama-3.1-70b-versatile"), \# ... stt, tts, etc .. ) Core plugins[](https://docs.livekit.io/agents/integrations/#core-plugins) -------------------------------------------------------------------------- The following are the core plugin types used in LiveKit Agents, to handle the primary voice AI tasks. Many providers are available for most functions. [Realtime models\ ---------------\ \ Plugins for multimodal speech-to-speech models like the OpenAI Realtime API.](https://docs.livekit.io/agents/integrations/realtime/) [Large language models (LLM)\ ---------------------------\ \ Plugins for AI models from OpenAI, Anthropic, and more.](https://docs.livekit.io/agents/integrations/llm/) [Speech-to-text (STT)\ --------------------\ \ Plugins for speech-to-text solutions like Deepgram, Whisper, and more.](https://docs.livekit.io/agents/integrations/stt/) [Text-to-speech (TTS)\ --------------------\ \ Plugins for text-to-speech solutions like Cartesia, ElevenLabs, and more.](https://docs.livekit.io/agents/integrations/tts/) [Virtual Avatars\ ---------------\ \ Plugins for virtual avatar solutions like Hedra, Tavus, and more.](https://docs.livekit.io/agents/integrations/avatar/) Additional plugins[](https://docs.livekit.io/agents/integrations/#additional-plugins) -------------------------------------------------------------------------------------- LiveKit Agents also includes the following additional specialized plugins, which are recommended for most voice AI use cases. Each runs locally and requires no additional API keys. [Silero VAD\ ----------\ \ Voice activity detection with Silero VAD.](https://docs.livekit.io/agents/build/turns/vad/) [LiveKit turn detector\ ---------------------\ \ A custom LiveKit model for improved end-of-turn detection.](https://docs.livekit.io/agents/build/turns/turn-detector/) [Enhanced noise cancellation\ ---------------------------\ \ LiveKit Cloud enhanced noise cancellation to improve voice AI performance.](https://pypi.org/project/livekit-plugins-noise-cancellation/) Build your own plugin[](https://docs.livekit.io/agents/integrations/#contribute) --------------------------------------------------------------------------------- The LiveKit Agents plugin framework is extensible and community-driven. Your plugin can integrate with new providers or directly load models for local inference. LiveKit especially welcomes new TTS, STT, and LLM plugins. To learn more, see the guidelines for contributions to the [Python](https://github.com/livekit/agents/blob/main/CONTRIBUTING.md) and [Node.js](https://github.com/livekit/agents-js/blob/main/CONTRIBUTING.md) SDKs. On this page [Overview](https://docs.livekit.io/agents/integrations/#overview) [Installing plugins](https://docs.livekit.io/agents/integrations/#install) [Using plugins](https://docs.livekit.io/agents/integrations/#using-plugins) [OpenAI API compatibility](https://docs.livekit.io/agents/integrations/#openai-api-compatibility) [Core plugins](https://docs.livekit.io/agents/integrations/#core-plugins) [Additional plugins](https://docs.livekit.io/agents/integrations/#additional-plugins) [Build your own plugin](https://docs.livekit.io/agents/integrations/#contribute) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/integrations/) Search --- # LiveKit Cloud | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/cloud/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/cloud/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Why Choose LiveKit Cloud?](https://docs.livekit.io/home/cloud/#why-choose-livekit-cloud-) [Comparing Open Source and Cloud](https://docs.livekit.io/home/cloud/#comparing-open-source-and-cloud) Copy pageSee more page options LiveKit Cloud is a fully-managed, globally distributed mesh network of LiveKit servers that provides all the power of the open-source platform with none of the operational complexity. It allows you to focus on building your application while LiveKit handles deployment, scaling, and maintenance. [Dashboard\ ---------\ \ Sign up for LiveKit Cloud to manage your projects, view analytics, and configure your LiveKit Cloud deployment.](https://cloud.livekit.io/) [Pricing\ -------\ \ View LiveKit Cloud pricing plans and choose the right option for your application's needs.](https://livekit.io/pricing) Why Choose LiveKit Cloud?[](https://docs.livekit.io/home/cloud/#why-choose-livekit-cloud-) ------------------------------------------------------------------------------------------- * **Zero operational overhead**: No need to manage servers, scaling, or infrastructure. * **Global edge network**: Users connect to the closest server for minimal latency. * **Unlimited scale**: Support for rooms with unlimited participants through our mesh architecture. * **Enterprise-grade reliability**: 99.99% uptime guarantee with redundant infrastructure. * **Comprehensive analytics**: Monitor usage, performance, and quality metrics through the Cloud dashboard. * **Same APIs and SDKs**: Use the exact same code whether you're on Cloud or self-hosted. LiveKit Cloud runs the same open-source servers that you can find on GitHub. It provides the same APIs and supports all of the same SDKs. An open source user can migrate to Cloud, and a Cloud customer can switch to self-hosted at any moment. As far as your code is concerned, the only difference is the URL that it connects to. For more details on LiveKit Cloud's architecture, see [Cloud Architecture](https://docs.livekit.io/home/cloud/architecture/) . Comparing Open Source and Cloud[](https://docs.livekit.io/home/cloud/#comparing-open-source-and-cloud) ------------------------------------------------------------------------------------------------------- When building with LiveKit, you can either self-host the open-source server or use the managed LiveKit Cloud service: | | Open Source | Cloud | | --- | --- | --- | | **Realtime features** | Full support | Full support | | **Egress (recording, streaming)** | Full support | Full support | | **Ingress (RTMP, WHIP, SRT ingest)** | Full support | Full support | | **SIP (telephony integration)** | Full support | Full support | | **Agents framework** | Full support | Full support | | **Who manages it** | You | LiveKit | | **Architecture** | Single-home SFU | Mesh SFU | | **Connection model** | Users in the same room connect to the same server | Each user connects to the closest server | | **Max users per room** | Up to ~3,000 | No limit | | **Analytics & telemetry** | N/A | Cloud dashboard | | **Uptime guarantees** | N/A | 99.99% | On this page [Why Choose LiveKit Cloud?](https://docs.livekit.io/home/cloud/#why-choose-livekit-cloud-) [Comparing Open Source and Cloud](https://docs.livekit.io/home/cloud/#comparing-open-source-and-cloud) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/cloud/) Search --- # LiveKit Agents | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Introduction](https://docs.livekit.io/agents/#introduction) [Use cases](https://docs.livekit.io/agents/#use-cases) [Framework overview](https://docs.livekit.io/agents/#framework-overview) [How agents connect to LiveKit](https://docs.livekit.io/agents/#how-agents-connect-to-livekit) [Getting started](https://docs.livekit.io/agents/#getting-started) Copy pageSee more page options Introduction[](https://docs.livekit.io/agents/#introduction) ------------------------------------------------------------- The Agents framework allows you to add a Python or Node.js program to any LiveKit room as a full realtime participant. The SDK includes a complete set of tools and abstractions that make it easy to feed realtime media and data through an AI pipeline that works with any provider, and to publish realtime results back to the room. If you want to get your hands on the code right away, follow this quickstart guide. It takes just a few minutes to build your first voice agent. [Voice AI quickstart\ -------------------\ \ Build a simple voice assistant with Python in less than 10 minutes.](https://docs.livekit.io/agents/start/voice-ai/) [Deeplearning.ai course\ ----------------------\ \ Learn to build and deploy voice agents with LiveKit in this free course from Deeplearning.ai.](https://www.deeplearning.ai/short-courses/building-ai-voice-agents-for-production/) [GitHub\ \ GitHub repository\ -----------------\ \ Python source code and examples for the LiveKit Agents SDK.](https://github.com/livekit/agents) [SDK reference\ -------------\ \ Python reference docs for the LiveKit Agents SDK.](https://docs.livekit.io/reference/python/v1/livekit/agents/index.html) Use cases[](https://docs.livekit.io/agents/#use-cases) ------------------------------------------------------- Some applications for agents include: * **Multimodal assistant**: Talk, text, or screen share with an AI assistant. * **Telehealth**: Bring AI into realtime telemedicine consultations, with or without humans in the loop. * **Call center**: Deploy AI to the front lines of customer service with inbound and outbound call support. * **Realtime translation**: Translate conversations in realtime. * **NPCs**: Add lifelike NPCs backed by language models instead of static scripts. * **Robotics**: Put your robot's brain in the cloud, giving it access to the most powerful models. The following [recipes](https://docs.livekit.io/recipes/) demonstrate some of these use cases: [Medical Office Triage\ ---------------------\ \ Agent that triages patients based on symptoms and medical history.](https://github.com/livekit-examples/python-agents-examples/tree/main/complex-agents/medical_office_triage) [Restaurant Agent\ ----------------\ \ A restaurant front-of-house agent that can take orders, add items to a shared cart, and checkout.](https://github.com/livekit/agents/blob/main/examples/voice_agents/restaurant_agent.py) [Company Directory\ -----------------\ \ Build a AI company directory agent. The agent can respond to DTMF tones and voice prompts, then redirect callers.](https://docs.livekit.io/recipes/company-directory/) [Pipeline Translator\ -------------------\ \ Implement translation in the processing pipeline.](https://github.com/livekit-examples/python-agents-examples/tree/main/translators/pipeline_translator.py) Framework overview[](https://docs.livekit.io/agents/#framework-overview) ------------------------------------------------------------------------- ![Diagram showing framework overview.](https://docs.livekit.io/images/agents/framework-overview.svg)![Diagram showing framework overview.](https://docs.livekit.io/images/agents/framework-overview.svg) Your agent code operates as a stateful, realtime bridge between powerful AI models and your users. While AI models typically run in data centers with reliable connectivity, users often connect from mobile networks with varying quality. WebRTC ensures smooth communication between agents and users, even over unstable connections. LiveKit WebRTC is used between the frontend and the agent, while the agent communicates with your backend using HTTP and WebSockets. This setup provides the benefits of WebRTC without its typical complexity. The agents SDK includes components for handling the core challenges of realtime voice AI, such as streaming audio through an STT-LLM-TTS pipeline, reliable turn detection, handling interruptions, and LLM orchestration. It supports plugins for most major AI providers, with more continually added. The framework is fully open source and supported by an active community. Other framework features include: * **Voice, video, and text**: Build agents that can process realtime input and produce output in any modality. * **Tool use**: Define tools that are compatible with any LLM, and even forward tool calls to your frontend. * **Multi-agent handoff**: Break down complex workflows into simpler tasks. * **Extensive integrations**: Integrate with nearly every AI provider there is for LLMs, STT, TTS, and more. * **State-of-the-art turn detection**: Use the custom turn detection model for lifelike conversation flow. * **Made for developers**: Build your agents in code, not configuration. * **Production ready**: Includes built-in worker orchestration, load balancing, and Kubernetes compatibility. * **Open source**: The framework and entire LiveKit ecosystem are open source under the Apache 2.0 license. How agents connect to LiveKit[](https://docs.livekit.io/agents/#how-agents-connect-to-livekit) ----------------------------------------------------------------------------------------------- ![Diagram showing a high-level view of how agents work.](https://docs.livekit.io/images/agents/agents-jobs-overview.svg)![Diagram showing a high-level view of how agents work.](https://docs.livekit.io/images/agents/agents-jobs-overview.svg) When your agent code starts, it first registers with a LiveKit server (either [self hosted](https://docs.livekit.io/home/self-hosting/deployment/) or [LiveKit Cloud](https://cloud.livekit.io/) ) to run as a "worker" process. The worker waits until it receives a dispatch request. To fulfill this request, the worker boots a "job" subprocess which joins the room. By default, your workers are dispatched to each new room created in your LiveKit project. To learn more about workers, see the [Worker lifecycle](https://docs.livekit.io/agents/worker/) guide. After your agent and user join a room, the agent and your frontend app can communicate using LiveKit WebRTC. This enables reliable and fast realtime communication in any network conditions. LiveKit also includes full support for telephony, so the user can join the call from a phone instead of a frontend app. To learn more about how LiveKit works overall, see the [Intro to LiveKit](https://docs.livekit.io/home/get-started/intro-to-livekit/) guide. Getting started[](https://docs.livekit.io/agents/#getting-started) ------------------------------------------------------------------- Follow these guides to learn more and get started with LiveKit Agents. [Voice AI quickstart\ -------------------\ \ Build a simple voice assistant with Python in less than 10 minutes.](https://docs.livekit.io/agents/start/voice-ai/) [Recipes\ -------\ \ A comprehensive collection of examples, guides, and recipes for LiveKit Agents.](https://docs.livekit.io/recipes/) [Intro to LiveKit\ ----------------\ \ An overview of the LiveKit ecosystem.](https://docs.livekit.io/home/get-started/intro-to-livekit/) [Web and mobile frontends\ ------------------------\ \ Put your agent in your pocket with a custom web or mobile app.](https://docs.livekit.io/agents/start/frontend/) [Telephony integration\ ---------------------\ \ Your agent can place and receive calls with LiveKit's SIP integration.](https://docs.livekit.io/agents/start/telephony/) [Building voice agents\ ---------------------\ \ Comprehensive documentation to build advanced voice AI apps with LiveKit.](https://docs.livekit.io/agents/build/) [Worker lifecycle\ ----------------\ \ Learn how to manage your agents with workers and jobs.](https://docs.livekit.io/agents/worker/) [Deploying to production\ -----------------------\ \ Guide to deploying your voice agent in a production environment.](https://docs.livekit.io/agents/ops/deployment/) [Integration guides\ ------------------\ \ Explore the full list of AI providers available for LiveKit Agents.](https://docs.livekit.io/agents/integrations/) On this page [Introduction](https://docs.livekit.io/agents/#introduction) [Use cases](https://docs.livekit.io/agents/#use-cases) [Framework overview](https://docs.livekit.io/agents/#framework-overview) [How agents connect to LiveKit](https://docs.livekit.io/agents/#how-agents-connect-to-livekit) [Getting started](https://docs.livekit.io/agents/#getting-started) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/) Search --- # SDK migration from v1 to v2 | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/migrate-from-v1/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/migrate-from-v1/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Breaking changes across SDKs](https://docs.livekit.io/reference/migrate-from-v1/#breaking-changes-across-sdks) [room.participants -> room.remoteParticipants](https://docs.livekit.io/reference/migrate-from-v1/#room-participants-room-remoteparticipants) [track -> trackPublication](https://docs.livekit.io/reference/migrate-from-v1/#track-trackpublication) [Updated publishData API](https://docs.livekit.io/reference/migrate-from-v1/#updated-publishdata-api) [Async room SID](https://docs.livekit.io/reference/migrate-from-v1/#async-room-sid) [Removed VideoQuality.OFF from VideoQuality enum](https://docs.livekit.io/reference/migrate-from-v1/#removed-videoquality-off-from-videoquality-enum) [Platform specific changes](https://docs.livekit.io/reference/migrate-from-v1/#platform-specific-changes) [Android](https://docs.livekit.io/reference/migrate-from-v1/#android) [Flutter](https://docs.livekit.io/reference/migrate-from-v1/#flutter) [Javascript/Typescript](https://docs.livekit.io/reference/migrate-from-v1/#javascript-typescript) [Swift](https://docs.livekit.io/reference/migrate-from-v1/#swift) [Go](https://docs.livekit.io/reference/migrate-from-v1/#go) Copy pageSee more page options Version 2 of the LiveKit SDKs include a small number of breaking changes, focused on: * Streamlining APIs to reduce confusion and improve naming consistency. * Updated APIs to accept a participant's identity instead of their SID, offering a more intuitive experience as identities are application-provided. * Enabling the coexistence of multiple libraries dependent on libwebrtc with LiveKit native SDKs. Breaking changes across SDKs[](https://docs.livekit.io/reference/migrate-from-v1/#breaking-changes-across-sdks) ---------------------------------------------------------------------------------------------------------------- This section outlines changes applicable to all frontend/client SDKs. ### `room.participants` -> `room.remoteParticipants`[](https://docs.livekit.io/reference/migrate-from-v1/#room-participants-room-remoteparticipants) In v2, we've updated the participants map on the room object, with key changes to note: * Clarification: `localParticipant` has always been excluded from this map, so the term `participants` was previously misleading. * Map key change: Instead of using the participant's `SID` as the map key, we now use their `identity`. AndroidFlutterGoJavaScriptSwift // legacy v1: in v1 participants were stored in a map with keys representing their SID. This led to unnecessary complications e.g. when trying to filter for a list of identities val alice \= room.remoteParticipants\['PA\_8sMkEu4vhz4v'\]; // new in v2: you can now use a participant's identity (encoded in the token) to directly access it from the remoteParticipants map val alice \= room.remoteParticipants\[Participant.Identity('alice')\]; ### `track` -> `trackPublication`[](https://docs.livekit.io/reference/migrate-from-v1/#track-trackpublication) In version 1, our SDKs used the term `track` ambiguously, referring to both `TrackPublication` and `Track`. In version 2, we've simplified this terminology: now, all API references to publications explicitly use `trackPublications`. For instance, * `participant.tracks` -> `participant.trackPublications` * `participant.getTrack` -> `participant.getTrackPublication` * `participant.videoTracks` -> `participant.videoTrackPublications` AndroidFlutterGoJavaScriptSwift // v1 val trackPublications \= room.localParticipant.tracks // v2 val trackPublications \= room.localParticipant.trackPublications ### Updated publishData API[](https://docs.livekit.io/reference/migrate-from-v1/#updated-publishdata-api) We've streamlined the `publishData` API in v2, reducing its arguments to: 1. The payload (data being sent) 2. A `DataPublishOptions` object for advanced features `DataPublishOptions` now allows you to: * specify a list of recipient participants using their identities * set a topic * choose if the data should be delivered reliably (slower, with retries) or not (faster) In our effort to remove server identities from user facing APIs, we've removed the need to specify participant SIDs for recipients. In v2, simply use participant identities, which are stable across reconnects. AndroidFlutterGoJavaScriptSwift // v1 room.localParticipant.publishData( data \= msg, destination \= listOf(participantSid) ) // v2 room.localParticipant.publishData( data \= msg, identities \= listOf(Participant.Identity(identity)) ) ### Async room SID[](https://docs.livekit.io/reference/migrate-from-v1/#async-room-sid) In order to speed up the initial connection, the room SID may not be immediately available upon connection. It's instead received later (typically within 300ms). To handle this, getting the room SID is done asynchronously in v2. AndroidFlutterGoJavaScriptSwift // v1 val roomSid \= room.sid // v2 coroutineScope { // room.getSid() is a suspend function val roomSid \= room.getSid() } ### Removed `VideoQuality.OFF` from VideoQuality enum[](https://docs.livekit.io/reference/migrate-from-v1/#removed-videoquality-off-from-videoquality-enum) In v2 we've removed the `OFF` option on the VideoQuality enum. Previously, setting OFF via the setQuality APIs had no effect and was confusing to users. AndroidFlutterGoJavaScriptSwift // v1 import livekit.LivekitModels.VideoQuality // v2 the enum has moved to a different package, with OFF option removed import io.livekit.android.room.track.VideoQuality Platform specific changes[](https://docs.livekit.io/reference/migrate-from-v1/#platform-specific-changes) ---------------------------------------------------------------------------------------------------------- ### Android[](https://docs.livekit.io/reference/migrate-from-v1/#android) #### Removal of previously deprecated APIs[](https://docs.livekit.io/reference/migrate-from-v1/#removal-of-previously-deprecated-apis) * `LiveKit.connect` - Please use `LiveKit.create` and `Room.connect` instead. * `Room.listener` - Please use `Room.events` instead. * `Participant.listener` - Please use `Participant.events` instead. #### Renaming of org.webrtc package to livekit.org.webrtc[](https://docs.livekit.io/reference/migrate-from-v1/#renaming-of-org-webrtc-package-to-livekit-org-webrtc) We've renamed our internal `org.webrtc` package to `livekit.org.webrtc` to prevent conflicts with other WebRTC implementations. If your code references this package, update your import as follows: // v1 import org.webrtc.\* // v2 import livekit.org.webrtc.\* #### Moved composables into a separate package[](https://docs.livekit.io/reference/migrate-from-v1/#moved-composables-into-a-separate-package) Composables, including `VideoRenderer` have been moved into a separate package, `components-android`. Previously the SDK depended on Jetpack Compose, causing View-based apps to depend on an unnecessary package. By moving these components to a separate package, only Compose-based apps will need to depend on it. To migrate, add in your `build.gradle`: dependencies { implementation "io.livekit:livekit-android-compose-components:1.0.0" } The `VideoRenderer` composable has also been renamed to `VideoTrackView` to maintain parity with other platforms. #### Participant.Sid and Identity inline value classes[](https://docs.livekit.io/reference/migrate-from-v1/#participant-sid-and-identity-inline-value-classes) To avoid confusion between participant `sid` and `identity` which shared the `String` type, we've added the `Participant.Sid` and `Participant.Identity` inline value classes. This will prevent inadvertantly using one in place of the other. ### Flutter[](https://docs.livekit.io/reference/migrate-from-v1/#flutter) #### Removal of previously deprecated APIs[](https://docs.livekit.io/reference/migrate-from-v1/#removal-of-previously-deprecated-apis) * `LiveKitClient.connect` - Please use `var room = Room(...)` and `room.connect` instead. * `track` in `TrackMutedEvent/TrackUnmutedEvent` - Use `publication` instead * `TrackStreamStateUpdatedEvent.trackPublication` - Use `TrackStreamStateUpdatedEvent.publication` instead * `RemotePublication.videoQuality` - Use `RemotePublication.setVideoQuality(quality)` instead * `RemotePublication.subscribed` - Use `RemotePublication.subscribe()` or `unsubscribe()` instead * `RemotePublication.enabled` - Use `RemotePublication.enable()` or `disable()` instead * `Participant.unpublishTrack` - Use `Participant.removePublishedTrack` instead * Removed `AudioPublishOptions.stopMicTrackOnMute` ### Javascript/Typescript[](https://docs.livekit.io/reference/migrate-from-v1/#javascript-typescript) #### `webAudioMix` is no longer experimental[](https://docs.livekit.io/reference/migrate-from-v1/#webaudiomix-is-no-longer-experimental) For this release, we're removing the `experimental` notion of the `expWebAudioMix` room option. When using web audio mixing, setting volume directly on the HTMLAudioElements would no longer have any effects. Instead, you can use `setVolume` methods that exist on both `RemoteParticipant` and `RemoteAudioTrack` to control the output volume. #### Removal of previously deprecated APIs[](https://docs.livekit.io/reference/migrate-from-v1/#removal-of-previously-deprecated-apis) * `RoomConnectOptions.publishOnly` - The publishOnly mode has been deprecated even before v1.0, finally removing those bits in the code * `RoomState` - Use `ConnectionState` instead * `RoomEvent.StateChanged` - Use `RoomEvent.ConnectionStateChanged` instead * `TrackPublishOptions.audioBitrate` - Use `TrackPublishOptions.audioPreset` instead * `room.getActiveAudioOutputDevice()` - Use `room.getActiveDevice('audiooutput')` instead ### Swift[](https://docs.livekit.io/reference/migrate-from-v1/#swift) #### Swift concurrency support[](https://docs.livekit.io/reference/migrate-from-v1/#swift-concurrency-support) Swift SDK v2 has migrated to [Swift Concurrency(async/await)](https://developer.apple.com/documentation/swift/updating_an_app_to_use_swift_concurrency) from [Google Promises](https://github.com/google/promises) . #### Renamed APIs[](https://docs.livekit.io/reference/migrate-from-v1/#renamed-apis) * WebRTC types such as `RTCVideoFrame` are now _not exported_ by the SDK, use new types defined by the SDK(`VideoFrame` etc) instead. * `LocalParticipant.publish(track:publishOptions:)` has been renamed to `LocalParticipant.publish(track:options:)`. * `RoomDelegate` and `ParticipantDelegate` signatures have been renamed. Xcode compiler will fail and suggest a rename if any of the previous delegates are used. * Legacy statistics (`TrackStats`) has been repalced with `TrackStatistics`. ### Go[](https://docs.livekit.io/reference/migrate-from-v1/#go) #### CreateRoom -> NewRoom[](https://docs.livekit.io/reference/migrate-from-v1/#createroom-newroom) The `CreateRoom` function has been renamed to `NewRoom` to disambiguate it from the `RoomService.CreateRoom` API in the server SDK. On this page [Breaking changes across SDKs](https://docs.livekit.io/reference/migrate-from-v1/#breaking-changes-across-sdks) [room.participants -> room.remoteParticipants](https://docs.livekit.io/reference/migrate-from-v1/#room-participants-room-remoteparticipants) [track -> trackPublication](https://docs.livekit.io/reference/migrate-from-v1/#track-trackpublication) [Updated publishData API](https://docs.livekit.io/reference/migrate-from-v1/#updated-publishdata-api) [Async room SID](https://docs.livekit.io/reference/migrate-from-v1/#async-room-sid) [Removed VideoQuality.OFF from VideoQuality enum](https://docs.livekit.io/reference/migrate-from-v1/#removed-videoquality-off-from-videoquality-enum) [Platform specific changes](https://docs.livekit.io/reference/migrate-from-v1/#platform-specific-changes) [Android](https://docs.livekit.io/reference/migrate-from-v1/#android) [Flutter](https://docs.livekit.io/reference/migrate-from-v1/#flutter) [Javascript/Typescript](https://docs.livekit.io/reference/migrate-from-v1/#javascript-typescript) [Swift](https://docs.livekit.io/reference/migrate-from-v1/#swift) [Go](https://docs.livekit.io/reference/migrate-from-v1/#go) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/migrate-from-v1/) Search --- # SIP inbound trunk | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/trunk-inbound/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/trunk-inbound/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/sip/trunk-inbound/#overview) [Restricting calls to a region](https://docs.livekit.io/sip/trunk-inbound/#region-pinning) [Inbound trunk example](https://docs.livekit.io/sip/trunk-inbound/#inbound-trunk-example) [Accepting calls from specific phone numbers](https://docs.livekit.io/sip/trunk-inbound/#accepting-calls-from-specific-phone-numbers) [List inbound trunks](https://docs.livekit.io/sip/trunk-inbound/#list-inbound-trunks) [Update inbound trunk](https://docs.livekit.io/sip/trunk-inbound/#update-inbound-trunk) [Update specific fields of an inbound trunk](https://docs.livekit.io/sip/trunk-inbound/#update-specific-fields-of-an-inbound-trunk) [Replace inbound trunk](https://docs.livekit.io/sip/trunk-inbound/#replace-inbound-trunk) Copy pageSee more page options Overview[](https://docs.livekit.io/sip/trunk-inbound/#overview) ---------------------------------------------------------------- After you purchase a phone number and [configure your SIP trunking provider](https://docs.livekit.io/sip/quickstarts/configuring-sip-trunk/) , you must create an inbound trunk and [dispatch rule](https://docs.livekit.io/sip/dispatch-rule/) to accept incoming calls. The inbound trunk allows you to limit incoming calls to those coming from your SIP trunking provider. You can also configure additional properties for all incoming calls that match the trunk including SIP headers, participant metadata and attributes, and session properties. For a full list of available parameters, see [`CreateSIPInboundTrunk`](https://docs.livekit.io/sip/api/#createsipinboundtrunk) . **Note** LiveKit supports username and password authentication for inbound trunks, but your SIP trunking provider must also support it. Support varies by provider—for example, Twilio Elastic SIP Trunking doesn’t support it, though you can use username and password authentication with [TwiML](https://docs.livekit.io/sip/accepting-calls-twilio-voice/) . Check with your provider to confirm. To learn more about LiveKit SIP, see [SIP overview](https://docs.livekit.io/sip/) . To learn more about SIP API endpoints and types, see [SIP API](https://docs.livekit.io/sip/api/) . Restricting calls to a region[](https://docs.livekit.io/sip/trunk-inbound/#region-pinning) ------------------------------------------------------------------------------------------- When you configure your SIP trunking provider for inbound calls, you need to specify the LiveKit SIP endpoint to use. By default, this is a global endpoint and incoming calls are routed to the region closest to the call's origination point—typically the region where your telephony provider initiated the call. You can limit calls to a specific region using [region pinning](https://docs.livekit.io/sip/cloud/) . Inbound trunk example[](https://docs.livekit.io/sip/trunk-inbound/#inbound-trunk-example) ------------------------------------------------------------------------------------------ The following examples create an inbound trunk that accepts calls made to the number `+1-510-555-0100` and enables Krisp [noise cancellation](https://docs.livekit.io/sip/#noise-cancellation-for-calls) . This phone number is the number purchased from your SIP trunking provider. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud 1. Create a file named `inbound-trunk.json` with the following content: { "trunk": { "name": "My trunk", "numbers": \[\ \ "+15105550100"\ \ \], "krispEnabled": true } } **Important** If you're using Telnyx, the leading `+` in the phone number assumes the `Destination Number Format` is set to `+E.164` for your number. 2. Create the inbound trunk using `lk`: lk sip inbound create inbound-trunk.json Accepting calls from specific phone numbers[](https://docs.livekit.io/sip/trunk-inbound/#accepting-calls-from-specific-phone-numbers) -------------------------------------------------------------------------------------------------------------------------------------- The configuration for inbound trunk accepts inbound calls to number `+1-510-555-0100` from caller numbers `+1-310-555-1100` and `+1-714-555-0100`. **Important** Remember to replace the numbers in the example with actual phone numbers when creating your trunks. **Tip** You can also filter allowed caller numbers with a [Dispatch Rule](https://docs.livekit.io/sip/dispatch-rule/) . LiveKit CLINode.jsPythonRubyGoLiveKit Cloud 1. Create a file named `inbound-trunk.json` with the following content: { "trunk": { "name": "My trunk", "numbers": \[\ \ "+15105550100"\ \ \], "allowedNumbers": \[\ \ "+13105550100",\ \ "+17145550100"\ \ \] } } **Important** If you're using Telnyx, the leading `+` in the phone number assumes the `Destination Number Format` is set to `+E.164` for your number. 2. Create the inbound trunk using `lk`: lk sip inbound create inbound-trunk.json List inbound trunks[](https://docs.livekit.io/sip/trunk-inbound/#list-inbound-trunks) -------------------------------------------------------------------------------------- Use the [`ListSIPInboundTrunk`](https://docs.livekit.io/sip/api/#listsipinboundtrunk) API to list all inbound trunks and trunk parameters. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud lk sip inbound list Update inbound trunk[](https://docs.livekit.io/sip/trunk-inbound/#update-inbound-trunk) ---------------------------------------------------------------------------------------- Use the [`UpdateSIPInboundTrunk`](https://docs.livekit.io/sip/api/#updatesipinboundtrunk) API to update specific fields of an inbound trunk or [replace](https://docs.livekit.io/sip/trunk-inbound/#replace-inbound-trunk) an inbound trunk with a new one. ### Update specific fields of an inbound trunk[](https://docs.livekit.io/sip/trunk-inbound/#update-specific-fields-of-an-inbound-trunk) The `UpdateSIPInboundTrunkFields` API allows you to update specific fields of an inbound trunk without affecting other fields. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud 1. Create a file named `inbound-trunk.json` with the following content: { "name": "My trunk", "numbers": \[\ \ "+15105550100"\ \ \] } **Important** If you're using Telnyx, the leading `+` in the phone number assumes the `Destination Number Format` is set to `+E.164` for your number. Update the inbound trunk using `lk`: lk sip inbound update \--id inbound-trunk.json ### Replace inbound trunk[](https://docs.livekit.io/sip/trunk-inbound/#replace-inbound-trunk) The `UpdateSIPInboundTrunk` API allows you to replace an existing inbound trunk with a new one using the same trunk ID. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud The CLI doesn't support replacing inbound trunks. On this page [Overview](https://docs.livekit.io/sip/trunk-inbound/#overview) [Restricting calls to a region](https://docs.livekit.io/sip/trunk-inbound/#region-pinning) [Inbound trunk example](https://docs.livekit.io/sip/trunk-inbound/#inbound-trunk-example) [Accepting calls from specific phone numbers](https://docs.livekit.io/sip/trunk-inbound/#accepting-calls-from-specific-phone-numbers) [List inbound trunks](https://docs.livekit.io/sip/trunk-inbound/#list-inbound-trunks) [Update inbound trunk](https://docs.livekit.io/sip/trunk-inbound/#update-inbound-trunk) [Update specific fields of an inbound trunk](https://docs.livekit.io/sip/trunk-inbound/#update-specific-fields-of-an-inbound-trunk) [Replace inbound trunk](https://docs.livekit.io/sip/trunk-inbound/#replace-inbound-trunk) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/trunk-inbound/) Search --- # SDK Quickstarts | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/quickstarts/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/quickstarts/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) Platform-specific quickstart guides =================================== LiveKit has SDKs for most major platforms and languages. Quickly integrate realtime AI, audio, or video into your app by selecting your platform below. Web SDKs[](https://docs.livekit.io/home/quickstarts/#web) ---------------------------------------------------------- For browser-based applications. [![Next.js](https://docs.livekit.io/images/sdks/nextjs.svg)![Next.js](https://docs.livekit.io/images/sdks/nextjs.svg)\ \ ### Next.js](https://docs.livekit.io/home/quickstarts/nextjs/) [![React](https://docs.livekit.io/images/sdks/react.svg)![React](https://docs.livekit.io/images/sdks/react.svg)\ \ ### React](https://docs.livekit.io/home/quickstarts/react/) [![JavaScript](https://docs.livekit.io/images/sdks/js.svg)![JavaScript](https://docs.livekit.io/images/sdks/js.svg)\ \ ### JavaScript](https://docs.livekit.io/home/quickstarts/javascript/) [![Unity (WebGL)](https://docs.livekit.io/images/sdks/unity.svg)![Unity (WebGL)](https://docs.livekit.io/images/sdks/unity-light.svg)\ \ ### Unity (WebGL)](https://docs.livekit.io/home/quickstarts/unity-web/) Native SDKs[](https://docs.livekit.io/home/quickstarts/#native) ---------------------------------------------------------------- For native applications on mobile, desktop, and more. [![Swift](https://docs.livekit.io/images/sdks/swift.svg)![Swift](https://docs.livekit.io/images/sdks/swift.svg)\ \ ### Swift](https://docs.livekit.io/home/quickstarts/swift/) [![Android (Compose)](https://docs.livekit.io/images/sdks/androidcompose.svg)![Android (Compose)](https://docs.livekit.io/images/sdks/androidcompose.svg)\ \ ### Android (Compose)](https://docs.livekit.io/home/quickstarts/android-compose/) [![Android](https://docs.livekit.io/images/sdks/android.svg)![Android](https://docs.livekit.io/images/sdks/android.svg)\ \ ### Android](https://docs.livekit.io/home/quickstarts/android/) [![Flutter](https://docs.livekit.io/images/sdks/flutter.svg)![Flutter](https://docs.livekit.io/images/sdks/flutter.svg)\ \ ### Flutter](https://docs.livekit.io/home/quickstarts/flutter/) [![React Native](https://docs.livekit.io/images/sdks/react.svg)![React Native](https://docs.livekit.io/images/sdks/react.svg)\ \ ### React Native](https://docs.livekit.io/home/quickstarts/react-native/) [![Expo](https://docs.livekit.io/images/sdks/expo.svg)![Expo](https://docs.livekit.io/images/sdks/expo-light.svg)\ \ ### Expo](https://docs.livekit.io/home/quickstarts/expo/) Other SDKs[](https://docs.livekit.io/home/quickstarts/#other) -------------------------------------------------------------- Don’t see your platform listed? * View the full [list of supported SDKs](https://docs.livekit.io/reference/) . * Integrate with a [telephone system using SIP](https://docs.livekit.io/sip/) . * Join the [LiveKit Slack community](https://livekit.io/join-slack) to share what you’re building. On this page [Web SDKs](https://docs.livekit.io/home/quickstarts/#web) [Native SDKs](https://docs.livekit.io/home/quickstarts/#native) [Other platforms](https://docs.livekit.io/home/quickstarts/#other) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/quickstarts/) Search --- # Agents Overview | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/overview/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/overview/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [What is LiveKit Agents?](https://docs.livekit.io/agents/v0/overview/#what-is-livekit-agents-) [What you can do with agents](https://docs.livekit.io/agents/v0/overview/#what-you-can-do-with-agents) [How agents connect to LiveKit](https://docs.livekit.io/agents/v0/overview/#how-agents-connect-to-livekit) [How to create an agent](https://docs.livekit.io/agents/v0/overview/#how-to-create-an-agent) [Agents framework features](https://docs.livekit.io/agents/v0/overview/#agents-framework-features) Copy pageSee more page options **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [LiveKit Agents 1.0](https://docs.livekit.io/agents/) . _v1.0 for Node.js is coming soon._ What is LiveKit Agents?[](https://docs.livekit.io/agents/v0/overview/#what-is-livekit-agents-) ----------------------------------------------------------------------------------------------- LiveKit Agents is a framework for building programmable, multimodal AI agents that orchestrate LLMs and other AI models to accomplish tasks. This framework allows you to build agents using Python or Node.js. Unlike traditional HTTP servers, agents operate as stateful, long-running processes. They connect to the LiveKit network via WebRTC, enabling low-latency, realtime media and data exchange with frontend applications. ![Diagram showing a high-level view of how agents work.](https://docs.livekit.io/images/agents/agents-overview.svg)![Diagram showing a high-level view of how agents work.](https://docs.livekit.io/images/agents/agents-overview.svg) The Agents framework overcomes several key limitations of traditional architectures: * **Multimodal**: Agents can exchange voice, video, and text with users. * **Simpler frontend**: Frontend applications use LiveKit’s SDKs to handle the complexities of WebRTC transport, media device management, and audio/video encoding and decoding. * **Low-latency**: The [LiveKit Cloud](https://cloud.livekit.io/) global mesh network connects each user to their nearest edge server, minimizing transport latency. * **Centralized business logic**: Keeping business logic within the agent process allows it to support clients across platforms, including telephony integrations. * **Stateful**: End-user interactions are inherently stateful. Rather than synchronizing client-side state through request/response cycles, agents provide a more intuitive way to manage these interactions. What you can do with agents[](https://docs.livekit.io/agents/v0/overview/#what-you-can-do-with-agents) ------------------------------------------------------------------------------------------------------- The LiveKit Agents framework is designed to give you flexibility when building server side, programmable participants. You can create multiple frontends that all connect to the same backend agent. Some great use cases for agents include: * **[AI voice agents](https://docs.livekit.io/agents/v0/voice-agent/) **: An agent that has natural voice conversations with users. * **Call center**: Answer incoming calls, or make outbound calls with AI agents. * **Transcription**: Realtime voice-to-text transcription. * **Object detection/recognition**: Identify objects over realtime video. * **AI-driven avatars**: Generated avatars using prompts. * **Translation**: Realtime translation. * **Video manipulation**: Realtime video filters and transforms. How agents connect to LiveKit[](https://docs.livekit.io/agents/v0/overview/#how-agents-connect-to-livekit) ----------------------------------------------------------------------------------------------------------- ![Diagram showing a high-level view of how agents work.](https://docs.livekit.io/images/agents/agents-jobs-overview.svg)![Diagram showing a high-level view of how agents work.](https://docs.livekit.io/images/agents/agents-jobs-overview.svg) When you start running your agent code, it registers itself with a LiveKit server (either [self hosted](https://docs.livekit.io/home/self-hosting/deployment/) or [LiveKit Cloud](https://cloud.livekit.io/) and runs as a background "worker" process. The worker waits on standby for users to connect. Once a end-user session is initiated (that is, a room is created for the user), an available worker dispatches an agent to the room. Users connect to a LiveKit room using a frontend application. Each user is a [participant](https://docs.livekit.io/home/get-started/api-primitives/#participant) in the room and the agent is an AI participant. How the agent interacts with end-user participants depends on the custom code you write. How to create an agent[](https://docs.livekit.io/agents/v0/overview/#how-to-create-an-agent) --------------------------------------------------------------------------------------------- To create an agent using the framework, you’ll need to write a Python or Node.js application (your agent) and a frontend for your users: * Write the application code for your agent. The configuration, functions, and plugin options are all part of your agent code. You can use plugins included in the framework for LLM, STT, TTS, VAD, and utilities for working with text, or write your own custom plugins. Define the entrypoint function that executes when a connection is made. You can also define optional functions to preprocess connections and set connection thresholds or permissions for the worker process. To learn more, see [Integrations](https://docs.livekit.io/agents/v0/integrations/overview/) and [Worker options](https://docs.livekit.io/agents/v0/build/anatomy/#worker-options) . * Create a frontend for users to connect to your agent in a LiveKit room. For development and testing, you can use the [Agents Playground](https://docs.livekit.io/agents/v0/playground/) . Agents framework features[](https://docs.livekit.io/agents/v0/overview/#agents-framework-features) --------------------------------------------------------------------------------------------------- * **LiveKit audio/video transport**: Use the same [LiveKit API primitives](https://docs.livekit.io/home/concepts/api-primitives/) to transport voice and video from your frontend to your application server in realtime. * **Abstractions over common tasks**: Tasks such as speech-to-text, text-to-speech, and using LLMs are simplified so you can focus on your core application logic. * **Extensive and extensible plugins**: Prebuilt integrations with OpenAI, Deepgram, Google, ElevenLabs, and more. You can create a plugin to integrate any other provider. * **End-to-end dev experience**: Compatible with [LiveKit server](https://github.com/livekit/livekit) and [LiveKit Cloud](https://cloud.livekit.io/) . Develop locally and deploy to production without changing a single line of code. * **Orchestration and scaling**: Built-in worker service for agent orchestration and load balancing. To scale, just add more servers. * **Open Source**: Like the rest of LiveKit, the Agents framework is Apache 2.0. * **Edge optimized**: When using LiveKit Cloud, your agents transmit voice and video over LiveKit's global edge network, ensuring minimal latency for users worldwide. On this page [What is LiveKit Agents?](https://docs.livekit.io/agents/v0/overview/#what-is-livekit-agents-) [What you can do with agents](https://docs.livekit.io/agents/v0/overview/#what-you-can-do-with-agents) [How agents connect to LiveKit](https://docs.livekit.io/agents/v0/overview/#how-agents-connect-to-livekit) [How to create an agent](https://docs.livekit.io/agents/v0/overview/#how-to-create-an-agent) [Agents framework features](https://docs.livekit.io/agents/v0/overview/#agents-framework-features) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/overview/) Search --- # LiveKit integration guides | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/integrations/overview/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/integrations/overview/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Realtime API](https://docs.livekit.io/agents/v0/integrations/overview/#realtime-api-integrations) [LLM integrations](https://docs.livekit.io/agents/v0/integrations/overview/#llm-integrations) [STT integrations](https://docs.livekit.io/agents/v0/integrations/overview/#stt-integrations) [TTS integrations](https://docs.livekit.io/agents/v0/integrations/overview/#tts-integrations) **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [Integration guides](https://docs.livekit.io/agents/integrations/) . _v1.0 for Node.js is coming soon._ The Agents framework supports integrations for providers using plugins. This topic describes LLM, STT, and TTS provider plugins. Additional plugins include support for Retrieval-Augmented Generation (RAG), Natural Language Toolkit (NLTK), LlamaIndex, Silero VAD, turn detection, and more. To see the list of additional plugins and learn more about how LiveKit plugins work, see [Working with plugins](https://docs.livekit.io/agents/v0/integrations/plugins/) . If you want to use a provider not listed in the following sections, contributions for plugins are always welcome. To learn more, see the the guidelines for contributions to the [Python repository](https://github.com/livekit/agents/blob/0.x/CONTRIBUTING.md) or the [Node.js repository](https://github.com/livekit/agents-js/blob/main/CONTRIBUTING.md) . Realtime API integrations[](https://docs.livekit.io/agents/v0/integrations/overview/#realtime-api-integrations) ---------------------------------------------------------------------------------------------------------------- Realtime APIs are designed for ulta-low-latency AI responses and use multimodal models. They can be better at understanding a user and their emotions resulting in more natural interactions. [![OpenAI logo](https://docs.livekit.io/images/icons/icon-logo-open-ai.svg)\ \ OpenAI Realtime API\ -------------------\ \ Support for the OpenAI Realtime API.](https://docs.livekit.io/agents/v0/integrations/openai/realtime/) [Gemini Live API\ ---------------\ \ Support for Google's Gemini Live API.](https://docs.livekit.io/agents/v0/integrations/google/#gemini-live-api) [Azure OpenAI Realtime API\ -------------------------\ \ OpenAI Realtime API hosted on Azure.](https://docs.livekit.io/agents/v0/integrations/azure/#azure-realtime-api) LLM integrations[](https://docs.livekit.io/agents/v0/integrations/overview/#llm-integrations) ---------------------------------------------------------------------------------------------- You can create an instance of an LLM to use in a `VoicePipelineAgent` for the following providers using a plugin: | | | | | --- | --- | --- | | [Anthropic](https://docs.livekit.io/reference/python/livekit/plugins/anthropic/index.html#livekit.plugins.anthropic.LLM) | [Azure](https://docs.livekit.io/agents/v0/integrations/azure/#azure-openai-ll) | [Amazon Web Services (AWS)](https://docs.livekit.io/reference/python/livekit/plugins/aws/llm.html) | | [Cerebras](https://docs.livekit.io/agents/v0/integrations/cerebras/#llm) | [Deepseek](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | [Fireworks](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | | [Gemini](https://docs.livekit.io/agents/v0/integrations/google/#gemini-llm) | [Groq](https://docs.livekit.io/agents/v0/integrations/groq/#llm) | [LlamaIndex](https://pypi.org/project/livekit-plugins-llama-index/) | | [Octo](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | [Ollama](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | [Openai](https://docs.livekit.io/agents/v0/integrations/openai/overview/#llm) | | [Perplexity](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | [Telnyx](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | [Together](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | | [xAI](https://docs.livekit.io/agents/v0/integrations/openai-compatible-llms/#supported%20llms) | | | STT integrations[](https://docs.livekit.io/agents/v0/integrations/overview/#stt-integrations) ---------------------------------------------------------------------------------------------- LiveKit plugins support the following STT providers. Create an instance of an STT to use in a \`VoicePipelineAgent\` or as a standalone transcription service. | Provider | Supported frameworks | | --- | --- | | AssemblyAI | [Python](https://docs.livekit.io/reference/python/livekit/plugins/assemblyai/index.html#livekit.plugins.assemblyai.STT) | | [Azure](https://docs.livekit.io/agents/v0/integrations/azure/#azure-speech-stt) | Python, Node.js | | AWS | [Python](https://docs.livekit.io/reference/python/livekit/plugins/aws/stt.html) | | Clova | [Python](https://github.com/livekit/agents/tree/0.x/livekit-plugins/livekit-plugins-clova) | | [Deepgram](https://docs.livekit.io/agents/v0/integrations/deepgram/#stt) | Python, Node.js | | fal | [Python](https://docs.livekit.io/reference/python/livekit/plugins/fal/stt.html) | | [Google](https://docs.livekit.io/agents/v0/integrations/google/#google-cloud-stt-and-tts) | Python | | [Groq](https://docs.livekit.io/agents/v0/integrations/groq/#stt) | Python, Node.js | | [OpenAI](https://docs.livekit.io/agents/v0/integrations/openai/overview/#stt) | Python, Node.js | | [Speechmatics](https://docs.livekit.io/agents/v0/integrations/speechmatics/) | Python | TTS integrations[](https://docs.livekit.io/agents/v0/integrations/overview/#tts-integrations) ---------------------------------------------------------------------------------------------- LiveKit plugins support the following TTS providers. Create an instance of a TTS to use in a \`VoicePipelineAgent\` or for speech generation in your apps. | Provider | Supported frameworks | | --- | --- | | [Azure](https://docs.livekit.io/agents/v0/integrations/azure/#azure-speech-tts) | Python | | AWS | [Python](https://docs.livekit.io/reference/python/livekit/plugins/aws/tts.html) | | [Cartesia](https://docs.livekit.io/agents/v0/integrations/cartesia/) | Python, Node.js | | [Deepgram](https://docs.livekit.io/agents/v0/integrations/deepgram/#tts) | Python, Node.js | | [Elevenlabs](https://docs.livekit.io/agents/v0/integrations/elevenlabs/) | Python, Node.js | | Neuphonic | [Python](https://docs.livekit.io/reference/python/livekit/plugins/neuphonic/index.html#livekit.plugins.neuphonic.TTS) | | [OpenAI](https://docs.livekit.io/agents/v0/integrations/openai/overview/#tts) | Python, Node.js | | [PlayHT](https://docs.livekit.io/agents/v0/integrations/playai/) | Python | | [Rime](https://docs.livekit.io/agents/v0/integrations/rime/) | Python | On this page [Realtime API](https://docs.livekit.io/agents/v0/integrations/overview/#realtime-api-integrations) [LLM integrations](https://docs.livekit.io/agents/v0/integrations/overview/#llm-integrations) [STT integrations](https://docs.livekit.io/agents/v0/integrations/overview/#stt-integrations) [TTS integrations](https://docs.livekit.io/agents/v0/integrations/overview/#tts-integrations) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/integrations/overview/) Search --- # Making outbound calls | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/making-calls/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/making-calls/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Outbound call workflow](https://docs.livekit.io/sip/making-calls/#outbound-call-workflow) [Setup for making calls](https://docs.livekit.io/sip/making-calls/#setup-for-making-calls) [SIP trunking provider setup](https://docs.livekit.io/sip/making-calls/#sip-trunking-provider-setup) [LiveKit SIP configuration](https://docs.livekit.io/sip/making-calls/#livekit-sip-configuration) [Make an outbound call](https://docs.livekit.io/sip/making-calls/#make-an-outbound-call) [Next steps](https://docs.livekit.io/sip/making-calls/#next-steps) Copy pageSee more page options Outbound call workflow[](https://docs.livekit.io/sip/making-calls/#outbound-call-workflow) ------------------------------------------------------------------------------------------- To make an outbound call, you create a [SIP participant](https://docs.livekit.io/sip/sip-participant/) with the user's phone number. When you execute the [`CreateSIPParticipant`](https://docs.livekit.io/sip/api/#createsipparticipant) request, LiveKit SIP sends an INVITE request to your SIP provider. If the SIP provider accepts the call, the SIP participant is added to the LiveKit room. ![LiveKit outbound SIP workflow](https://docs.livekit.io/images/sip/outbound-sip-workflow.svg)![LiveKit outbound SIP workflow](https://docs.livekit.io/images/sip/outbound-sip-workflow.svg) 1. Call the `CreateSIPParticipant` API to create a SIP participant. 2. LiveKit SIP sends an INVITE request to the SIP trunking provider. 3. SIP trunking provider validates trunk credentials and accepts the call. 4. LiveKit server places SIP participant in the LiveKit room specified in the `CreateSIPParticipant` request. Setup for making calls[](https://docs.livekit.io/sip/making-calls/#setup-for-making-calls) ------------------------------------------------------------------------------------------- The following sections outline the steps required to make an outbound SIP call. ### SIP trunking provider setup[](https://docs.livekit.io/sip/making-calls/#sip-trunking-provider-setup) 1. Purchase a phone number from a SIP Provider. For a list of tested providers, see the table in [Using LiveKit SIP](https://docs.livekit.io/sip/#using-livekit-sip) . 2. Configure the SIP Trunk on the provider to send SIP traffic to accept SIP traffic from the LiveKit SIP service. For instructions for setting up a SIP trunk, see [Configuring a SIP provider trunk](https://docs.livekit.io/sip/quickstarts/configuring-sip-trunk/) . ### LiveKit SIP configuration[](https://docs.livekit.io/sip/making-calls/#livekit-sip-configuration) Create an [outbound trunk](https://docs.livekit.io/sip/trunk-outbound/) associated with your SIP provider phone number. This is the number that is used to dial out to the user. Include the authentication credentials required by your SIP trunking provider to make calls. ### Make an outbound call[](https://docs.livekit.io/sip/making-calls/#make-an-outbound-call) Create a SIP participant. When the `CreateSIPParticipant` request is executed, a SIP call is initiated: 1. An INVITE request is sent to the SIP trunk provider. The provider checks authentication credentials and returns a response to LiveKit. 2. If the call is accepted, LiveKit dials the user and creates a SIP participant in the LiveKit room. If the call is not accepted by the SIP trunk provider, the `CreateSIPParticipant` request fails. After the call starts ringing, you can check the call status by listening to [participant events](https://docs.livekit.io/home/client/events/#events) : * If the `sip.callStatus` participant attribute is updated to `active`, the call has connected. * If the call fails, the participant is disconnected and leaves the room. Next steps[](https://docs.livekit.io/sip/making-calls/#next-steps) ------------------------------------------------------------------- See the following guide to create an AI agent that makes outbound calls. [Voice AI telephony guide\ ------------------------\ \ Create an AI agent to make outbound calls.](https://docs.livekit.io/agents/start/telephony/) On this page [Outbound call workflow](https://docs.livekit.io/sip/making-calls/#outbound-call-workflow) [Setup for making calls](https://docs.livekit.io/sip/making-calls/#setup-for-making-calls) [SIP trunking provider setup](https://docs.livekit.io/sip/making-calls/#sip-trunking-provider-setup) [LiveKit SIP configuration](https://docs.livekit.io/sip/making-calls/#livekit-sip-configuration) [Make an outbound call](https://docs.livekit.io/sip/making-calls/#make-an-outbound-call) [Next steps](https://docs.livekit.io/sip/making-calls/#next-steps) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/making-calls/) Search --- # Recording and composition | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/egress/overview/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/overview/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Introduction](https://docs.livekit.io/home/egress/overview/#introduction) [Egress types](https://docs.livekit.io/home/egress/overview/#egress-types) [Room composite egress](https://docs.livekit.io/home/egress/overview/#room-composite-egress) [Web egress](https://docs.livekit.io/home/egress/overview/#web-egress) [Participant egress](https://docs.livekit.io/home/egress/overview/#participant-egress) [Track composite egress](https://docs.livekit.io/home/egress/overview/#track-composite-egress) [Track egress](https://docs.livekit.io/home/egress/overview/#track-egress) [Service architecture](https://docs.livekit.io/home/egress/overview/#service-architecture) Copy pageSee more page options Introduction[](https://docs.livekit.io/home/egress/overview/#introduction) --------------------------------------------------------------------------- LiveKit Egress gives you a powerful and consistent set of APIs to export any room or individual tracks from a LiveKit session. It supports recording to a MP4 file or HLS segments, as well as exporting to livestreaming services like YouTube Live, Twitch, and Facebook via RTMP(s). For LiveKit Cloud customers, Egress is ready to use with your project without additional configuration. When self-hosting LiveKit, Egress is a separate component that needs to be [deployed](https://docs.livekit.io/home/self-hosting/egress/) . Egress types[](https://docs.livekit.io/home/egress/overview/#egress-types) --------------------------------------------------------------------------- ### Room composite egress[](https://docs.livekit.io/home/egress/overview/#room-composite-egress) Export an entire room's video and/or audio using a web layout rendered by Chrome. Room composites are tied to a room's lifecycle, and will stop automatically when the room ends. Composition templates are customizable web pages that can be hosted anywhere. Example use case: recording a meeting for team members to watch later. ### Web egress[](https://docs.livekit.io/home/egress/overview/#web-egress) Similar to Room Composite, but allows you to record and export any web page. Web Egress are not tied to LiveKit rooms, and can be used to record non-LiveKit content. Example use case: restreaming content from a third-party source to YouTube and Twitch. ### Participant egress[](https://docs.livekit.io/home/egress/overview/#participant-egress) Export a participant's video and audio together. This is a newer API and is designed to be easier to use than Track Composite Egress. Example use case: record the teacher's video in an online class. ### Track composite egress[](https://docs.livekit.io/home/egress/overview/#track-composite-egress) Sync and export up to one audio and one video track. Will transcode and mux. Example use case: exporting audio+video from many cameras at once during a production, for use in additional post-production. ### Track egress[](https://docs.livekit.io/home/egress/overview/#track-egress) Export individual tracks directly. Video tracks are not transcoded. Example use case: streaming an audio track to a captioning service via websocket. Service architecture[](https://docs.livekit.io/home/egress/overview/#service-architecture) ------------------------------------------------------------------------------------------- Depending on your request type, the egress service will either launch a web template in Chrome and connect to the room (room composite requests), or it will use the sdk directly (track and track composite requests). It uses GStreamer to encode, and can output to a file or to one or more streams. ![Egress instance](https://docs.livekit.io/images/diagrams/egress-instance.svg)![Egress instance](https://docs.livekit.io/images/diagrams/egress-instance-light.svg) On this page [Introduction](https://docs.livekit.io/home/egress/overview/#introduction) [Egress types](https://docs.livekit.io/home/egress/overview/#egress-types) [Room composite egress](https://docs.livekit.io/home/egress/overview/#room-composite-egress) [Web egress](https://docs.livekit.io/home/egress/overview/#web-egress) [Participant egress](https://docs.livekit.io/home/egress/overview/#participant-egress) [Track composite egress](https://docs.livekit.io/home/egress/overview/#track-composite-egress) [Track egress](https://docs.livekit.io/home/egress/overview/#track-egress) [Service architecture](https://docs.livekit.io/home/egress/overview/#service-architecture) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/overview/) Search --- # Intro to LiveKit | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/get-started/intro-to-livekit/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/get-started/intro-to-livekit/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Why choose LiveKit?](https://docs.livekit.io/home/get-started/intro-to-livekit/#why-choose-livekit-) [What is WebRTC?](https://docs.livekit.io/home/get-started/intro-to-livekit/#what-is-webrtc) [LiveKit ecosystem](https://docs.livekit.io/home/get-started/intro-to-livekit/#livekit-ecosystem) [Deployment options](https://docs.livekit.io/home/get-started/intro-to-livekit/#deployment-options) [What can you build with LiveKit?](https://docs.livekit.io/home/get-started/intro-to-livekit/#what-can-you-build-with-livekit-) Copy pageSee more page options LiveKit is an open source platform for developers building realtime media applications. It makes it easy to integrate audio, video, text, data, and AI models while offering scalable realtime infrastructure built on top of WebRTC. Why choose LiveKit?[](https://docs.livekit.io/home/get-started/intro-to-livekit/#why-choose-livekit-) ------------------------------------------------------------------------------------------------------ LiveKit provides a complete solution for realtime applications with several key advantages: * **Developer-friendly**: Consistent APIs across platforms with comprehensive and well-documented SDKs. * **Open source**: No vendor lock-in with complete transparency and flexibility. * **AI-native**: First-class support for integrating AI models into realtime experiences. * **Scalable**: Can support anywhere from a handful of users to thousands of concurrent participants, or more. * **Deployment flexibility**: Choose between fully-managed cloud or self-hosted options. * **Private and secure**: End-to-end encryption, HIPAA-compliance, and more. * **Built on WebRTC**: The most robust realtime media protocol for peak performance in any network condition. ### What is WebRTC?[](https://docs.livekit.io/home/get-started/intro-to-livekit/#what-is-webrtc) [WebRTC](https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API) provides significant advantages over other options for building realtime applications such as websockets. * **Optimized for media**: Purpose-built for audio and video with advanced codecs and compression algorithms. * **Network resilient**: Performs reliably even in challenging network conditions due to UDP, adaptive bitrate, and more. * **Broad compatibility**: Natively supported in all modern browsers. LiveKit handles all of the complexity of running production-grade WebRTC infrastructure while extending support to mobile apps, backends, and telephony. LiveKit ecosystem[](https://docs.livekit.io/home/get-started/intro-to-livekit/#livekit-ecosystem) -------------------------------------------------------------------------------------------------- The LiveKit platform consists of these core components: * **LiveKit Server**: An open-source media server that enables realtime communication between participants. Use LiveKit's fully-managed global cloud, or self-host your own. * **LiveKit SDKs**: Full-featured web, native, and backend SDKs that make it easy to join rooms and publish and consume realtime media and data. * **LiveKit Agents**: A framework for building realtime multimodal AI agents, with an extensive collection of plugins for nearly every AI provider. * **Telephony**: A flexible SIP integration for inbound or outbound calling into any LiveKit room or agent session. * **Egress**: Record and export realtime media from LiveKit rooms. * **Ingress**: Ingest external streams (such as RTMP and WHIP) into LiveKit rooms. * **Server APIs**: A REST API for managing rooms, and more. Includes SDKs and a CLI. Deployment options[](https://docs.livekit.io/home/get-started/intro-to-livekit/#deployment-options) ---------------------------------------------------------------------------------------------------- LiveKit offers two deployment options for LiveKit Server to fit your needs: * **LiveKit Cloud**: A fully-managed, globally distributed service with automatic scaling and high reliability. Trusted by companies of all sizes, from startups to enterprises. * **Self-hosted**: Run the open source LiveKit server on your own infrastructure for maximum control and customization. Both options provide the same core platform features and use the same SDKs. What can you build with LiveKit?[](https://docs.livekit.io/home/get-started/intro-to-livekit/#what-can-you-build-with-livekit-) -------------------------------------------------------------------------------------------------------------------------------- * **AI assistants**: Voice and video agents powered by any AI model. * **Video conferencing**: Secure, private meetings for teams of any size. * **Interactive livestreaming**: Broadcast to audiences with realtime engagement. * **Robotics**: Integrate realtime video and powerful AI models into real-world devices. * **Healthcare**: HIPAA-compliant telehealth with AI and humans in the loop. * **Customer service**: Flexible and observable web, mobile, and telephone support options. Whatever your use case, LiveKit makes it easy to build innovative, intelligent realtime applications without worrying about scaling media infrastructure. [Get started with LiveKit today](https://docs.livekit.io/home/) . On this page [Why choose LiveKit?](https://docs.livekit.io/home/get-started/intro-to-livekit/#why-choose-livekit-) [What is WebRTC?](https://docs.livekit.io/home/get-started/intro-to-livekit/#what-is-webrtc) [LiveKit ecosystem](https://docs.livekit.io/home/get-started/intro-to-livekit/#livekit-ecosystem) [Deployment options](https://docs.livekit.io/home/get-started/intro-to-livekit/#deployment-options) [What can you build with LiveKit?](https://docs.livekit.io/home/get-started/intro-to-livekit/#what-can-you-build-with-livekit-) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/get-started/intro-to-livekit/) Search --- # Rooms, participants, and tracks | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/get-started/api-primitives/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/get-started/api-primitives/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/home/get-started/api-primitives/#overview) [Room](https://docs.livekit.io/home/get-started/api-primitives/#room) [Participant](https://docs.livekit.io/home/get-started/api-primitives/#participant) [Participant fields](https://docs.livekit.io/home/get-started/api-primitives/#participant-fields) [Types of participants](https://docs.livekit.io/home/get-started/api-primitives/#types-of-participants) [Track](https://docs.livekit.io/home/get-started/api-primitives/#track) [TrackPublication fields](https://docs.livekit.io/home/get-started/api-primitives/#trackpublication-fields) [Track subscription](https://docs.livekit.io/home/get-started/api-primitives/#track-subscription) Copy pageSee more page options Overview[](https://docs.livekit.io/home/get-started/api-primitives/#overview) ------------------------------------------------------------------------------ LiveKit has only three core constructs: a room, participant, and track. A room is simply a realtime session between one or more participants. A participant can publish one or more tracks and/or subscribe to one or more tracks from another participant. Room[](https://docs.livekit.io/home/get-started/api-primitives/#room) ---------------------------------------------------------------------- A `Room` is a container object representing a LiveKit session. Each participant in a room receives updates about changes to other participants in the same room. For example, when a participant adds, removes, or modifies the state (for example, mute) of a track, other participants are notified of this change. This is a powerful mechanism for synchronizing state and fundamental to building any realtime experience. A room can be created manually via [server API](https://docs.livekit.io/home/server/managing-rooms/#create-a-room) , or automatically, when the first participant joins it. Once the last participant leaves a room, it closes after a short delay. Participant[](https://docs.livekit.io/home/get-started/api-primitives/#participant) ------------------------------------------------------------------------------------ A `Participant` is a user or process that is participating in a realtime session. They are represented by a unique developer-provided `identity` and a server-generated `sid`. A participant object also contains metadata about its state and tracks they've published. **Important** A participant's identity is unique per room. Thus, if participants with the same identity join a room, only the most recent one to join will remain; the server automatically disconnects other participants using that identity. There are two kinds of participant objects in the SDKs: * A `LocalParticipant` represents the current user who, by default, can publish tracks in a room. * A `RemoteParticipant` represents a remote user. The local participant, by default, can subscribe to any tracks published by a remote participant. A participant may also [exchange data](https://docs.livekit.io/home/client/data/) with one or many other participants. ### Participant fields[](https://docs.livekit.io/home/get-started/api-primitives/#participant-fields) | Field | Type | Description | | --- | --- | --- | | sid | string | A UID for this particular participant, generated by LiveKit server. | | identity | string | Unique identity of the participant, as specified when connecting. | | name | string | Optional display name. | | state | ParticipantInfo.State | JOINING, JOINED, ACTIVE, or DISCONNECTED. | | kind | ParticipantInfo.Kind | The type of participant; more below. | | attributes | string | User-specified [attributes](https://docs.livekit.io/home/client/data/)
for the participant. | | permission | ParticipantInfo.Permission | Permissions granted to the participant. | ### Types of participants[](https://docs.livekit.io/home/get-started/api-primitives/#types-of-participants) In a realtime session, a participant could represent an end-user, as well as a server-side process. It's possible to distinguish between them with the `kind` field: * `STANDARD`: A regular participant, typically an end-user in your application. * `AGENT`: An agent spawned with the [Agents framework](https://docs.livekit.io/agents/) . * `SIP`: A telephony user connected via [SIP](https://docs.livekit.io/sip/) . * `EGRESS`: A server-side process that is recording the session using [LiveKit Egress](https://docs.livekit.io/home/egress/overview/) . * `INGRESS`: A server-side process that is ingesting media into the session using [LiveKit Ingress](https://docs.livekit.io/home/ingress/overview/) . Track[](https://docs.livekit.io/home/get-started/api-primitives/#track) ------------------------------------------------------------------------ A `Track` represents a stream of information, be it audio, video or custom data. By default, a participant in a room may publish tracks, such as their camera or microphone streams and subscribe to one or more tracks published by other participants. In order to model a track which may not be subscribed to by the local participant, all track objects have a corresponding `TrackPublication` object: * `Track`: a wrapper around the native WebRTC `MediaStreamTrack`, representing a playable track. * `TrackPublication`: a track that's been published to the server. If the track is subscribed to by the local participant and available for playback locally, it will have a `.track` attribute representing the associated `Track` object. We can now list and manipulate tracks (via track publications) published by other participants, even if the local participant is not subscribed to them. ### TrackPublication fields[](https://docs.livekit.io/home/get-started/api-primitives/#trackpublication-fields) A `TrackPublication` contains information about its associated track: | Field | Type | Description | | --- | --- | --- | | sid | string | A UID for this particular track, generated by LiveKit server. | | kind | Track.Kind | The type of track, whether it be audio, video or arbitrary data. | | source | Track.Source | Source of media: Camera, Microphone, ScreenShare, or ScreenShareAudio. | | name | string | The name given to this particular track when initially published. | | subscribed | boolean | Indicates whether or not this track has been subscribed to by the local participant. | | track | Track | If the local participant is subscribed, the associated `Track` object representing a WebRTC track. | | muted | boolean | Whether this track is muted or not by the local participant. While muted, it won't receive new bytes from the server. | ### Track subscription[](https://docs.livekit.io/home/get-started/api-primitives/#track-subscription) When a participant is subscribed to a track (which hasn't been muted by the publishing participant), they continuously receive its data. If the participant unsubscribes, they stop receiving media for that track and may resubscribe to it at any time. When a participant creates or joins a room, the `autoSubscribe` option is set to `true` by default. This means the participant automatically subscribes to all existing tracks being published and any track published in the future. For more fine-grained control over track subscriptions, you can set `autoSubscribe` to `false` and instead use [selective subscriptions](https://docs.livekit.io/home/client/receive/#selective-subscription) . **Note** For most use cases, muting a track on the publisher side or unsubscribing from it on the subscriber side is typically recommended over unpublishing it. Publishing a track requires a negotiation phase and consequently has worse time-to-first-byte performance. On this page [Overview](https://docs.livekit.io/home/get-started/api-primitives/#overview) [Room](https://docs.livekit.io/home/get-started/api-primitives/#room) [Participant](https://docs.livekit.io/home/get-started/api-primitives/#participant) [Participant fields](https://docs.livekit.io/home/get-started/api-primitives/#participant-fields) [Types of participants](https://docs.livekit.io/home/get-started/api-primitives/#types-of-participants) [Track](https://docs.livekit.io/home/get-started/api-primitives/#track) [TrackPublication fields](https://docs.livekit.io/home/get-started/api-primitives/#trackpublication-fields) [Track subscription](https://docs.livekit.io/home/get-started/api-primitives/#track-subscription) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/get-started/api-primitives/) Search --- # Inbound calls with Twilio Voice | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/accepting-calls-twilio-voice/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Inbound calls with Twilio programmable voice](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#inbound-calls-with-twilio-programmable-voice) [Step 1. Purchase a phone number from Twilio](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-1-purchase-a-phone-number-from-twilio) [Step 2. Set up a TwiML Bin](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-2-set-up-a-twiml-bin) [Step 3. Direct phone number to the TwiML Bin](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-3-direct-phone-number-to-the-twiml-bin) [Step 4. Create a LiveKit inbound trunk](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-4-create-a-livekit-inbound-trunk) [Step 5. Create a dispatch rule to place each caller into their own room.](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-5-create-a-dispatch-rule-to-place-each-caller-into-their-own-room-) [Testing with an agent](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#testing-with-an-agent) [Connecting to a Twilio phone conference](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#connecting-to-a-twilio-phone-conference) [Step 1. Set Twilio environment variables](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-1-set-twilio-environment-variables) [Step 2. Bridge a Twilio conference and LiveKit SIP](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-2-bridge-a-twilio-conference-and-livekit-sip) [Step 3. Execute the file](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-3-execute-the-file) Copy pageSee more page options Inbound calls with Twilio programmable voice[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#inbound-calls-with-twilio-programmable-voice) ------------------------------------------------------------------------------------------------------------------------------------------------------- Accept inbound calls using Twilio programmable voice. All you need is an inbound trunk and a dispatch rule created using the LiveKit CLI (or SDK) to accept calls and route callers to LiveKit rooms. The following steps guide you through the process. **Note** This method doesn't support [SIP REFER](https://docs.livekit.io/sip/transfer-cold/) . To set up Elastic SIP Trunking, see the [Configuring Twilio SIP trunks](https://docs.livekit.io/sip/quickstarts/configuring-twilio-trunk/) quickstart. ### Step 1. Purchase a phone number from Twilio[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-1-purchase-a-phone-number-from-twilio) If you don't already have a phone number, see [How to Search for and Buy a Twilio Phone Number From Console](https://help.twilio.com/articles/223135247-How-to-Search-for-and-Buy-a-Twilio-Phone-Number-from-Console) . ### Step 2. Set up a TwiML Bin[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-2-set-up-a-twiml-bin) TwiML Bins are a simple way to test TwiML responses. Use a TwiML Bin to redirect an inbound call to LiveKit. To create a TwiML Bin, follow these steps: 1. Navigate to your [TwiML Bins](https://console.twilio.com/us1/develop/twiml-bins/twiml-bins?frameUrl=/console/twiml-bins) page. 2. Create a TwiML Bin and add the following contents: sip:@ ### Step 3. Direct phone number to the TwiML Bin[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-3-direct-phone-number-to-the-twiml-bin) Configure incoming calls to a specific phone number to use the TwiML Bin you just created: 1. Navigate to the [Manage numbers](https://console.twilio.com/us1/develop/phone-numbers/manage/incoming) page and select the purchased phone number. 2. In the **Voice Configuration** section, edit the **A call comes in** fields. After you select **TwiML Bin**. select the TwiML Bin created in the previous step. ### Step 4. Create a LiveKit inbound trunk[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-4-create-a-livekit-inbound-trunk) Use the LiveKit CLI to create an [inbound trunk](https://docs.livekit.io/sip/trunk-inbound/) for the purchased phone number. 1. Create an `inbound-trunk.json` file with the following contents. Replace the phone number and add a `username` and `password` of your choosing: { "trunk": { "name": "My inbound trunk", "auth\_username": "", "auth\_password": "" } } **Note** Be sure to use the same username and password that's specified in the TwiML Bin. 2. Use the CLI to create an inbound trunk: lk sip inbound create inbound-trunk.json ### Step 5. Create a dispatch rule to place each caller into their own room.[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-5-create-a-dispatch-rule-to-place-each-caller-into-their-own-room-) Use the LiveKit CLI to create a [dispatch rule](https://docs.livekit.io/sip/dispatch-rule/) that places each caller into individual rooms named with the prefix `call`. 1. Create a `dispatch-rule.json` file with the following contents: { "dispatch\_rule": { "rule": { "dispatchRuleIndividual": { "roomPrefix": "call-" } } } } 2. Create the dispatch rule using the CLI: lk sip dispatch create dispatch-rule.json ### Testing with an agent[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#testing-with-an-agent) Follow the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai/) to create an agent that responds to incoming calls. Then call the phone number and your agent should pick up the call. Connecting to a Twilio phone conference[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#connecting-to-a-twilio-phone-conference) --------------------------------------------------------------------------------------------------------------------------------------------- You can bridge Twilio conferencing to LiveKit via SIP, allowing you to add agents and other LiveKit clients to an existing Twilio conference. This requires the following setup: * [Twilio conferencing](https://www.twilio.com/docs/voice/conference) . * LiveKit [inbound trunk](https://docs.livekit.io/sip/trunk-inbound/) . * LiveKit [voice AI agent](https://docs.livekit.io/agents/start/voice-ai/) . The example in this section uses [Node](https://nodejs.org/) and the [Twilio Node SDK](https://www.twilio.com/docs/libraries) . ### Step 1. Set Twilio environment variables[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-1-set-twilio-environment-variables) You can find these values in your [Twilio Console](https://console.twilio.com/) : export TWILIO\_ACCOUNT\_SID\= export TWILIO\_AUTH\_TOKEN\= ### Step 2. Bridge a Twilio conference and LiveKit SIP[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-2-bridge-a-twilio-conference-and-livekit-sip) Create a `bridge.js` file and update the `twilioPhoneNumber`, `conferenceSid`, `sipHost`, and `from` field for the API call in the following code: **Note** If you're signed in to [LiveKit Cloud](https://cloud.livekit.io/) , your sip host is filled in below. import twilio from 'twilio'; const accountSid \= process.env.TWILIO\_ACCOUNT\_SID; const authToken \= process.env.TWILIO\_AUTH\_TOKEN; const twilioClient \= twilio(accountSid, authToken); /\*\* \* Phone number bought from Twilio that is associated with a LiveKit trunk. \* For example, +14155550100. \* See https://docs.livekit.io/sip/quickstarts/configuring-twilio-trunk/ \*/ const twilioPhoneNumber \= ''; /\*\* \* SIP host is available in your LiveKit Cloud project settings. \* This is your project domain without the leading "sip:". \*/ const sipHost \= ''; /\*\* \* The conference SID from Twilio that you want to add the agent to. You \* likely want to obtain this from your conference status callback webhook handler. \* The from field must contain the phone number, client identifier, or username \* portion of the SIP address that made this call. \* See https://www.twilio.com/docs/voice/api/conference-participant-resource#request-body-parameters \*/ const conferenceSid \= ''; await twilioClient.conferences(conferenceSid).participants.create({ from: '', to: \`sip:${twilioPhoneNumber}@${sipHost}\`, }); ### Step 3. Execute the file[](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-3-execute-the-file) When you run the file, it bridges the Twilio conference to a new LiveKit session using the previously configured dispatch rule. This allows you to automatically [dispatch an agent](https://docs.livekit.io/agents/worker/agent-dispatch/) to the Twilio conference. node bridge.js On this page [Inbound calls with Twilio programmable voice](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#inbound-calls-with-twilio-programmable-voice) [Step 1. Purchase a phone number from Twilio](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-1-purchase-a-phone-number-from-twilio) [Step 2. Set up a TwiML Bin](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-2-set-up-a-twiml-bin) [Step 3. Direct phone number to the TwiML Bin](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-3-direct-phone-number-to-the-twiml-bin) [Step 4. Create a LiveKit inbound trunk](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-4-create-a-livekit-inbound-trunk) [Step 5. Create a dispatch rule to place each caller into their own room.](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-5-create-a-dispatch-rule-to-place-each-caller-into-their-own-room-) [Testing with an agent](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#testing-with-an-agent) [Connecting to a Twilio phone conference](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#connecting-to-a-twilio-phone-conference) [Step 1. Set Twilio environment variables](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-1-set-twilio-environment-variables) [Step 2. Bridge a Twilio conference and LiveKit SIP](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-2-bridge-a-twilio-conference-and-livekit-sip) [Step 3. Execute the file](https://docs.livekit.io/sip/accepting-calls-twilio-voice/#step-3-execute-the-file) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/accepting-calls-twilio-voice/) Search --- # OpenAI integration guide | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/integrations/openai/overview/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/integrations/openai/overview/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/v0/integrations/openai/overview/#overview) [Quick reference](https://docs.livekit.io/agents/v0/integrations/openai/overview/#quick-reference) [LLM](https://docs.livekit.io/agents/v0/integrations/openai/overview/#llm) [STT](https://docs.livekit.io/agents/v0/integrations/openai/overview/#stt) [TTS](https://docs.livekit.io/agents/v0/integrations/openai/overview/#tts) Copy pageSee more page options **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [OpenAI integration](https://docs.livekit.io/agents/integrations/openai/) . _v1.0 for Node.js is coming soon._ Overview[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#overview) ------------------------------------------------------------------------------------- LiveKit's OpenAI integration provides support for LLM, STT, TTS, and the OpenAI Realtime API. The [Quick reference](https://docs.livekit.io/agents/v0/integrations/openai/overview/#quick-reference) section in this topic describes using OpenAI for LLM, STT, and TTS. For a guide on using the Realtime API, see the [OpenAI Realtime API integration guide](https://docs.livekit.io/agents/v0/integrations/openai/realtime/) . Use OpenAI STT, TTS, and LLM to create agents using the [VoicePipelineAgent](https://docs.livekit.io/agents/v0/voice-agent/voice-pipeline/) class. To use the Realtime API, you can create an agent using the [MultimodalAgent](https://docs.livekit.io/agents/v0/voice-agent/multimodal-agent/) class. **Note** The following quickstart guides are available to get you started creating an AI voice assistant with OpenAI: * [Voice agent quickstart](https://docs.livekit.io/agents/v0/quickstarts/voice-agent/) using the `VoicePipelineAgent` class. * [Speech-to-speech quickstart](https://docs.livekit.io/agents/v0/quickstarts/s2s/) using the `MultimodalAgent` class. Quick reference[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#quick-reference) --------------------------------------------------------------------------------------------------- The following sections provide a quick reference for integrating OpenAI with LiveKit. For the complete reference, see the links provided in each section. ### LLM[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#llm) LiveKit's OpenAI plugin provides support for creating an instance of an LLM class to be used in a [`VoicePipelineAgent`](https://docs.livekit.io/agents/v0/voice-agent/voice-pipeline/) . #### LLM class usage[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#llm-class-usage) Create an instance of OpenAI LLM: agent.py .env.local PythonNode.js from livekit.plugins.openai import llm openai\_llm \= llm.LLM( model\="gpt-4o", temperature\=0.8, ) #### LLM parameters[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#llm-parameters) This section describes some of the available parameters. For a complete reference of all available parameters, see the [plugin reference](https://docs.livekit.io/reference/python/livekit/plugins/openai/index.html#livekit.plugins.openai.LLM) . **model**`string | ChatModels``Optional`Default: `gpt-4o` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#model) ID of the model to use for inference. For a list of supported models, see `ChatModels` in the respective GitHub repository: [Python](https://github.com/livekit/agents/blob/b283301b6f7a29fe6208ac2c81b626ed0be813a0/livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/models.py#L7C1-L31C2) , [Node.js](https://github.com/livekit/agents-js/blob/e18d8777e4a19f32b50fffa9e3b61f5e5351631b/plugins/openai/src/models.ts#L5C1-L28C30) . To learn more about available models, see [Models](https://platform.openai.com/docs/models) . **api\_key**`string``Optional`Env: `OPENAI_API_KEY` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#api_key) OpenAI API key. Required if the environment variable is not set. **temperature**`float``Optional`Default: `1.0` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#temperature) A measure of randomness of completions. A lower temperature is more deterministic. To learn more, see [chat completions](https://console.groq.com/docs/api-reference#chat-create) . ### STT[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#stt) LiveKit's OpenAI plugin allows you to create an instance of OpenAI STT that can be used as the first stage in a [`VoicePipelineAgent`](https://docs.livekit.io/agents/v0/voice-agent/voice-pipeline/) or as a standalone transcription service. #### STT usage[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#stt-usage) Create an OpenAI STT: agent.py .env.local PythonNode.js from livekit.plugins.openai import stt openai\_stt \= stt.STT( language\="en", model\="whisper-1", ) #### STT parameters[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#stt-parameters) This section describes some of the available parameters. For a complete reference of all available parameters, see the [plugin reference](https://docs.livekit.io/reference/python/livekit/plugins/openai/index.html#livekit.plugins.openai.STT) . **model**`WhisperModels | string``Optional`Default: `whisper-1` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#model) ID of the model to use for speech recognition. To learn more, see [Whisper](https://platform.openai.com/docs/models#whisper) . **language**`string``Optional`Default: `en` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#language) Language of input audio in [ISO-639-1](https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes) format. See OpenAI's documentation for a list of [supported languages](https://platform.openai.com/docs/guides/speech-to-text#supported-languages) . ### TTS[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#tts) LiveKit's OpenAI plugin allows you to create an instance of OpenAI TTS that can be used in a [`VoicePipelineAgent`](https://docs.livekit.io/agents/v0/voice-agent/voice-pipeline/) or as a standalone speech generator. #### TTS usage[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#tts-usage) Create an OpenAI TTS: agent.py .env.local PythonNode.js from livekit.plugins.openai import tts openai\_tts \= tts.TTS( model\="tts-1", voice\="nova", ) #### TTS parameters[](https://docs.livekit.io/agents/v0/integrations/openai/overview/#tts-parameters) This section describes some of the available parameters. For a complete reference of all available parameters, see the [plugin reference](https://docs.livekit.io/reference/python/livekit/plugins/openai/index.html#livekit.plugins.openai.TTS) . **model**`TTSModels | string``Optional`Default: `tts-1` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#model) ID of the model to use for speech generation. To learn more, see [TTS models](https://platform.openai.com/docs/models#tts) . **voice**`TTSVoice | string``Optional`Default: `alloy` [#](https://docs.livekit.io/agents/v0/integrations/openai/overview/#voice) ID of the voice used for speech generation. To learn more, see [TTS voice options](https://platform.openai.com/docs/guides/text-to-speech#voice-options) . On this page [Overview](https://docs.livekit.io/agents/v0/integrations/openai/overview/#overview) [Quick reference](https://docs.livekit.io/agents/v0/integrations/openai/overview/#quick-reference) [LLM](https://docs.livekit.io/agents/v0/integrations/openai/overview/#llm) [STT](https://docs.livekit.io/agents/v0/integrations/openai/overview/#stt) [TTS](https://docs.livekit.io/agents/v0/integrations/openai/overview/#tts) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/integrations/openai/overview/) Search --- # SIP dispatch rule | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/dispatch-rule/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/dispatch-rule/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Caller dispatch rule (individual)](https://docs.livekit.io/sip/dispatch-rule/#caller-dispatch-rule-individual-) [Direct dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#direct-dispatch-rule) [Pin-protected room](https://docs.livekit.io/sip/dispatch-rule/#pin-protected-room) [Callee dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#callee-dispatch-rule) [Setting custom attributes on inbound SIP participants](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-attributes-on-inbound-sip-participants) [Setting custom metadata on inbound SIP participants](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-metadata-on-inbound-sip-participants) [Update dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#update-dispatch-rule) [Update specific fields of a dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#update-specific-fields-of-a-dispatch-rule) [Replace dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#replace-dispatch-rule) [List dispatch rules](https://docs.livekit.io/sip/dispatch-rule/#list-dispatch-rules) Copy pageSee more page options To create a dispatch rule with the SIP service, use the `CreateSIPDispatchRule` API. It returns a `SIPDispatchRuleInfo` object that describes the created `SIPDispatchRule`. By default, a dispatch rule is matched against all your trunks and makes a caller's phone number visible to others in the room. You can change these default behaviors using dispatch rule options. See the [`CreateSIPDispatchRule`](https://docs.livekit.io/sip/api/#createsipdispatchrule) API reference for full list of available options. To learn more about SIP and dispatch rules, see [SIP overview](https://docs.livekit.io/sip/) . To learn more about SIP API endpoints and types, see [SIP API](https://docs.livekit.io/sip/api/) . Caller dispatch rule (individual)[](https://docs.livekit.io/sip/dispatch-rule/#caller-dispatch-rule-individual-) ----------------------------------------------------------------------------------------------------------------- An `SIPDispatchRuleIndividual` rule creates a new room for each caller. The name of the created room is the phone number of the caller plus a random suffix. You can optionally add a specific prefix to the room name by using the `roomPrefix` option. The following examples dispatch callers into individual rooms prefixed with `call-`, and [dispatches an agent](https://docs.livekit.io/agents/worker/agent-dispatch/) named `inbound-agent` to newly created rooms: LiveKit CLINode.jsPythonRubyGoLiveKit Cloud { "dispatch\_rule": { "rule": { "dispatchRuleIndividual": { "roomPrefix": "call-" } }, "name": "My dispatch rule", "roomConfig": { "agents": \[{\ \ "agentName": "inbound-agent",\ \ "metadata": "job dispatch metadata"\ \ }\] } } } **Note** When you omit the `trunk_ids` field, the dispatch rule matches calls from all inbound trunks. Direct dispatch rule[](https://docs.livekit.io/sip/dispatch-rule/#direct-dispatch-rule) ---------------------------------------------------------------------------------------- A direct dispatch rule places all callers into a specified room. You can optionally protect room access by adding a pin in the `pin` field: In the following examples, all calls are immediately connected to room `open-room` on LiveKit. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud 1. Create a file named `dispatch-rule.json` and add the following: { "dispatch\_rule": { "rule": { "dispatchRuleDirect": { "roomName": "open-room" } }, "name": "My dispatch rule" } } 2. Create the dispatch rule using `lk`: lk sip dispatch create dispatch-rule.json ### Pin-protected room[](https://docs.livekit.io/sip/dispatch-rule/#pin-protected-room) Add a `pin` to a room to require callers to enter a pin to connect to a room in LiveKit. The following example requires callers to enter `12345#` on the phone to enter `safe-room`: { "dispatch\_rule": { "trunk\_ids": \[\], "rule": { "dispatchRuleDirect": { "roomName": "safe-room", "pin": "12345" } }, "name": "My dispatch rule" } } Callee dispatch rule[](https://docs.livekit.io/sip/dispatch-rule/#callee-dispatch-rule) ---------------------------------------------------------------------------------------- This creates a dispatch rule that puts callers into rooms based on the called number. The name of the room is the called phone number plus an optional prefix (if `roomPrefix` is set). You can optionally add a random suffix for each caller by setting `randomize` to true, making a separate room per caller. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud { "dispatch\_rule": { "rule": { "dispatchRuleCallee": { "roomPrefix": "number-", "randomize": false } }, "name": "My dispatch rule" } } Setting custom attributes on inbound SIP participants[](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-attributes-on-inbound-sip-participants) ---------------------------------------------------------------------------------------------------------------------------------------------------------- LiveKit participants have an `attributes` field that stores key-value pairs. You can add custom attributes for SIP participants in the dispatch rule. These attributes are inherited by all SIP participants created by the dispatch rule. To learn more, see [SIP participant attributes](https://docs.livekit.io/sip/sip-participant/#sip-participant-attributes) . The following examples add two attributes to SIP participants created by this dispatch rule: LiveKit CLINode.jsPythonRubyGoLiveKit Cloud { "dispatch\_rule": { "attributes": { "": "", "": "" }, "rule": { "dispatchRuleIndividual": { "roomPrefix": "call-" } }, "name": "My dispatch rule" } } Setting custom metadata on inbound SIP participants[](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-metadata-on-inbound-sip-participants) ------------------------------------------------------------------------------------------------------------------------------------------------------ LiveKit participants have a `metadata` field that can store arbitrary data for your application (typically JSON). It can also be set on SIP participants created by a dispatch rule. Specifically, `metadata` set on a dispatch rule will be inherited by all SIP participants created by it. The following examples add the metadata, `{"is_internal": true}`, to all SIP participants created from an inbound call by this dispatch rule: LiveKit CLINode.jsPythonRubyGoLiveKit Cloud { "dispatch\_rule": { "metadata": "{\\"is\_internal\\": true}", "rule": { "dispatchRuleIndividual": { "roomPrefix": "call-" } }, "name": "My dispatch rule" } } Update dispatch rule[](https://docs.livekit.io/sip/dispatch-rule/#update-dispatch-rule) ---------------------------------------------------------------------------------------- Use the [`UpdateSIPDispatchRule`](https://docs.livekit.io/sip/api/#updatesipdispatchrule) API to update specific fields of a dispatch rule or [replace](https://docs.livekit.io/sip/dispatch-rule/#replace-dispatch-rule) a dispatch rule with a new one. ### Update specific fields of a dispatch rule[](https://docs.livekit.io/sip/dispatch-rule/#update-specific-fields-of-a-dispatch-rule) The `UpdateSIPDispatchRuleFields` API allows you to update specific fields of a dispatch rule without affecting other fields. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud Create a file named `dispatch-rule.json` with the following content: { "name": "My updated dispatch rule", "rule": { "dispatchRuleCallee": { "roomPrefix": "number-", "randomize": false, "pin": "1234" } } } Update the dispatch rule using `lk`. You can update the `trunks` parameter to a comma-separated string of trunks IDs if the rule matches specific trunks. lk sip dispatch update \--id \\ \--trunks "\[\]" \\ dispatch-rule.json ### Replace dispatch rule[](https://docs.livekit.io/sip/dispatch-rule/#replace-dispatch-rule) The `UpdateSIPDispatchRule` API allows you to replace an existing dispatch rule with a new one using the same dispatch rule ID. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud The instructions for replacing a dispatch rule are the same as for [updating a dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#update-specific-fields-of-a-dispatch-rule) . List dispatch rules[](https://docs.livekit.io/sip/dispatch-rule/#list-dispatch-rules) -------------------------------------------------------------------------------------- Use the [`ListSIPDispatchRule`](https://docs.livekit.io/sip/api/#listsipdispatchrule) API to list all dispatch rules. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud lk sip dispatch list On this page [Caller dispatch rule (individual)](https://docs.livekit.io/sip/dispatch-rule/#caller-dispatch-rule-individual-) [Direct dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#direct-dispatch-rule) [Pin-protected room](https://docs.livekit.io/sip/dispatch-rule/#pin-protected-room) [Callee dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#callee-dispatch-rule) [Setting custom attributes on inbound SIP participants](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-attributes-on-inbound-sip-participants) [Setting custom metadata on inbound SIP participants](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-metadata-on-inbound-sip-participants) [Update dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#update-dispatch-rule) [Update specific fields of a dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#update-specific-fields-of-a-dispatch-rule) [Replace dispatch rule](https://docs.livekit.io/sip/dispatch-rule/#replace-dispatch-rule) [List dispatch rules](https://docs.livekit.io/sip/dispatch-rule/#list-dispatch-rules) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/dispatch-rule/) Search --- # Authentication | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/get-started/authentication/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/get-started/authentication/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/home/get-started/authentication/#overview) [Creating a token](https://docs.livekit.io/home/get-started/authentication/#creating-a-token) [Token example](https://docs.livekit.io/home/get-started/authentication/#token-example) [Video grant](https://docs.livekit.io/home/get-started/authentication/#video-grant) [Example: subscribe-only token](https://docs.livekit.io/home/get-started/authentication/#example-subscribe-only-token) [Example: camera-only](https://docs.livekit.io/home/get-started/authentication/#example-camera-only) [SIP grant](https://docs.livekit.io/home/get-started/authentication/#sip-grant) [Creating a token with SIP grants](https://docs.livekit.io/home/get-started/authentication/#creating-a-token-with-sip-grants) [Room configuration](https://docs.livekit.io/home/get-started/authentication/#room-configuration) [Creating a token with room configuration](https://docs.livekit.io/home/get-started/authentication/#creating-a-token-with-room-configuration) [Token refresh](https://docs.livekit.io/home/get-started/authentication/#token-refresh) [Updating permissions](https://docs.livekit.io/home/get-started/authentication/#updating-permissions) Copy pageSee more page options Overview[](https://docs.livekit.io/home/get-started/authentication/#overview) ------------------------------------------------------------------------------ For a LiveKit SDK to successfully connect to the server, it must pass an access token with the request. This token encodes the identity of a participant, name of the room, capabilities and permissions. Access tokens are JWT-based and signed with your API secret to prevent forgery. Access tokens also carry an expiration time, after which the server will reject connections with that token. Note: expiration time only impacts the initial connection, and not subsequent reconnects. Creating a token[](https://docs.livekit.io/home/get-started/authentication/#creating-a-token) ---------------------------------------------------------------------------------------------- LiveKit CLINode.jsGoRubyJavaPythonRustOther lk token create \\ --api-key \\ --api-secret \\ \--identity \\ \--room \\ \--join \\ --valid-for 1h ### Token example[](https://docs.livekit.io/home/get-started/authentication/#token-example) Here's an example of the decoded body of a join token: { "exp": 1621657263, "iss": "APIMmxiL8rquKztZEoZJV9Fb", "sub": "myidentity", "nbf": 1619065263, "video": { "room": "myroom", "roomJoin": true }, "metadata": "" } | field | description | | --- | --- | | exp | Expiration time of token | | nbf | Start time that the token becomes valid | | iss | API key used to issue this token | | sub | Unique identity for the participant | | metadata | Participant metadata | | attributes | Participant attributes (key/value pairs of strings) | | video | Video grant, including room permissions (see below) | | sip | SIP grant | Video grant[](https://docs.livekit.io/home/get-started/authentication/#video-grant) ------------------------------------------------------------------------------------ Room permissions are specified in the `video` field of a decoded join token. It may contain one or more of the following properties: | field | type | description | | --- | --- | --- | | roomCreate | bool | Permission to create or delete rooms | | roomList | bool | Permission to list available rooms | | roomJoin | bool | Permission to join a room | | roomAdmin | bool | Permission to moderate a room | | roomRecord | bool | Permissions to use Egress service | | ingressAdmin | bool | Permissions to use Ingress service | | room | string | Name of the room, required if join or admin is set | | canPublish | bool | Allow participant to publish tracks | | canPublishData | bool | Allow participant to publish data to the room | | canPublishSources | string\[\] | Requires `canPublish` to be true. When set, only listed source can be published. (camera, microphone, screen\_share, screen\_share\_audio) | | canSubscribe | bool | Allow participant to subscribe to tracks | | canUpdateOwnMetadata | bool | Allow participant to update its own metadata | | hidden | bool | Hide participant from others in the room | | kind | string | Type of participant (standard, ingress, egress, sip, or agent). this field is typically set by LiveKit internals. | ### Example: subscribe-only token[](https://docs.livekit.io/home/get-started/authentication/#example-subscribe-only-token) To create a token where the participant can only subscribe, and not publish into the room, you would use the following grant: { ... "video": { "room": "myroom", "roomJoin": true, "canSubscribe": true, "canPublish": false, "canPublishData": false } } ### Example: camera-only[](https://docs.livekit.io/home/get-started/authentication/#example-camera-only) Allow the participant to publish camera, but disallow other sources { ... "video": { "room": "myroom", "roomJoin": true, "canSubscribe": true, "canPublish": true, "canPublishSources": \["camera"\] } } SIP grant[](https://docs.livekit.io/home/get-started/authentication/#sip-grant) -------------------------------------------------------------------------------- In order to interact with the SIP service, permission must be granted in the `sip` field of the JWT. It may contain the following properties: | field | type | description | | --- | --- | --- | | admin | bool | Permission to manage SIP trunks and dispatch rules. | | call | bool | Permission to make SIP calls via `CreateSIPParticipant`. | ### Creating a token with SIP grants[](https://docs.livekit.io/home/get-started/authentication/#creating-a-token-with-sip-grants) Node.jsGoRubyJavaPythonRust import { AccessToken, SIPGrant, VideoGrant } from 'livekit-server-sdk'; const roomName \= 'name-of-room'; const participantName \= 'user-name'; const at \= new AccessToken('api-key', 'secret-key', { identity: participantName, }); const sipGrant: SIPGrant \= { admin: true, call: true, }; const videoGrant: VideoGrant \= { room: roomName, roomJoin: true, }; at.addGrant(sipGrant); at.addGrant(videoGrant); const token \= await at.toJwt(); console.log('access token', token); Room configuration[](https://docs.livekit.io/home/get-started/authentication/#room-configuration) -------------------------------------------------------------------------------------------------- You can create an access token for a user that includes room configuration options. When a room is created for a user, the room is created using the configuration stored in the token. For example, you can use this to [explicitly dispatch an agent](https://docs.livekit.io/agents/worker/agent-dispatch/) when a user joins a room. For the full list of `RoomConfiguration` fields, see [RoomConfiguration](https://docs.livekit.io/reference/server/server-apis/#roomconfiguration) . ### Creating a token with room configuration[](https://docs.livekit.io/home/get-started/authentication/#creating-a-token-with-room-configuration) Node.jsGoRubyPythonRust For a full example of explicit agent dispatch, see the [example](https://github.com/livekit/node-sdks/blob/main/examples/agent-dispatch/index.ts) in GitHub. import { AccessToken, SIPGrant, VideoGrant } from 'livekit-server-sdk'; import { RoomAgentDispatch, RoomConfiguration } from '@livekit/protocol'; const roomName \= 'name-of-room'; const participantName \= 'user-name'; const agentName \= 'my-agent'; const at \= new AccessToken('api-key', 'secret-key', { identity: participantName, }); const videoGrant: VideoGrant \= { room: roomName, roomJoin: true, }; at.addGrant(videoGrant); at.roomConfig \= new RoomConfiguration ( agents: \[\ \ new RoomAgentDispatch({\ \ agentName: "test-agent",\ \ metadata: "test-metadata"\ \ })\ \ \] ); const token \= await at.toJwt(); console.log('access token', token); Token refresh[](https://docs.livekit.io/home/get-started/authentication/#token-refresh) ---------------------------------------------------------------------------------------- LiveKit server proactively issues refreshed tokens to connected clients, ensuring they can reconnect if disconnected. These refreshed access tokens have a 10-minute expiration. Additionally, tokens are refreshed when there are changes to a participant's name, permissions or metadata. Updating permissions[](https://docs.livekit.io/home/get-started/authentication/#updating-permissions) ------------------------------------------------------------------------------------------------------ A participant's permissions can be updated at any time, even after they've already connected. This is useful in applications where the participant's role could change during the session, such as in a participatory livestream. It's possible to issue a token with `canPublish: false` initially, and then updating it to `canPublish: true` during the session. Permissions can be changed with the [UpdateParticipant](https://docs.livekit.io/home/server/managing-participants/#updating-permissions) server API. On this page [Overview](https://docs.livekit.io/home/get-started/authentication/#overview) [Creating a token](https://docs.livekit.io/home/get-started/authentication/#creating-a-token) [Token example](https://docs.livekit.io/home/get-started/authentication/#token-example) [Video grant](https://docs.livekit.io/home/get-started/authentication/#video-grant) [Example: subscribe-only token](https://docs.livekit.io/home/get-started/authentication/#example-subscribe-only-token) [Example: camera-only](https://docs.livekit.io/home/get-started/authentication/#example-camera-only) [SIP grant](https://docs.livekit.io/home/get-started/authentication/#sip-grant) [Creating a token with SIP grants](https://docs.livekit.io/home/get-started/authentication/#creating-a-token-with-sip-grants) [Room configuration](https://docs.livekit.io/home/get-started/authentication/#room-configuration) [Creating a token with room configuration](https://docs.livekit.io/home/get-started/authentication/#creating-a-token-with-room-configuration) [Token refresh](https://docs.livekit.io/home/get-started/authentication/#token-refresh) [Updating permissions](https://docs.livekit.io/home/get-started/authentication/#updating-permissions) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/get-started/authentication/) Search --- # Ingress overview | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/ingress/overview/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/ingress/overview/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Introduction](https://docs.livekit.io/home/ingress/overview/#introduction) [Supported Sources](https://docs.livekit.io/home/ingress/overview/#supported-sources) [Workflow](https://docs.livekit.io/home/ingress/overview/#workflow) [WHIP / RTMP](https://docs.livekit.io/home/ingress/overview/#whip-rtmp) [URL Input](https://docs.livekit.io/home/ingress/overview/#url-input) [API](https://docs.livekit.io/home/ingress/overview/#api) [CreateIngress](https://docs.livekit.io/home/ingress/overview/#createingress) [ListIngress](https://docs.livekit.io/home/ingress/overview/#listingress) [UpdateIngress](https://docs.livekit.io/home/ingress/overview/#updateingress) [DeleteIngress](https://docs.livekit.io/home/ingress/overview/#deleteingress) [Using video presets](https://docs.livekit.io/home/ingress/overview/#using-video-presets) [Custom settings](https://docs.livekit.io/home/ingress/overview/#custom-settings) [Enabling transcoding for WHIP sessions](https://docs.livekit.io/home/ingress/overview/#enabling-transcoding-for-whip-sessions) [Service architecture](https://docs.livekit.io/home/ingress/overview/#service-architecture) Copy pageSee more page options Introduction[](https://docs.livekit.io/home/ingress/overview/#introduction) ---------------------------------------------------------------------------- LiveKit Ingress lets you import video from another source into a LiveKit room. While WebRTC is a versatile and scalable transport protocol for both media ingestion and delivery, some applications require integrating with existing workflows or equipment that do not support WebRTC. Perhaps your users want to publish video from OBS Studio or a dedicated hardware device, or maybe they want to stream the content of media file hosted on a HTTP server to a room. LiveKit Ingress makes these integrations easy. LiveKit Ingress can automatically transcode the source media to ensure compatibility with LiveKit clients. It can publish multiple layers with [Simulcast](https://blog.livekit.io/an-introduction-to-webrtc-simulcast-6c5f1f6402eb/) . The parameters of the different video layers can be defined at ingress creation time. Presets are provided to make encoding settings configuration easy. The optional ability to provide custom encoding parameters enables more specialized use cases. For LiveKit Cloud customers, Ingress is ready to use with your project without additional configuration. When self-hosting LiveKit, Ingress is deployed as a separate service. Supported Sources[](https://docs.livekit.io/home/ingress/overview/#supported-sources) -------------------------------------------------------------------------------------- * RTMP/RTMPS * WHIP * Media files fetched from any HTTP server. The following media formats are supported: * HTTP Live Streaming (HLS) * ISO MPEG-4 (MP4) * Apple Quicktime (MOV) * Matroska (MKV/WEBM) * OGG audio * MP3 audio * M4A audio * Media served by a SRT server Workflow[](https://docs.livekit.io/home/ingress/overview/#workflow) -------------------------------------------------------------------- ### WHIP / RTMP[](https://docs.livekit.io/home/ingress/overview/#whip-rtmp) A typical push Ingress goes like this: 1. Your app creates an Ingress with `CreateIngress` API, which returns a URL and stream key of the Ingress 2. Your user copies and pastes the URL and key into your streaming workflow 3. Your user starts their stream 4. The Ingress Service starts transcoding their stream, or forwards media unchanged if transcoding is disabled. 5. The Ingress Service joins the LiveKit room and publishes the media for other Participants 6. When the stream source disconnects from the Ingress service, the Ingress Service participant leaves the room. 7. The Ingress remains valid, in a disconnected state, allowing it to be reused with the same stream key ### URL Input[](https://docs.livekit.io/home/ingress/overview/#url-input) When pulling media from a HTTP or SRT server, Ingress has a slightly different lifecycle: it will start immediately after calling CreateIngress. 1. Your app creates an Ingress with `CreateIngress` API 2. The Ingress Service starts fetching the file or media and transcoding it 3. The Ingress Service joins the LiveKit room and publishes the transcoded media for other Participants 4. When the media is completely consumed, or if `DeleteIngress` is called, the Ingress Service participant leaves the room. API[](https://docs.livekit.io/home/ingress/overview/#api) ---------------------------------------------------------- ### CreateIngress[](https://docs.livekit.io/home/ingress/overview/#createingress) #### WHIP / RTMP example[](https://docs.livekit.io/home/ingress/overview/#whip-rtmp-example) To provision an Ingress with the Ingress Service, use the CreateIngress API. It returns an `IngressInfo` object that describes the created Ingress, along with connection settings. These parameters can also be queried at any time using the `ListIngress` API LiveKit CLIJavaScriptGoRuby Create a file at `ingress.json` with the following content: { "input\_type": 0 for RTMP, 1 for WHIP "name": "Name of the Ingress goes here", "room\_name": "Name of the room to connect to", "participant\_identity": "Unique identity for the room participant the Ingress service will connect as", "participant\_name": "Name displayed in the room for the participant", "enable\_transcoding": true // Transcode the input stream. Can only be false for WHIP, } Then create the Ingress using `lk`: export LIVEKIT\_URL\=https://my-livekit-host export LIVEKIT\_API\_KEY\=livekit-api-key export LIVEKIT\_API\_SECRET\=livekit-api-secret lk ingress create ingress.json #### URL Input example[](https://docs.livekit.io/home/ingress/overview/#url-input-example) With URL Input, Ingress will begin immediately after `CreateIngress` is called. URL\_INPUT Ingress cannot be re-used. LiveKit CLIJavaScriptGoRuby Create a file at `ingress.json` with the following content: { "input\_type": "URL\_INPUT", // or 2 "name": "Name of the Ingress goes here", "room\_name": "Name of the room to connect to", "participant\_identity": "Unique identity for the room participant the Ingress service will connect as", "participant\_name": "Name displayed in the room for the participant", "url": "HTTP(S) or SRT url to the file or stream" } Then create the Ingress using `lk`: export LIVEKIT\_URL\=https://my-livekit-host export LIVEKIT\_API\_KEY\=livekit-api-key export LIVEKIT\_API\_SECRET\=livekit-api-secret lk ingress create ingress.json ### ListIngress[](https://docs.livekit.io/home/ingress/overview/#listingress) LiveKit CLIJavaScriptGoRuby lk ingress list The optional `--room` option allows to restrict the output to the Ingress associated to a given room. The `--id` option can check if a specific ingress is active. ### UpdateIngress[](https://docs.livekit.io/home/ingress/overview/#updateingress) The Ingress configuration can be updated using the `UpdateIngress` API. This enables the ability to re-use the same Ingress URL to publish to different rooms. Only reusable Ingresses, such as RTMP or WHIP, can be updated. LiveKit CLIJavaScriptGoRuby Create a file at `ingress.json` with the fields to be updated. { "ingress\_id": "Ingress ID of the Ingress to update", "name": "Name of the Ingress goes here", "room\_name": "Name of the room to connect to", "participant\_identity": "Unique identity for the room participant the Ingress service will connect as", "participant\_name": "Name displayed in the room for the participant" } The only required field is `ingress_id`. Non provided fields are left unchanged. lk ingress update ingress.json ### DeleteIngress[](https://docs.livekit.io/home/ingress/overview/#deleteingress) An Ingress can be reused multiple times. When not needed anymore, it can be deleted using the `DeleteIngress` API: LiveKit CLIJavaScriptGoRuby lk ingress delete Using video presets[](https://docs.livekit.io/home/ingress/overview/#using-video-presets) ------------------------------------------------------------------------------------------ The Ingress service can transcode the media being received. This is the only supported behavior for RTMP and URL inputs. WHIP ingresses are not transcoded by default, but transcoding can be enabled by setting the `enable_transcoding` parameter. When transcoding is enabled, The default settings enable [video simulcast](https://blog.livekit.io/an-introduction-to-webrtc-simulcast-6c5f1f6402eb/) to ensure media can be consumed by all viewers, and should be suitable for most use cases. In some situations however, adjusting these settings may be desirable to match source content or the viewer conditions better. For this purpose, LiveKit Ingress defines several presets, both for audio and video. Presets define both the characteristics of the media (codec, dimesions, framerate, channel count, sample rate) and the bitrate. For video, a single preset defines the full set of simulcast layers. A preset can be chosen at Ingress creation time from the [constants in the Ingress protocol definition](https://github.com/livekit/protocol/blob/main/protobufs/livekit_ingress.proto) : LiveKit CLIJavaScriptGoRuby Create a file at `ingress.json` with the following content: { "name": "Name of the egress goes here", "room\_name": "Name of the room to connect to", "participant\_identity": "Unique identity for the room participant the Ingress service will connect as", "participant\_name": "Name displayed in the room for the participant" "video": { "name": "track name", "source": "SCREEN\_SHARE", "preset": "Video preset enum value" }, "audio": { "name": "track name", "source": "SCREEN\_SHARE\_AUDIO", "preset": "Audio preset enum value" } } Then create the Ingress using `lk`: lk ingress create ingress.json Custom settings[](https://docs.livekit.io/home/ingress/overview/#custom-settings) ---------------------------------------------------------------------------------- For specialized use cases, it is also possible to specify fully custom encoding parameters. In this case, all video layers need to be defined if simulcast is desired. LiveKit CLIJavaScriptGoRuby Create a file at `ingress.json` with the following content: { "name": "Name of the egress goes here", "room\_name": "Name of the room to connect to", "participant\_identity": "Unique identity for the room participant the Ingress service will connect as", "participant\_name": "Name displayed in the room for the participant", "video": { "options": { "video\_codec": "video codec ID from the \[VideoCodec enum\](https://github.com/livekit/protocol/blob/main/protobufs/livekit\_models.proto)", "frame\_rate": "desired framerate in frame per second", "layers": \[\ \ {\ \ "quality": "ID for one of the LOW, MEDIUM or HIGH VideoQualitu definitions",\ \ "witdh": "width of the layer in pixels",\ \ "height": "height of the layer in pixels",\ \ "bitrate": "video bitrate for the layer in bit per second"\ \ }\ \ \] } }, "audio": { "options": { "audio\_codec": "audio codec ID from the \[AudioCodec enum\](https://github.com/livekit/protocol/blob/main/protobufs/livekit\_models.proto)", "bitrate": "audio bitrate for the layer in bit per second", "channels": "audio channel count, 1 for mono, 2 for stereo", "disable\_dtx": "wether to disable the \[DTX feature\](https://www.rfc-editor.org/rfc/rfc6716#section-2.1.9) for the OPUS codec" } } } Then create the Ingress using `lk`: lk ingress create ingress.json Enabling transcoding for WHIP sessions[](https://docs.livekit.io/home/ingress/overview/#enabling-transcoding-for-whip-sessions) -------------------------------------------------------------------------------------------------------------------------------- By default, WHIP ingress sessions forward incoming audio and video media unmodified from the source to LiveKit clients. This behavior allows the lowest possible end to end latency between the media source and the viewers. This however requires the source encoder to be configured with settings that are compatible with all the subscribers, and ensure the right trade offs between quality and reach for clients with variable connection quality. This is best achieved when the source encoder is configured with simulcast enabled. If the source encoder cannot be setup easily to achieve such tradeoffs, or if the available uplink bandwidth is insufficient to send all required simulcast layers, WHIP ingresses can be configured to transcode the source media similarly to other source types. This is done by setting the `enable_transcoding` option on the ingress. The encoder settings can then be configured in the `audio` and `video` settings in the same manner as for other inputs types. LiveKit CLIJavaScriptGoRuby Create a file at `ingress.json` with the following content: { "input\_type": 1 (WHIP only) "name": "Name of the egress goes here", "room\_name": "Name of the room to connect to", "participant\_identity": "Unique identity for the room participant the Ingress service will connect as", "participant\_name": "Name displayed in the room for the participant", "enable\_transcoding": true "video": { "name": "track name", "source": "SCREEN\_SHARE", "preset": "Video preset enum value" }, "audio": { "name": "track name", "source": "SCREEN\_SHARE\_AUDIO", "preset": "Audio preset enum value" } } Then create the Ingress using `lk`: lk ingress create ingress.json Service architecture[](https://docs.livekit.io/home/ingress/overview/#service-architecture) -------------------------------------------------------------------------------------------- LiveKit Ingress exposes public RTMP and WHIP endpoints streamers can connect to. On initial handshake, the Ingress service validates the incoming request and retrieves the corresponding Ingress metadata, including what LiveKit room the stream belongs to. The Ingress server then sets up a GStreamer based media processing pipeline to transcode the incoming media to a format compatible with LiveKit WebRTC clients, publishes the resulting media to the LiveKit room. ![Ingress instance](https://docs.livekit.io/images/diagrams/ingress-instance.svg)![Ingress instance](https://docs.livekit.io/images/diagrams/ingress-instance.svg) On this page [Introduction](https://docs.livekit.io/home/ingress/overview/#introduction) [Supported Sources](https://docs.livekit.io/home/ingress/overview/#supported-sources) [Workflow](https://docs.livekit.io/home/ingress/overview/#workflow) [WHIP / RTMP](https://docs.livekit.io/home/ingress/overview/#whip-rtmp) [URL Input](https://docs.livekit.io/home/ingress/overview/#url-input) [API](https://docs.livekit.io/home/ingress/overview/#api) [CreateIngress](https://docs.livekit.io/home/ingress/overview/#createingress) [ListIngress](https://docs.livekit.io/home/ingress/overview/#listingress) [UpdateIngress](https://docs.livekit.io/home/ingress/overview/#updateingress) [DeleteIngress](https://docs.livekit.io/home/ingress/overview/#deleteingress) [Using video presets](https://docs.livekit.io/home/ingress/overview/#using-video-presets) [Custom settings](https://docs.livekit.io/home/ingress/overview/#custom-settings) [Enabling transcoding for WHIP sessions](https://docs.livekit.io/home/ingress/overview/#enabling-transcoding-for-whip-sessions) [Service architecture](https://docs.livekit.io/home/ingress/overview/#service-architecture) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/ingress/overview/) Search --- # Make outbound calls | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/outbound-calls/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/outbound-calls/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Creating a SIP participant](https://docs.livekit.io/sip/outbound-calls/#creating-a-sip-participant) [Making a call with extension codes (DTMF)](https://docs.livekit.io/sip/outbound-calls/#dtmf) [Playing dial tone while the call is dialing](https://docs.livekit.io/sip/outbound-calls/#playing-dial-tone-while-the-call-is-dialing) Copy pageSee more page options The following sections include examples for making an outbound call by creating a LiveKit SIP participant and configuring call settings for dialing out. To create an AI agent to make outbound calls on your behalf, see the [Voice AI telephony guide](https://docs.livekit.io/agents/start/telephony/) . Creating a SIP participant[](https://docs.livekit.io/sip/outbound-calls/#creating-a-sip-participant) ----------------------------------------------------------------------------------------------------- To make outbound calls with SIP Service, create a SIP participant with the [`CreateSIPParticipant`](https://docs.livekit.io/sip/api/#createsipparticipant) API. It returns an `SIPParticipantInfo` object that describes the participant. Outbound calling requires at least one [Outbound Trunk](https://docs.livekit.io/sip/trunk-outbound/) . LiveKit CLINode.jsPythonRubyGo 1. Create a `sip-participant.json` file with the following participant details: { "sip\_trunk\_id": "", "sip\_call\_to": "", "room\_name": "my-sip-room", "participant\_identity": "sip-test", "participant\_name": "Test Caller", "krisp\_enabled": true, "wait\_until\_answered": true } 2. Create the SIP Participant using the CLI. After you run this command, the participant makes a call to the `sip_call_to` number configured in your outbound trunk. When you set `wait_until_answered` to `true`, the command waits until the callee picks up the call before returning. You can also monitor the call status using the [SIP participant attributes](https://docs.livekit.io/sip/sip-participant/#sip-attributes) . When the callee picks up the call, the `sip.callStatus` attribute is `active`. lk sip participant create sip-participant.json Once the user picks up, they will be connected to `my-sip-room`. Making a call with extension codes (DTMF)[](https://docs.livekit.io/sip/outbound-calls/#dtmf) ---------------------------------------------------------------------------------------------- To make outbound calls with fixed extension codes (DTMF tones), set `dtmf` field in `CreateSIPParticipant` request: LiveKit CLINode.jsPythonRubyGo { "sip\_trunk\_id": "", "sip\_call\_to": "", "dtmf": "\*123#ww456", "room\_name": "my-sip-room", "participant\_identity": "sip-test", "participant\_name": "Test Caller" } **Tip** Character `w` can be used to delay DTMF by 0.5 sec. This example will dial a specified number and will send the following DTMF tones: * `*123#` * Wait 1 sec * `456` Playing dial tone while the call is dialing[](https://docs.livekit.io/sip/outbound-calls/#playing-dial-tone-while-the-call-is-dialing) --------------------------------------------------------------------------------------------------------------------------------------- SIP participants emit no audio by default while the call connects. This can be changed by setting `play_dialtone` field in `CreateSIPParticipant` request: LiveKit CLINode.jsPythonRubyGo { "sip\_trunk\_id": "", "sip\_call\_to": "", "room\_name": "my-sip-room", "participant\_identity": "sip-test", "participant\_name": "Test Caller", "play\_dialtone": true } If `play_dialtone` is enabled, the SIP Participant plays a dial tone to the room until the phone is picked up. On this page [Creating a SIP participant](https://docs.livekit.io/sip/outbound-calls/#creating-a-sip-participant) [Making a call with extension codes (DTMF)](https://docs.livekit.io/sip/outbound-calls/#dtmf) [Playing dial tone while the call is dialing](https://docs.livekit.io/sip/outbound-calls/#playing-dial-tone-while-the-call-is-dialing) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/outbound-calls/) Search --- # Transferring calls | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/transfer-cold/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/transfer-cold/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Transferring a SIP participant using SIP REFER](https://docs.livekit.io/sip/transfer-cold/#transferring-a-sip-participant-using-sip-refer) [Enable call transfers for your Twilio SIP trunk](https://docs.livekit.io/sip/transfer-cold/#enable-call-transfers-for-your-twilio-sip-trunk) [TransferSIPParticipant server API parameters](https://docs.livekit.io/sip/transfer-cold/#transfersipparticipant-server-api-parameters) [Usage](https://docs.livekit.io/sip/transfer-cold/#usage) Copy pageSee more page options A "cold transfer" refers to transferring a caller (SIP participant) to another number or SIP endpoint without a hand off. A cold transfer shuts down the room (that is, the session) of the original call. Transferring a SIP participant using SIP REFER[](https://docs.livekit.io/sip/transfer-cold/#transferring-a-sip-participant-using-sip-refer) -------------------------------------------------------------------------------------------------------------------------------------------- REFER is a SIP method that allows you to move an active session to another endpoint (that is, transfer a call). For LiveKit telephony apps, you can use the `TransferSIPParticipant` server API to transfer a caller to another phone number or SIP endpoint. In order to successfully transfer calls, you must configure your provider trunks to allow call transfers. ### Enable call transfers for your Twilio SIP trunk[](https://docs.livekit.io/sip/transfer-cold/#enable-call-transfers-for-your-twilio-sip-trunk) Enable call transfer and PSTN transfers for your Twilio SIP trunk. To learn more, see Twilio's [Call Transfer via SIP REFER](https://www.twilio.com/docs/sip-trunking/call-transfer) documentation. When you transfer a call, you have the option to set the caller ID to display the phone number of the transferee (the caller) or the transferor (the phone number associated with your LiveKit trunk). CLIConsole The following command enables call transfers and sets the caller ID to display the number of the transferee: **Note** * To list trunks, execute `twilio api trunking v1 trunks list`. * To set the caller ID to the transferor, set `transfer-caller-id` to `from-transferor`. twilio api trunking v1 trunks update \--sid \\ \--transfer-mode enable-all \\ \--transfer-caller-id from-transferee ### TransferSIPParticipant server API parameters[](https://docs.livekit.io/sip/transfer-cold/#transfersipparticipant-server-api-parameters) **transfer\_to**`string``Required` [#](https://docs.livekit.io/sip/transfer-cold/#transfer_to) The `transfer_to` value can either be a valid telephone number or a SIP URI. The following examples are valid values: * `tel:+15105550100` * `sip:+15105550100@sip.telnyx.com` * `sip:+15105550100@my-livekit-demo.pstn.twilio.com` **participant\_identity**`string``Required` [#](https://docs.livekit.io/sip/transfer-cold/#participant_identity) Identity of the SIP participant that should be transferred. **room\_name**`string``Required` [#](https://docs.livekit.io/sip/transfer-cold/#room_name) Source room name for the transfer. **play\_dialtone**`bool``Required` [#](https://docs.livekit.io/sip/transfer-cold/#play_dialtone) Play dial tone to the user being transferred when a transfer is initiated. ### Usage[](https://docs.livekit.io/sip/transfer-cold/#usage) Set up the following environment variables: export LIVEKIT\_URL\= export LIVEKIT\_API\_KEY\= export LIVEKIT\_API\_SECRET\= Reveal API Key and Secret Node.jsPythonRubyGo This example uses the LiveKit URL, API key, and secret set as environment variables. import { SipClient } from 'livekit-server-sdk'; // ... async function transferParticipant(participant) { console.log("transfer participant initiated"); const sipTransferOptions \= { playDialtone: false }; const sipClient \= new SipClient(process.env.LIVEKIT\_URL, process.env.LIVEKIT\_API\_KEY, process.env.LIVEKIT\_API\_SECRET); const transferTo \= "tel:+15105550100"; await sipClient.transferSipParticipant('open-room', participant.identity, transferTo, sipTransferOptions); console.log('transfer participant'); } On this page [Transferring a SIP participant using SIP REFER](https://docs.livekit.io/sip/transfer-cold/#transferring-a-sip-participant-using-sip-refer) [Enable call transfers for your Twilio SIP trunk](https://docs.livekit.io/sip/transfer-cold/#enable-call-transfers-for-your-twilio-sip-trunk) [TransferSIPParticipant server API parameters](https://docs.livekit.io/sip/transfer-cold/#transfersipparticipant-server-api-parameters) [Usage](https://docs.livekit.io/sip/transfer-cold/#usage) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/transfer-cold/) Search --- # Handling DTMF | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/dtmf/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/dtmf/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Sending DTMF](https://docs.livekit.io/sip/dtmf/#sending-dtmf) [Receiving DTMF](https://docs.livekit.io/sip/dtmf/#receiving-dtmf) Copy pageSee more page options LiveKit's Telephony stack fully supports DTMF tones, enabling integration with legacy IVR systems. It also enables agents to receive DTMF tones from telephone users. Sending DTMF[](https://docs.livekit.io/sip/dtmf/#sending-dtmf) --------------------------------------------------------------- To send DTMF tones, use the `publishDtmf` API on the `localParticipant`. This API transmits DTMF tones to the room; tones can be sent by any participant in the room. SIP participants in the room receive the tones and relay them to the telephone user. Node.jsPythonGo // publishes 123# in DTMF await localParticipant.publishDtmf(1, '1'); await localParticipant.publishDtmf(2, '2'); await localParticipant.publishDtmf(3, '3'); await localParticipant.publishDtmf(11, '#'); **Tip** Sending DTMF tones requires both a numeric code and a string representation to ensure compatibility with various SIP implementations. Special characters like `*` and `#` are mapped to their respective numeric codes. See [RFC 4733](https://datatracker.ietf.org/doc/html/rfc4733#section-3.2) for details. Receiving DTMF[](https://docs.livekit.io/sip/dtmf/#receiving-dtmf) ------------------------------------------------------------------- When SIP receives DTMF tones, they are relayed to the room as events that participants can listen for. Node.jsPythonGo room.on(RoomEvent.DtmfReceived, (code, digit, participant) \=> { console.log('DTMF received from participant', participant.identity, code, digit); }); On this page [Sending DTMF](https://docs.livekit.io/sip/dtmf/#sending-dtmf) [Receiving DTMF](https://docs.livekit.io/sip/dtmf/#receiving-dtmf) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/dtmf/) Search --- # SIP outbound trunk | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/trunk-outbound/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/trunk-outbound/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Restricting calls to a region](https://docs.livekit.io/sip/trunk-outbound/#region-pinning) [Create an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#create-an-outbound-trunk) [Configuring an outbound trunk for any phone number](https://docs.livekit.io/sip/trunk-outbound/#configuring-an-outbound-trunk-for-any-phone-number) [List outbound trunks](https://docs.livekit.io/sip/trunk-outbound/#list-outbound-trunks) [Update an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#update-an-outbound-trunk) [Update specific fields of an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#update-specific-fields-of-an-outbound-trunk) [Replace an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#replace-an-outbound-trunk) [IP address range for LiveKit Cloud SIP](https://docs.livekit.io/sip/trunk-outbound/#ip-address-range-for-livekit-cloud-sip) Copy pageSee more page options To provision an outbound trunk with the SIP Service, use the [`CreateSIPOutboundTrunk`](https://docs.livekit.io/sip/api/#createsipoutboundtrunk) API. It returns an `SIPOutboundTrunkInfo` object that describes the created SIP trunk. You can query these parameters any time using the `ListSIPOutboundTrunk` API. Restricting calls to a region[](https://docs.livekit.io/sip/trunk-outbound/#region-pinning) -------------------------------------------------------------------------------------------- To originate calls from the same region as the destination phone number, set the `destination_country` parameter for an outbound trunk. This applies region pinning to all calls made through the trunk. When `destination_country` is enabled, outbound calls are routed based on location: * For countries that LiveKit operates data centers in, calls originate from a server within the country. * For other countries, calls originate from a server that is closest to that country. In the unlikely event that the preferred region is non-operational or offline, calls originate from another region nearby. For a full list of supported regions, see [Available regions](https://docs.livekit.io/sip/cloud/#available-regions) . The `destination_country` parameter accepts a two-letter country code. To learn more, see [CreateSIPOutboundTrunk](https://docs.livekit.io/sip/api/#createsipoutboundtrunk) . Create an outbound trunk[](https://docs.livekit.io/sip/trunk-outbound/#create-an-outbound-trunk) ------------------------------------------------------------------------------------------------- The following creates a SIP outbound trunk with username and password authentication. It makes outbound calls from number `+15105550100`. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud 1. Create a file named `outbound-trunk.json` using your phone number, trunk domain name, and `username` and `password`: TwilioTelnyx { "trunk": { "name": "My outbound trunk", "address": ".pstn.twilio.com", "numbers": \["+15105550100"\], "authUsername": "", "authPassword": "" } } 2. Create the outbound trunk using the CLI: lk sip outbound create outbound-trunk.json The output of the command returns the trunk ID. Copy it for the next step: SIPTrunkID: ### Configuring an outbound trunk for any phone number[](https://docs.livekit.io/sip/trunk-outbound/#configuring-an-outbound-trunk-for-any-phone-number) The `numbers` parameter for outbound trunks is a required field. However, you can set the parameter to any string (for example, `*`) to use the outbound trunk for calls from any number. This is useful if you want to use the same outbound trunk for all calls or if you want to use a different phone number for each call. Instead of setting the number on the trunk, you can set the phone number to call from using the `sip_number` parameter for the [CreateSIPParticipant](https://docs.livekit.io/sip/api/#createsipparticipant) API. The following example creates an outbound trunk that allows calling from any number, then initiates a call using the outbound trunk. 1. Create an outbound trunk using the CLI. Create a file named `outbound-trunk.json` and copy and paste the following content: { "trunk": { "name": "My outbound trunk", "address": ".pstn.twilio.com", "numbers": \["\*"\], "auth\_username": "", "auth\_password": "" } } Create the outbound trunk using the CLI: lk sip outbound create outbound-trunk.json 2. Initiate a call from the number `+15105550100` using the CLI. This number is the phone number configured with your SIP trunk provider. Use the from the output of the previous step. Create a file named `participant.json` and copy and paste the following content: { "sip\_number": "+15105550100", "sip\_trunk\_id": "", "sip\_call\_to": "+12135550100", "room\_name": "open-room", "participant\_identity": "sip-test", "participant\_name": "Test call participant", "wait\_until\_answered": true } **Important** If you're using Telnyx, the leading `+` in the phone number assumes the `Destination Number Format` is set to `+E.164` for your number. Initiate the call using the CLI: lk sip participant create participant.json After you run the command, a call from the number `+15105550100` to `+12135550100` is initiated. Output from the command returns when the call is answered. List outbound trunks[](https://docs.livekit.io/sip/trunk-outbound/#list-outbound-trunks) ----------------------------------------------------------------------------------------- Use the [`ListSIPOutboundTrunk`](https://docs.livekit.io/sip/api/#listsipoutboundtrunk) API to list all outbound trunks and trunk parameters. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud lk sip outbound list Update an outbound trunk[](https://docs.livekit.io/sip/trunk-outbound/#update-an-outbound-trunk) ------------------------------------------------------------------------------------------------- The [`UpdateSIPOutboundTrunk`](https://docs.livekit.io/sip/api/#updatesipoutboundtrunk) API allows you to update specific fields of an outbound trunk or [replace](https://docs.livekit.io/sip/trunk-outbound/#replace-sip-outbound-trunk) an outbound trunk with a new one. ### Update specific fields of an outbound trunk[](https://docs.livekit.io/sip/trunk-outbound/#update-specific-fields-of-an-outbound-trunk) The `UpdateSIPOutboundTrunkFields` API allows you to update specific fields of an outbound trunk without affecting other fields. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud 1. Create a file named `outbound-trunk.json` with the fields you want to update. The following example updates the name and phone numbers for the trunk: TwilioTelnyx { "name": "My updated outbound trunk", "address": ".pstn.twilio.com", "numbers": \["+15105550100"\] } 2. Update the outbound trunk using the CLI: lk sip outbound update \--id outbound-trunk.json The output of the command returns the trunk ID: SIPTrunkID: ### Replace an outbound trunk[](https://docs.livekit.io/sip/trunk-outbound/#replace-an-outbound-trunk) The `UpdateSIPOutboundTrunk` API allows you to replace an existing outbound trunk with a new one using the same trunk ID. LiveKit CLINode.jsPythonRubyGoLiveKit Cloud The CLI doesn't support replacing outbound trunks. IP address range for LiveKit Cloud SIP[](https://docs.livekit.io/sip/trunk-outbound/#ip-address-range-for-livekit-cloud-sip) ----------------------------------------------------------------------------------------------------------------------------- LiveKit Cloud nodes do not have a static IP address range, thus there's no way currently to use IP range for outbound authentication. Thus, prefer setting user/password authentication on SIP trunk Provider. If it's unavailable, or IP range is required in addition to user/password, set range(s) that include all IPs: e.g. `0.0.0.0/0` or `0.0.0.0/1`+`128.0.0.0/1`. On this page [Restricting calls to a region](https://docs.livekit.io/sip/trunk-outbound/#region-pinning) [Create an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#create-an-outbound-trunk) [Configuring an outbound trunk for any phone number](https://docs.livekit.io/sip/trunk-outbound/#configuring-an-outbound-trunk-for-any-phone-number) [List outbound trunks](https://docs.livekit.io/sip/trunk-outbound/#list-outbound-trunks) [Update an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#update-an-outbound-trunk) [Update specific fields of an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#update-specific-fields-of-an-outbound-trunk) [Replace an outbound trunk](https://docs.livekit.io/sip/trunk-outbound/#replace-an-outbound-trunk) [IP address range for LiveKit Cloud SIP](https://docs.livekit.io/sip/trunk-outbound/#ip-address-range-for-livekit-cloud-sip) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/trunk-outbound/) Search --- # HD voice for SIP | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/hd-voice/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/hd-voice/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Configuring Telnyx](https://docs.livekit.io/sip/hd-voice/#configuring-telnyx) [Other Providers](https://docs.livekit.io/sip/hd-voice/#other-providers) Copy pageSee more page options Telephone calls have traditionally been routed through the Public Switched Telephone Network (PSTN), a technology for landlines dating back over a century. PSTN calls are limited to an 8kHz sample rate using a narrowband audio codec, resulting in audio that typically sounds muffled or lacks range. Modern cell phones can use VoIP for calls when connected via Wi-Fi or mobile data. VoIP can leverage wideband audio codecs that transmit audio at a higher sample rate, resulting in much higher quality audio, often referred to as HD Voice. LiveKit SIP supports wideband audio codecs such as G.722 out of the box, providing higher quality audio when used with HD Voice-capable SIP trunks or endpoints. Configuring Telnyx[](https://docs.livekit.io/sip/hd-voice/#configuring-telnyx) ------------------------------------------------------------------------------- Telnyx supports HD Voice for customers in the US. To enable HD Voice with Telnyx, ensure the following are configured in your Telnyx portal: * `HD Voice feature` is enabled on the phone number you are trying to use (under Number -> Voice) * `G.722` codec is enabled on your SIP Trunk (under SIP Connection -> Inbound) * We recommend leaving G.711U enabled for compatibility. Other Providers[](https://docs.livekit.io/sip/hd-voice/#other-providers) ------------------------------------------------------------------------- Currently, Twilio does not support HD voice. If you find other providers that support HD voice, please let us know so we can update this guide. On this page [Configuring Telnyx](https://docs.livekit.io/sip/hd-voice/#configuring-telnyx) [Other Providers](https://docs.livekit.io/sip/hd-voice/#other-providers) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/hd-voice/) Search --- # livekit_client - Dart API docs ![The LiveKit icon, the name of the repository and some sample code in the background.](https://docs.livekit.io/.github/banner_light.png) [![pub package](https://img.shields.io/pub/v/livekit_client?label=livekit_client&color=blue)](https://pub.dev/packages/livekit_client) LiveKit Flutter SDK =================== Use this SDK to add real-time video, audio and data features to your Flutter app. By connecting to a self- or cloud-hosted [LiveKit](https://livekit.io/) server, you can quickly build applications like interactive live streaming or video calls with just a few lines of code. This package is published to pub.dev as [livekit\_client](https://pub.dev/packages/livekit_client) . Docs ---- More Docs and guides are available at [https://docs.livekit.io](https://docs.livekit.io/) Current supported features -------------------------- | Feature | Subscribe/Publish | Simulcast | Background audio | Screen sharing | End to End Encryption | Multi Codec Simulcast | | --- | --- | --- | --- | --- | --- | --- | | iOS | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | | Android | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | | Mac | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | | Windows | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | | Linux | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 = Available 🟡 = Coming soon (Work in progress) 🔴 = Not currently available (Possibly in the future) Example app ----------- We built a multi-user conferencing app as an example in the [example/](https://docs.livekit.io/reference/client-sdk-flutter/example/) folder. You can join the same room from any supported LiveKit clients. Installation ------------ Include this package to your `pubspec.yaml` --- dependencies: livekit_client: ### iOS Camera and microphone usage need to be declared in your `Info.plist` file. ... NSCameraUsageDescription $(PRODUCT_NAME) uses your camera NSMicrophoneUsageDescription $(PRODUCT_NAME) uses your microphone Your application can still run the voice call when it is switched to the background if the background mode is enabled. Select the app target in Xcode, click the Capabilities tab, enable Background Modes, and check **Audio, AirPlay, and Picture in Picture**. Your `Info.plist` should have the following entries. ... UIBackgroundModes audio #### Notes Since [xcode 14](https://developer.apple.com/news/upcoming-requirements/?id=06062022a) no longer supports 32bit builds, and our latest version is based on libwebrtc m104+ the iOS framework no longer supports 32bit builds, we strongly recommend upgrading to flutter 3.3.0+. if you are using flutter 3.0.0 or below, there is a high chance that your flutter app cannot be compiled correctly due to the missing i386 and arm 32bit framework [#132](https://github.com/livekit/client-sdk-flutter/issues/132) [#172](https://github.com/livekit/client-sdk-flutter/issues/172) . You can try to modify your `{projects_dir}/ios/Podfile` to fix this issue. post_install do |installer| installer.pods_project.targets.each do |target| flutter_additional_ios_build_settings(target) target.build_configurations.each do |config| # Workaround for https://github.com/flutter/flutter/issues/64502 config.build_settings['ONLY_ACTIVE_ARCH'] = 'YES' # <= this line end end end For iOS, the minimum supported deployment target is `12.1`. You will need to add the following to your Podfile. platform :ios, '12.1' You may need to delete `Podfile.lock` and re-run `pod install` after updating deployment target. ### Android We require a set of permissions that need to be declared in your `AppManifest.xml`. These are required by Flutter WebRTC, which we depend on. ... For using the bluetooth headset correctly on the android device, you need to add `permission_handler` to your project. And call the following code after launching your app for the first time. import 'package:permission_handler/permission_handler.dart'; Future _checkPermissions() async { var status = await Permission.bluetooth.request(); if (status.isPermanentlyDenied) { print('Bluetooth Permission disabled'); } status = await Permission.bluetoothConnect.request(); if (status.isPermanentlyDenied) { print('Bluetooth Connect Permission disabled'); } } void main() async { WidgetsFlutterBinding.ensureInitialized(); await _checkPermissions(); runApp(MyApp()); } #### Audio Modes By default, we use the `communication` audio mode on Android which works best for two-way voice communication. If your app is media playback oriented and does not need the use of the device's microphone, you can use the `media` audio mode which will provide better audio quality. import 'package:flutter_webrtc/flutter_webrtc.dart' as webrtc; Future _initializeAndroidAudioSettings() async { await webrtc.WebRTC.initialize(options: { 'androidAudioConfiguration': webrtc.AndroidAudioConfiguration.media.toMap() }); webrtc.Helper.setAndroidAudioConfiguration( webrtc.AndroidAudioConfiguration.media); } void main() async { await _initializeAudioSettings(); runApp(const MyApp()); } Note: the audio routing will become controlled by the system and cannot be manually changed with functions like `Hardware.selectAudioOutput`. ### Desktop support In order to enable Flutter desktop development, please follow [instructions here](https://docs.flutter.dev/desktop#set-up) . On M1 Macs, you will also need to install x86\_64 version of FFI: sudo arch -x86_64 gem install ffi On Windows [VS 2019](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=community&rel=16) is needed (link in flutter docs will download VS 2022). Usage ----- ### Connecting to a room, publish video & audio final roomOptions = RoomOptions( adaptiveStream: true, dynacast: true, // ... your room options ) final room = await LiveKitClient.connect(url, token, roomOptions: roomOptions); try { // video will fail when running in ios simulator await room.localParticipant.setCameraEnabled(true); } catch (error) { print('Could not publish video, error: $error'); } await room.localParticipant.setMicrophoneEnabled(true); ### Screen sharing Screen sharing is supported across all platforms. You can enable it with: room.localParticipant.setScreenShareEnabled(true); #### Android On Android, you would have to define a foreground service in your AndroidManifest.xml. ... #### iOS On iOS, a broadcast extension is needed in order to capture screen content from other apps. See [setup guide](https://github.com/flutter-webrtc/flutter-webrtc/wiki/iOS-Screen-Sharing#broadcast-extension-quick-setup) for instructions. #### Desktop(Windows/macOS) On dekstop you can use `ScreenSelectDialog` to select the window or screen you want to share. try { final source = await showDialog( context: context, builder: (context) => ScreenSelectDialog(), ); if (source == null) { print('cancelled screenshare'); return; } print('DesktopCapturerSource: ${source.id}'); var track = await LocalVideoTrack.createScreenShareTrack( ScreenShareCaptureOptions( sourceId: source.id, maxFrameRate: 15.0, ), ); await room.localParticipant.publishVideoTrack(track); } catch (e) { print('could not publish screen sharing: $e'); } ### End to End Encryption LiveKit supports end-to-end encryption for audio/video data sent over the network. By default, the native platform can support E2EE without any settings, but for flutter web, you need to use the following steps to create `e2ee.worker.dart.js` file. # for example app dart compile js .\web\e2ee.worker.dart -o .\example\web\e2ee.worker.dart.js # for your project export YOU_PROJECT_DIR=your_project_dir git clone https://github.com/livekit/client-sdk-flutter.git cd client-sdk-flutter && flutter pub get dart compile js .\web\e2ee.worker.dart -o ${YOU_PROJECT_DIR}\web\e2ee.worker.dart.js ### Advanced track manipulation The setCameraEnabled/setMicrophoneEnabled helpers are wrappers around the Track API. You can also manually create and publish tracks: var localVideo = await LocalVideoTrack.createCameraTrack(); await room.localParticipant.publishVideoTrack(localVideo); ### Rendering video Each track can be rendered separately with the provided `VideoTrackRenderer` widget. VideoTrack? track; @override Widget build(BuildContext context) { if (track != null) { return VideoTrackRenderer(track); } else { return Container( color: Colors.grey, ); } } ### Audio handling Audio tracks are played automatically as long as you are subscribed to them. ### Handling changes LiveKit client makes it simple to build declarative UI that reacts to state changes. It notifies changes in two ways * `ChangeNotifier` - generic notification of changes. This is useful when you are building reactive UI and only care about changes that may impact rendering. * `EventsListener` - listener pattern to listen to specific events (see [events.dart](https://github.com/livekit/client-sdk-flutter/blob/main/lib/src/events.dart) ). This example will show you how to use both to react to room events. class RoomWidget extends StatefulWidget { final Room room; RoomWidget(this.room); @override State createState() { return _RoomState(); } } class _RoomState extends State { late final EventsListener _listener = widget.room.createListener(); @override void initState() { super.initState(); // used for generic change updates widget.room.addListener(_onChange); // used for specific events _listener ..on((_) { // handle disconnect }) ..on((e) { print("participant joined: ${e.participant.identity}"); }) } @override void dispose() { // be sure to dispose listener to stop listening to further updates _listener.dispose(); widget.room.removeListener(_onChange); super.dispose(); } void _onChange() { // perform computations and then call setState // setState will trigger a build setState(() { // your updates here }); } @override Widget build(BuildContext context) { // your build function } } Similarly, you could do the same when rendering participants. Reacting to changes makes it possible to handle tracks published/unpublished or re-ordering participants in your UI. class VideoView extends StatefulWidget { final Participant participant; VideoView(this.participant); @override State createState() { return _VideoViewState(); } } class _VideoViewState extends State { TrackPublication? videoPub; @override void initState() { super.initState(); widget.participant.addListener(this._onParticipantChanged); // trigger initial change _onParticipantChanged(); } @override void dispose() { widget.participant.removeListener(this._onParticipantChanged); super.dispose(); } @override void didUpdateWidget(covariant VideoView oldWidget) { oldWidget.participant.removeListener(_onParticipantChanged); widget.participant.addListener(_onParticipantChanged); _onParticipantChanged(); super.didUpdateWidget(oldWidget); } void _onParticipantChanged() { var subscribedVideos = widget.participant.videoTracks.values.where((pub) { return pub.kind == TrackType.VIDEO && !pub.isScreenShare && pub.subscribed; }); setState(() { if (subscribedVideos.length > 0) { var videoPub = subscribedVideos.first; // when muted, show placeholder if (!videoPub.muted) { this.videoPub = videoPub; return; } } this.videoPub = null; }); } @override Widget build(BuildContext context) { var videoPub = this.videoPub; if (videoPub != null) { return VideoTrackRenderer(videoPub.track as VideoTrack); } else { return Container( color: Colors.grey, ); } } } ### Mute, unmute local tracks On `LocalTrackPublication`s, you could control if the track is muted by setting its `muted` property. Changing the mute status will generate an `onTrackMuted` or `onTrack Unmuted` delegate call for the local participant. Other participant will receive the status change as well. // mute track trackPub.muted = true; // unmute track trackPub.muted = false; ### Subscriber controls When subscribing to remote tracks, the client has precise control over status of its subscriptions. You could subscribe or unsubscribe to a track, change its quality, or disabling the track temporarily. These controls are accessible on the `RemoteTrackPublication` object. For more info, see [Subscriber controls](https://docs.livekit.io/guides/room/receive#subscriber-controls) . Getting help / Contributing --------------------------- Please join us on [Slack](https://livekit.io/join-slack) to get help from our devs / community members. We welcome your contributions(PRs) and details can be discussed there. License ------- Apache License 2.0 Thanks ------ A huge thank you to [flutter-webrtc](https://github.com/flutter-webrtc/flutter-webrtc) for making it possible to use WebRTC in Flutter. | LiveKit Ecosystem | | | --- | --- | | Client SDKs | [Components](https://github.com/livekit/components-js)
· [JavaScript](https://github.com/livekit/client-sdk-js)
· [iOS/macOS](https://github.com/livekit/client-sdk-swift)
· [Android](https://github.com/livekit/client-sdk-android)
· **Flutter** · [React Native](https://github.com/livekit/client-sdk-react-native)
· [Rust](https://github.com/livekit/client-sdk-rust)
· [Python](https://github.com/livekit/client-sdk-python)
· [Unity (web)](https://github.com/livekit/client-sdk-unity-web)
· [Unity (beta)](https://github.com/livekit/client-sdk-unity) | | Server SDKs | [Node.js](https://github.com/livekit/server-sdk-js)
· [Golang](https://github.com/livekit/server-sdk-go)
· [Ruby](https://github.com/livekit/server-sdk-ruby)
· [Java/Kotlin](https://github.com/livekit/server-sdk-kotlin)
· [PHP (community)](https://github.com/agence104/livekit-server-sdk-php)
· [Python (community)](https://github.com/tradablebits/livekit-server-sdk-python) | | Services | [Livekit server](https://github.com/livekit/livekit)
· [Egress](https://github.com/livekit/egress)
· [Ingress](https://github.com/livekit/ingress) | | Resources | [Docs](https://docs.livekit.io/)
· [Example apps](https://github.com/livekit-examples)
· [Cloud](https://livekit.io/cloud)
· [Self-hosting](https://docs.livekit.io/oss/deployment)
· [CLI](https://github.com/livekit/livekit-cli) | Libraries --------- [livekit\_client](https://docs.livekit.io/reference/client-sdk-flutter/livekit_client/livekit_client-library.html) Flutter Client SDK to LiveKit. [livekit\_client\_web](https://docs.livekit.io/reference/client-sdk-flutter/livekit_client_web/livekit_client_web-library.html) --- # SIP participant | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/sip-participant/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/sip-participant/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [SIP participant attributes](https://docs.livekit.io/sip/sip-participant/#sip-participant-attributes) [SIP attributes](https://docs.livekit.io/sip/sip-participant/#sip-attributes) [Twilio attributes](https://docs.livekit.io/sip/sip-participant/#twilio-attributes) [Custom attributes](https://docs.livekit.io/sip/sip-participant/#custom-attributes) [Examples](https://docs.livekit.io/sip/sip-participant/#examples) [Basic example](https://docs.livekit.io/sip/sip-participant/#basic-example) [Modify voice AI agent based on caller attributes](https://docs.livekit.io/sip/sip-participant/#modify-voice-ai-agent-based-on-caller-attributes) [Creating a SIP participant to make outbound calls](https://docs.livekit.io/sip/sip-participant/#creating-a-sip-participant-to-make-outbound-calls) Copy pageSee more page options **Note** To create a SIP participant to make outbound calls, see [Make outbound calls](https://docs.livekit.io/sip/outbound-calls/) . Each user in a LiveKit telephony app is a [LiveKit participant](https://docs.livekit.io/home/get-started/api-primitives/#participant) . This includes end users who call in using your inbound trunk, the participant you use to make outbound calls, and if you're using an agent, the AI voice agent that interacts with callers. SIP participants are managed like any other participant using the [participant management commands](https://docs.livekit.io/home/server/managing-participants/) . SIP participant attributes[](https://docs.livekit.io/sip/sip-participant/#sip-participant-attributes) ------------------------------------------------------------------------------------------------------ SIP participants can be identified using the `kind` field for participants, which identifies the [type of participant](https://docs.livekit.io/home/get-started/api-primitives/#types-of-participants) in a LiveKit room (i.e. session). For SIP participants, this is `Participant.Kind == SIP`. The participant `attributes` field contains SIP specific attributes that identify the caller and call details. You can use SIP participant attributes to create different workflows based on the caller. For example, look up customer information in a database to identify the caller. ### SIP attributes[](https://docs.livekit.io/sip/sip-participant/#sip-attributes) All SIP participants have the following attributes: | Attribute | Description | | --- | --- | | `sip.callID` | LiveKit's SIP call ID. A unique ID used as a SIP call tag to identify a conversation (i.e. match requests and responses). | | `sip.callIDFull` | Trunk provider SIP call ID. A globally unique ID to identify a specific SIP call. | | `sip.callStatus` | Current call status for the SIP call associated with this participant. Valid values are:

* `active`: Participant is connected and the call is active.
* `automation`: For outbound calls using Dual-Tone Multi-Frequency (DTMF), this status indicates the call has successfully connected, but is still dialing DTMF numbers. After all the numbers are dialed, the status changes to `active`.
* `dialing`: Call is dialing and waiting to be picked up.
* `hangup`: Call has been ended by a participant.
* `ringing`: Inbound call is ringing for the caller. Status changes to `active` when the SIP participant subscribes to any remote audio tracks. | | `sip.phoneNumber` | User's phone number. For inbound trunks, this is the phone number the call originates from. For outbound SIP, this is the number dialed by the SIP participant.

**Note**

This attribute isn't available if `HidePhoneNumber` is set in the dispatch rule. | | `sip.ruleID` | SIP `DispatchRule` ID used for the inbound call. This field is empty for outbound calls. | | `sip.trunkID` | The inbound or outbound SIP trunk ID used for the call. | | `sip.trunkPhoneNumber` | Phone number associated with SIP trunk. For inbound trunks, this is the number dialed in to by an end user. For outbound trunks, this is the number a call originates from. | ### Twilio attributes[](https://docs.livekit.io/sip/sip-participant/#twilio-attributes) If you're using Twilio SIP trunks, the following additional attributes are included: | Attribute | Description | | --- | --- | | `sip.twilio.accountSid` | Twilio account SID. | | `sip.twilio.callSid` | Twilio call SID. | ### Custom attributes[](https://docs.livekit.io/sip/sip-participant/#custom-attributes) You can add custom SIP participant attributes in one of two ways: * Adding attributes to the dispatch rule. To learn more, see [Setting custom attributes on inbound SIP participants](https://docs.livekit.io/sip/dispatch-rule/#setting-custom-attributes-on-inbound-sip-participants) . * Using SIP headers: For any `X-*` SIP headers, you can configure your trunk with `headers_to_attributes` and a key/value pair mapping. For example: TelnyxTwilio { "trunk": { "name": "Demo inbound trunk", "numbers": \["+15105550100"\], "headers\_to\_attributes": { "X-": "", } } } **Caution** Note the leading `+` assumes the `Destination Number Format` is set to `+E.164` for your Telnyx number. Examples[](https://docs.livekit.io/sip/sip-participant/#examples) ------------------------------------------------------------------ The following examples use SIP participant attributes. ### Basic example[](https://docs.livekit.io/sip/sip-participant/#basic-example) Node.jsPython This example logs the Twilio call SID if the user is a SIP participant. if (participant.kind \== ParticipantKind.SIP) { console.log(participant.attributes\['sip.twilio.callSid'\]); }; ### Modify voice AI agent based on caller attributes[](https://docs.livekit.io/sip/sip-participant/#modify-voice-ai-agent-based-on-caller-attributes) Follow the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai/) to create an agent that responds to incoming calls. Then modify the agent to use SIP participant attributes. PythonNode.js Before starting your `AgentSession`, select the best Deepgram STT model for the participant. Add this code to your `entrypoint` function: \# Add this import to the top of your file from livekit import rtc participant \= await ctx.wait\_for\_participant() dg\_model \= "nova-2-general" \# Check if the participant is a SIP participant if participant.kind \== rtc.ParticipantKind.PARTICIPANT\_KIND\_SIP: \# Use a Deepgram model better suited for phone calls dg\_model \= "nova-2-phonecall" if participant.attributes\['sip.phoneNumber'\] \== '+15105550100': logger.info("Caller phone number is +1-510-555-0100") \# Add other logic here to modify the agent based on the caller's phone number session \= AgentSession( stt\=deepgram.STT(model\=dg\_model), \# ... llm, vad, tts, etc. ) \# ... rest of your entrypoint, including \`await session.start(...)\` Creating a SIP participant to make outbound calls[](https://docs.livekit.io/sip/sip-participant/#creating-a-sip-participant-to-make-outbound-calls) ---------------------------------------------------------------------------------------------------------------------------------------------------- To make outbound calls, create a SIP participant. To learn more, see [Make outbound calls](https://docs.livekit.io/sip/outbound-calls/) . On this page [SIP participant attributes](https://docs.livekit.io/sip/sip-participant/#sip-participant-attributes) [SIP attributes](https://docs.livekit.io/sip/sip-participant/#sip-attributes) [Twilio attributes](https://docs.livekit.io/sip/sip-participant/#twilio-attributes) [Custom attributes](https://docs.livekit.io/sip/sip-participant/#custom-attributes) [Examples](https://docs.livekit.io/sip/sip-participant/#examples) [Basic example](https://docs.livekit.io/sip/sip-participant/#basic-example) [Modify voice AI agent based on caller attributes](https://docs.livekit.io/sip/sip-participant/#modify-voice-ai-agent-based-on-caller-attributes) [Creating a SIP participant to make outbound calls](https://docs.livekit.io/sip/sip-participant/#creating-a-sip-participant-to-make-outbound-calls) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/sip-participant/) Search --- # SIP APIs | LiveKit Docs [Skip to main content](https://docs.livekit.io/sip/api/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/api/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/sip/api/#overview) [Using endpoints](https://docs.livekit.io/sip/api/#using-endpoints) [SIPService APIs](https://docs.livekit.io/sip/api/#sipservice-apis) [CreateSIPInboundTrunk](https://docs.livekit.io/sip/api/#createsipinboundtrunk) [CreateSIPOutboundTrunk](https://docs.livekit.io/sip/api/#createsipoutboundtrunk) [CreateSIPDispatchRule](https://docs.livekit.io/sip/api/#createsipdispatchrule) [CreateSIPParticipant](https://docs.livekit.io/sip/api/#createsipparticipant) [DeleteSIPDispatchRule](https://docs.livekit.io/sip/api/#deletesipdispatchrule) [DeleteSIPTrunk](https://docs.livekit.io/sip/api/#deletesiptrunk) [GetSIPInboundTrunk](https://docs.livekit.io/sip/api/#getsipinboundtrunk) [GetSIPOutboundTrunk](https://docs.livekit.io/sip/api/#getsipoutboundtrunk) [ListSIPDispatchRule](https://docs.livekit.io/sip/api/#listsipdispatchrule) [ListSIPInboundTrunk](https://docs.livekit.io/sip/api/#listsipinboundtrunk) [ListSIPOutboundTrunk](https://docs.livekit.io/sip/api/#listsipoutboundtrunk) [TransferSIPParticipant](https://docs.livekit.io/sip/api/#transfersipparticipant) [UpdateSIPDispatchRule](https://docs.livekit.io/sip/api/#updatesipdispatchrule) [UpdateSIPInboundTrunk](https://docs.livekit.io/sip/api/#updatesipinboundtrunk) [UpdateSIPOutboundTrunk](https://docs.livekit.io/sip/api/#updatesipoutboundtrunk) [Types](https://docs.livekit.io/sip/api/#types) [GetSIPInboundTrunkResponse](https://docs.livekit.io/sip/api/#getsipinboundtrunkresponse) [GetSIPOutboundTrunkResponse](https://docs.livekit.io/sip/api/#getsipoutboundtrunkresponse) [SIPDispatchRule](https://docs.livekit.io/sip/api/#sipdispatchrule) [SIPHeaderOptions](https://docs.livekit.io/sip/api/#sipheaderoptions) [SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) [SIPDispatchRuleUpdate](https://docs.livekit.io/sip/api/#sipdispatchruleupdate) [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) [SIPInboundTrunkUpdate](https://docs.livekit.io/sip/api/#sipinboundtrunkupdate) [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) [SIPOutboundTrunkUpdate](https://docs.livekit.io/sip/api/#sipoutboundtrunkupdate) [SIPParticipantInfo](https://docs.livekit.io/sip/api/#sipparticipantinfo) [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) [SIPTransport](https://docs.livekit.io/sip/api/#siptransport) [SIPTrunkInfo](https://docs.livekit.io/sip/api/#siptrunkinfo) [TrunkKind](https://docs.livekit.io/sip/api/#trunkkind) [UpdateSIPDispatchRuleRequest](https://docs.livekit.io/sip/api/#updatesipdispatchrulerequest) [UpdateSIPInboundTrunkRequest](https://docs.livekit.io/sip/api/#updatesipinboundtrunkrequest) [UpdateSIPOutboundTrunkRequest](https://docs.livekit.io/sip/api/#updatesipoutboundtrunkrequest) Copy pageSee more page options Overview[](https://docs.livekit.io/sip/api/#overview) ------------------------------------------------------ LiveKit has built-in APIs that let you to manage SIP trunks, dispatch rules, and SIP participants. The SIP API is available with LiveKit server SDKs and CLI: * [Go SIP client](https://pkg.go.dev/github.com/livekit/server-sdk-go/v2#SIPClient) * [JS SIP client](https://docs.livekit.io/server-sdk-js/classes/SipClient.html) * [Ruby SIP client](https://github.com/livekit/server-sdk-ruby/blob/main/lib/livekit/sip_service_client.rb) * [Python SIP client](https://docs.livekit.io/reference/python/v1/livekit/api/sip_service.html) * [Java SIP client](https://github.com/livekit/server-sdk-kotlin/blob/main/src/main/kotlin/io/livekit/server/SipServiceClient.kt) * [CLI](https://github.com/livekit/livekit-cli/blob/main/cmd/lk/sip.go) **Important** Requests to the SIP API require the SIP `admin` permission unless otherwise noted. To create a token with the appropriate grant, see [SIP grant](https://docs.livekit.io/home/get-started/authentication/#sip-grant) . To learn more about additional APIs, see [Server APIs](https://docs.livekit.io/reference/server/server-apis/) . ### Using endpoints[](https://docs.livekit.io/sip/api/#using-endpoints) The SIP API is accessible via `/twirp/livekit.SIP/`. For example, if you're using LiveKit Cloud the following URL is for the [CreateSIPInboundTrunk](https://docs.livekit.io/sip/api/#createsipinoundtrunk) API endpoint: https:///twirp/livekit.SIP/CreateSIPInboundTrunk #### Authorization header[](https://docs.livekit.io/sip/api/#authorization-header) All endpoints require a signed access token. This token should be set via HTTP header: Authorization: Bearer LiveKit server SDKs automatically include the above header. #### Post body[](https://docs.livekit.io/sip/api/#post-body) Twirp expects an HTTP POST request. The body of the request must be a JSON object (`application/json`) containing parameters specific to that request. Use an empty `{}` body for requests that don't require parameters. #### Examples[](https://docs.livekit.io/sip/api/#examples) For example, create an inbound trunk using [CreateSIPInboundTrunk](https://docs.livekit.io/sip/api/#createsipinboundtrunk) : curl \-X POST https:///twirp/livekit.SIP/CreateSIPInboundTrunk \\ \-H "Authorization: Bearer " \\ \-H 'Content-Type: application/json' \\ \-d '{ "name": "My trunk", "numbers": \["+15105550100"\] }' List inbound trunks using [ListSIPInboundTrunk](https://docs.livekit.io/sip/api/#listsipinboundtrunk) API endpoint to list inbound trunks: curl \-X POST https:///twirp/livekit.SIP/ListSIPInboundTrunk \\ \-H "Authorization: Bearer " \\ \-H 'Content-Type: application/json' \\ \-d '{}' SIPService APIs[](https://docs.livekit.io/sip/api/#sipservice-apis) -------------------------------------------------------------------- The SIPService APIs allow you to manage trunks, dispatch rules, and SIP participants. **Tip** All RPC definitions and options can be found [here](https://github.com/livekit/protocol/blob/main/protobufs/livekit_sip.proto) . ### CreateSIPInboundTrunk[](https://docs.livekit.io/sip/api/#createsipinboundtrunk) Create an inbound trunk with the specified settings. Returns [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | name | string | yes | name of the trunk. | | metadata | string | | Initial metadata to assign to the trunk. This metadata is added to every SIP participant that uses the trunk. | | numbers | array | yes | Array of provider phone numbers associated with the trunk. | | allowed\_addresses | array | | List of IP addresses that are allowed to use the trunk. Each item in the list can be an individual IP address or a Classless Inter-Domain Routing notation representing a CIDR block. | | allowed\_numbers | array | | List of phone numbers that are allowed to use the trunk. | | auth\_username | string | | If configured, the username for authorized use of the provider's SIP trunk. | | auth\_password | string | | If configured, the password for authorized use of the provider's SIP trunk. | | headers | map | | SIP X-\* headers for INVITE request. These headers are sent as-is and may help identify this call as coming from LiveKit for the other SIP endpoint. | | headers\_to\_attributes | map | | Key-value mapping of SIP X-\* header names to participant attribute names. | | attributes\_to\_headers | map | | Map SIP headers from INVITE request to `sip.h.*` participant attributes. If the names of the required headers is known, use `headers_to_attributes` instead. | | include\_headers | [SIPHeaderOptions](https://docs.livekit.io/sip/api/#sipheaderoptions) | | Specify how SIP headers should be mapped to attributes. | | ringing\_timeout | [google.protobuf.Duration](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/duration.proto) | | Maximum time for the call to ring. | | max\_call\_duration | google.protobuf.Duration | | Maximum call duration. | | krisp\_enabled | bool | | True to enable [Krisp noise cancellation](https://docs.livekit.io/sip/#noise-cancellation-for-calls)
for the caller. | | media\_encryption | [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) | | Whether or not to encrypt media. | ### CreateSIPOutboundTrunk[](https://docs.livekit.io/sip/api/#createsipoutboundtrunk) Create an outbound trunk with the specified settings. Returns [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | name | string | yes | name of the trunk. | | metadata | string | | User-defined metadata for the trunk. This metadata is added to every SIP participant that uses the trunk. | | address | string | yes | Hostname or IP the SIP INVITE is sent to. This is _not_ a SIP URI and shouldn't contain the `sip:` protocol. | | destination\_country | string | yes | Two letter [country code](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2)
for the country the call terminates in. LiveKit uses the country code to route calls. To learn more, see [Restricting calls to a region](https://docs.livekit.io/sip/trunk-outbound/#region-pinning)
. | | numbers | array | yes | List of provider phone numbers associated with the trunk that can be used as a caller id. | | transport | [SIPTransport](https://docs.livekit.io/sip/api/#siptransport) | | Protocol to use for SIP transport: auto, TCP, or UDP. | | auth\_username | string | | If configured, the username for authorized use of the provider's SIP trunk. | | auth\_password | string | | If configured, the password for authorized use of the provider's SIP trunk. | | headers | map | | SIP X-\* headers for INVITE request. These headers are sent as-is and may help identify this call as coming from LiveKit for the other SIP endpoint. | | headers\_to\_attributes | map | | Key-value mapping of SIP X-\* header names to participant attribute names. | | media\_encryption | [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) | | Whether or not to encrypt media. | ### CreateSIPDispatchRule[](https://docs.livekit.io/sip/api/#createsipdispatchrule) Create dispatch rule. Returns [SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | dispatch\_rule | [SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) | yes | Dispatch rule to create. | | trunk\_ids | array | | List of associated trunk IDs. If empty, all trunks match this dispatch rule. | | hide\_phone\_number | bool | | If true, use a random value for participant identity and phone number ommitted from attributes. By default, the participant identity is created using the phone number (if the participant identity isn't explicitly set). | | inbound\_numbers | array | | If set, the dispatch rule only accepts calls made to numbers in the list. | | name | string | yes | Human-readable name for the dispatch rule. | | metadata | string | | Optional metadata for the dispatch rule. If defined, participants created by the rule inherit this metadata. | | attributes | map | | Key-value mapping of user-defined attributes. Participants created by this rule inherit these attributes. | | room\_preset | string | | Only for LiveKit Cloud: Config preset to use. | | room\_config | [RoomConfiguration](https://docs.livekit.io/reference/server/server-apis/#roomconfiguration) | | Room configuration to use if the participant initiates the room. | ### CreateSIPParticipant[](https://docs.livekit.io/sip/api/#createsipparticipant) **Note** Requires SIP `call` grant on authorization token. Create a SIP participant to make outgoing calls. Returns [SIPParticipantInfo](https://docs.livekit.io/sip/api/#sipparticipantinfo) | Parameter | Type | Required | Description | | --- | --- | --- | --- | | sip\_trunk\_id | string | yes | ID for SIP trunk used to dial user. | | sip\_call\_to | string | yes | Phone number to call. | | sip\_number | string | | SIP number to call from. If empty, use trunk number. | | room\_name | string | yes | Name of the room to connect the participant to. | | participant\_identity | string | | Identity of the participant. | | participant\_name | string | | Name of the participant. | | participant\_metadata | string | | User-defined metadata that is attached to created participant. | | participant\_attributes | map | | Key-value mapping of user-defined attributes to attach to created participant. | | dtmf | string | | DTMF digits (extension codes) to use when making the call. Use character `w` to add a 0.5 second delay. | | play\_dialtone | bool | | Optionally play dial tone in the room in the room as an audible indicator for existing participants. | | hide\_phone\_number | bool | | If true, use a random value for participant identity and phone number ommitted from attributes. By default, the participant identity is created using the phone number (if the participant identity isn't explicitly set). | | headers | map | | SIP X-\* headers for INVITE request. These headers are sent as-is and may help identify this call as coming from LiveKit. | | include\_headers | [SIPHeaderOptions](https://docs.livekit.io/sip/api/#sipheaderoptions) | | Specify how SIP headers should be mapped to attributes. | | ringing\_timeout | google.protobuf.Duration | | Maximum time for the callee to answer the call. | | max\_call\_duration | google.protobuf.Duration | | Maximum call duration. | | krisp\_enabled | bool | | True to enable [Krisp noise cancellation](https://docs.livekit.io/sip/#noise-cancellation-for-calls)
for the callee. | | media\_encryption | [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) | | Whether or not to encrypt media. | | wait\_until\_answered | bool | | If true, return after the call is answered — including if it goes to voicemail. | ### DeleteSIPDispatchRule[](https://docs.livekit.io/sip/api/#deletesipdispatchrule) Delete a dispatch rule. Returns [SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | sip\_dispatch\_rule\_id | string | | ID of dispatch rule. | ### DeleteSIPTrunk[](https://docs.livekit.io/sip/api/#deletesiptrunk) Delete a trunk. Returns [SIPTrunkInfo](https://docs.livekit.io/sip/api/#siptrunkinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | sip\_trunk\_id | string | yes | ID of trunk. | ### GetSIPInboundTrunk[](https://docs.livekit.io/sip/api/#getsipinboundtrunk) Get inbound trunk. Returns [GetSIPInboundTrunkResponse](https://docs.livekit.io/sip/api/#getsipinboundtrunkresponse) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | sip\_trunk\_id | string | yes | ID of trunk. | ### GetSIPOutboundTrunk[](https://docs.livekit.io/sip/api/#getsipoutboundtrunk) Get outbound trunk. Returns [GetSIPOutboundTrunkResponse](https://docs.livekit.io/sip/api/#getsipoutboundtrunkresponse) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | sip\_trunk\_id | string | yes | ID of trunk. | ### ListSIPDispatchRule[](https://docs.livekit.io/sip/api/#listsipdispatchrule) List dispatch rules. Returns array<[SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) \>. ### ListSIPInboundTrunk[](https://docs.livekit.io/sip/api/#listsipinboundtrunk) List inbound trunks. Returns array<[SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) \>. ### ListSIPOutboundTrunk[](https://docs.livekit.io/sip/api/#listsipoutboundtrunk) List outbound trunks. Returns array<[SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) \>. ### TransferSIPParticipant[](https://docs.livekit.io/sip/api/#transfersipparticipant) **Note** Requires SIP `call` grant on authorization token. Transfer call to another number or SIP endpoint. Returns [google.protobuf.Empty](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/empty.proto) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | participant\_identity | string | yes | Identity of the participant to transfer. | | room\_name | string | yes | Name of the room the participant is currently in. | | transfer\_to | string | yes | Phone number or SIP endpoint to transfer participant to. | | play\_dialtone | bool | | Optionally play dial tone during the transfer. By default, the room audio is played during the transfer. | ### UpdateSIPDispatchRule[](https://docs.livekit.io/sip/api/#updatesipdispatchrule) Update a dispatch rule. Returns [SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | req | [UpdateSIPDispatchRuleRequest](https://docs.livekit.io/sip/api/#updatesipdispatchrulerequest) | yes | Update or replace request. | ### UpdateSIPInboundTrunk[](https://docs.livekit.io/sip/api/#updatesipinboundtrunk) Update an inbound trunk. Returns [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | req | [UpdateSIPInboundTrunkRequest](https://docs.livekit.io/sip/api/#updatesipinboundtrunkrequest) | yes | Update or replace request. | ### UpdateSIPOutboundTrunk[](https://docs.livekit.io/sip/api/#updatesipoutboundtrunk) Update an outbound trunk. Returns [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) . | Parameter | Type | Required | Description | | --- | --- | --- | --- | | req | [UpdateSIPOutboundTrunkRequest](https://docs.livekit.io/sip/api/#updatesipoutboundtrunkrequest) | yes | Update or replace request. | Types[](https://docs.livekit.io/sip/api/#types) ------------------------------------------------ The SIP service includes the following types. ### GetSIPInboundTrunkResponse[](https://docs.livekit.io/sip/api/#getsipinboundtrunkresponse) | Field | Type | Description | | --- | --- | --- | | trunk | [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) | Inbound trunk. | ### GetSIPOutboundTrunkResponse[](https://docs.livekit.io/sip/api/#getsipoutboundtrunkresponse) | Field | Type | Description | | --- | --- | --- | | trunk | [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) | Outbound trunk. | ### SIPDispatchRule[](https://docs.livekit.io/sip/api/#sipdispatchrule) Valid values include: | Name | Type | Value | Description | | --- | --- | --- | --- | | dispatch\_rule\_direct | SIPDispatchRuleDirect | 1 | Dispatches callers into an existing room. You can optionally require a pin before caller enters the room. | | dispatch\_rule\_individual | SIPDispatchRuleIndividual | 2 | Creates a new room for each caller. | | dispatch\_rule\_callee | SIPDispatchRuleCallee | 3 | Creates a new room for each callee. | ### SIPHeaderOptions[](https://docs.livekit.io/sip/api/#sipheaderoptions) Enum. Valid values are as follows: | Name | Value | Description | | --- | --- | --- | | SIP\_NO\_HEADERS | 0 | Don't map any headers except those mapped explicitly. | | SIP\_X\_HEADERS | 1 | Map all `X-*` headers to `sip.h.*` attributes. | | SIP\_ALL\_HEADERS | 2 | Map all headers to `sip.h.*` attributes. | ### SIPDispatchRuleInfo[](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) | Field | Type | Description | | --- | --- | --- | | sip\_dispatch\_rule\_id | string | Dispatch rule ID. | | rule | [SIPDispatchRule](https://docs.livekit.io/sip/api/#sipdispatchrule) | Type of dispatch rule. | | trunk\_ids | array | List of associated trunk IDs. | | hide\_phone\_number | bool | If true, hides phone number. | | inbound\_numbers | array | If this list is included, the dispatch rule only accepts calls made to the numbers in the list. | | name | string | Human-readable name for the dispatch rule. | | metadata | string | User-defined metadata for the dispatch rule. Participants created by this rule inherit this metadata. | | headers | map | Custom SIP X-\* headers to included in the 200 OK response. | | attributes | map | Key-value mapping of user-defined attributes. Participants created by this rule inherit these attributes. | | room\_preset | string | Only for LiveKit Cloud: Config preset to use. | | room\_config | [RoomConfiguration](https://docs.livekit.io/reference/server/server-apis/#roomconfiguration) | Room configuration object associated with the dispatch rule. | ### SIPDispatchRuleUpdate[](https://docs.livekit.io/sip/api/#sipdispatchruleupdate) | Field | Type | Description | | --- | --- | --- | | trunk\_ids | array | List of trunk IDs to associate with the dispatch rule. | | rule | [SIPDispatchRule](https://docs.livekit.io/sip/api/#sipdispatchrule) | Type of dispatch rule. | | name | string | Human-readable name for the dispatch rule. | | metadata | string | User-defined metadata for the dispatch rule. Participants created by this rule inherit this metadata. | | attributes | map | Key-value mapping of user-defined attributes. Participants created by this rule inherit these attributes. | | media\_encryption | [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) | Whether or not to encrypt media. | ### SIPInboundTrunkInfo[](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) | Field | Type | Description | | --- | --- | --- | | sip\_trunk\_id | string | Trunk ID | | name | string | Human-readable name for the trunk. | | numbers | array | Phone numbers associated with the trunk. The trunk only accepts calls made to the phone numbers in the list. | | allowed\_addresses | array | IP addresses or CIDR blocks that are allowed to use the trunk. If this list is populated, the trunk only accepts traffic from the IP addresses in the list. | | allowed\_numbers | array | Phone numbers that are allowed to dial in. If this list is populated, the trunk only accepts calls from the numbers in the list. | | auth\_username | string | Username used to authenticate inbound SIP invites. | | auth\_password | string | Password used to authenticate inbound SIP invites. | | headers | map | Custom SIP X-\* headers to included in the 200 OK response. | | headers\_to\_attributes | map | Custom SIP X-\* headers that map to SIP participant attributes. | | ringing\_timeout | [google.protobuf.Duration](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/duration.proto) | Maximum time for the caller to wait for track subscription (that is, for the call to be picked up). | | max\_call\_duration | google.protobuf.Duration | Maximum call duration. | | krisp\_enabled | Boolean | True if Krisp noise cancellation is enabled for the call. | ### SIPInboundTrunkUpdate[](https://docs.livekit.io/sip/api/#sipinboundtrunkupdate) | Field | Type | Description | | --- | --- | --- | | numbers | list\[\] | List of phone numbers associated with the trunk. | | allowed\_addresses | list\[\] | List of IP addresses or CIDR blocks that are allowed to use the trunk. | | allowed\_numbers | list\[\] | List of phone numbers that are allowed to use the trunk. | | auth\_username | string | Username used to authenticate inbound SIP invites. | | auth\_password | string | Password used to authenticate inbound SIP invites. | | name | string | Human-readable name for the trunk. | | metadata | string | User-defined metadata for the trunk. | | media\_encryption | [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) | Whether or not to encrypt media. | ### SIPOutboundTrunkInfo[](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) | Field | Type | Description | | --- | --- | --- | | sip\_trunk\_id | string | Trunk ID. | | name | string | Trunk name. | | metadata | string | User-defined metadata for trunk. | | address | string | Hostname or IP address the SIP request message (SIP INVITE) is sent to. | | destination\_country | string | Two letter [country code](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2)
for the country the call terminates in. LiveKit uses the country code to route calls. To learn more, see [Restricting calls to a region](https://docs.livekit.io/sip/trunk-outbound/#region-pinning)
. | | transport | [SIPTransport](https://docs.livekit.io/sip/api/#siptransport) | Protocol to use for SIP transport: auto, TCP, or UDP. | | numbers | array | Phone numbers used to make calls. A random number in the list is selected whenever a call is made. | | auth\_username | string | Username used to authenticate with the SIP server. | | auth\_password | string | Password used to authenticate with the SIP server. | | headers | map | Custom SIP X-\* headers to included in the 200 OK response. | | headers\_to\_attributes | map | Custom SIP X-\* headers that map to SIP participant attributes. | ### SIPOutboundTrunkUpdate[](https://docs.livekit.io/sip/api/#sipoutboundtrunkupdate) | Field | Type | Description | | --- | --- | --- | | address | string | Hostname or IP address the SIP request message (SIP INVITE) is sent to. | | transport | [SIPTransport](https://docs.livekit.io/sip/api/#siptransport) | Protocol to use for SIP transport: auto, TCP, or UDP. | | destination\_country | string | Two letter [country code](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2)
for the country the call terminates in. LiveKit uses the country code to route calls. To learn more, see [Restricting calls to a region](https://docs.livekit.io/sip/trunk-outbound/#region-pinning)
. | | numbers | array | Phone numbers used to make calls. A random number in the list is selected whenever a call is made. | | auth\_username | string | Username used to authenticate with the SIP server. | | auth\_password | string | Password used to authenticate with the SIP server. | | name | string | Human-readable name for the trunk. | | metadata | string | User-defined metadata for the trunk. | | media\_encryption | [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) | Whether or not to encrypt media. | ### SIPParticipantInfo[](https://docs.livekit.io/sip/api/#sipparticipantinfo) | Field | Type | Description | | --- | --- | --- | | participant\_id | string | Participant ID. | | participant\_identity | string | Participant name. | | room\_name | string | Name of the room. | | sip\_call\_id | string | SIP call ID. | ### SIPMediaEncryption[](https://docs.livekit.io/sip/api/#sipmediaencryption) Enum. Valid values are as follows: | Name | Value | Description | | --- | --- | --- | | SIP\_MEDIA\_ENCRYPT\_DISABLE | 0 | Don't turn on encryption. | | SIP\_MEDIA\_ENCRYPT\_ALLOW | 1 | Use encryption if available. | | SIP\_MEDIA\_ENCRYPT\_REQUIRE | 2 | Require encryption. | ### SIPTransport[](https://docs.livekit.io/sip/api/#siptransport) Enum. Valid values are as follows: | Name | Value | Description | | --- | --- | --- | | SIP\_TRANSPORT\_AUTO | 0 | Detect automatically. | | SIP\_TRANSPORT\_UDP | 1 | UDP | | SIP\_TRANSPORT\_TCP | 2 | TCP | | SIP\_TRANSPORT\_TLS | 3 | TLS | ### SIPTrunkInfo[](https://docs.livekit.io/sip/api/#siptrunkinfo) **Note** This type is deprecated. See [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) and [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) . | Field | Type | Description | | --- | --- | --- | | sip\_trunk\_id | string | Trunk ID. | | kind | [TrunkKind](https://docs.livekit.io/sip/api/#trunkkind) | Type of trunk. | | inbound\_addresses | array | IP addresses or CIDR blocks that are allowed to use the trunk. If this list is populated, the trunk only accepts traffic from the IP addresses in the list. | | outbound\_address | string | IP address that the SIP INVITE is sent to. | | outbound\_number | string | Phone number used to make outbound calls. | | transport | [SIPTransport](https://docs.livekit.io/sip/api/#siptransport) | Protocol to use for SIP transport: auto, TCP, or UDP. | | inbound\_numbers | array | If this list is populated, the trunk only accepts calls to the numbers in this list. | | inbound\_username | string | Username used to authenticate inbound SIP invites. | | inbound\_password | string | Password used to authenticate inbound SIP invites. | | outbound\_username | string | Username used to authenticate outbound SIP invites. | | outbound\_password | string | Password used to authenticate outbound SIP invites. | | name | string | Trunk name. | | metadata | string | Initial metadata to assign to the trunk. This metadata is added to every SIP participant that uses the trunk. | ### TrunkKind[](https://docs.livekit.io/sip/api/#trunkkind) Enum. Valid values are as follows: | Name | Value | Description | | --- | --- | --- | | TRUNK\_LEGACY | 0 | Legacy trunk. | | TRUNK\_INBOUND | 1 | [Inbound trunk](https://docs.livekit.io/sip/trunk-inbound/)
. | | TRUNK\_OUTBOUND | 2 | [Outbound trunk](https://docs.livekit.io/sip/trunk-outbound/)
. | ### UpdateSIPDispatchRuleRequest[](https://docs.livekit.io/sip/api/#updatesipdispatchrulerequest) | Field | Type | Description | | --- | --- | --- | | sip\_dispatch\_rule\_id | string | Dispatch rule ID. | | action | [SIPDispatchRule](https://docs.livekit.io/sip/api/#sipdispatchrule)
\| [SIPDispatchRuleUpdate](https://docs.livekit.io/sip/api/#sipdispatchruleupdate) | Dispatch rule for replacement or update. | ### UpdateSIPInboundTrunkRequest[](https://docs.livekit.io/sip/api/#updatesipinboundtrunkrequest) | Field | Type | Description | | --- | --- | --- | | sip\_trunk\_id | string | Trunk ID. | | action | [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo)
\| [SIPInboundTrunkUpdate](https://docs.livekit.io/sip/api/#sipinboundtrunkupdate) | Trunk info for replacement or update. | ### UpdateSIPOutboundTrunkRequest[](https://docs.livekit.io/sip/api/#updatesipoutboundtrunkrequest) | Field | Type | Description | | --- | --- | --- | | sip\_trunk\_id | string | Trunk ID. | | action | [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo)
\| [SIPOutboundTrunkUpdate](https://docs.livekit.io/sip/api/#sipoutboundtrunkupdate) | Trunk info for replacement or update. | On this page [Overview](https://docs.livekit.io/sip/api/#overview) [Using endpoints](https://docs.livekit.io/sip/api/#using-endpoints) [SIPService APIs](https://docs.livekit.io/sip/api/#sipservice-apis) [CreateSIPInboundTrunk](https://docs.livekit.io/sip/api/#createsipinboundtrunk) [CreateSIPOutboundTrunk](https://docs.livekit.io/sip/api/#createsipoutboundtrunk) [CreateSIPDispatchRule](https://docs.livekit.io/sip/api/#createsipdispatchrule) [CreateSIPParticipant](https://docs.livekit.io/sip/api/#createsipparticipant) [DeleteSIPDispatchRule](https://docs.livekit.io/sip/api/#deletesipdispatchrule) [DeleteSIPTrunk](https://docs.livekit.io/sip/api/#deletesiptrunk) [GetSIPInboundTrunk](https://docs.livekit.io/sip/api/#getsipinboundtrunk) [GetSIPOutboundTrunk](https://docs.livekit.io/sip/api/#getsipoutboundtrunk) [ListSIPDispatchRule](https://docs.livekit.io/sip/api/#listsipdispatchrule) [ListSIPInboundTrunk](https://docs.livekit.io/sip/api/#listsipinboundtrunk) [ListSIPOutboundTrunk](https://docs.livekit.io/sip/api/#listsipoutboundtrunk) [TransferSIPParticipant](https://docs.livekit.io/sip/api/#transfersipparticipant) [UpdateSIPDispatchRule](https://docs.livekit.io/sip/api/#updatesipdispatchrule) [UpdateSIPInboundTrunk](https://docs.livekit.io/sip/api/#updatesipinboundtrunk) [UpdateSIPOutboundTrunk](https://docs.livekit.io/sip/api/#updatesipoutboundtrunk) [Types](https://docs.livekit.io/sip/api/#types) [GetSIPInboundTrunkResponse](https://docs.livekit.io/sip/api/#getsipinboundtrunkresponse) [GetSIPOutboundTrunkResponse](https://docs.livekit.io/sip/api/#getsipoutboundtrunkresponse) [SIPDispatchRule](https://docs.livekit.io/sip/api/#sipdispatchrule) [SIPHeaderOptions](https://docs.livekit.io/sip/api/#sipheaderoptions) [SIPDispatchRuleInfo](https://docs.livekit.io/sip/api/#sipdispatchruleinfo) [SIPDispatchRuleUpdate](https://docs.livekit.io/sip/api/#sipdispatchruleupdate) [SIPInboundTrunkInfo](https://docs.livekit.io/sip/api/#sipinboundtrunkinfo) [SIPInboundTrunkUpdate](https://docs.livekit.io/sip/api/#sipinboundtrunkupdate) [SIPOutboundTrunkInfo](https://docs.livekit.io/sip/api/#sipoutboundtrunkinfo) [SIPOutboundTrunkUpdate](https://docs.livekit.io/sip/api/#sipoutboundtrunkupdate) [SIPParticipantInfo](https://docs.livekit.io/sip/api/#sipparticipantinfo) [SIPMediaEncryption](https://docs.livekit.io/sip/api/#sipmediaencryption) [SIPTransport](https://docs.livekit.io/sip/api/#siptransport) [SIPTrunkInfo](https://docs.livekit.io/sip/api/#siptrunkinfo) [TrunkKind](https://docs.livekit.io/sip/api/#trunkkind) [UpdateSIPDispatchRuleRequest](https://docs.livekit.io/sip/api/#updatesipdispatchrulerequest) [UpdateSIPInboundTrunkRequest](https://docs.livekit.io/sip/api/#updatesipinboundtrunkrequest) [UpdateSIPOutboundTrunkRequest](https://docs.livekit.io/sip/api/#updatesipoutboundtrunkrequest) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/sip/api/) Search --- # Android Components | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/components/android/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/android/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Installation](https://docs.livekit.io/reference/components/android/#installation) Copy pageSee more page options Installation[](https://docs.livekit.io/reference/components/android/#installation) ----------------------------------------------------------------------------------- GroovyKotlin In your app's `build.gradle` file: dependencies { implementation "io.livekit:livekit-android-compose-components:" } See our [releases page](https://github.com/livekit/components-android/releases) for information on the latest version of the SDK. You'll also need JitPack as one of your repositories. GroovyKotlin In your root project's `settings.gradle` file: dependencyResolutionManagement { repositories { google() mavenCentral() maven { url 'https://jitpack.io' } } } On this page [Installation](https://docs.livekit.io/reference/components/android/#installation) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/android/) Search --- # LiveKit JS Client SDK - v2.15.4 LiveKit JS Client SDK - v2.15.4 =============================== ![The LiveKit icon, the name of the repository and some sample code in the background.](https://raw.githubusercontent.com/livekit/client-sdk-js/main/.github/banner_light.png) JavaScript/TypeScript client SDK for LiveKit[](https://docs.livekit.io/reference/client-sdk-js/#javascripttypescript-client-sdk-for-livekit) ============================================================================================================================================= Use this SDK to add realtime video, audio and data features to your JavaScript/TypeScript app. By connecting to [LiveKit](https://livekit.io/) Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI, live streaming, or video calls with just a few lines of code. Docs[](https://docs.livekit.io/reference/client-sdk-js/#docs) -------------------------------------------------------------- Docs and guides at [https://docs.livekit.io](https://docs.livekit.io/) [SDK reference](https://docs.livekit.io/client-sdk-js/) Note This is v2 of `livekit-client`. When migrating from v1.x to v2.x you might encounter a small set of breaking changes. Read the [migration guide](https://docs.livekit.io/recipes/migrate-from-v1/) for a detailed overview of what has changed. Installation[](https://docs.livekit.io/reference/client-sdk-js/#installation) ------------------------------------------------------------------------------ ### Yarn[](https://docs.livekit.io/reference/client-sdk-js/#yarn) yarn add livekit-client ### NPM[](https://docs.livekit.io/reference/client-sdk-js/#npm) npm install livekit-client --save ### Minified JS[](https://docs.livekit.io/reference/client-sdk-js/#minified-js) To use the SDK without a package manager, you can include it with a script tag: The module will be exported under `LivekitClient` in the global namespace. When accessing symbols from the class, you'd need to prefix them with `LivekitClient.`. For example, `Room` becomes `LivekitClient.Room`. Usage[](https://docs.livekit.io/reference/client-sdk-js/#usage) ---------------------------------------------------------------- Examples below are in TypeScript, if using JS/CommonJS imports replace import with: const livekit = require('livekit-client');const room = new livekit.Room(...);// call this some time before actually connecting to speed up the actual connectionroom.prepareConnection(url, token);await room.connect(...); ### Connecting to a room, publish video & audio[](https://docs.livekit.io/reference/client-sdk-js/#connecting-to-a-room-publish-video--audio) import { LocalParticipant, LocalTrackPublication, Participant, RemoteParticipant, RemoteTrack, RemoteTrackPublication, Room, RoomEvent, Track, VideoPresets,} from 'livekit-client';// creates a new room with optionsconst room = new Room({ // automatically manage subscribed video quality adaptiveStream: true, // optimize publishing bandwidth and CPU for published tracks dynacast: true, // default capture settings videoCaptureDefaults: { resolution: VideoPresets.h720.resolution, },});// pre-warm connection, this can be called as early as your page is loadedroom.prepareConnection(url, token);// set up event listenersroom .on(RoomEvent.TrackSubscribed, handleTrackSubscribed) .on(RoomEvent.TrackUnsubscribed, handleTrackUnsubscribed) .on(RoomEvent.ActiveSpeakersChanged, handleActiveSpeakerChange) .on(RoomEvent.Disconnected, handleDisconnect) .on(RoomEvent.LocalTrackUnpublished, handleLocalTrackUnpublished);// connect to roomawait room.connect('ws://localhost:7800', token);console.log('connected to room', room.name);// publish local camera and mic tracksawait room.localParticipant.enableCameraAndMicrophone();function handleTrackSubscribed( track: RemoteTrack, publication: RemoteTrackPublication, participant: RemoteParticipant,) { if (track.kind === Track.Kind.Video || track.kind === Track.Kind.Audio) { // attach it to a new HTMLVideoElement or HTMLAudioElement const element = track.attach(); parentElement.appendChild(element); }}function handleTrackUnsubscribed( track: RemoteTrack, publication: RemoteTrackPublication, participant: RemoteParticipant,) { // remove tracks from all attached elements track.detach();}function handleLocalTrackUnpublished( publication: LocalTrackPublication, participant: LocalParticipant,) { // when local tracks are ended, update UI to remove them from rendering publication.track.detach();}function handleActiveSpeakerChange(speakers: Participant[]) { // show UI indicators when participant is speaking}function handleDisconnect() { console.log('disconnected from room');} In order to connect to a room, you need to first create an access token. See [authentication docs](https://docs.livekit.io/home/get-started/authentication/) for details ### Handling common track types[](https://docs.livekit.io/reference/client-sdk-js/#handling-common-track-types) While LiveKit is designed to be flexible, we've added a few shortcuts that makes working with common track types simple. For a user's camera, microphone, and screen share, you can enable them with the following `LocalParticipant` methods: const p = room.localParticipant;// turn on the local user's camera and mic, this may trigger a browser prompt// to ensure permissions are grantedawait p.setCameraEnabled(true);await p.setMicrophoneEnabled(true);// start sharing the user's screen, this will trigger a browser prompt to select// the screen to share.await p.setScreenShareEnabled(true);// disable camera to mute them, when muted, the user's camera indicator will be turned offawait p.setCameraEnabled(false); Similarly, you can access these common track types on the other participants' end. // get a RemoteParticipant by their identityconst p = room.remoteParticipants.get('participant-identity');if (p) { // if the other user has enabled their camera, attach it to a new HTMLVideoElement if (p.isCameraEnabled) { const publication = p.getTrackPublication(Track.Source.Camera); if (publication?.isSubscribed) { const videoElement = publication.videoTrack?.attach(); // do something with the element } }} ### Creating a track prior to creating a room[](https://docs.livekit.io/reference/client-sdk-js/#creating-a-track-prior-to-creating-a-room) In some cases, it may be useful to create a track before creating a room. For example, when building a staging area so the user may check their own camera. You can use our global track creation functions for this: const tracks = await createLocalTracks({ audio: true, video: true,}); ### Publish tracks from any source[](https://docs.livekit.io/reference/client-sdk-js/#publish-tracks-from-any-source) LiveKit lets you publish any track as long as it can be represented by a MediaStreamTrack. You can specify a name on the track in order to identify it later. const pub = await room.localParticipant.publishTrack(mediaStreamTrack, { name: 'mytrack', simulcast: true, // if this should be treated like a camera feed, tag it as such // supported known sources are .Camera, .Microphone, .ScreenShare source: Track.Source.Camera,});// you may mute or unpublish the track laterpub.setMuted(true);room.localParticipant.unpublishTrack(mediaStreamTrack); ### Device management APIs[](https://docs.livekit.io/reference/client-sdk-js/#device-management-apis) Users may have multiple input and output devices available. LiveKit will automatically use the one that's deemed as the `default` device on the system. You may also list and specify an alternative device to use. We use the same deviceId as one returned by [MediaDevices.enumerateDevices()](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/enumerateDevices) . #### Example listing and selecting a camera device[](https://docs.livekit.io/reference/client-sdk-js/#example-listing-and-selecting-a-camera-device) // list all microphone devicesconst devices = await Room.getLocalDevices('audioinput');// select last deviceconst device = devices[devices.length - 1];// in the current room, switch to the selected device and set// it as default audioinput in the future.await room.switchActiveDevice('audioinput', device.deviceId); You can also switch devices given a constraint. This could be useful on mobile devices to switch to a back-facing camera: await videoTrack.restartTrack({ facingMode: 'environment',}); #### Handling device failures[](https://docs.livekit.io/reference/client-sdk-js/#handling-device-failures) When creating tracks using LiveKit APIs (`connect`, `createLocalTracks`, `setCameraEnabled`, etc), it's possible to encounter errors with the underlying media device. In those cases, LiveKit will emit `RoomEvent.MediaDevicesError`. You can use the helper `MediaDeviceFailure.getFailure(error)` to determine specific reason for the error. * `PermissionDenied` - the user disallowed capturing devices * `NotFound` - the particular device isn't available * `DeviceInUse` - device is in use by another process (happens on Windows) These distinctions enables you to provide more specific messaging to the user. You could also retrieve the last error with `LocalParticipant.lastCameraError` and `LocalParticipant.lastMicrophoneError`. ### Audio playback[](https://docs.livekit.io/reference/client-sdk-js/#audio-playback) Browsers can be restrictive with regards to audio playback that is not initiated by user interaction. What each browser considers as user interaction can vary by vendor (for example, Safari on iOS is very restrictive). LiveKit will attempt to autoplay all audio tracks when you attach them to audio elements. However, if that fails, we'll notify you via `RoomEvent.AudioPlaybackStatusChanged`. `Room.canPlaybackAudio` will indicate if audio playback is permitted. LiveKit takes an optimistic approach so it's possible for this value to change from `true` to `false` when we encounter a browser error. In the case user interaction is required, LiveKit provides `Room.startAudio` to start audio playback. This function must be triggered in an onclick or ontap event handler. In the same session, once audio playback is successful, additional audio tracks can be played without further user interactions. room.on(RoomEvent.AudioPlaybackStatusChanged, () => { if (!room.canPlaybackAudio) { // UI is necessary. ... button.onclick = () => { // startAudio *must* be called in an click/tap handler. room.startAudio().then(() => { // successful, UI can be removed now button.remove(); }); } }}); ### Configuring logging[](https://docs.livekit.io/reference/client-sdk-js/#configuring-logging) This library uses [loglevel](https://github.com/pimterry/loglevel) for its internal logs. You can change the effective log level with the `logLevel` field in `ConnectOptions`. The method `setLogExtension` allows to hook into the livekit internal logs and send them to some third party logging service setLogExtension((level: LogLevel, msg: string, context: object) => { const enhancedContext = { ...context, timeStamp: Date.now() }; if (level >= LogLevel.debug) { console.log(level, msg, enhancedContext); }}); ### RPC[](https://docs.livekit.io/reference/client-sdk-js/#rpc) Perform your own predefined method calls from one participant to another. This feature is especially powerful when used with [Agents](https://docs.livekit.io/agents) , for instance to forward LLM function calls to your client application. #### Registering an RPC method[](https://docs.livekit.io/reference/client-sdk-js/#registering-an-rpc-method) The participant who implements the method and will receive its calls must first register support: room.localParticipant?.registerRpcMethod( // method name - can be any string that makes sense for your application 'greet', // method handler - will be called when the method is invoked by a RemoteParticipant async (data: RpcInvocationData) => { console.log(`Received greeting from ${data.callerIdentity}: ${data.payload}`); return `Hello, ${data.callerIdentity}!`; },); In addition to the payload, your handler will also receive `responseTimeout`, which informs you the maximum time available to return a response. If you are unable to respond in time, the call will result in an error on the caller's side. #### Performing an RPC request[](https://docs.livekit.io/reference/client-sdk-js/#performing-an-rpc-request) The caller may then initiate an RPC call like so: try { const response = await room.localParticipant!.performRpc({ destinationIdentity: 'recipient-identity', method: 'greet', payload: 'Hello from RPC!', }); console.log('RPC response:', response);} catch (error) { console.error('RPC call failed:', error);} You may find it useful to adjust the `responseTimeout` parameter, which indicates the amount of time you will wait for a response. We recommend keeping this value as low as possible while still satisfying the constraints of your application. #### Errors[](https://docs.livekit.io/reference/client-sdk-js/#errors) LiveKit is a dynamic realtime environment and calls can fail for various reasons. You may throw errors of the type `RpcError` with a string `message` in an RPC method handler and they will be received on the caller's side with the message intact. Other errors will not be transmitted and will instead arrive to the caller as `1500` ("Application Error"). Other built-in errors are detailed in `RpcError`. Error Codes[](https://docs.livekit.io/reference/client-sdk-js/#error-codes) ---------------------------------------------------------------------------- | Code | Name | Reason | | --- | --- | --- | | 1 | `ConnectionError` | 0: `NotAllowed`
1: `ServerUnreachable`
2: `InternalError`
3: `Cancelled`
4:`LeaveRequest` | | 10 | `UnsupportedServer` | | | 12 | `UnexpectedConnectionState` | | | 13 | `NegotiationError` | | | 14 | `PublishDataError` | | | 15 | `SignalRequestError` | | | 20 | `TrackInvalidError` | | | 21 | `DeviceUnsupportedError` | | | 40 | `CryptorError` | | Examples[](https://docs.livekit.io/reference/client-sdk-js/#examples) ---------------------------------------------------------------------- ### Demo App[](https://docs.livekit.io/reference/client-sdk-js/#demo-app) [examples/demo](https://docs.livekit.io/reference/client-sdk-js/media/demo) contains a demo webapp that uses the SDK. Run it with `pnpm install && pnpm examples:demo` ### RPC Demo[](https://docs.livekit.io/reference/client-sdk-js/#rpc-demo) [examples/rpc](https://docs.livekit.io/reference/client-sdk-js/media/rpc) contains a demo webapp that uses the SDK to showcase the RPC capabilities. Run it with `pnpm install && pnpm dev` from the `examples/rpc` directory. Browser Support[](https://docs.livekit.io/reference/client-sdk-js/#browser-support) ------------------------------------------------------------------------------------ | Browser | Desktop OS | Mobile OS | | --- | --- | --- | | Chrome | Windows, macOS, Linux | Android | | Firefox | Windows, macOS, Linux | Android | | Safari | macOS | iOS | | Edge (Chromium) | Windows, macOS | | We aim to support a broad range of browser versions by transpiling the library code with babel. You can have a look at the `"browerslist"` section of `package.json` for more details. > Note that the library requires some specific browser APIs to be present. You can check general compatibility with the helper function `isBrowserSupported()`. Support for more modern features like adaptiveStream and dynacast can be checked for with `supportsAdaptiveStream()` and `supportsDynacast()`. If you are targeting legacy browsers, but still want adaptiveStream functionality you'll likely need to use polyfills for [ResizeObserver](https://www.npmjs.com/package/resize-observer-polyfill) and [IntersectionObserver](https://www.npmjs.com/package/intersection-observer) . Also when targeting legacy browsers, older than the ones specified in our browserslist target, make sure to transpile the library code to your desired target and include required polyfills with babel and/or corejs. | LiveKit Ecosystem | | | --- | --- | | LiveKit SDKs | **Browser** · [iOS/macOS/visionOS](https://github.com/livekit/client-sdk-swift)
· [Android](https://github.com/livekit/client-sdk-android)
· [Flutter](https://github.com/livekit/client-sdk-flutter)
· [React Native](https://github.com/livekit/client-sdk-react-native)
· [Rust](https://github.com/livekit/rust-sdks)
· [Node.js](https://github.com/livekit/node-sdks)
· [Python](https://github.com/livekit/python-sdks)
· [Unity](https://github.com/livekit/client-sdk-unity)
· [Unity (WebGL)](https://github.com/livekit/client-sdk-unity-web)
· [ESP32](https://github.com/livekit/client-sdk-esp32) | | Server APIs | [Node.js](https://github.com/livekit/node-sdks)
· [Golang](https://github.com/livekit/server-sdk-go)
· [Ruby](https://github.com/livekit/server-sdk-ruby)
· [Java/Kotlin](https://github.com/livekit/server-sdk-kotlin)
· [Python](https://github.com/livekit/python-sdks)
· [Rust](https://github.com/livekit/rust-sdks)
· [PHP (community)](https://github.com/agence104/livekit-server-sdk-php)
· [.NET (community)](https://github.com/pabloFuente/livekit-server-sdk-dotnet) | | UI Components | [React](https://github.com/livekit/components-js)
· [Android Compose](https://github.com/livekit/components-android)
· [SwiftUI](https://github.com/livekit/components-swift)
· [Flutter](https://github.com/livekit/components-flutter) | | Agents Frameworks | [Python](https://github.com/livekit/agents)
· [Node.js](https://github.com/livekit/agents-js)
· [Playground](https://github.com/livekit/agent-playground) | | Services | [LiveKit server](https://github.com/livekit/livekit)
· [Egress](https://github.com/livekit/egress)
· [Ingress](https://github.com/livekit/ingress)
· [SIP](https://github.com/livekit/sip) | | Resources | [Docs](https://docs.livekit.io/)
· [Example apps](https://github.com/livekit-examples)
· [Cloud](https://livekit.io/cloud)
· [Self-hosting](https://docs.livekit.io/home/self-hosting/deployment)
· [CLI](https://github.com/livekit/livekit-cli) | ### Settings Member Visibility * Inherited ThemeOSLightDark ### On This Page [JavaScript/TypeScript client SDK for LiveKit](https://docs.livekit.io/reference/client-sdk-js/#javascripttypescript-client-sdk-for-livekit) * [Docs](https://docs.livekit.io/reference/client-sdk-js/#docs) * [Installation](https://docs.livekit.io/reference/client-sdk-js/#installation) * * [Yarn](https://docs.livekit.io/reference/client-sdk-js/#yarn) * [NPM](https://docs.livekit.io/reference/client-sdk-js/#npm) * [Minified JS](https://docs.livekit.io/reference/client-sdk-js/#minified-js) * [Usage](https://docs.livekit.io/reference/client-sdk-js/#usage) * * [Connecting to a room, publish video & audio](https://docs.livekit.io/reference/client-sdk-js/#connecting-to-a-room-publish-video--audio) * [Handling common track types](https://docs.livekit.io/reference/client-sdk-js/#handling-common-track-types) * [Creating a track prior to creating a room](https://docs.livekit.io/reference/client-sdk-js/#creating-a-track-prior-to-creating-a-room) * [Publish tracks from any source](https://docs.livekit.io/reference/client-sdk-js/#publish-tracks-from-any-source) * [Device management APIs](https://docs.livekit.io/reference/client-sdk-js/#device-management-apis) * * [Example listing and selecting a camera device](https://docs.livekit.io/reference/client-sdk-js/#example-listing-and-selecting-a-camera-device) * [Handling device failures](https://docs.livekit.io/reference/client-sdk-js/#handling-device-failures) * [Audio playback](https://docs.livekit.io/reference/client-sdk-js/#audio-playback) * [Configuring logging](https://docs.livekit.io/reference/client-sdk-js/#configuring-logging) * [RPC](https://docs.livekit.io/reference/client-sdk-js/#rpc) * * [Registering an RPC method](https://docs.livekit.io/reference/client-sdk-js/#registering-an-rpc-method) * [Performing an RPC request](https://docs.livekit.io/reference/client-sdk-js/#performing-an-rpc-request) * [Errors](https://docs.livekit.io/reference/client-sdk-js/#errors) * [Error Codes](https://docs.livekit.io/reference/client-sdk-js/#error-codes) * [Examples](https://docs.livekit.io/reference/client-sdk-js/#examples) * * [Demo App](https://docs.livekit.io/reference/client-sdk-js/#demo-app) * [RPC Demo](https://docs.livekit.io/reference/client-sdk-js/#rpc-demo) * [Browser Support](https://docs.livekit.io/reference/client-sdk-js/#browser-support) --- # livekit-android-sdk livekit-android-sdk =================== Android Client SDK to [LiveKit](https://github.com/livekit/livekit-server) . Packages -------- [io.livekit.android](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android/index.html) Link copied to clipboard SDK This package contains the initial `connect` function. [io.livekit.android.annotations](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.annotations/index.html) Link copied to clipboard SDK [io.livekit.android.audio](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/index.html) Link copied to clipboard SDK [io.livekit.android.coroutines](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.coroutines/index.html) Link copied to clipboard SDK [io.livekit.android.e2ee](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.e2ee/index.html) Link copied to clipboard SDK [io.livekit.android.events](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.events/index.html) Link copied to clipboard SDK [io.livekit.android.renderer](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.renderer/index.html) Link copied to clipboard SDK [io.livekit.android.room](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/index.html) Link copied to clipboard SDK Room is the primary class that manages the connection to the LiveKit Room. It exposes listeners that lets you hook into room events. [io.livekit.android.room.datastream](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream/index.html) Link copied to clipboard SDK [io.livekit.android.room.datastream.incoming](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/index.html) Link copied to clipboard SDK [io.livekit.android.room.datastream.outgoing](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.outgoing/index.html) Link copied to clipboard SDK [io.livekit.android.room.network](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.network/index.html) Link copied to clipboard SDK [io.livekit.android.room.participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/index.html) Link copied to clipboard SDK [io.livekit.android.room.provisions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.provisions/index.html) Link copied to clipboard SDK [io.livekit.android.room.rpc](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.rpc/index.html) Link copied to clipboard SDK [io.livekit.android.room.track](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track/index.html) Link copied to clipboard SDK `AudioTrack` and `VideoTrack` are the classes that represent the types of media streams that can be subscribed and published. [io.livekit.android.room.track.screencapture](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track.screencapture/index.html) Link copied to clipboard SDK [io.livekit.android.room.track.video](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track.video/index.html) Link copied to clipboard SDK [io.livekit.android.room.types](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.types/index.html) Link copied to clipboard SDK [io.livekit.android.rpc](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.rpc/index.html) Link copied to clipboard SDK [io.livekit.android.stats](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.stats/index.html) Link copied to clipboard SDK [io.livekit.android.webrtc](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.webrtc/index.html) Link copied to clipboard SDK [io.livekit.android.webrtc.peerconnection](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.webrtc.peerconnection/index.html) Link copied to clipboard SDK --- # LiveKit SFU | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/internals/livekit-sfu/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/internals/livekit-sfu/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [P2P vs. MCU vs. SFU](https://docs.livekit.io/reference/internals/livekit-sfu/#p2p-vs-mcu-vs-sfu) [LiveKit SFU Architecture](https://docs.livekit.io/reference/internals/livekit-sfu/#livekit-sfu-architecture) Copy pageSee more page options P2P vs. MCU vs. SFU[](https://docs.livekit.io/reference/internals/livekit-sfu/#p2p-vs-mcu-vs-sfu) -------------------------------------------------------------------------------------------------- Plain WebRTC is a peer-to-peer (P2P) protocol. When two peers connect with one another, they exchange—ignoring data channels—audio and video ("media") directly. This works well for a set of 2-3 peers, but how many people have an internet connection which can consistently upload five 720p (1.5Mbps) video streams simultaneously? Not many. Thus, scaling WebRTC to groups of more than 2-3 people requires a client-server model. One popular flavor of this model is an Multipoint Conferencing Unit (MCU) architecture. In an MCU setup, a user in a conference sends media streams, each encoded and compressed, to a central server (the "MCU"). The MCU decompresses and decodes each stream it receives, "glues" them together with incoming streams from other users (collectively referred to as "publishers"), and transmits a single media stream down to each recipient (a "subscriber"). For audio, streams are mixed together, and for video, they're typically composited into a predefined layout, like a row or grid of tiles. The clear advantages of an MCU approach are each publisher need only send one copy of a media stream, and each subscriber receives just a single, composite stream; a huge savings in bandwidth on either end. A key tradeoff is flexibility. If your application relies on being able to tweak the volume of an individual audio stream, you're out of luck. If your app's UI doesn't map to a row or grid of videos, you'll need to either compromise on your interface design or write code to segment the single video stream from the server back into individual tiles. Another major disadvantage of the MCU approach is scale. You'll need a beefy machine to decode, composite and re-encode all those streams, and if a session grows too large to fit on one server, then what? We chose to base LiveKit on another common client-server architecture: a Selective Forwarding Unit (SFU). You can think of an SFU as a specialized router, one optimized for low-latency, high-bandwidth media forwarding. In this setup, a publisher sends media streams—once again, encoded and compressed—to a server (the "SFU"), except this time, the server forwards a copy of each stream (in WebRTC parlance, a "track") to each interested subscriber without manipulating any underlying packets. Similar to an MCU, a publisher need only transmit a single copy of their media streams, saving a client significant upstream bandwidth. However, an SFU trades downstream bandwidth efficiency for flexibility and scalability by contrast. A user subscribed to camera feeds of five others would pull down five individual video streams (as opposed to one with an MCU). The benefit is your application is no longer tightly-coupled to side-effects of your media infrastructure — you have complete control over every individual audio and video track. If a session exhausts the resources of one server, there are options for splitting it across multiple nodes. LiveKit's SFU also contains smarts on both the server and client (via SDK) to automatically (and invisibly) measure a subscriber's downstream bandwidth and adjust track parameters (e.g. resolution or bitrate) accordingly. As a developer, you'll rarely, if ever, have to think about how many tracks your application is pulling down. LiveKit SFU Architecture[](https://docs.livekit.io/reference/internals/livekit-sfu/#livekit-sfu-architecture) -------------------------------------------------------------------------------------------------------------- LiveKit is written in Go, leveraging [Pion](https://github.com/pion/webrtc) 's Go-based WebRTC implementation. The SFU is horizontally-scalable: you can run it on one or one hundred nodes with an identical configuration. Nodes use peer-to-peer routing via Redis, ensuring clients joining a particular room all connect to the same node. When running LiveKit as a single node, there are no external dependencies, but Redis is required for distributed, multi-node setups. ![LiveKit Architecture Diagram](https://docs.livekit.io/images/diagrams/architecture.svg)![LiveKit Architecture Diagram](https://docs.livekit.io/images/diagrams/architecture-light.svg) On this page [P2P vs. MCU vs. SFU](https://docs.livekit.io/reference/internals/livekit-sfu/#p2p-vs-mcu-vs-sfu) [LiveKit SFU Architecture](https://docs.livekit.io/reference/internals/livekit-sfu/#livekit-sfu-architecture) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/internals/livekit-sfu/) Search --- # server-sdk-js | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/server-sdk-js/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/server-sdk-js/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Installation](https://docs.livekit.io/reference/server-sdk-js/#installation) Copy pageSee more page options Installation[](https://docs.livekit.io/reference/server-sdk-js/#installation) ------------------------------------------------------------------------------ yarnnpm yarn add livekit-server-sdk On this page [Installation](https://docs.livekit.io/reference/server-sdk-js/#installation) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/server-sdk-js/) Search --- # Client Protocol | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/internals/client-protocol/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/internals/client-protocol/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Basics](https://docs.livekit.io/reference/internals/client-protocol/#basics) [Protobufs](https://docs.livekit.io/reference/internals/client-protocol/#protobufs) [Dedicated PeerConnections](https://docs.livekit.io/reference/internals/client-protocol/#dedicated-peerconnections) [Joining a room](https://docs.livekit.io/reference/internals/client-protocol/#joining-a-room) [WebSocket Parameters](https://docs.livekit.io/reference/internals/client-protocol/#websocket-parameters) [Publishing](https://docs.livekit.io/reference/internals/client-protocol/#publishing) [Receiving tracks](https://docs.livekit.io/reference/internals/client-protocol/#receiving-tracks) [Server events](https://docs.livekit.io/reference/internals/client-protocol/#server-events) [SpeakersChanged](https://docs.livekit.io/reference/internals/client-protocol/#speakerschanged) [Client-initiated control](https://docs.livekit.io/reference/internals/client-protocol/#client-initiated-control) [Mute/unmute local tracks](https://docs.livekit.io/reference/internals/client-protocol/#mute-unmute-local-tracks) [Changing quality of streams](https://docs.livekit.io/reference/internals/client-protocol/#changing-quality-of-streams) [Subscription control](https://docs.livekit.io/reference/internals/client-protocol/#subscription-control) Copy pageSee more page options **Note** Using LiveKit in your app does not require you to understand the underlying protocol. This is one of our design goals. Basics[](https://docs.livekit.io/reference/internals/client-protocol/#basics) ------------------------------------------------------------------------------ LiveKit clients use a WebSocket to communicate with the server over Protocol Buffers. Client could establish up to two WebRTC PeerConnections with the SFUs, used for publishing and receiving streams, respectively. By default, the subscriber PeerConnection will always be open upon connection. The publisher PeerConnection will be established only when the client is ready to publish. ![Client-Server Connection](https://docs.livekit.io/images/diagrams/client-server-connection.svg)![Client-Server Connection](https://docs.livekit.io/images/diagrams/client-server-connection-light.svg) ### Protobufs[](https://docs.livekit.io/reference/internals/client-protocol/#protobufs) LiveKit uses Protocol Buffers for all of its communications. Communication happens asynchronously: one side may send a message to the other at any time, without the expectation of an immediate response. LiveKit protobufs reside in the [livekit/protocol repo](https://github.com/livekit/protocol) . As a convention, a client always sends a `SignalRequest` and the server replies with a `SignalResponse`. ### Dedicated PeerConnections[](https://docs.livekit.io/reference/internals/client-protocol/#dedicated-peerconnections) For each client connected to the server, we use up to two separate `PeerConnection` objects. One for publishing tracks to the server, and the other for receiving subscribed tracks. Using separate peer connections simplifies the negotiation process and eliminates negotiation [Glares](https://www.ietf.org/proceedings/82/slides/rtcweb-10.pdf) . The side sending tracks to the other will be the one that initiates the offer. Joining a room[](https://docs.livekit.io/reference/internals/client-protocol/#joining-a-room) ---------------------------------------------------------------------------------------------- 1. client initiates WebSocket connection to `/rtc` 2. server sends a `JoinResponse`, which includes room information, the current participant's data, and information about other participants in the room 3. server initiates the subscriber `PeerConnection`, sends `offer` to client * if `AutoSubscribe` is enabled, this offer will contain existing tracks in the room. * the offer will include two data channels as part of the connection 4. client and server will exchange ICE candidates via `trickle` 5. client accepts the subscriber connection, sends an `answer` 6. ICE connectivity is established 7. server notifies other participants of the new participant ### WebSocket Parameters[](https://docs.livekit.io/reference/internals/client-protocol/#websocket-parameters) Websocket endpoint `/rtc` is the initial step that the client connects to. It takes in several parameters to give the server information about the client and its capabilities: * access\_token: an encoded JWT access token * reconnect: true if client is trying to resume to an existing connection. when this is set, server will attempt to perform a ICE restart after connection is established. * auto\_subscribe: true by default. If true, server will automatically subscribe client to all tracks in the room * sdk: indicates the SDK it's using. (js, ios, android, etc) * protocol: indicates the protocol version. this document descriibes the latest protocol version: 9 * version: version of the client SDK Publishing[](https://docs.livekit.io/reference/internals/client-protocol/#publishing) -------------------------------------------------------------------------------------- To publish a track, a client must first notify the server of its intent and send up any client-defined metadata about the track. 1. client sends a `AddTrackRequest` with track metadata 2. server sends back a `TrackPublishedResponse` 3. client adds a transceiver to the `PeerConnection`, along with the media track 4. client initiates `offer`, sends to server 5. server answers the offer and starts receiving the track 6. if server subscribes other participants to the track Receiving tracks[](https://docs.livekit.io/reference/internals/client-protocol/#receiving-tracks) -------------------------------------------------------------------------------------------------- LiveKit server sends down track metadata to all participants in a room as soon as it's published, then it adds the track to each client's subscriber `PeerConnection`. Server events[](https://docs.livekit.io/reference/internals/client-protocol/#server-events) -------------------------------------------------------------------------------------------- The client must also be ready to act upon other changes in the room. The server will notify clients of: * `ParticipantUpdate`: when other participants join or leave, or if there are changes to their tracks * `LeaveRequest`: when the participant should immediately disconnect * `SpeakersChanged`: when the active speakers in the room changes For all server events, clients should handle them in an idempotent way. For example, it's possible to receive multiple ParticipantUpdates with identical metadata. ### SpeakersChanged[](https://docs.livekit.io/reference/internals/client-protocol/#speakerschanged) Server will send down a list of `SpeakerInfo` that has changed from the last update. Clients are responsible for applying the deltas and firing the appropriate events. Client-initiated control[](https://docs.livekit.io/reference/internals/client-protocol/#client-initiated-control) ------------------------------------------------------------------------------------------------------------------ ### Mute/unmute local tracks[](https://docs.livekit.io/reference/internals/client-protocol/#mute-unmute-local-tracks) WebRTC doesn't natively support muting tracks. When a track is disabled, it will continue to periodically send "empty" packets. With LiveKit (and SFUs, in general), we want a discrete mute event in order to notify other participants of the change and to optimize network consumption by suppressing empty packets. To mute a track, set `MediaStreamTrack.enabled` to false, and subsequently send a `MuteTrackRequest` to the server with that track's `sid`. ### Changing quality of streams[](https://docs.livekit.io/reference/internals/client-protocol/#changing-quality-of-streams) For a particular client, `UpdateTrackSettings` informs the server whether a subscribed track should be temporarily paused, or if the server should send down a stream of differing quality. This is especially useful for larger rooms, when the client wants to optimize how much data it's receiving at once. For example, offscreen clients could have their streams temporarily paused. ### Subscription control[](https://docs.livekit.io/reference/internals/client-protocol/#subscription-control) Clients also have the ability to control which tracks they're subscribed to. An `UpdateSubscription` message allows the client to subscribe or unsubscribe to published tracks. On this page [Basics](https://docs.livekit.io/reference/internals/client-protocol/#basics) [Protobufs](https://docs.livekit.io/reference/internals/client-protocol/#protobufs) [Dedicated PeerConnections](https://docs.livekit.io/reference/internals/client-protocol/#dedicated-peerconnections) [Joining a room](https://docs.livekit.io/reference/internals/client-protocol/#joining-a-room) [WebSocket Parameters](https://docs.livekit.io/reference/internals/client-protocol/#websocket-parameters) [Publishing](https://docs.livekit.io/reference/internals/client-protocol/#publishing) [Receiving tracks](https://docs.livekit.io/reference/internals/client-protocol/#receiving-tracks) [Server events](https://docs.livekit.io/reference/internals/client-protocol/#server-events) [SpeakersChanged](https://docs.livekit.io/reference/internals/client-protocol/#speakerschanged) [Client-initiated control](https://docs.livekit.io/reference/internals/client-protocol/#client-initiated-control) [Mute/unmute local tracks](https://docs.livekit.io/reference/internals/client-protocol/#mute-unmute-local-tracks) [Changing quality of streams](https://docs.livekit.io/reference/internals/client-protocol/#changing-quality-of-streams) [Subscription control](https://docs.livekit.io/reference/internals/client-protocol/#subscription-control) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/internals/client-protocol/) Search --- # AI voice agents | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/voice-agent/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/voice-agent/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Features](https://docs.livekit.io/agents/v0/voice-agent/#features) [Multimodal or voice pipeline](https://docs.livekit.io/agents/v0/voice-agent/#multimodal-or-voice-pipeline) [Handling background noise](https://docs.livekit.io/agents/v0/voice-agent/#handling-background-noise) [Turn detection](https://docs.livekit.io/agents/v0/voice-agent/#turn-detection) [Voice activity detection (VAD)](https://docs.livekit.io/agents/v0/voice-agent/#voice-activity-detection-vad-) [Turn detection model](https://docs.livekit.io/agents/v0/voice-agent/#turn-detection-model) [Agent state](https://docs.livekit.io/agents/v0/voice-agent/#agent-state) [Transcriptions](https://docs.livekit.io/agents/v0/voice-agent/#transcriptions) Copy pageSee more page options **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [Building voice agents](https://docs.livekit.io/agents/build/) . _v1.0 for Node.js is coming soon._ Companies like OpenAI, Character.ai, Retell, and Speak have built their conversational AI products on the LiveKit platform. AI voice agents are one of the primary use cases for LiveKit's Agents framework. Features[](https://docs.livekit.io/agents/v0/voice-agent/#features) -------------------------------------------------------------------- * Programmable conversation flows * Integrated LLM function calls * Provide context to the conversation via RAG * Leverage connectors from an open-source plugin ecosystem * Send synchronized transcriptions to your frontend Multimodal or voice pipeline[](https://docs.livekit.io/agents/v0/voice-agent/#multimodal-or-voice-pipeline) ------------------------------------------------------------------------------------------------------------ LiveKit offers two types of voice agents: `MultimodalAgent` and `VoicePipelineAgent`. * [MultimodalAgent](https://docs.livekit.io/agents/v0/voice-agent/multimodal-agent/) uses OpenAI’s multimodal model and realtime API to directly process user audio and generate audio responses, similar to OpenAI’s advanced voice mode, producing more natural-sounding speech. * [VoicePipelineAgent](https://docs.livekit.io/agents/v0/voice-agent/voice-pipeline/) uses a pipeline of STT, LLM, and TTS models, providing greater control over the conversation flow by allowing applications to modify the text returned by the LLM. | | Multimodal | Voice pipeline | | --- | --- | --- | | Python | ✅ | ✅ | | Node.JS | ✅ | ✅ | | Model type | single multimodal | STT, LLM, TTS | | Function calling | ✅ | ✅ | | RAG | via function calling | ✅ | | Natural speech | more natural | | | Modify LLM response | | ✅ | | Model vendors | OpenAI | various | | Turn detection | VAD | VAD and turn detection model | Handling background noise[](https://docs.livekit.io/agents/v0/voice-agent/#handling-background-noise) ------------------------------------------------------------------------------------------------------ While humans can easily ignore background noise, AI models often struggle, leading to misinterpretations or unnecessary pauses when detecting non-speech sounds. Although WebRTC includes built-in noise suppression, it often falls short in real-world environments. To address this, LiveKit has partnered with [Krisp](https://krisp.ai/) to bring best-in-class noise suppression technology to AI agents. For instructions on enabling Krisp, see [Krisp integration guide](https://docs.livekit.io/home/client/tracks/noise-cancellation/) Turn detection[](https://docs.livekit.io/agents/v0/voice-agent/#turn-detection) -------------------------------------------------------------------------------- **Endpointing** is the process of detecting the start and end of speech in an audio stream. This is crucial for conversational AI agents to understand when a user has finished speaking and when to start responding to user input. Determining the end of a turn is particularly challenging for AI agents. Humans rely on multiple cues, such as pauses, speech tone, and content, to recognize when someone has finished speaking. LiveKit employs two primary strategies to approximate how humans determine turn boundaries: ### Voice activity detection (VAD)[](https://docs.livekit.io/agents/v0/voice-agent/#voice-activity-detection-vad-) LiveKit Agents uses VAD to detect when the user has finished speaking. The agent waits for a minimum duration of silence before considering the turn complete. * Both VoicePipelineAgent and MultimodalAgent use VAD for turn detection. * For OpenAI Multimodal configuration, refer to the [MultimodalAgent turn detection](https://docs.livekit.io/agents/v0/integrations/openai/customize/vad/) docs. * VoicePipelineAgent uses Silero VAD to detect end of speech. The `min_endpointing_delay` parameter in the agent constructor specifies the minimum silence duration to consider the end of a turn. ### Turn detection model[](https://docs.livekit.io/agents/v0/voice-agent/#turn-detection-model) While VAD provides a simple approximation of turn completion, it lacks contextual awareness. In natural conversations, pauses often occur as people think or formulate responses. To address this, LiveKit has developed a custom, open-weights language model to incorporate conversational context as an additional signal to VAD. The [turn-detector](https://github.com/livekit/agents/tree/0.x/livekit-plugins/livekit-plugins-turn-detector) plugin uses this model to predict whether a user is done speaking. When the model predicts that the user is **not done** with their turn, the agent will wait for a significantly longer period of silence before responding. This helps to prevent unwanted interruptions during natural pauses in speech. Here's [a demo](https://youtu.be/EYDrSSEP0h0) of the model in action. #### Benchmarks[](https://docs.livekit.io/agents/v0/voice-agent/#benchmarks) In our testing, the turn detector model demonstrated the following performance: * **85% true positive rate**: avoids early interruptions by correctly identifying when the user is not done speaking. * **97% true negative rate**: accurately determines the end of a turn when the user has finished speaking. #### Using turn detector[](https://docs.livekit.io/agents/v0/voice-agent/#using-turn-detector) Currently, this model is supported for `VoicePipelineAgent` in Python. To use it, install the `livekit-plugins-turn-detector` package. Then, initialize the agent with the turn detector: from livekit.plugins import turn\_detector agent \= VoicePipelineAgent( ... turn\_detector\=turn\_detector.EOUModel(), ) Before running the agent for the first time, download the model weights: python my\_agent.py download-files Agent state[](https://docs.livekit.io/agents/v0/voice-agent/#agent-state) -------------------------------------------------------------------------- Voice agents automatically publish their current state to your frontend, making it easy to build UI that reflects the agent’s status. The state is passed to your frontend as a [participant attribute](https://docs.livekit.io/home/client/data/#participant-attributes-and-metadata) on the agent participant. Components like `useVoiceAssistant` expose the following states: * `disconnected`: either agent or user is disconnected * `connecting`: agent is being connected with the user * `initializing`: agent is connected, but not yet ready * `listening`: agent is listening for user input * `thinking`: agent is performing inference on user input * `speaking`: agent is playing out a response Transcriptions[](https://docs.livekit.io/agents/v0/voice-agent/#transcriptions) -------------------------------------------------------------------------------- LiveKit provides realtime transcriptions for both the agent and the user, which are sent to your frontend via the [transcription protocol](https://docs.livekit.io/agents/v0/voice-agent/transcriptions/) . User speech transcriptions are delivered as soon as they are processed by STT. Since the agent’s text response is available before speech synthesis, we manually synchronize the text transcription with audio playback. On this page [Features](https://docs.livekit.io/agents/v0/voice-agent/#features) [Multimodal or voice pipeline](https://docs.livekit.io/agents/v0/voice-agent/#multimodal-or-voice-pipeline) [Handling background noise](https://docs.livekit.io/agents/v0/voice-agent/#handling-background-noise) [Turn detection](https://docs.livekit.io/agents/v0/voice-agent/#turn-detection) [Voice activity detection (VAD)](https://docs.livekit.io/agents/v0/voice-agent/#voice-activity-detection-vad-) [Turn detection model](https://docs.livekit.io/agents/v0/voice-agent/#turn-detection-model) [Agent state](https://docs.livekit.io/agents/v0/voice-agent/#agent-state) [Transcriptions](https://docs.livekit.io/agents/v0/voice-agent/#transcriptions) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/voice-agent/) Search --- # Agents playground | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/start/playground/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/playground/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/start/playground/#overview) [Links](https://docs.livekit.io/agents/start/playground/#links) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/start/playground/#overview) ---------------------------------------------------------------------- The LiveKit Agents playground is a versatile web frontend that makes it easy to test your multimodal AI agent without having to worry about UI until you're happy with your AI. To use the playground, you first need to have an agent running in `dev` or `start` mode. If you haven't done that yet, first follow the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai/) . | Feature | Notes | | --- | --- | | Audio | Mic input and speaker output with visualizer | | Text | Live transcription and chat input | | Video | Live webcam input, live output | Links[](https://docs.livekit.io/agents/start/playground/#links) ---------------------------------------------------------------- Follow these links to get started with the playground. [Hosted playground\ -----------------\ \ A hosted playground that seamlessly integrates with LiveKit Cloud.](https://agents-playground.livekit.io/) [Source code\ -----------\ \ Run the playground yourself or use it as a starting point for your own application.](https://github.com/livekit/agents-playground/) On this page [Overview](https://docs.livekit.io/agents/start/playground/#overview) [Links](https://docs.livekit.io/agents/start/playground/#links) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/playground/) Search --- # Agents Playground | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/playground/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/playground/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Using the hosted playground](https://docs.livekit.io/agents/v0/playground/#using-the-hosted-playground) [Self-hosted playground](https://docs.livekit.io/agents/v0/playground/#self-hosted-playground) [1\. Clone the project](https://docs.livekit.io/agents/v0/playground/#1-clone-the-project) [2\. Create environment variables](https://docs.livekit.io/agents/v0/playground/#2-create-environment-variables) [3\. Customizing configuration](https://docs.livekit.io/agents/v0/playground/#3-customizing-configuration) [4\. Run the playground](https://docs.livekit.io/agents/v0/playground/#4-run-the-playground) [5\. Deploying the playground](https://docs.livekit.io/agents/v0/playground/#5-deploying-the-playground) Copy pageSee more page options **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [Agents playground](https://docs.livekit.io/agents/start/playground/) . _v1.0 for Node.js is coming soon._ To ease the process of building and testing an agent, we've developed a versatile web frontend called "playground". This app is available for use or customization according to your specific requirements. It can also serve as a starting point for your application. * Playground source code: [https://github.com/livekit/agents-playground/](https://github.com/livekit/agents-playground/) * Built with Next.js and [LiveKit Components](https://github.com/livekit/components-js) . Using the hosted playground[](https://docs.livekit.io/agents/v0/playground/#using-the-hosted-playground) --------------------------------------------------------------------------------------------------------- A hosted version of the playground is available at [https://agents-playground.livekit.io](https://agents-playground.livekit.io/) . This version is compatible with any LiveKit server instance (including one running locally). To use it, you'll need to input your LiveKit server's URL and a corresponding access token. Self-hosted playground[](https://docs.livekit.io/agents/v0/playground/#self-hosted-playground) ----------------------------------------------------------------------------------------------- When you run your own version of the playground, it can be configured to automatically generate access tokens for your users. This setup streamlines the user experience, removing the need for manual token entry. ### 1\. Clone the project[](https://docs.livekit.io/agents/v0/playground/#1-clone-the-project) git clone https://github.com/livekit/agents-playground.git ### 2\. Create environment variables[](https://docs.livekit.io/agents/v0/playground/#2-create-environment-variables) Create an `.env.local` file in the root of your project folder with the following variables: LIVEKIT\_API\_KEY\=YOUR\_API\_KEY LIVEKIT\_API\_SECRET\=YOUR\_API\_SECRET \# Public configuration NEXT\_PUBLIC\_LIVEKIT\_URL\=wss://YOUR\_LIVEKIT\_URL NEXT\_PUBLIC\_APP\_CONFIG\=" title: 'LiveKit Agent Playground' description: 'LiveKit Agent Playground allows you to test your LiveKit Agent integration by connecting to your LiveKit Cloud or self-hosted instance.' github\_link: 'https://github.com/livekit/agents-playground' video\_fit: 'cover' # 'contain' or 'cover' settings: editable: true # Should the user be able to edit settings in-app theme\_color: 'cyan' chat: true # Enable or disable chat feature outputs: audio: true # Enable or disable audio output video: true # Enable or disable video output inputs: mic: true # Enable or disable microphone input camera: true # Enable or disable camera input sip: true # Enable or disable SIP input " 3\. Customizing configuration[](https://docs.livekit.io/agents/v0/playground/#3-customizing-configuration) ----------------------------------------------------------------------------------------------------------- `NEXT_PUBLIC_APP_CONFIG` is a configurable YAML string designed to tailor the playground's capabilities. By default, the playground is set up to publish both the user's camera and microphone when they connect to a room. However if, for example, your agent doesn't need vision, you can disable the camera by setting `inputs.camera` to `false`. Similarly, agents have varying output requirements. If your agent does not provide voice/audio feedback to the user, you can set `outputs.audio` to `false`. This adjustment will consequently remove the related audio component from the playground UI. 4\. Run the playground[](https://docs.livekit.io/agents/v0/playground/#4-run-the-playground) --------------------------------------------------------------------------------------------- That's it! You're ready to run your own version of the playground. npm install npm run dev 5\. Deploying the playground[](https://docs.livekit.io/agents/v0/playground/#5-deploying-the-playground) --------------------------------------------------------------------------------------------------------- To deploy your version of the playground, you can use any hosting provider that supports Node.js. We're hosting the public version of it on [Vercel](https://vercel.com/) . [Deploy with Vercel](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Flivekit%2Fagent-playground&env=LIVEKIT_API_KEY,LIVEKIT_API_SECRET,NEXT_PUBLIC_LIVEKIT_URL,NEXT_PUBLIC_APP_CONFIG&project-name=my-agent-playground) On this page [Using the hosted playground](https://docs.livekit.io/agents/v0/playground/#using-the-hosted-playground) [Self-hosted playground](https://docs.livekit.io/agents/v0/playground/#self-hosted-playground) [1\. Clone the project](https://docs.livekit.io/agents/v0/playground/#1-clone-the-project) [2\. Create environment variables](https://docs.livekit.io/agents/v0/playground/#2-create-environment-variables) [3\. Customizing configuration](https://docs.livekit.io/agents/v0/playground/#3-customizing-configuration) [4\. Run the playground](https://docs.livekit.io/agents/v0/playground/#4-run-the-playground) [5\. Deploying the playground](https://docs.livekit.io/agents/v0/playground/#5-deploying-the-playground) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/playground/) Search --- # Server APIs | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/server/server-apis/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/server/server-apis/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/reference/server/server-apis/#overview) [Implementation details](https://docs.livekit.io/reference/server/server-apis/#implementation-details) [Endpoints](https://docs.livekit.io/reference/server/server-apis/#endpoints) [Authorization header](https://docs.livekit.io/reference/server/server-apis/#authorization-header) [Post body](https://docs.livekit.io/reference/server/server-apis/#post-body) [Egress APIs](https://docs.livekit.io/reference/server/server-apis/#egress-apis) [Ingress APIs](https://docs.livekit.io/reference/server/server-apis/#ingress-apis) [RoomService APIs](https://docs.livekit.io/reference/server/server-apis/#roomservice-apis) [CreateRoom](https://docs.livekit.io/reference/server/server-apis/#createroom) [ListRooms](https://docs.livekit.io/reference/server/server-apis/#listrooms) [DeleteRoom](https://docs.livekit.io/reference/server/server-apis/#deleteroom) [ListParticipants](https://docs.livekit.io/reference/server/server-apis/#listparticipants) [GetParticipant](https://docs.livekit.io/reference/server/server-apis/#getparticipant) [RemoveParticipant](https://docs.livekit.io/reference/server/server-apis/#removeparticipant) [MutePublishedTrack](https://docs.livekit.io/reference/server/server-apis/#mutepublishedtrack) [UpdateParticipant](https://docs.livekit.io/reference/server/server-apis/#updateparticipant) [UpdateSubscriptions](https://docs.livekit.io/reference/server/server-apis/#updatesubscriptions) [UpdateRoomMetadata](https://docs.livekit.io/reference/server/server-apis/#updateroommetadata) [SendData](https://docs.livekit.io/reference/server/server-apis/#senddata) [SIP APIs](https://docs.livekit.io/reference/server/server-apis/#sip-apis) [Types](https://docs.livekit.io/reference/server/server-apis/#types) [Room](https://docs.livekit.io/reference/server/server-apis/#room) [RoomAgentDispatch](https://docs.livekit.io/reference/server/server-apis/#roomagentdispatch) [RoomConfiguration](https://docs.livekit.io/reference/server/server-apis/#roomconfiguration) [ParticipantInfo](https://docs.livekit.io/reference/server/server-apis/#participantinfo) [TrackInfo](https://docs.livekit.io/reference/server/server-apis/#trackinfo) [ParticipantPermission](https://docs.livekit.io/reference/server/server-apis/#participantpermission) [VideoLayer](https://docs.livekit.io/reference/server/server-apis/#videolayer) [RoomEgress](https://docs.livekit.io/reference/server/server-apis/#roomegress) [AutoTrackEgress](https://docs.livekit.io/reference/server/server-apis/#autotrackegress) [ParticipantInfo\_State](https://docs.livekit.io/reference/server/server-apis/#participantinfo-state) [TrackSource](https://docs.livekit.io/reference/server/server-apis/#tracksource) [TrackSource](https://docs.livekit.io/reference/server/server-apis/#tracksource) [TrackType](https://docs.livekit.io/reference/server/server-apis/#tracktype) [VideoQuality](https://docs.livekit.io/reference/server/server-apis/#videoquality) Copy pageSee more page options Overview[](https://docs.livekit.io/reference/server/server-apis/#overview) --------------------------------------------------------------------------- LiveKit has built-in APIs that let you to manage rooms, participants, tracks, and SIP-based apps. These APIs are designed for use by your backend and are fully distributed across multiple nodes: any instance is capable of fulfilling requests about any room, participant, track, trunk, or dispatch rule. Implementation details[](https://docs.livekit.io/reference/server/server-apis/#implementation-details) ------------------------------------------------------------------------------------------------------- We provide [server sdks](https://docs.livekit.io/reference/#backend) that make it easy to use these APIs. If you prefer to implement your own client, read on. ### Endpoints[](https://docs.livekit.io/reference/server/server-apis/#endpoints) Server APIs are built with [Twirp](https://twitchtv.github.io/twirp/docs/intro.html) , and differ from a traditional REST interface. Arguments are passed by POSTing a JSON body to an endpoint. Each API is accessible via `/twirp/livekit./` * Egress: `/twirp/livekit.Egress/` * Ingress: `/twirp/livekit.Ingress/` * Room Service: `/twirp/livekit.RoomService/` * SIP: `/twirp/livekit.SIP/` ### Authorization header[](https://docs.livekit.io/reference/server/server-apis/#authorization-header) All endpoints require a signed access token. This token should be set via HTTP header: Authorization: Bearer LiveKit's server sdks automatically include the above header. ### Post body[](https://docs.livekit.io/reference/server/server-apis/#post-body) Twirp expects an HTTP POST request. The body of the request must be a JSON object (`application/json`) containing parameters specific to that request. Use an empty `{}` body for requests that don't require parameters. For example, the following lists the room : curl \-X POST /twirp/livekit.RoomService/ListRooms \\ \-H "Authorization: Bearer " \\ \-H 'Content-Type: application/json' \\ \-d '{ "names": \[""\] }' When passing in parameters, the server accepts either `snake_case` or `camelCase` for keys. Egress APIs[](https://docs.livekit.io/reference/server/server-apis/#egress-apis) --------------------------------------------------------------------------------- See [Egress API](https://docs.livekit.io/home/egress/api/) . Ingress APIs[](https://docs.livekit.io/reference/server/server-apis/#ingress-apis) ----------------------------------------------------------------------------------- See [Ingress API](https://docs.livekit.io/home/ingress/overview/#api) . RoomService APIs[](https://docs.livekit.io/reference/server/server-apis/#roomservice-apis) ------------------------------------------------------------------------------------------- The RoomService API allows you to manage rooms, participants, tracks, and data. ### CreateRoom[](https://docs.livekit.io/reference/server/server-apis/#createroom) Create a room with the specified settings. Requires `roomCreate` permission. This method is optional; a room is created automatically when the first participant joins it. When creating a room, it's possible to configure automatic recording of the room or individually published tracks. See [Auto Egress](https://docs.livekit.io/home/egress/autoegress/) docs. Returns [Room](https://docs.livekit.io/reference/server/server-apis/#room) | Parameter | Type | Required | Description | | --- | --- | --- | --- | | name | string | yes | Name of the room. | | empty\_timeout | uint32 | | Number of seconds to keep the room open if no one joins. Default is 300 seconds. | | departure\_timeout | uint32 | | Number of seconds the room remains open after the last participant leaves. Default is 20 seconds. | | max\_participants | uint32 | | Limit number of participants that can be in the room. Default is 0. | | node\_id | string | | Override node selection (note: for advanced users). | | metadata | string | | Initial metadata to assign to the room. | | egress | [RoomEgress](https://docs.livekit.io/reference/server/server-apis/#roomegress) | | Set the room to be recorded or streamed. | | min\_playout\_delay | uint32 | | Minimum playout delay in ms. Default is 0 ms. | | max\_playout\_delay | uint32 | | Maximum playout delay in ms. Default is 0 ms. | ### ListRooms[](https://docs.livekit.io/reference/server/server-apis/#listrooms) List rooms that are active/open. Requires `roomList` permission. Returns List<[Room](https://docs.livekit.io/reference/server/server-apis/#room) \> | Parameter | Type | Required | Description | | --- | --- | --- | --- | | names | List | | when passed in, only returns rooms matching one or more specified names | ### DeleteRoom[](https://docs.livekit.io/reference/server/server-apis/#deleteroom) Delete an existing room. Requires `roomCreate` permission. DeleteRoom will forcibly disconnect all participants currently in the room. | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | name of the room | ### ListParticipants[](https://docs.livekit.io/reference/server/server-apis/#listparticipants) List participants in a room, Requires `roomAdmin` | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | name of the room | Returns List<[ParticipantInfo](https://docs.livekit.io/reference/server/server-apis/#ParticipantInfo) \> ### GetParticipant[](https://docs.livekit.io/reference/server/server-apis/#getparticipant) Get information about a specific participant in a room, Requires `roomAdmin` | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | name of the room | | identity | string | yes | identity of the participant | Returns [ParticipantInfo](https://docs.livekit.io/reference/server/server-apis/#ParticipantInfo) ### RemoveParticipant[](https://docs.livekit.io/reference/server/server-apis/#removeparticipant) Remove a participant from a room. Requires `roomAdmin` | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | name of the room | | identity | string | yes | identity of the participant | ### MutePublishedTrack[](https://docs.livekit.io/reference/server/server-apis/#mutepublishedtrack) Mute or unmute a participant's track. Requires `roomAdmin` For privacy reasons, LiveKit server is configured by default to disallow the remote unmuting of tracks. To enable it, set [enable\_remote\_unmute](https://github.com/livekit/livekit/blob/4b630d2156265b9dc5ba6c6f786a408cf1a670a4/config-sample.yaml#L134) to true. | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | name of the room | | identity | string | yes | | | track\_sid | string | yes | sid of the track to mute | | muted | bool | yes | set to true to mute, false to unmute | ### UpdateParticipant[](https://docs.livekit.io/reference/server/server-apis/#updateparticipant) Update information for a participant. Updating metadata will broadcast the change to all other participants in the room. Requires `roomAdmin` | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | | | identity | string | yes | | | metadata | string | | user-provided payload, an empty value is equivalent to a no-op | | permission | [ParticipantPermission](https://docs.livekit.io/reference/server/server-apis/#ParticipantPermission) | | set to update the participant's permissions | ### UpdateSubscriptions[](https://docs.livekit.io/reference/server/server-apis/#updatesubscriptions) Subscribe or unsubscribe a participant from one or more published tracks. Requires `roomAdmin`. As an admin, you can subscribe a participant to a track even if they do not have `canSubscribe` permission. | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | | | identity | string | yes | | | track\_sids | List | yes | list of sids of tracks | | subscribe | bool | yes | set to true to subscribe and false to unsubscribe from tracks | ### UpdateRoomMetadata[](https://docs.livekit.io/reference/server/server-apis/#updateroommetadata) Update room metadata. A metadata update will be broadcast to all participants in the room. Requires `roomAdmin` | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | | | metadata | string | yes | user-provided payload; opaque to LiveKit | ### SendData[](https://docs.livekit.io/reference/server/server-apis/#senddata) Send data packets to one or more participants in a room. See the [data packet docs](https://docs.livekit.io/home/client/data/packets/) for more details and examples of client-side integration. | Parameter | Type | Required | Description | | --- | --- | --- | --- | | room | string | yes | The room to send the packet in | | data | bytes | yes | The raw packet bytes | | kind | enum | yes | `reliable` or `lossy` delivery mode | | destination\_identities | List<\[string\]> | yes | List of participant identities to receive packet, leave blank to send the packet to everyone | | topic | string | | Topic for the packet | SIP APIs[](https://docs.livekit.io/reference/server/server-apis/#sip-apis) --------------------------------------------------------------------------- See [SIP APIs](https://docs.livekit.io/sip/api/) . Types[](https://docs.livekit.io/reference/server/server-apis/#types) --------------------------------------------------------------------- ### Room[](https://docs.livekit.io/reference/server/server-apis/#room) | Field | Type | Description | | --- | --- | --- | | sid | string | Unique session ID. | | name | string | | | empty\_timeout | uint32 | Number of seconds the room remains open if no one joins. | | departure\_timeout | uint32 | Number of seconds the room remains open after the last participant leaves. | | max\_participants | uint32 | Maximum number of participants that can be in the room (0 = no limit). | | creation\_time | int64 | Unix timestamp (seconds since epoch) when this room was created. | | turn\_password | string | Password that the embedded TURN server requires. | | metadata | string | User-specified metadata, opaque to LiveKit. | | num\_participants | uint32 | Number of participants currently in the room, excludes hidden participants. | | active\_recording | bool | True if a participant with `recorder` permission is currently in the room. | ### RoomAgentDispatch[](https://docs.livekit.io/reference/server/server-apis/#roomagentdispatch) A `RoomAgentDispatch` object can be passed to automatically [dispatch a named agent](https://docs.livekit.io/agents/worker/agent-dispatch/#explicit) to a room. | Field | Type | Description | | --- | --- | --- | | agent\_name | string | Name of agent to dispatch to room. | | metadata | string | User-specified metadata, opaque to LiveKit. | ### RoomConfiguration[](https://docs.livekit.io/reference/server/server-apis/#roomconfiguration) A `RoomConfiguration` object can be passed when you create an [access token](https://docs.livekit.io/home/get-started/authentication/#room-configuration) or [SIP dispatch rule](https://docs.livekit.io/sip/dispatch-rule/) , or to automatically [dispatch an agent](https://docs.livekit.io/agents/worker/agent-dispatch/) to a room. | Field | Type | Description | | --- | --- | --- | | name | string | | | empty\_timeout | int | Number of seconds the room remains open if no one joins. | | departure\_timeout | int | Number of seconds the room remains open after the last participant leaves. | | max\_participants | int | Maximum number of participants that can be in the room (0 = no limit). | | egress | [RoomEgress](https://docs.livekit.io/reference/server/server-apis/#roomegress) | If set, automatically start recording or streaming when room is created. | | min\_playout\_delay | int | Minimum playout delay in ms. | | max\_playout\_delay | int | Maximum playout delay in ms. | | sync\_streams | bool | If true, enable A/V sync for playout delays >200ms. | | agents | List<\[[RoomAgentDispatch](https://docs.livekit.io/reference/server/server-apis/#roomagentdispatch)
\]> | One or more agents to be dispatched to the room on connection. | ### ParticipantInfo[](https://docs.livekit.io/reference/server/server-apis/#participantinfo) | Field | Type | Description | | --- | --- | --- | | sid | string | server-generated identifier | | identity | string | user-specified unique identifier for the participant | | name | string | name given to the participant in access token (optional) | | state | [ParticipantInfo\_State](https://docs.livekit.io/reference/server/server-apis/#ParticipantInfo-State) | connection state of the participant | | tracks | List<[TrackInfo](https://docs.livekit.io/reference/server/server-apis/#TrackInfo)
\> | tracks published by the participant | | metadata | string | user-specified metadata for the participant | | joined\_at | int64 | timestamp when the participant joined room | | permission | ParticipantPermission | permission given to the participant via access token | | is\_publisher | bool | true if the participant has published media or data | ### TrackInfo[](https://docs.livekit.io/reference/server/server-apis/#trackinfo) | Field | Type | Description | | --- | --- | --- | | sid | string | server-generated identifier | | type | [TrackType](https://docs.livekit.io/reference/server/server-apis/#TrackType) | audio or video | | source | [TrackSource](https://docs.livekit.io/reference/server/server-apis/#TrackSource) | source of the Track | | name | string | name given at publish time (optional) | | mime\_type | string | mime type of codec used | | muted | bool | true if track has been muted by the publisher | | width | uint32 | original width of video (unset for audio) | | height | uint32 | original height of video (unset for audio) | | simulcast | bool | true if track is simulcasted | | disable\_dtx | bool | true if DTX is disabled | | layers | List<[VideoLayer](https://docs.livekit.io/reference/server/server-apis/#VideoLayer)
\> | simulcast or SVC layers in the track | ### ParticipantPermission[](https://docs.livekit.io/reference/server/server-apis/#participantpermission) | Field | Type | Description | | --- | --- | --- | | can\_subscribe | bool | allow the participant to subscribe to other tracks in the room | | can\_publish | bool | allow the participant to publish new tracks to the room | | can\_publish\_data | bool | allow the participant to publish data to the room | ### VideoLayer[](https://docs.livekit.io/reference/server/server-apis/#videolayer) Represents a single simulcast layer in a [Track](https://docs.livekit.io/reference/server/server-apis/#TrackInfo) | Field | Type | Description | | --- | --- | --- | | quality | [VideoQuality](https://docs.livekit.io/reference/server/server-apis/#VideoQuality) | high, medium, or low | | width | uint32 | | | height | uint32 | | ### RoomEgress[](https://docs.livekit.io/reference/server/server-apis/#roomegress) Used to specify Auto Egress settings when creating a room. | Field | Type | Description | | --- | --- | --- | | room | [RoomCompositeEgressRequest](https://docs.livekit.io/home/egress/room-composite/#starting-a-roomcomposite) | set to start a Room Composite Egress when participant joins, same parameters as `StartCompositeEgress` API | | tracks | [AutoTrackEgress](https://docs.livekit.io/reference/server/server-apis/#AutoTrackEgress) | set to export each published track automatically | ### AutoTrackEgress[](https://docs.livekit.io/reference/server/server-apis/#autotrackegress) | Field | Type | Description | | --- | --- | --- | | filepath | string | template to use for file name. see [Egress filenames](https://docs.livekit.io/home/egress/overview/#filename-templating) | | disable\_manifest | bool | when set to true, disables uploading of JSON manifests | | s3 | [S3Upload](https://github.com/livekit/protocol/blob/85bf30570f0f4ce1d06e40cd98222a6350013315/livekit_egress.proto#L112) | set when uploading to S3 | | gcp | [GCPUpload](https://github.com/livekit/protocol/blob/85bf30570f0f4ce1d06e40cd98222a6350013315/livekit_egress.proto#L121) | set when uploading to Google Cloud Storage | | azure | [AzureBlobUpload](https://github.com/livekit/protocol/blob/85bf30570f0f4ce1d06e40cd98222a6350013315/livekit_egress.proto#L126) | set when uploading to Azure Blob Storage | ### ParticipantInfo\_State[](https://docs.livekit.io/reference/server/server-apis/#participantinfo-state) Enum, valid values: * JOINING: 0 * JOINED: 1 * ACTIVE: 2 * DISCONNECTED: 3 ### TrackSource[](https://docs.livekit.io/reference/server/server-apis/#tracksource) Enum, valid values: * AUDIO: 0 * VIDEO: 1 ### TrackSource[](https://docs.livekit.io/reference/server/server-apis/#tracksource) Enum, valid values: * UNKNOWN: 0 * CAMERA: 1 * MICROPHONE: 2 * SCREEN\_SHARE: 3 * SCREEN\_SHARE\_AUDIO: 4 ### TrackType[](https://docs.livekit.io/reference/server/server-apis/#tracktype) Enum, valid values: * AUDIO: 0 * VIDEO: 1 ### VideoQuality[](https://docs.livekit.io/reference/server/server-apis/#videoquality) Enum, valid values: * LOW: 0 * MEDIUM: 1 * HIGH: 2 * OFF: 3 On this page [Overview](https://docs.livekit.io/reference/server/server-apis/#overview) [Implementation details](https://docs.livekit.io/reference/server/server-apis/#implementation-details) [Endpoints](https://docs.livekit.io/reference/server/server-apis/#endpoints) [Authorization header](https://docs.livekit.io/reference/server/server-apis/#authorization-header) [Post body](https://docs.livekit.io/reference/server/server-apis/#post-body) [Egress APIs](https://docs.livekit.io/reference/server/server-apis/#egress-apis) [Ingress APIs](https://docs.livekit.io/reference/server/server-apis/#ingress-apis) [RoomService APIs](https://docs.livekit.io/reference/server/server-apis/#roomservice-apis) [CreateRoom](https://docs.livekit.io/reference/server/server-apis/#createroom) [ListRooms](https://docs.livekit.io/reference/server/server-apis/#listrooms) [DeleteRoom](https://docs.livekit.io/reference/server/server-apis/#deleteroom) [ListParticipants](https://docs.livekit.io/reference/server/server-apis/#listparticipants) [GetParticipant](https://docs.livekit.io/reference/server/server-apis/#getparticipant) [RemoveParticipant](https://docs.livekit.io/reference/server/server-apis/#removeparticipant) [MutePublishedTrack](https://docs.livekit.io/reference/server/server-apis/#mutepublishedtrack) [UpdateParticipant](https://docs.livekit.io/reference/server/server-apis/#updateparticipant) [UpdateSubscriptions](https://docs.livekit.io/reference/server/server-apis/#updatesubscriptions) [UpdateRoomMetadata](https://docs.livekit.io/reference/server/server-apis/#updateroommetadata) [SendData](https://docs.livekit.io/reference/server/server-apis/#senddata) [SIP APIs](https://docs.livekit.io/reference/server/server-apis/#sip-apis) [Types](https://docs.livekit.io/reference/server/server-apis/#types) [Room](https://docs.livekit.io/reference/server/server-apis/#room) [RoomAgentDispatch](https://docs.livekit.io/reference/server/server-apis/#roomagentdispatch) [RoomConfiguration](https://docs.livekit.io/reference/server/server-apis/#roomconfiguration) [ParticipantInfo](https://docs.livekit.io/reference/server/server-apis/#participantinfo) [TrackInfo](https://docs.livekit.io/reference/server/server-apis/#trackinfo) [ParticipantPermission](https://docs.livekit.io/reference/server/server-apis/#participantpermission) [VideoLayer](https://docs.livekit.io/reference/server/server-apis/#videolayer) [RoomEgress](https://docs.livekit.io/reference/server/server-apis/#roomegress) [AutoTrackEgress](https://docs.livekit.io/reference/server/server-apis/#autotrackegress) [ParticipantInfo\_State](https://docs.livekit.io/reference/server/server-apis/#participantinfo-state) [TrackSource](https://docs.livekit.io/reference/server/server-apis/#tracksource) [TrackSource](https://docs.livekit.io/reference/server/server-apis/#tracksource) [TrackType](https://docs.livekit.io/reference/server/server-apis/#tracktype) [VideoQuality](https://docs.livekit.io/reference/server/server-apis/#videoquality) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/server/server-apis/) Search --- # Deployment and scaling | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/deployment/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/deployment/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Deployment best practices](https://docs.livekit.io/agents/v0/deployment/#deployment-best-practices) [Networking](https://docs.livekit.io/agents/v0/deployment/#networking) [Environment variables](https://docs.livekit.io/agents/v0/deployment/#environment-variables) [Storage](https://docs.livekit.io/agents/v0/deployment/#storage) [Memory and CPU](https://docs.livekit.io/agents/v0/deployment/#memory-and-cpu) [Rollout](https://docs.livekit.io/agents/v0/deployment/#rollout) [Load balancing](https://docs.livekit.io/agents/v0/deployment/#load-balancing) [Worker availability](https://docs.livekit.io/agents/v0/deployment/#worker-availability) [Autoscaling](https://docs.livekit.io/agents/v0/deployment/#autoscaling) [Where to deploy](https://docs.livekit.io/agents/v0/deployment/#where-to-deploy) [Recommendations](https://docs.livekit.io/agents/v0/deployment/#recommendations) Copy pageSee more page options **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [Deploying to production](https://docs.livekit.io/agents/ops/deployment/) . _v1.0 for Node.js is coming soon._ Agent workers are designed to be deployable and scalable by default. When you start your worker locally, you are adding it to a pool (of one). LiveKit automatically load balances across available workers in the pool. This works the same way in production, the only difference is the worker is deployed somewhere with more than one running instance. ![Diagram showing multiple workers running multiple instances of your Agent](https://docs.livekit.io/images/agents/agents-orchestration.svg)![Diagram showing multiple workers running multiple instances of your Agent](https://docs.livekit.io/images/agents/agents-orchestration.svg) Deployment best practices[](https://docs.livekit.io/agents/v0/deployment/#deployment-best-practices) ----------------------------------------------------------------------------------------------------- ### Networking[](https://docs.livekit.io/agents/v0/deployment/#networking) Workers communicate with LiveKit and accept incoming jobs via a WebSocket connection to a LiveKit server. This means that workers are not web servers and do not need to be exposed to the public internet. No inbound hosts or ports need to be exposed. Workers can optionally expose a health check endpoint for monitoring purposes. This is not required for normal operation. The default health check server listens on `http://0.0.0.0:8081/`. ### Environment variables[](https://docs.livekit.io/agents/v0/deployment/#environment-variables) Your production worker will need certain environment variables configured. A minimal worker requires the LiveKit URL, API key and secret: * `LIVEKIT_URL` * `LIVEKIT_API_KEY` * `LIVEKIT_API_SECRET` Depending on the plugins your agent uses, you might need additional environment variables: * `DEEPGRAM_API_KEY` * `CARTESIA_API_KEY` * `OPENAI_API_KEY` * etc. **Important** If you use a `LIVEKIT_URL`, `LIVEKIT_API_KEY`, and `LIVEKIT_API_SECRET` from the same project that you use for local development, your local worker will join the same pool as your production workers. This means real users could be connected to your local worker. This is usually not what you want so make sure to use a different project for local development. ### Storage[](https://docs.livekit.io/agents/v0/deployment/#storage) Workers are stateless and do not require persistent storage. The minimal docker image is about 1GB in size. For ephemeral storage, 10GB should be more than enough to account for the docker image size and any temporary files that are created. ### Memory and CPU[](https://docs.livekit.io/agents/v0/deployment/#memory-and-cpu) Different agents will have different memory and CPU requirements. To help guide your scaling decisions, we ran a load test that approximates the load of a voice-to-voice session on a 4-Core, 8GB machine. **Tip** During the automated load test we also added one human participant interacting with a voice assistant agent to make sure quality of service was maintained. This test created 30 agents corresponding to 30 users (so 60 participants in total). The users published looping speech audio. The agents were subscribed to their corresponding user's audio and running the Silero voice activity detection plugin against that audio. The agents also published their own audio which was a simple sine wave. In short, this test was designed to evaluate a voice assistant use case where the agent is listening to user speech, running VAD, and publishing audio back to the user. The results of running the above test on a 4-Core, 8GB machine are: CPU Usage: ~3.8 cores Memory usage: ~2.8GB To be safe and account for spikes, we recommend 4 cores for every 25 voice agents. ### Rollout[](https://docs.livekit.io/agents/v0/deployment/#rollout) Workers stop accepting jobs when they receive a SIGINT or SIGTERM. Agents that are still running on the worker continue to run. It's important that you configure a large enough grace period for your containers to allow agents to finish. Voice agents could require a 10+ minute grace period to allow for conversations to finish. Different deployment platforms have different ways of setting this grace period. In Kubernetes, it's the `terminationGracePeriodSeconds` field in the pod spec. Consult your deployment platform's documentation for more information. Load balancing[](https://docs.livekit.io/agents/v0/deployment/#load-balancing) ------------------------------------------------------------------------------- Workers don't need an external load balancer. They rely on a job distribution system embedded within LiveKit servers. This system is responsible for ensuring that when a job becomes available (e.g. a new room is created), it is dispatched to only one worker at a time. If a worker fails to accept the job within a predetermined timeout period, the job is routed to another available worker. In the case of LiveKit Cloud, the system prioritizes available workers at the "edge" or geographically closest to the end-user. Within a region, job distribution is uniform across workers. Worker availability[](https://docs.livekit.io/agents/v0/deployment/#worker-availability) ----------------------------------------------------------------------------------------- As mentioned in the Load Balancing section, LiveKit will automatically distribute load across available workers. This means that LiveKit needs a way to know which workers are available. This "worker availability" is defined by the `load_fnc` and `load_threshold` in the `WorkerOptions` configuration. The `load_fnc` returns a value between 0 and 1, indicating how busy the worker is while `load_threshold`, a value between 0 and 1, is that load value at which the worker will stop accepting new jobs. By default, the `load_fnc` returns the CPU usage of the worker and the `load_threshold` is 0.75. Autoscaling[](https://docs.livekit.io/agents/v0/deployment/#autoscaling) ------------------------------------------------------------------------- Many voice agent use cases have non-uniform load patterns over a day/week/month so it's a good idea to configure an autoscaler. An autoscaler should be configured at a _lower_ threshold than the worker's `load_threshold`. This allows for existing workers to continue to accept new jobs while additional workers are still starting up. Since voice agents are typically long running tasks (relative to typical web requests), rapid increases in load are more likely to be sustained. In technical terms: spikes are less spikey. For your autoscaling configuration, you should consider _reducing_ cooldown/stabilization periods when scaling up. When scaling down, consider _increasing_ cooldown/stabilization periods because workers take time to drain. For example, if deploying on Kubernetes using a Horizontal Pod Autoscaler, see [stabilizationWindowSeconds](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#default-behavior) . Where to deploy[](https://docs.livekit.io/agents/v0/deployment/#where-to-deploy) --------------------------------------------------------------------------------- There are many ways to deploy software to a production environment. We provide some platform-specific [deployment examples](https://github.com/livekit-examples/agent-deployment) . All of the examples assume Docker containers are used for deployment. ### Recommendations[](https://docs.livekit.io/agents/v0/deployment/#recommendations) * **Render.com**: We (and other builders in the LiveKit community) have found Render.com to be the easiest way to deploy and autoscale workers. We provide an example `render.yaml` and instructions in the [deployment examples](https://github.com/livekit-examples/agent-deployment) repo. * **Kubernetes**: If you have a running Kubernetes cluster, it makes sense to deploy your workers there. We provide an example manifest in the deployment example repo. On this page [Deployment best practices](https://docs.livekit.io/agents/v0/deployment/#deployment-best-practices) [Networking](https://docs.livekit.io/agents/v0/deployment/#networking) [Environment variables](https://docs.livekit.io/agents/v0/deployment/#environment-variables) [Storage](https://docs.livekit.io/agents/v0/deployment/#storage) [Memory and CPU](https://docs.livekit.io/agents/v0/deployment/#memory-and-cpu) [Rollout](https://docs.livekit.io/agents/v0/deployment/#rollout) [Load balancing](https://docs.livekit.io/agents/v0/deployment/#load-balancing) [Worker availability](https://docs.livekit.io/agents/v0/deployment/#worker-availability) [Autoscaling](https://docs.livekit.io/agents/v0/deployment/#autoscaling) [Where to deploy](https://docs.livekit.io/agents/v0/deployment/#where-to-deploy) [Recommendations](https://docs.livekit.io/agents/v0/deployment/#recommendations) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/deployment/) Search --- # Voice AI quickstart | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/start/voice-ai/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/voice-ai/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/start/voice-ai/#overview) [Requirements](https://docs.livekit.io/agents/start/voice-ai/#requirements) [Python](https://docs.livekit.io/agents/start/voice-ai/#python) [LiveKit server](https://docs.livekit.io/agents/start/voice-ai/#livekit-server) [AI providers](https://docs.livekit.io/agents/start/voice-ai/#ai-providers) [Setup](https://docs.livekit.io/agents/start/voice-ai/#setup) [Packages](https://docs.livekit.io/agents/start/voice-ai/#packages) [Environment variables](https://docs.livekit.io/agents/start/voice-ai/#environment-variables) [Agent code](https://docs.livekit.io/agents/start/voice-ai/#agent-code) [Download model files](https://docs.livekit.io/agents/start/voice-ai/#download-model-files) [Speak to your agent](https://docs.livekit.io/agents/start/voice-ai/#speak-to-your-agent) [Connect to playground](https://docs.livekit.io/agents/start/voice-ai/#connect-to-playground) [Next steps](https://docs.livekit.io/agents/start/voice-ai/#next-steps) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/start/voice-ai/#overview) -------------------------------------------------------------------- This guide walks you through the setup of your very first voice assistant using LiveKit Agents for Python. In less than 10 minutes, you'll have a voice assistant that you can speak to in your terminal, browser, telephone, or native app. [Python starter project\ ----------------------\ \ Prefer to just clone a repo? This repo is ready-to-go, will all the code you need to get started.\ \ GitHublivekit-examples/agent-starter-python](https://github.com/livekit-examples/agent-starter-python) [Deeplearning.ai course\ ----------------------\ \ Learn to build and deploy voice agents with LiveKit in this free course from Deeplearning.ai.](https://www.deeplearning.ai/short-courses/building-ai-voice-agents-for-production/) Requirements[](https://docs.livekit.io/agents/start/voice-ai/#requirements) ---------------------------------------------------------------------------- The following sections describe the minimum requirements to get started with LiveKit Agents. ### Python[](https://docs.livekit.io/agents/start/voice-ai/#python) LiveKit Agents requires Python 3.9 or later. **Looking for Node.js?** The Node.js beta is still in development and has not yet reached v1.0. See the [v0.x documentation](https://docs.livekit.io/agents/v0/) for Node.js reference and join the [LiveKit Community Slack](https://livekit.io/join-slack) to be the first to know when the next release is available. ### LiveKit server[](https://docs.livekit.io/agents/start/voice-ai/#livekit-server) You need a LiveKit server instance to transport realtime media between user and agent. The easiest way to get started is with a free [LiveKit Cloud](https://cloud.livekit.io/) account. Create a project and use the API keys in the following steps. You may also [self-host LiveKit](https://docs.livekit.io/home/self-hosting/local/) if you prefer. ### AI providers[](https://docs.livekit.io/agents/start/voice-ai/#ai-providers) LiveKit Agents [integrates with most AI model providers](https://docs.livekit.io/agents/integrations/) and supports both high-performance STT-LLM-TTS voice pipelines, as well as lifelike multimodal models. The rest of this guide assumes you use one of the following two starter packs, which provide the best combination of value, features, and ease of setup. STT-LLM-TTS pipelineRealtime model Your agent strings together three specialized providers into a high-performance voice pipeline. You need accounts and API keys for each. ![Diagram showing STT-LLM-TTS pipeline.](https://docs.livekit.io/images/agents/stt-llm-tts-pipeline.svg)![Diagram showing STT-LLM-TTS pipeline.](https://docs.livekit.io/images/agents/stt-llm-tts-pipeline.svg) | Component | Provider | Required Key | Alternatives | | --- | --- | --- | --- | | STT | [Deepgram](https://deepgram.com/) | `DEEPGRAM_API_KEY` | [STT integrations](https://docs.livekit.io/agents/integrations/stt/#providers) | | LLM | [OpenAI](https://platform.openai.com/) | `OPENAI_API_KEY` | [LLM integrations](https://docs.livekit.io/agents/integrations/llm/#providers) | | TTS | [Cartesia](https://cartesia.ai/) | `CARTESIA_API_KEY` | [TTS integrations](https://docs.livekit.io/agents/integrations/tts/#providers) | Setup[](https://docs.livekit.io/agents/start/voice-ai/#setup) -------------------------------------------------------------- Use the instructions in the following sections to set up your new project. ### Packages[](https://docs.livekit.io/agents/start/voice-ai/#packages) **Noise cancellation** This example integrates LiveKit Cloud [enhanced background voice/noise cancellation](https://docs.livekit.io/home/cloud/noise-cancellation/) , powered by Krisp. If you're not using LiveKit Cloud, omit the plugin and the `noise_cancellation` parameter from the following code. For telephony applications, use the `BVCTelephony` model for the best results. STT-LLM-TTS pipelineRealtime model Install the following packages to build a complete voice AI agent with your STT-LLM-TTS pipeline, noise cancellation, and [turn detection](https://docs.livekit.io/agents/build/turns/) : pip install \\ "livekit-agents\[deepgram,openai,cartesia,silero,turn-detector\]~=1.0" \\ "livekit-plugins-noise-cancellation~=0.2" \\ "python-dotenv" ### Environment variables[](https://docs.livekit.io/agents/start/voice-ai/#environment-variables) Create a file named `.env` and add your LiveKit credentials along with the necessary API keys for your AI providers. STT-LLM-TTS pipelineRealtime model .env DEEPGRAM\_API\_KEY\= OPENAI\_API\_KEY\= CARTESIA\_API\_KEY\= LIVEKIT\_API\_KEY\= LIVEKIT\_API\_SECRET\= LIVEKIT\_URL\= Reveal API Key and Secret ### Agent code[](https://docs.livekit.io/agents/start/voice-ai/#agent-code) Create a file named `agent.py` containing the following code for your first voice agent. STT-LLM-TTS pipelineRealtime model agent.py from dotenv import load\_dotenv from livekit import agents from livekit.agents import AgentSession, Agent, RoomInputOptions from livekit.plugins import ( openai, cartesia, deepgram, noise\_cancellation, silero, ) from livekit.plugins.turn\_detector.multilingual import MultilingualModel load\_dotenv() class Assistant(Agent): def \_\_init\_\_(self) \-\> None: super().\_\_init\_\_(instructions\="You are a helpful voice AI assistant.") async def entrypoint(ctx: agents.JobContext): session \= AgentSession( stt\=deepgram.STT(model\="nova-3", language\="multi"), llm\=openai.LLM(model\="gpt-4o-mini"), tts\=cartesia.TTS(model\="sonic-2", voice\="f786b574-daa5-4673-aa0c-cbe3e8534c02"), vad\=silero.VAD.load(), turn\_detection\=MultilingualModel(), ) await session.start( room\=ctx.room, agent\=Assistant(), room\_input\_options\=RoomInputOptions( \# LiveKit Cloud enhanced noise cancellation \# - If self-hosting, omit this parameter \# - For telephony applications, use \`BVCTelephony\` for best results noise\_cancellation\=noise\_cancellation.BVC(), ), ) await session.generate\_reply( instructions\="Greet the user and offer your assistance." ) if \_\_name\_\_ \== "\_\_main\_\_": agents.cli.run\_app(agents.WorkerOptions(entrypoint\_fnc\=entrypoint)) Download model files[](https://docs.livekit.io/agents/start/voice-ai/#download-model-files) -------------------------------------------------------------------------------------------- To use the `turn-detector`, `silero`, or `noise-cancellation` plugins, you first need to download the model files: python agent.py download-files Speak to your agent[](https://docs.livekit.io/agents/start/voice-ai/#speak-to-your-agent) ------------------------------------------------------------------------------------------ Start your agent in `console` mode to run inside your terminal: python agent.py console Your agent speaks to you in the terminal, and you can speak to it as well. ![Screenshot of the CLI console mode.](https://docs.livekit.io/images/agents/start/cli-console.png)![Screenshot of the CLI console mode.](https://docs.livekit.io/images/agents/start/cli-console.png) Connect to playground[](https://docs.livekit.io/agents/start/voice-ai/#connect-to-playground) ---------------------------------------------------------------------------------------------- Start your agent in `dev` mode to connect it to LiveKit and make it available from anywhere on the internet: python agent.py dev Use the [Agents playground](https://docs.livekit.io/agents/start/playground/) to speak with your agent and explore its full range of multimodal capabilities. Congratulations, your agent is up and running. Continue to use the playground or the `console` mode as you build and test your agent. **Agent CLI modes** In the `console` mode, the agent runs locally and is only available within your terminal. Run your agent in `dev` (development / debug) or `start` (production) mode to connect to LiveKit and join rooms. Next steps[](https://docs.livekit.io/agents/start/voice-ai/#next-steps) ------------------------------------------------------------------------ Follow these guides bring your voice AI app to life in the real world. [Web and mobile frontends\ ------------------------\ \ Put your agent in your pocket with a custom web or mobile app.](https://docs.livekit.io/agents/start/frontend/) [Telephony integration\ ---------------------\ \ Your agent can place and receive calls with LiveKit's SIP integration.](https://docs.livekit.io/agents/start/telephony/) [Testing your agent\ ------------------\ \ Add behavioral tests to fine-tune your agent's behavior.](https://docs.livekit.io/agents/build/testing/) [Building voice agents\ ---------------------\ \ Comprehensive documentation to build advanced voice AI apps with LiveKit.](https://docs.livekit.io/agents/build/) [Worker lifecycle\ ----------------\ \ Learn how to manage your agents with workers and jobs.](https://docs.livekit.io/agents/worker/) [Deploying to production\ -----------------------\ \ Deploy your new voice agent to production.](https://docs.livekit.io/agents/ops/deployment/) [Integration guides\ ------------------\ \ Explore the full list of AI providers available for LiveKit Agents.](https://docs.livekit.io/agents/integrations/) [Recipes\ -------\ \ A comprehensive collection of examples, guides, and recipes for LiveKit Agents.](https://docs.livekit.io/recipes/) On this page [Overview](https://docs.livekit.io/agents/start/voice-ai/#overview) [Requirements](https://docs.livekit.io/agents/start/voice-ai/#requirements) [Python](https://docs.livekit.io/agents/start/voice-ai/#python) [LiveKit server](https://docs.livekit.io/agents/start/voice-ai/#livekit-server) [AI providers](https://docs.livekit.io/agents/start/voice-ai/#ai-providers) [Setup](https://docs.livekit.io/agents/start/voice-ai/#setup) [Packages](https://docs.livekit.io/agents/start/voice-ai/#packages) [Environment variables](https://docs.livekit.io/agents/start/voice-ai/#environment-variables) [Agent code](https://docs.livekit.io/agents/start/voice-ai/#agent-code) [Download model files](https://docs.livekit.io/agents/start/voice-ai/#download-model-files) [Speak to your agent](https://docs.livekit.io/agents/start/voice-ai/#speak-to-your-agent) [Connect to playground](https://docs.livekit.io/agents/start/voice-ai/#connect-to-playground) [Next steps](https://docs.livekit.io/agents/start/voice-ai/#next-steps) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/voice-ai/) Search --- # Agents v0.x migration guide | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/start/v0-migration/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/v0-migration/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Unified agent interface](https://docs.livekit.io/agents/start/v0-migration/#unified-agent-interface) [Customizing pipeline behavior](https://docs.livekit.io/agents/start/v0-migration/#customizing-pipeline-behavior) [before\_llm\_cb -> llm\_node](https://docs.livekit.io/agents/start/v0-migration/#before-llm-cb-llm-node) [before\_tts\_cb -> tts\_node](https://docs.livekit.io/agents/start/v0-migration/#before-tts-cb-tts-node) [Tool definition and use](https://docs.livekit.io/agents/start/v0-migration/#tool-definition-and-use) [Chat context](https://docs.livekit.io/agents/start/v0-migration/#chat-context) [Updating chat context](https://docs.livekit.io/agents/start/v0-migration/#updating-chat-context) [Transcriptions](https://docs.livekit.io/agents/start/v0-migration/#transcriptions) [Accepting text input](https://docs.livekit.io/agents/start/v0-migration/#accepting-text-input) [State change events](https://docs.livekit.io/agents/start/v0-migration/#state-change-events) [User state](https://docs.livekit.io/agents/start/v0-migration/#user-state) [Agent state](https://docs.livekit.io/agents/start/v0-migration/#agent-state) [Other events](https://docs.livekit.io/agents/start/v0-migration/#other-events) [Removed features](https://docs.livekit.io/agents/start/v0-migration/#removed-features) Copy pageSee more page options Unified agent interface[](https://docs.livekit.io/agents/start/v0-migration/#unified-agent-interface) ------------------------------------------------------------------------------------------------------ Agents 1.0 introduces `AgentSession`, a single, unified [agent orchestrator](https://docs.livekit.io/agents/build/#agent-sessions) that serves as the foundation for all types of agents built using the framework. With this change, the `VoicePipelineAgent` and `MultimodalAgent` classes have been deprecated and 0.x agents will need to be updated to use `AgentSession` in order to be compatible with 1.0 and later. `AgentSession` contains a superset of the functionality of `VoicePipelineAgent` and `MultimodalAgent`, allowing you to switch between pipelined and speech-to-speech models without changing your core application logic. **Note** The following code highlights the differences between Agents v0.x and Agents 1.0. For a full working example, see the [Voice AI quickstart](https://docs.livekit.io/agents/starts/voice-ai/) . Version 0.xVersion 1.0 from livekit.agents import JobContext, llm from livekit.agents.pipeline import VoicePipelineAgent from livekit.plugins import ( cartesia, deepgram, google, silero, ) async def entrypoint(ctx: JobContext): initial\_ctx \= llm.ChatContext().append( role\="system", text\="You are a helpful voice AI assistant.", ) agent \= VoicePipelineAgent( vad\=silero.VAD.load(), stt\=deepgram.STT(), llm\=google.LLM(), tts\=cartesia.TTS(), ) await agent.start(room, participant) await agent.say("Hey, how can I help you today?", allow\_interruptions\=True) Customizing pipeline behavior[](https://docs.livekit.io/agents/start/v0-migration/#customizing-pipeline-behavior) ------------------------------------------------------------------------------------------------------------------ We’ve introduced more flexibility for developers to customize the behavior of agents built on 1.0 through the new concept of [pipeline nodes](https://docs.livekit.io/agents/build/nodes/) , which enable custom processing within the pipeline steps while also delegating to the default implementation of each node as needed. Pipeline nodes replaces the `before_llm_cb` and `before_tts_cb` callbacks. ### before\_llm\_cb -> llm\_node[](https://docs.livekit.io/agents/start/v0-migration/#before-llm-cb-llm-node) `before_llm_cb` has been replaced by `llm_node`. This node can be used to modify the chat context before sending it to LLM, or integrate with custom LLM providers without having to create a plugin. As long as it returns AsyncIterable\[llm.ChatChunk\], the LLM node will forward the chunks to the next node in the pipeline. Version 0.xVersion 1.0 async def add\_rag\_context(assistant: VoicePipelineAgent, chat\_ctx: llm.ChatContext): rag\_context: str \= retrieve(chat\_ctx) chat\_ctx.append(text\=rag\_context, role\="system") agent \= VoicePipelineAgent( ... before\_llm\_cb\=add\_rag\_context, ) ### before\_tts\_cb -> tts\_node[](https://docs.livekit.io/agents/start/v0-migration/#before-tts-cb-tts-node) `before_tts_cb` has been replaced by `tts_node`. This node gives greater flexibility in customizing the TTS pipeline. It's possible to modify the text before synthesis, as well as the audio buffers after synthesis. Version 0.xVersion 1.0 def \_before\_tts\_cb(agent: VoicePipelineAgent, text: str | AsyncIterable\[str\]): \# The TTS is incorrectly pronouncing "LiveKit", so we'll replace it with MFA-style IPA \# spelling for Cartesia return tokenize.utils.replace\_words( text\=text, replacements\={"livekit": r"<>"} ) agent \= VoicePipelineAgent( ... before\_tts\_cb\=\_before\_tts\_cb, ) Tool definition and use[](https://docs.livekit.io/agents/start/v0-migration/#tool-definition-and-use) ------------------------------------------------------------------------------------------------------ Agents 1.0 streamlines the way in which [tools](https://docs.livekit.io/agents/build/tools/) are defined for use within your agents, making it easier to add and maintain agent tools. When migrating from 0.x to 1.0, developers will need to make the following changes to existing use of functional calling within their agents in order to be compatible with versions 1.0 and later. * The `@llm.ai_callable` decorator for function definition has been replaced with the new `@function_tool` decorator. * If you define your functions within an `Agent` and use the `@function_tool` decorator, these tools are automatically accessible to the LLM. In this scenario, you no longer required to define your functions in a `llm.FunctionContext` class and pass them into the agent constructor. * Argument types are now inferred from the function signature and docstring. Annotated types are no longer supported. * Functions take in a `RunContext` object, which provides access to the current agent state. Version 0.xVersion 1.0 from livekit.agents import llm from livekit.agents.pipeline import VoicePipelineAgent from livekit.agents.multimodal import MultimodalAgent class AssistantFnc(llm.FunctionContext): @llm.ai\_callable() async def get\_weather( self, ... ) ... fnc\_ctx \= AssistantFnc() pipeline\_agent \= VoicePipelineAgent( ... fnc\_ctx\=fnc\_ctx, ) multimodal\_agent \= MultimodalAgent( ... fnc\_ctx\=fnc\_ctx, ) Chat context[](https://docs.livekit.io/agents/start/v0-migration/#chat-context) -------------------------------------------------------------------------------- ChatContext has been overhauled in 1.0 to provide a more powerful and flexible API for managing chat history. It now accounts for differences between LLM providers—such as stateless and stateful APIs—while exposing a unified interface. Chat history can now include three types of items: * `ChatMessage`: a message associated with a role (e.g., user, assistant). Each message includes a list of `content` items, which can contain text, images, or audio. * `FunctionCall`: a function call initiated by the LLM. * `FunctionCallOutput`: the result returned from a function call. ### Updating chat context[](https://docs.livekit.io/agents/start/v0-migration/#updating-chat-context) In 0.x, updating the chat context required modifying chat\_ctx.messages directly. This approach was error-prone and difficult to time correctly, especially with realtime APIs. In v1.x, there are two supported ways to update the chat context: * **Agent handoff** – [transferring control](https://docs.livekit.io/agents/build/workflows/#handing-off-control-to-another-agent) to a new agent, which will have its own chat context. * **Explicit update** - calling `agent.update_chat_ctx()` to modify the context directly. Transcriptions[](https://docs.livekit.io/agents/start/v0-migration/#transcriptions) ------------------------------------------------------------------------------------ Agents 1.0 brings some new changes to how [transcriptions](https://docs.livekit.io/agents/build/text/#transcriptions) are handled: * Transcriptions now use [text streams](https://docs.livekit.io/home/client/data/text-streams/) with topic `lk.transcription`. * The [old transcription protocol](https://docs.livekit.io/agents/v0/voice-agent/transcriptions/) is deprecated and will be removed in v1.1. * for now both protocols are used for backwards compatibility. * Upcoming versions SDKs/components standardize on text streams for transcriptions. Accepting text input[](https://docs.livekit.io/agents/start/v0-migration/#accepting-text-input) ------------------------------------------------------------------------------------------------ Agents 1.0 introduces [improved support for text input](https://docs.livekit.io/agents/build/text/#text-input) . Previously, text had to be manually intercepted and injected into the agent via `ChatManager`. In this version, agents automatically receive text input from a text stream on the `lk.chat` topic. The `ChatManager` has been removed in Python SDK v1.0. State change events[](https://docs.livekit.io/agents/start/v0-migration/#state-change-events) ---------------------------------------------------------------------------------------------- ### User state[](https://docs.livekit.io/agents/start/v0-migration/#user-state) `user_started_speaking` and `user_stopped_speaking` events are no longer emitted. They've been combined into a single `user_state_changed` event. Version 0.xVersion 1.0 @agent.on("user\_started\_speaking") def on\_user\_started\_speaking(): print("User started speaking") ### Agent state[](https://docs.livekit.io/agents/start/v0-migration/#agent-state) Version 0.xVersion 1.0 @agent.on("agent\_started\_speaking") def on\_agent\_started\_speaking(): \# Log transcribed message from user print("Agent started speaking") Other events[](https://docs.livekit.io/agents/start/v0-migration/#other-events) -------------------------------------------------------------------------------- Agent events were overhauled in version 1.0. For details, see the [events](https://docs.livekit.io/agents/build/events/) page. Removed features[](https://docs.livekit.io/agents/start/v0-migration/#removed-features) ---------------------------------------------------------------------------------------- * OpenAI Assistants API support has been removed in 1.0. The beta integration with the Assistants API in the OpenAI LLM plugin has been deprecated. Its stateful model made it difficult to manage state consistently between the API and agent. On this page [Unified agent interface](https://docs.livekit.io/agents/start/v0-migration/#unified-agent-interface) [Customizing pipeline behavior](https://docs.livekit.io/agents/start/v0-migration/#customizing-pipeline-behavior) [before\_llm\_cb -> llm\_node](https://docs.livekit.io/agents/start/v0-migration/#before-llm-cb-llm-node) [before\_tts\_cb -> tts\_node](https://docs.livekit.io/agents/start/v0-migration/#before-tts-cb-tts-node) [Tool definition and use](https://docs.livekit.io/agents/start/v0-migration/#tool-definition-and-use) [Chat context](https://docs.livekit.io/agents/start/v0-migration/#chat-context) [Updating chat context](https://docs.livekit.io/agents/start/v0-migration/#updating-chat-context) [Transcriptions](https://docs.livekit.io/agents/start/v0-migration/#transcriptions) [Accepting text input](https://docs.livekit.io/agents/start/v0-migration/#accepting-text-input) [State change events](https://docs.livekit.io/agents/start/v0-migration/#state-change-events) [User state](https://docs.livekit.io/agents/start/v0-migration/#user-state) [Agent state](https://docs.livekit.io/agents/start/v0-migration/#agent-state) [Other events](https://docs.livekit.io/agents/start/v0-migration/#other-events) [Removed features](https://docs.livekit.io/agents/start/v0-migration/#removed-features) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/v0-migration/) Search --- # Agent dispatch | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/worker/agent-dispatch/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/worker/agent-dispatch/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Dispatching agents](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatching-agents) [Automatic agent dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/#automatic) [Explicit agent dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/#explicit) [Dispatch via API](https://docs.livekit.io/agents/worker/agent-dispatch/#via-api) [Dispatch from inbound SIP calls](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatch-from-inbound-sip-calls) [Dispatch on participant connection](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatch-on-participant-connection) Copy pageSee more page options Dispatching agents[](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatching-agents) ----------------------------------------------------------------------------------------------- Dispatch is the process of assigning an agent to a room. LiveKit server manages this process as part of the [worker lifecycle](https://docs.livekit.io/agents/worker/) . LiveKit optimizes dispatch for high concurrency and low latency, typically supporting hundred of thousands of new connections per second with a max dispatch time under 150 ms. Automatic agent dispatch[](https://docs.livekit.io/agents/worker/agent-dispatch/#automatic) -------------------------------------------------------------------------------------------- By default, an agent is automatically dispatched to each new room. Automatic dispatch is the best option if you want to assign the same agent to all new participants. Explicit agent dispatch[](https://docs.livekit.io/agents/worker/agent-dispatch/#explicit) ------------------------------------------------------------------------------------------ Explicit dispatch is available for greater control over when and how agents join rooms. This approach leverages the same worker systems, allowing you to run agent workers in the same way. To use explicit dispatch, set the `agent_name` field in the `WorkerOptions`: PythonNode.js opts \= WorkerOptions( ... agent\_name\="test-agent", ) **Important** Automatic dispatch is disabled if the `agent_name` property is set. ### Dispatch via API[](https://docs.livekit.io/agents/worker/agent-dispatch/#via-api) Agent workers with the `agent_name` set can be explicitly dispatched to a room via `AgentDispatchService`. PythonNode.jsLiveKit CLIGo import asyncio from livekit import api room\_name \= "my-room" agent\_name \= "test-agent" async def create\_explicit\_dispatch(): lkapi \= api.LiveKitAPI() dispatch \= await lkapi.agent\_dispatch.create\_dispatch( api.CreateAgentDispatchRequest( agent\_name\=agent\_name, room\=room\_name, metadata\='{"user\_id": "12345"}' ) ) print("created dispatch", dispatch) dispatches \= await lkapi.agent\_dispatch.list\_dispatch(room\_name\=room\_name) print(f"there are {len(dispatches)} dispatches in {room\_name}") await lkapi.aclose() asyncio.run(create\_explicit\_dispatch()) The room, `my-room`, is automatically created during dispatch if it doesn't already exist, and the worker assigns `test-agent` to it. #### Job metadata[](https://docs.livekit.io/agents/worker/agent-dispatch/#job-metadata) Explicit dispatch allows you to pass metadata to the agent, available in the `JobContext`. This is useful for including details the like the user's ID, name, or phone number. The metadata field is a string. LiveKit recommends using JSON to pass structured data. The [examples](https://docs.livekit.io/agents/worker/agent-dispatch/#via-api) in the previous section demonstrate how to pass job metadata during dispatch. For information on consuming job metadata in an agent, see the following guide: [Job metadata\ ------------\ \ Learn how to consume job metadata in an agent.](https://docs.livekit.io/agents/worker/job/#metadata) ### Dispatch from inbound SIP calls[](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatch-from-inbound-sip-calls) Agents can be explicitly dispatched for inbound SIP calls. [SIP dispatch rules](https://docs.livekit.io/sip/dispatch-rule/) can define one or more agents using the `room_config.agents` field. LiveKit recommends explicit agent dispatch for SIP inbound calls rather than automatic agent dispatch as it allows multiple agents within a single project. ### Dispatch on participant connection[](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatch-on-participant-connection) You can configure a participant's token to dispatch one or more agents immediately upon connection. To dispatch multiple agents, include multiple `RoomAgentDispatch` entries in `RoomConfiguration`. The following example creates a token that dispatches the `test-agent` agent to the `my-room` room when the participant connects: PythonNode.jsGo from livekit.api import ( AccessToken, RoomAgentDispatch, RoomConfiguration, VideoGrants, ) room\_name \= "my-room" agent\_name \= "test-agent" def create\_token\_with\_agent\_dispatch() \-\> str: token \= ( AccessToken() .with\_identity("my\_participant") .with\_grants(VideoGrants(room\_join\=True, room\=room\_name)) .with\_room\_config( RoomConfiguration( agents\=\[\ \ RoomAgentDispatch(agent\_name\="test-agent", metadata\='{"user\_id": "12345"}')\ \ \], ), ) .to\_jwt() ) return token On this page [Dispatching agents](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatching-agents) [Automatic agent dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/#automatic) [Explicit agent dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/#explicit) [Dispatch via API](https://docs.livekit.io/agents/worker/agent-dispatch/#via-api) [Dispatch from inbound SIP calls](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatch-from-inbound-sip-calls) [Dispatch on participant connection](https://docs.livekit.io/agents/worker/agent-dispatch/#dispatch-on-participant-connection) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/worker/agent-dispatch/) Search --- # Web and mobile frontends | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/start/frontend/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/frontend/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/start/frontend/#overview) [Starter apps](https://docs.livekit.io/agents/start/frontend/#starter-apps) [Media and text](https://docs.livekit.io/agents/start/frontend/#media-and-text) [Data sharing](https://docs.livekit.io/agents/start/frontend/#data-sharing) [State and control](https://docs.livekit.io/agents/start/frontend/#state-and-control) [Audio visualizer](https://docs.livekit.io/agents/start/frontend/#audio-visualizer) [Authentication](https://docs.livekit.io/agents/start/frontend/#authentication) [Virtual avatars](https://docs.livekit.io/agents/start/frontend/#virtual-avatars) [Responsiveness tips](https://docs.livekit.io/agents/start/frontend/#responsiveness-tips) [Minimize connection time](https://docs.livekit.io/agents/start/frontend/#minimize-connection-time) [Connection indicators](https://docs.livekit.io/agents/start/frontend/#connection-indicators) [Effects](https://docs.livekit.io/agents/start/frontend/#effects) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/start/frontend/#overview) -------------------------------------------------------------------- LiveKit Agents is ready to integrate with your preferred frontend platform using the [LiveKit SDKs](https://docs.livekit.io/home/client/connect/) for JavaScript, Swift, Android, Flutter, React Native, and more. Your agent can communicate with your frontend through LiveKit WebRTC, which provides fast and reliable realtime connectivity. For example, a simple voice agent subscribes to the user's microphone track and publishes its own. [Text transcriptions](https://docs.livekit.io/agents/build/text/) are also available as text streams. A more complex agent with vision capabilities can subscribe to a video track published from the user's camera or shared screen. An agent can also publish its own video to implement a virtual avatar or other features. In all of these cases, the LiveKit SDKs are production grade and easy to use so you can build useful and advanced agents without worrying about the complexities of realtime media delivery. This topic contains resources and tips for building a high-quality frontend for your agent. Starter apps[](https://docs.livekit.io/agents/start/frontend/#starter-apps) ---------------------------------------------------------------------------- LiveKit recommends using one of the following starter apps to get up and running quickly on your preferred platform. Each app is open source under the MIT License so you can freely modify it to your own needs. The mobile apps require a hosted token server, but include a [LiveKit Cloud Sandbox](https://cloud.livekit.io/projects/p_/sandbox/templates/token-server) for development purposes. http://localhost:3000 ![](https://docs.livekit.io/images/agents/start/frontend/agent-starter-react-screenshot-dark.png)![](https://docs.livekit.io/images/agents/start/frontend/agent-starter-react-screenshot-light.png) [Swift\ \ SwiftUI Voice Agent\ -------------------\ \ A native iOS, macOS, and visionOS voice AI assistant built in SwiftUI.\ \ GitHublivekit-examples/agent-starter-swift](https://github.com/livekit-examples/agent-starter-swift) [Next.js\ \ Next.js Voice Agent\ -------------------\ \ A web voice AI assistant built with React and Next.js.\ \ GitHublivekit-examples/agent-starter-react](https://github.com/livekit-examples/agent-starter-react) [Flutter\ \ Flutter Voice Agent\ -------------------\ \ A cross-platform voice AI assistant app built with Flutter.\ \ GitHublivekit-examples/agent-starter-flutter](https://github.com/livekit-examples/agent-starter-flutter) [React\ \ React Native Voice Agent\ ------------------------\ \ A native voice AI assistant app built with React Native and Expo.\ \ GitHublivekit-examples/agent-starter-react-native](https://github.com/livekit-examples/agent-starter-react-native) [Android\ \ Android Voice Agent\ -------------------\ \ A native Android voice AI assistant app built with Kotlin and Jetpack Compose.\ \ GitHublivekit-examples/agent-starter-android](https://github.com/livekit-examples/agent-starter-android) [Web Embed Voice Agent\ ---------------------\ \ A voice AI agent that can be embedded in any web page.\ \ GitHublivekit-examples/agent-starter-embed](https://github.com/livekit-examples/agent-starter-embed) Media and text[](https://docs.livekit.io/agents/start/frontend/#media-and-text) -------------------------------------------------------------------------------- To learn more about realtime media and text streams, see the following documentation. [Media tracks\ ------------\ \ Use the microphone, speaker, cameras, and screenshare with your agent.](https://docs.livekit.io/home/client/tracks/) [Text streams\ ------------\ \ Send and receive realtime text and transcriptions.](https://docs.livekit.io/home/client/data/text-streams/) Data sharing[](https://docs.livekit.io/agents/start/frontend/#data-sharing) ---------------------------------------------------------------------------- To share images, files, or any other kind of data between your frontend and your agent, you can use the following features. [Byte streams\ ------------\ \ Send and receive images, files, or any other data.](https://docs.livekit.io/home/client/data/byte-streams/) [Data packets\ ------------\ \ Low-level API for sending and receiving any kind of data.](https://docs.livekit.io/home/client/data/packets/) State and control[](https://docs.livekit.io/agents/start/frontend/#state-and-control) -------------------------------------------------------------------------------------- In some cases, your agent and your frontend code might need a custom integration of state and configuration to meet your application's requirements. In these cases, the LiveKit realtime state and data features can be used to create a tightly-coupled and responsive experience. AgentSession automatically manages the `lk.agent.state` participant attribute to contain the appropriate string value from among `initializing`, `listening`, `thinking`, or `speaking`. [State synchronization\ ---------------------\ \ Share custom state between your frontend and agent.](https://docs.livekit.io/home/client/state/) [RPC\ ---\ \ Define and call methods on your agent or your frontend from the other side.](https://docs.livekit.io/home/client/data/rpc/) Audio visualizer[](https://docs.livekit.io/agents/start/frontend/#audio-visualizer) ------------------------------------------------------------------------------------ The LiveKit component SDKs for React, SwiftUI, Android Compose, and Flutter include an audio visualizer component that can be used to give your voice agent a visual presence in your application. For complete examples, see the sample apps listed above. The following documentation is a quick guide to using these components: ReactSwiftAndroidFlutter Install the [React components](https://github.com/livekit/components-js/tree/main/packages/react) and [styles](https://github.com/livekit/components-js/tree/main/packages/styles) packages to use the [useVoiceAssistant](https://docs.livekit.io/reference/components/react/hook/usevoiceassistant/) hook and the [BarVisualizer](https://docs.livekit.io/reference/components/react/component/barvisualizer/) . These components work automatically within a [LiveKitRoom](https://docs.livekit.io/reference/components/react/component/livekitroom/) or [RoomContext.Provider](https://docs.livekit.io/reference/components/react/component/roomcontext/) ). Also see [VoiceAssistantControlBar](https://docs.livekit.io/reference/components/react/component/voiceassistantcontrolbar/) , which provides a simple set of common UI controls for voice agent applications. "use client"; import "@livekit/components-styles"; import { useVoiceAssistant, BarVisualizer, } from "@livekit/components-react"; export default function SimpleVoiceAssistant() { // Get the agent's audio track and current state const { state, audioTrack } \= useVoiceAssistant(); return (

{state} ); } Authentication[](https://docs.livekit.io/agents/start/frontend/#authentication) -------------------------------------------------------------------------------- The LiveKit SDKs require a [token](https://docs.livekit.io/home/get-started/authentication/) to connect to a room. In web apps, you can typically include a simple token endpoint as part of the app. For mobile apps, you need a separate [token server](https://docs.livekit.io/home/server/generating-tokens/) . Virtual avatars[](https://docs.livekit.io/agents/start/frontend/#virtual-avatars) ---------------------------------------------------------------------------------- Your frontend can include a video representation of your agent using a virtual avatar from a supported provider. LiveKit includes full support for video rendering on all supported platforms. The [starter apps](https://docs.livekit.io/agents/start/frontend/#starter-apps) include support for virtual avatars. For more information and a list of supported providers, consult the documentation: [Virtual avatars\ ---------------\ \ Use a virtual avatar to give your agent a visual presence in your app.](https://docs.livekit.io/agents/integrations/avatar/) Responsiveness tips[](https://docs.livekit.io/agents/start/frontend/#responsiveness-tips) ------------------------------------------------------------------------------------------ This section contains some suggestions to make your app feel more responsive to the user. ### Minimize connection time[](https://docs.livekit.io/agents/start/frontend/#minimize-connection-time) To connect your user to your agent, these steps must all occur: 1. Fetch an access token. 2. The user connects to the room. 3. Dispatch an agent process. 4. The agent connects to the room. 5. User and agent publish and subscribe to each other's media tracks. If done in sequence, this takes up to a few seconds to complete. You can reduce this time by eliminating or parallelizing these steps. **Option 1: "Warm" token** In this case, your application will generate a token for the user at login with a long expiration time. When you need to connect to the room, the token is already available in your frontend. **Option 2: Dispatch agent during token generation** In this case, your application will optimistically create a room and dispatch the agent at the same time the token is generated, using [explicit agent dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/#explicit) . This allows the user and the agent to connect to the room at the same time. ### Connection indicators[](https://docs.livekit.io/agents/start/frontend/#connection-indicators) Make your app feel more responsive, even when slow to connect, by linking various events into only one or two status indicators for the user rather than a number of discrete steps and UI changes. Refer to the [event handling](https://docs.livekit.io/home/client/events/) documentation for more information on how to monitor the connection state and other events. In the case that your agent fails to connect, you should notify the user and allow them to try again rather than leaving them to speak into an empty room. * **Room connection**: The `room.connect` method can be awaited in most SDKs, and most also provide a `room.connectionState` property. Also monitor the `Disconnected` event to know when the connection is lost. * **Agent presence**: Monitor `ParticipantConnected` events with `participant.kind === ParticipantKind.AGENT` * **Agent state**: Access the agent's state (`initializing`, `listening`, `thinking`, or `speaking`) * **Track subscription**: Listen for `TrackSubscribed` events to know when your media has been subscribed to. ### Effects[](https://docs.livekit.io/agents/start/frontend/#effects) You should use sound effects, haptic feedback, and visual effects to make your agent feel more responsive. This is especially important during long thinking states (for instance, when performing external lookups or tool use). The [visualizer](https://docs.livekit.io/agents/start/frontend/#audio-visualizer) includes basic "thinking" state indication and also allows the user to notice when their audio is not working. For more advanced effects, use the [state and control](https://docs.livekit.io/agents/start/frontend/#state-control) features to trigger effects in your frontend. On this page [Overview](https://docs.livekit.io/agents/start/frontend/#overview) [Starter apps](https://docs.livekit.io/agents/start/frontend/#starter-apps) [Media and text](https://docs.livekit.io/agents/start/frontend/#media-and-text) [Data sharing](https://docs.livekit.io/agents/start/frontend/#data-sharing) [State and control](https://docs.livekit.io/agents/start/frontend/#state-and-control) [Audio visualizer](https://docs.livekit.io/agents/start/frontend/#audio-visualizer) [Authentication](https://docs.livekit.io/agents/start/frontend/#authentication) [Virtual avatars](https://docs.livekit.io/agents/start/frontend/#virtual-avatars) [Responsiveness tips](https://docs.livekit.io/agents/start/frontend/#responsiveness-tips) [Minimize connection time](https://docs.livekit.io/agents/start/frontend/#minimize-connection-time) [Connection indicators](https://docs.livekit.io/agents/start/frontend/#connection-indicators) [Effects](https://docs.livekit.io/agents/start/frontend/#effects) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/start/frontend/) Search --- # Deploying to production | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/ops/deployment/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/ops/deployment/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/ops/deployment/#overview) [Project setup](https://docs.livekit.io/agents/ops/deployment/#project-setup) [Where to deploy](https://docs.livekit.io/agents/ops/deployment/#where-to-deploy) [Networking](https://docs.livekit.io/agents/ops/deployment/#networking) [Environment variables](https://docs.livekit.io/agents/ops/deployment/#environment-variables) [Storage](https://docs.livekit.io/agents/ops/deployment/#storage) [Memory and CPU](https://docs.livekit.io/agents/ops/deployment/#memory-and-cpu) [Rollout](https://docs.livekit.io/agents/ops/deployment/#rollout) [Load balancing](https://docs.livekit.io/agents/ops/deployment/#load-balancing) [Worker availability](https://docs.livekit.io/agents/ops/deployment/#worker-availability) [Autoscaling](https://docs.livekit.io/agents/ops/deployment/#autoscaling) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/ops/deployment/#overview) -------------------------------------------------------------------- LiveKit agents are ready to deploy to any container orchestration system such as Kubernetes. The framework uses a worker pool model and job dispatch is automatically balanced by LiveKit server across available workers. The workers themselves spawn a new sub-process for each job, and that job is where your code and agent participant run. Project setup[](https://docs.livekit.io/agents/ops/deployment/#project-setup) ------------------------------------------------------------------------------ Deploying to a production environment generally requires a simple `Dockerfile` that builds and runs an agent worker, and a deployment platform that scales your worker pool based on load. The following starter projects each include a working Dockerfile and CI configuration. [Python Voice Agent\ ------------------\ \ A production-ready voice AI starter project for Python.\ \ GitHublivekit-examples/agent-starter-python](https://github.com/livekit-examples/agent-starter-python) [Node.js Voice Agent\ -------------------\ \ A production-ready voice AI starter project for Node.js.\ \ GitHublivekit-examples/agent-starter-node](https://github.com/livekit-examples/agent-starter-node) Where to deploy[](https://docs.livekit.io/agents/ops/deployment/#where-to-deploy) ---------------------------------------------------------------------------------- LiveKit Agents can be deployed almost anywhere. The LiveKit team and community have found the following deployment platforms to be the easiest and most cost-effective to use. [New\ \ LiveKit Cloud Agents Beta\ -------------------------\ \ Run your agent on the same network and infrastructure that serves LiveKit Cloud, with builds, deployment, and scaling handled for you. Sign up for the public beta to get started.](https://livekit.io/cloud-agents-beta) [![/images/icons/logo-kubernetes.svg](https://docs.livekit.io/images/icons/logo-kubernetes.svg)\ \ Kubernetes\ ----------\ \ Sample configuration for deploying and autoscaling LiveKit Agents on Kubernetes.](https://github.com/livekit-examples/agent-deployment/tree/main/kubernetes) [Render.com\ ----------\ \ Sample configuration for deploying and autoscaling LiveKit Agents on Render.com.](https://github.com/livekit-examples/agent-deployment/tree/main/render.com) [More deployment examples\ ------------------------\ \ Example `Dockerfile` and configuration files for a variety of deployment platforms.](https://github.com/livekit-examples/agent-deployment) Networking[](https://docs.livekit.io/agents/ops/deployment/#networking) ------------------------------------------------------------------------ Workers use a WebSocket connection to register with LiveKit server and accept incoming jobs. This means that workers do not need to expose any inbound hosts or ports to the public internet. You may optionally expose a private health check endpoint for monitoring, but this is not required for normal operation. The default health check server listens on `http://0.0.0.0:8081/`. Environment variables[](https://docs.livekit.io/agents/ops/deployment/#environment-variables) ---------------------------------------------------------------------------------------------- It is best to configure your worker with environment variables for secrets like API keys. In addition to the LiveKit variables, you are likely to need additional keys for external services your agent depends on. For instance, an agent built with the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai/) needs the following keys at a minimum: .env DEEPGRAM\_API\_KEY\= OPENAI\_API\_KEY\= CARTESIA\_API\_KEY\= LIVEKIT\_API\_KEY\= LIVEKIT\_API\_SECRET\= LIVEKIT\_URL\= Reveal API Key and Secret **Project environments** It's recommended to use a separate LiveKit instance for staging, production, and development environments. This ensures you can continue working on your agent locally without accidentally processing real user traffic. In LiveKit Cloud, make a separate project for each environment. Each has a unique URL, API key, and secret. For self-hosted LiveKit server, use a separate deployment for staging and production and a local server for development. Storage[](https://docs.livekit.io/agents/ops/deployment/#storage) ------------------------------------------------------------------ Worker and job processes have no particular storage requirements beyond the size of the Docker image itself (typically <1GB). 10GB of ephemeral storage should be more than enough to account for this and any temporary storage needs your app has. Memory and CPU[](https://docs.livekit.io/agents/ops/deployment/#memory-and-cpu) -------------------------------------------------------------------------------- Memory and CPU requirements vary significantly based on the specific details of your app. For instance, agents that use [enhanced noise cancellation](https://docs.livekit.io/cloud/noise-cancellation/) or the [LiveKit turn detector](https://docs.livekit.io/agents/build/turns/turn-detector/) require more CPU and memory than those that don't. In some cases, the memory requirements might exceed the amount available on a cloud provider's free tier. LiveKit recommends 4 cores and 8GB per worker as a starting rule for most voice AI apps. This worker can handle 10-25 concurrent jobs, depending on the components in use. **Real world load test results** LiveKit ran a load test to evaluate the memory and CPU requirements of a typical voice-to-voice app. * 30 agents each placed in their own LiveKit Cloud room. * 30 simulated user participants, one in each room. * Each simulated participant published looping speech audio to the agents. * Each agent subscribed to the incoming audio of the user and ran the Silero VAD plugin. * Each agent published their own audio (simple looping sine wave). * One additional user participant with a corresponding voice AI agent to ensure subjective quality of service. This test ran all agents on a single 4-Core, 8GB machine. This machine reached peak usage of: * CPU: ~3.8 cores utilized * Memory: ~2.8GB used Rollout[](https://docs.livekit.io/agents/ops/deployment/#rollout) ------------------------------------------------------------------ Workers stop accepting jobs upon `SIGINT` or `SIGTERM`. Any job still running on the worker continues to run to completion. It's important that you configure a large enough grace period such that your jobs can finish without interrupting the user experience. Voice AI apps might require a 10+ minute grace period to allow for conversations to finish. Different deployment platforms have different ways of setting this grace period. In Kubernetes, it's the `terminationGracePeriodSeconds` field in the pod spec. Consult your deployment platform's documentation for more information. Load balancing[](https://docs.livekit.io/agents/ops/deployment/#load-balancing) -------------------------------------------------------------------------------- LiveKit server includes a built-in balanced job distribution system. This system peforms round-robin distribution with a single-assignment principle that ensures each job is assigned to only one worker. If a worker fails to accept the job within a predetermined timeout period, the job is sent to another available worker instead. LiveKit Cloud additionally exercises geographic affinity to prioritize matching users and workers that are geographically closest to each other. This ensures the lowest possible latency between users and agents. Worker availability[](https://docs.livekit.io/agents/ops/deployment/#worker-availability) ------------------------------------------------------------------------------------------ Worker availability is defined by the `load_fnc` and `load_threshold` parameters in the `WorkerOptions` configuration. The `load_fnc` must return a value between 0 and 1, indicating how busy the worker is. `load_threshold` is the load value above which the worker stops accepting new jobs. The default `load_fnc` is overall CPU utilization, and the default `load_threshold` is `0.7`. In a custom deployment, you can override `load_fnc` and `load_threshold` to match the scaling behavior of your environment and application. Autoscaling[](https://docs.livekit.io/agents/ops/deployment/#autoscaling) -------------------------------------------------------------------------- To handle variable traffic patterns, add an autoscaling strategy to your deployment platform. Your autoscaler should use the same underlying metrics as your `load_fnc` (the default is CPU utilization) but should scale up at a _lower_ threshold than your worker's `load_threshold`. This ensures continuity of service by adding new workers before existing ones go out of service. For example, if your `load_threshold` is `0.7`, you should scale up at `0.5`. Since voice agents are typically long running tasks (relative to typical web requests), rapid increases in load are more likely to be sustained. In technical terms: spikes are less spikey. For your autoscaling configuration, you should consider _reducing_ cooldown/stabilization periods when scaling up. When scaling down, consider _increasing_ cooldown/stabilization periods because workers take time to drain. For example, if deploying on Kubernetes using a Horizontal Pod Autoscaler, see [stabilizationWindowSeconds](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#default-behavior) . On this page [Overview](https://docs.livekit.io/agents/ops/deployment/#overview) [Project setup](https://docs.livekit.io/agents/ops/deployment/#project-setup) [Where to deploy](https://docs.livekit.io/agents/ops/deployment/#where-to-deploy) [Networking](https://docs.livekit.io/agents/ops/deployment/#networking) [Environment variables](https://docs.livekit.io/agents/ops/deployment/#environment-variables) [Storage](https://docs.livekit.io/agents/ops/deployment/#storage) [Memory and CPU](https://docs.livekit.io/agents/ops/deployment/#memory-and-cpu) [Rollout](https://docs.livekit.io/agents/ops/deployment/#rollout) [Load balancing](https://docs.livekit.io/agents/ops/deployment/#load-balancing) [Worker availability](https://docs.livekit.io/agents/ops/deployment/#worker-availability) [Autoscaling](https://docs.livekit.io/agents/ops/deployment/#autoscaling) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/ops/deployment/) Search --- # Logs, metrics, and telemetry | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/build/metrics/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/build/metrics/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/build/metrics/#overview) [Logging events](https://docs.livekit.io/agents/build/metrics/#logging-events) [Aggregating metrics](https://docs.livekit.io/agents/build/metrics/#aggregating-metrics) [Metrics reference](https://docs.livekit.io/agents/build/metrics/#metrics-reference) [Speech-to-text (STT)](https://docs.livekit.io/agents/build/metrics/#speech-to-text-stt-) [LLM](https://docs.livekit.io/agents/build/metrics/#llm) [Text-to-speech (TTS)](https://docs.livekit.io/agents/build/metrics/#text-to-speech-tts-) [End-of-utterance (EOU)](https://docs.livekit.io/agents/build/metrics/#end-of-utterance-eou-) [Measuring conversation latency](https://docs.livekit.io/agents/build/metrics/#measuring-conversation-latency) [Telemetry](https://docs.livekit.io/agents/build/metrics/#telemetry) [Collected data](https://docs.livekit.io/agents/build/metrics/#collected-data) [Enabling telemetry](https://docs.livekit.io/agents/build/metrics/#enabling-telemetry) [Trace example](https://docs.livekit.io/agents/build/metrics/#trace-example) [Example](https://docs.livekit.io/agents/build/metrics/#example) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/build/metrics/#overview) ------------------------------------------------------------------- LiveKit Agents provides built-in support for logging, collecting, and analyzing metrics to help you monitor and optimize your agent's performance. Agent sessions emit structured metrics events that can be logged in real time or aggregated to analyze latency and usage patterns. In addition to per-event metrics, LiveKit’s OpenTelemetry integration provides trace-based observability. This enables you to capture the execution flow of an agents's lifecycle—from session start to individual [node](https://docs.livekit.io/agents/build/nodes/) operations. You can use any OpenTelemetry-compatible provider to collect and analyze telemetry data, giving you insight into conversation latency, tool usage, and performance bottlenecks. For information on log levels, see the [worker options](https://docs.livekit.io/agents/worker/options/#log-levels) page. Logging events[](https://docs.livekit.io/agents/build/metrics/#logging-events) ------------------------------------------------------------------------------- Agent metrics events are fired by the `AgentSession` whenever there is a new metrics object available during an active session. A `log_metrics` helper function is also provided to format logging output for each metric type. from livekit.agents import metrics, MetricsCollectedEvent ... @session.on("metrics\_collected") def \_on\_metrics\_collected(ev: MetricsCollectedEvent): metrics.log\_metrics(ev.metrics) Aggregating metrics[](https://docs.livekit.io/agents/build/metrics/#aggregating-metrics) ----------------------------------------------------------------------------------------- The `metrics` module also includes a `UsageCollector` helper class for aggregating usage metrics across a session. It tracks metrics such as LLM, TTS, and STT API usage, which can help estimate session cost. from livekit.agents import metrics, MetricsCollectedEvent ... usage\_collector \= metrics.UsageCollector() @session.on("metrics\_collected") def \_on\_metrics\_collected(ev: MetricsCollectedEvent): usage\_collector.collect(ev.metrics) async def log\_usage(): summary \= usage\_collector.get\_summary() logger.info(f"Usage: {summary}") \# At shutdown, generate and log the summary from the usage collector ctx.add\_shutdown\_callback(log\_usage) Metrics reference[](https://docs.livekit.io/agents/build/metrics/#metrics-reference) ------------------------------------------------------------------------------------- ![Diagram where metrics are measured.](https://docs.livekit.io/images/agents/agents-capturing-metrics-v1.svg)![Diagram where metrics are measured.](https://docs.livekit.io/images/agents/agents-capturing-metrics-v1.svg) ### Speech-to-text (STT)[](https://docs.livekit.io/agents/build/metrics/#speech-to-text-stt-) `STTMetrics` is emitted after the STT model has processed the audio input. This metrics is only available when an STT component is used, which does not apply to Realtime APIs. | Metric | Description | | --- | --- | | `audio_duration` | The duration (seconds) of the audio input received by the STT model. | | `duration` | For non-streaming STT, the amount of time (seconds) it took to create the transcript. Always `0` for streaming STT. | | `streamed` | `True` if the STT is in streaming mode. | ### LLM[](https://docs.livekit.io/agents/build/metrics/#llm) `LLMMetrics` is emitted after each LLM inference completes. If the response includes tool calls, the event does not include the time taken to execute those calls. Each tool call response triggers a separate `LLMMetrics` event. | Metric | Description | | --- | --- | | `duration` | The amount of time (seconds) it took for the LLM to generate the entire completion. | | `completion_tokens` | The number of tokens generated by the LLM in the completion. | | `prompt_tokens` | The number of tokens provided in the prompt sent to the LLM. | | `prompt_cached_tokens` | The number of cached tokens in the input prompt. | | `speech_id` | An unique identifier representing a turn in the user input. | | `total_tokens` | Total token usage for the completion. | | `tokens_per_second` | The rate of token generation (tokens/second) by the LLM to generate the completion. | | `ttft` | The amount of time (seconds) that it took for the LLM to generate the first token of the completion. | ### Text-to-speech (TTS)[](https://docs.livekit.io/agents/build/metrics/#text-to-speech-tts-) `TTSMetrics` is emitted after a TTS has generated speech from text input. | Metric | Description | | --- | --- | | `audio_duration` | The duration (seconds) of the audio output generated by the TTS model. | | `characters_count` | The number of characters in the text input to the TTS model. | | `duration` | The amount of time (seconds) it took for the TTS model to generate the entire audio output. | | `ttfb` | The amount of time (seconds) that it took for the TTS model to generate the first byte of its audio output. | | `speech_id` | An identifier linking to a user's turn. | | `streamed` | `True` if the TTS is in streaming mode. | ### End-of-utterance (EOU)[](https://docs.livekit.io/agents/build/metrics/#end-of-utterance-eou-) `EOUMetrics` is emitted when the user is determined to have finished speaking. It includes metrics related to end-of-turn detection and transcription latency. This event is only available in Realtime APIs when `turn_detection` is set to either VAD or LiveKit's turn detector plugin. When using server-side turn detection, EOUMetrics is not emitted, as this information is not available. | Metric | Description | | --- | --- | | `end_of_utterance_delay` | Time (in seconds) from the end of speech (as detected by VAD) to the point when the user's turn is considered complete. This includes any `transcription_delay`. | | `transcription_delay` | Time (seconds) between the end of speech and when final transcript is available | | `on_user_turn_completed_delay` | Time (in seconds) taken to execute the `on_user_turn_completed` callback. | | `speech_id` | A unique identifier indicating the user's turn. | Measuring conversation latency[](https://docs.livekit.io/agents/build/metrics/#measuring-conversation-latency) --------------------------------------------------------------------------------------------------------------- Total conversation latency is defined as the time it takes for the agent to respond to a user's utterance. Given the metrics above, it can be computed as follows: total\_latency \= eou.end\_of\_utterance\_delay + llm.ttft + tts.ttfb Telemetry[](https://docs.livekit.io/agents/build/metrics/#telemetry) --------------------------------------------------------------------- LiveKit's [OpenTelemetry](https://opentelemetry.io/docs/) integration automatically collects telemetry data from your agents and publishes it to any OpenTelemetry-compatible provider you choose. This enables monitoring and analysis of your agent's behavior and performance. ### Collected data[](https://docs.livekit.io/agents/build/metrics/#collected-data) A **trace** represents the execution flow of a single request within an LLM application. It captures all relevant steps, including duration and metadata. Agent telemetry records traces for the following activities: * Session start * Agent turn * LLM node * Function tool * TTS node * End-of-turn detection * LLM and TTS metrics ### Enabling telemetry[](https://docs.livekit.io/agents/build/metrics/#enabling-telemetry) To enable telemetry, configure a tracer provider using `set_tracer_provider` in your entrypoint function. You can use any [OpenTelemetry-compatible provider](https://opentelemetry.io/ecosystem/vendors/) . The following example uses [LangFuse](https://langfuse.com/docs/opentelemetry/get-started) . Set the required public key, secret key, and host as environment variables: import base64 import os from livekit.agents.telemetry import set\_tracer\_provider def setup\_langfuse( host: str | None \= None, public\_key: str | None \= None, secret\_key: str | None \= None ): from opentelemetry.exporter.otlp.proto.http.trace\_exporter import OTLPSpanExporter from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor public\_key \= public\_key or os.getenv("LANGFUSE\_PUBLIC\_KEY") secret\_key \= secret\_key or os.getenv("LANGFUSE\_SECRET\_KEY") host \= host or os.getenv("LANGFUSE\_HOST") if not public\_key or not secret\_key or not host: raise ValueError("LANGFUSE\_PUBLIC\_KEY, LANGFUSE\_SECRET\_KEY, and LANGFUSE\_HOST must be set") langfuse\_auth \= base64.b64encode(f"{public\_key}:{secret\_key}".encode()).decode() os.environ\["OTEL\_EXPORTER\_OTLP\_ENDPOINT"\] \= f"{host.rstrip('/')}/api/public/otel" os.environ\["OTEL\_EXPORTER\_OTLP\_HEADERS"\] \= f"Authorization=Basic {langfuse\_auth}" trace\_provider \= TracerProvider() trace\_provider.add\_span\_processor(BatchSpanProcessor(OTLPSpanExporter())) set\_tracer\_provider(trace\_provider) async def entrypoint(ctx: JobContext): setup\_langfuse() \# set up the langfuse tracer provider \# ... ### Trace example[](https://docs.livekit.io/agents/build/metrics/#trace-example) The following diagram shows a trace of an agent session with user turns. ![Diagram showing a trace of an agent session with two user turns.](https://docs.livekit.io/images/agents/agents-telemetry-trace-example.png)![Diagram showing a trace of an agent session with two user turns.](https://docs.livekit.io/images/agents/agents-telemetry-trace-example.png) ### Example[](https://docs.livekit.io/agents/build/metrics/#example) For a full example, see the following in the LiveKit Agents GitHub repository. [LangFuse trace example\ ----------------------\ \ An example of an agent using LangFuse as the tracer provider.](https://github.com/livekit/agents/blob/main/examples/voice_agents/langfuse_trace.py) On this page [Overview](https://docs.livekit.io/agents/build/metrics/#overview) [Logging events](https://docs.livekit.io/agents/build/metrics/#logging-events) [Aggregating metrics](https://docs.livekit.io/agents/build/metrics/#aggregating-metrics) [Metrics reference](https://docs.livekit.io/agents/build/metrics/#metrics-reference) [Speech-to-text (STT)](https://docs.livekit.io/agents/build/metrics/#speech-to-text-stt-) [LLM](https://docs.livekit.io/agents/build/metrics/#llm) [Text-to-speech (TTS)](https://docs.livekit.io/agents/build/metrics/#text-to-speech-tts-) [End-of-utterance (EOU)](https://docs.livekit.io/agents/build/metrics/#end-of-utterance-eou-) [Measuring conversation latency](https://docs.livekit.io/agents/build/metrics/#measuring-conversation-latency) [Telemetry](https://docs.livekit.io/agents/build/metrics/#telemetry) [Collected data](https://docs.livekit.io/agents/build/metrics/#collected-data) [Enabling telemetry](https://docs.livekit.io/agents/build/metrics/#enabling-telemetry) [Trace example](https://docs.livekit.io/agents/build/metrics/#trace-example) [Example](https://docs.livekit.io/agents/build/metrics/#example) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/build/metrics/) Search --- # Samples | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/components/android/samples/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/android/samples/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) Copy pageSee more page options See our sample apps for some examples of usage of the Android Components SDK: * [Meet Example App](https://github.com/livekit-examples/android-components-meet) : A simple teleconferencing app that connects to a LiveKit server. * [Livestreaming Example App](https://github.com/livekit-examples/android-livestream) : A fleshed out livestreaming experience that allows a user to broadcast to viewers as well as let them join the stream as a host. HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/android/samples/) Search --- # Unknown \# LiveKit Docs > LiveKit is an open source platform for developers building realtime media applications. It makes it easy to integrate audio, video, text, data, and AI models while offering scalable realtime infrastructure built on top of WebRTC. ## Overview LiveKit contains these primary components: - \[Open source WebRTC SFU\](https://github.com/livekit/livekit), with a hosted global mesh version available as \[LiveKit Cloud\](https://cloud.livekit.io) - \[Open source AI Agents Framework\](https://github.com/livekit/agents) for building realtime and Voice AI agents in Python (Node.js beta also \[available\](https://github.com/livekit/agents-js)) - \[Realtime SDKs\](https://docs.livekit.io/home/client/connect.md) to make it easy to add realitme audio, video, and data to your apps (available for Web, iOS, Android, Flutter, React Native, Unity, Python, Node.js, Rust, and more)) - \[Telephony integration\](https://docs.livekit.io/sip.md) built on SIP for integrating telephony into LiveKit rooms For greater detail, see \[Intro to LiveKit\](https://docs.livekit.io/home/get-started/intro-to-livekit.md). The following document is a comprehensive list of all available documentation and examples for LiveKit. ## Home ### Get Started - \[Intro to LiveKit\](https://docs.livekit.io/home/get-started/intro-to-livekit.md): An overview of the LiveKit ecosystem. - \[Rooms, participants, and tracks\](https://docs.livekit.io/home/get-started/api-primitives.md): Guide to the core API primitives in LiveKit. - \[Authentication\](https://docs.livekit.io/home/get-started/authentication.md): Learn how to authenticate your users to LiveKit sessions. ### CLI - \[Installing CLI\](https://docs.livekit.io/home/cli/cli-setup.md): Install the LiveKit CLI and test your setup using an example frontend application. - \[Bootstrapping an application\](https://docs.livekit.io/home/cli/templates.md): Create and initialize an app from a convenient set of templates. ### LiveKit SDKs - \[Connecting to LiveKit\](https://docs.livekit.io/home/client/connect.md): Learn how to connect with realtime SDKs. #### Realtime media - \[Overview\](https://docs.livekit.io/home/client/tracks.md): Audio and video media exchange between participants. - \[Camera & microphone\](https://docs.livekit.io/home/client/tracks/publish.md): Publish realtime audio and video from any device. - \[Screen sharing\](https://docs.livekit.io/home/client/tracks/screenshare.md): Publish your screen with LiveKit. - \[Subscribing to tracks\](https://docs.livekit.io/home/client/tracks/subscribe.md): Play and render realtime media tracks in your application. - \[Noise & echo cancellation\](https://docs.livekit.io/home/client/tracks/noise-cancellation.md): Achieve crystal-clear audio for video conferencing and voice AI. - \[End-to-end encryption\](https://docs.livekit.io/home/client/tracks/encryption.md): Secure your realtime media tracks with E2EE. - \[Codecs & more\](https://docs.livekit.io/home/client/tracks/advanced.md): Advanced audio and video topics. #### Realtime text & data - \[Overview\](https://docs.livekit.io/home/client/data.md): Exchange text, files, and custom data between participants. - \[Sending text\](https://docs.livekit.io/home/client/data/text-streams.md): Use text streams to send any amount of text between participants. - \[Sending files & bytes\](https://docs.livekit.io/home/client/data/byte-streams.md): Use byte streams to send files, images, or any other kind of data between participants. - \[Remote method calls\](https://docs.livekit.io/home/client/data/rpc.md): Use RPC to execute custom methods on other participants in the room and await a response. - \[Data packets\](https://docs.livekit.io/home/client/data/packets.md): Low-level API for high frequency or advanced use cases. #### State synchronization - \[Overview\](https://docs.livekit.io/home/client/state.md) - \[Participant attributes\](https://docs.livekit.io/home/client/state/participant-attributes.md): A key-value store for per-participant state. - \[Room metadata\](https://docs.livekit.io/home/client/state/room-metadata.md): Share application-specific state with all participants. - \[Handling events\](https://docs.livekit.io/home/client/events.md): Observe and respond to events in the LiveKit SDK. #### Platform-specific quickstarts - \[Overview\](https://docs.livekit.io/home/quickstarts.md) - \[Next.js\](https://docs.livekit.io/home/quickstarts/nextjs.md): Get started with LiveKit and Next.js - \[React\](https://docs.livekit.io/home/quickstarts/react.md): Get started with LiveKit and React. - \[JavaScript\](https://docs.livekit.io/home/quickstarts/javascript.md): Get started with LiveKit and JavaScript - \[Unity (WebGL)\](https://docs.livekit.io/home/quickstarts/unity-web.md): Get started with LiveKit and Unity (WebGL) - \[Swift\](https://docs.livekit.io/home/quickstarts/swift.md): Get started with LiveKit on iOS using SwiftUI - \[Android (Compose)\](https://docs.livekit.io/home/quickstarts/android-compose.md): Get started with LiveKit and Android using Jetpack Compose - \[Android\](https://docs.livekit.io/home/quickstarts/android.md): Get started with LiveKit and Android - \[Flutter\](https://docs.livekit.io/home/quickstarts/flutter.md): Get started with LiveKit and Flutter - \[React Native\](https://docs.livekit.io/home/quickstarts/react-native.md): Get started with LiveKit and React Native - \[Expo\](https://docs.livekit.io/home/quickstarts/expo.md): Get started with LiveKit and Expo on React Native ### Server APIs - \[Token generation\](https://docs.livekit.io/home/server/generating-tokens.md): Generate tokens for your frontend - \[Room management\](https://docs.livekit.io/home/server/managing-rooms.md): Create, list, and delete Rooms from your backend server. - \[Participant management\](https://docs.livekit.io/home/server/managing-participants.md): List, remove, and mute from your backend server. - \[Webhooks\](https://docs.livekit.io/home/server/webhooks.md): Configure LiveKit to notify your server when room events take place. ### Recording & Composition - \[Overview\](https://docs.livekit.io/home/egress/overview.md): Use LiveKit's egress service to record or livestream a room. - \[Composite & web recordings\](https://docs.livekit.io/home/egress/composite-recording.md): LiveKit web-based recorder gives you flexible compositing options - \[Recording participants\](https://docs.livekit.io/home/egress/participant.md): Record participants individually with the Egress API. - \[Recording individual tracks\](https://docs.livekit.io/home/egress/track.md): Track egress allows you export a single track without transcoding. - \[Output and streaming options\](https://docs.livekit.io/home/egress/outputs.md): Export content anywhere, in any format. - \[Auto Egress\](https://docs.livekit.io/home/egress/autoegress.md): Automatically start recording with a room. - \[Custom recording templates\](https://docs.livekit.io/home/egress/custom-template.md): Create your own recording layout to use with Room Composite Egress. - \[Egress API\](https://docs.livekit.io/home/egress/api.md): Use LiveKit's egress service to record or livestream a Room. - \[Examples\](https://docs.livekit.io/home/egress/examples.md): Usage examples for Egress APIs. ### Stream ingest - \[Overview\](https://docs.livekit.io/home/ingress/overview.md): Use LiveKit's ingress service to bring live streams from non-WebRTC sources into LiveKit rooms. - \[Encoder configuration\](https://docs.livekit.io/home/ingress/configure-streaming-software.md): How to configure streaming software to work with LiveKit Ingress. ### Cloud - \[Overview\](https://docs.livekit.io/home/cloud.md): The fully-managed, globally distributed LiveKit deployment option. - \[Architecture\](https://docs.livekit.io/home/cloud/architecture.md): LiveKit Cloud gives you the flexibility of LiveKit's WebRTC stack, combined with global, CDN-scale infrastructure offering 99.99% uptime. - \[Sandbox\](https://docs.livekit.io/home/cloud/sandbox.md): Rapidly prototype your apps and share them with others, cutting out the boilerplate. - \[Quotas & limits\](https://docs.livekit.io/home/cloud/quotas-and-limits.md): Guide to the quotas and limits for LiveKit Cloud plans. - \[Billing\](https://docs.livekit.io/home/cloud/billing.md): Learn how LiveKit Cloud billing works. - \[Configuring firewalls\](https://docs.livekit.io/home/cloud/firewall.md): Learn how to configure firewalls for LiveKit Cloud. - \[Region pinning\](https://docs.livekit.io/home/cloud/region-pinning.md): Learn how to isolate LiveKit traffic to a specific region. - \[Analytics API\](https://docs.livekit.io/home/cloud/analytics-api.md): Get information about your LiveKit sessions and participants - \[Enhanced noise cancellation\](https://docs.livekit.io/home/cloud/noise-cancellation.md): LiveKit Cloud offers AI-powered noise cancellation for realtime audio. ### Self-hosting - \[Running locally\](https://docs.livekit.io/home/self-hosting/local.md): This will get a LiveKit instance up and running, ready to receive audio and video streams from participants. - \[Deployment overview\](https://docs.livekit.io/home/self-hosting/deployment.md): WebRTC servers can be tricky to deploy because of their use of UDP ports and having to know their own public IP address. This guide will help you get a secure LiveKit deployment up and running. - \[Virtual machine\](https://docs.livekit.io/home/self-hosting/vm.md): This guide helps you to set up a production-ready LiveKit server on a cloud virtual machine. - \[Kubernetes\](https://docs.livekit.io/home/self-hosting/kubernetes.md): Deploy LiveKit to Kubernetes. - \[Distributed multi-region\](https://docs.livekit.io/home/self-hosting/distributed.md): LiveKit is architected to be distributed, with homogeneous instances running across many servers. In distributed mode, Redis is required as shared data store and message bus. - \[Firewall configuration\](https://docs.livekit.io/home/self-hosting/ports-firewall.md): Reference for ports and suggested firewall rules for LiveKit. - \[Benchmarks\](https://docs.livekit.io/home/self-hosting/benchmark.md): Guide to load-testing and benchmarking your LiveKit installation. - \[Egress\](https://docs.livekit.io/home/self-hosting/egress.md): The Egress service uses redis messaging queues to load balance and communicate with your LiveKit server. - \[Ingress\](https://docs.livekit.io/home/self-hosting/ingress.md): The Ingress service uses Redis messaging queues to communicate with your LiveKit server. - \[SIP server\](https://docs.livekit.io/home/self-hosting/sip-server.md): Setting up and configuring a self-hosted SIP server for LiveKit telephony apps. ## Agents ### Getting started - \[Introduction\](https://docs.livekit.io/agents.md): Realtime framework for production-grade multimodal and voice AI agents. - \[Voice AI quickstart\](https://docs.livekit.io/agents/start/voice-ai.md): Build a simple voice assistant with Python in less than 10 minutes. - \[Telephony integration\](https://docs.livekit.io/agents/start/telephony.md): Enable your voice AI agent to make and receive phone calls. - \[Web & mobile frontends\](https://docs.livekit.io/agents/start/frontend.md): Bring your agent to life through a web or mobile app. - \[Agents playground\](https://docs.livekit.io/agents/start/playground.md): A virtual workbench to test your multimodal AI agent. - \[Migrating from v0.x\](https://docs.livekit.io/agents/start/v0-migration.md): Migrate your Python-based agents from version v0.x to 1.0. - \[Deeplearning.ai course\](https://www.deeplearning.ai/short-courses/building-ai-voice-agents-for-production/) ### Building voice agents - \[Overview\](https://docs.livekit.io/agents/build.md): In-depth guide to voice AI with LiveKit Agents. - \[Workflows\](https://docs.livekit.io/agents/build/workflows.md): How to model repeatable, accurate workflows through agents, handoffs, and tasks. - \[Speech & audio\](https://docs.livekit.io/agents/build/audio.md): Speech and audio capabilities for LiveKit agents. - \[Vision\](https://docs.livekit.io/agents/build/vision.md): Enhance your agent with visual understanding from images and live video. - \[Tool definition & use\](https://docs.livekit.io/agents/build/tools.md): Let your agents call external tools and more. - \[Pipeline nodes & hooks\](https://docs.livekit.io/agents/build/nodes.md): Learn how to customize the behavior of your agent with nodes and hooks in the voice pipeline. - \[Text & transcriptions\](https://docs.livekit.io/agents/build/text.md): Integrate realtime text features into your agent. #### Turn detection & interruptions - \[Overview\](https://docs.livekit.io/agents/build/turns.md): Guide to managing conversation turns in voice AI. - \[Turn detector plugin\](https://docs.livekit.io/agents/build/turns/turn-detector.md): Open-weights model for contextually-aware voice AI turn detection. - \[Silero VAD plugin\](https://docs.livekit.io/agents/build/turns/vad.md): High-performance voice activity detection for LiveKit Agents. - \[External data & RAG\](https://docs.livekit.io/agents/build/external-data.md): Best practices for adding context and taking external actions. - \[Logs, metrics, & telemetry\](https://docs.livekit.io/agents/build/metrics.md): Collecting logs, metrics, and telemetry data from your agent for debugging and insights. - \[Events & error handling\](https://docs.livekit.io/agents/build/events.md): Guides and reference for events and error handling in LiveKit Agents. - \[Testing & evaluation\](https://docs.livekit.io/agents/build/testing.md): Write tests to control and evaluate agent behavior. ### Worker lifecycle - \[Overview\](https://docs.livekit.io/agents/worker.md): How the worker coordinates with LiveKit server to manage agent jobs. - \[Agent dispatch\](https://docs.livekit.io/agents/worker/agent-dispatch.md): Specifying how and when your agents are assigned to rooms. - \[Job lifecycle\](https://docs.livekit.io/agents/worker/job.md): Learn more about the entrypoint function and how to end and clean up LiveKit sessions. - \[Worker options\](https://docs.livekit.io/agents/worker/options.md): Learn about the options available for creating a worker. ### Deployment & operations - \[Deploying to production\](https://docs.livekit.io/agents/ops/deployment.md): Guide to running LiveKit Agents in a production environment. #### Deploying to LiveKit Cloud - \[Session recording & transcripts\](https://docs.livekit.io/agents/ops/recording.md): Export session data in video, audio, or text format. ### Partner spotlight #### OpenAI - \[Overview\](https://docs.livekit.io/agents/integrations/openai.md): Build world-class realtime AI apps with OpenAI and LiveKit Agents. - \[Realtime API\](https://docs.livekit.io/agents/integrations/realtime/openai.md): How to use the OpenAI Realtime API with LiveKit Agents. - \[OpenAI LLM\](https://docs.livekit.io/agents/integrations/llm/openai.md): How to use the OpenAI LLM plugin for LiveKit Agents. - \[OpenAI TTS\](https://docs.livekit.io/agents/integrations/tts/openai.md): How to use the OpenAI TTS plugin for LiveKit Agents. - \[OpenAI STT\](https://docs.livekit.io/agents/integrations/stt/openai.md): How to use the OpenAI STT plugin for LiveKit Agents. #### Google - \[Overview\](https://docs.livekit.io/agents/integrations/google.md): Build world-class realtime AI apps with Google AI and LiveKit Agents. - \[Gemini Live API\](https://docs.livekit.io/agents/integrations/realtime/gemini.md): How to use the Gemini Live API with LiveKit Agents. - \[Gemini LLM\](https://docs.livekit.io/agents/integrations/llm/gemini.md): A guide to using Google Gemini with LiveKit Agents. - \[Google Cloud TTS\](https://docs.livekit.io/agents/integrations/tts/google.md): How to use the Google Cloud TTS plugin for LiveKit Agents. - \[Google Cloud STT\](https://docs.livekit.io/agents/integrations/stt/google.md): How to use the Google Cloud STT plugin for LiveKit Agents. #### Azure - \[Overview\](https://docs.livekit.io/agents/integrations/azure.md): An overview of the Azure AI integrations with LiveKit Agents. - \[Azure AI Speech TTS\](https://docs.livekit.io/agents/integrations/tts/azure.md): How to use the Azure Speech TTS plugin for LiveKit Agents. - \[Azure AI Speech STT\](https://docs.livekit.io/agents/integrations/stt/azure.md): How to use the Azure Speech STT plugin for LiveKit Agents. - \[Azure OpenAI Realtime API\](https://docs.livekit.io/agents/integrations/realtime/azure-openai.md): How to use the Azure OpenAI Realtime API with LiveKit Agents. - \[Azure OpenAI LLM\](https://docs.livekit.io/agents/integrations/llm/azure-openai.md): How to use the Azure OpenAI LLM plugin for LiveKit Agents. - \[Azure OpenAI TTS\](https://docs.livekit.io/agents/integrations/tts/azure-openai.md): How to use the Azure OpenAI TTS plugin for LiveKit Agents. - \[Azure OpenAI STT\](https://docs.livekit.io/agents/integrations/stt/azure-openai.md): How to use the Azure OpenAI STT plugin for LiveKit Agents. #### AWS - \[Overview\](https://docs.livekit.io/agents/integrations/aws.md): An overview of the AWS AI integrations with LiveKit Agents. - \[Amazon Bedrock LLM\](https://docs.livekit.io/agents/integrations/llm/aws.md): How to use the Amazon Bedrock LLM plugin for LiveKit Agents. - \[Amazon Polly TTS\](https://docs.livekit.io/agents/integrations/tts/aws.md): How to use the Amazon Polly TTS plugin for LiveKit Agents. - \[Amazon Transcribe STT\](https://docs.livekit.io/agents/integrations/stt/aws.md): How to use the Amazon Transcribe STT plugin for LiveKit Agents. - \[Amazon Nova Sonic\](https://docs.livekit.io/agents/integrations/realtime/nova-sonic.md): How to use the Amazon Nova Sonic model with LiveKit Agents. #### Groq - \[Overview\](https://docs.livekit.io/agents/integrations/groq.md): Ship lightning-fast voice AI with Groq and LiveKit Agents. - \[Groq LLM\](https://docs.livekit.io/agents/integrations/llm/groq.md): How to use the Groq LLM plugin for LiveKit Agents. - \[Groq TTS\](https://docs.livekit.io/agents/integrations/tts/groq.md): How to use the Groq TTS plugin for LiveKit Agents. - \[Groq STT\](https://docs.livekit.io/agents/integrations/stt/groq.md): How to use the Groq STT plugin for LiveKit Agents. - \[Cerebras\](https://docs.livekit.io/agents/integrations/cerebras.md): Build voice AI on the world's fastest inference. - \[Llama\](https://docs.livekit.io/agents/integrations/llama.md): Build voice AI on open source models from Meta AI. ### Integration guides - \[Overview\](https://docs.livekit.io/agents/integrations.md): Guides for integrating supported AI providers into LiveKit Agents. #### Realtime models - \[Overview\](https://docs.livekit.io/agents/integrations/realtime.md): Guides for adding realtime model integrations to your agents. - \[Gemini Live API\](https://docs.livekit.io/agents/integrations/realtime/gemini.md): How to use the Gemini Live API with LiveKit Agents. - \[Amazon Nova Sonic\](https://docs.livekit.io/agents/integrations/realtime/nova-sonic.md): How to use the Amazon Nova Sonic model with LiveKit Agents. - \[OpenAI Realtime API\](https://docs.livekit.io/agents/integrations/realtime/openai.md): How to use the OpenAI Realtime API with LiveKit Agents. - \[Azure OpenAI Realtime API\](https://docs.livekit.io/agents/integrations/realtime/azure-openai.md): How to use the Azure OpenAI Realtime API with LiveKit Agents. #### Large language models (LLM) - \[Overview\](https://docs.livekit.io/agents/integrations/llm.md): Guides for adding LLM integrations to your agents. - \[Anthropic\](https://docs.livekit.io/agents/integrations/llm/anthropic.md): How to use the Anthropic Claude LLM plugin for LiveKit Agents. - \[Amazon Bedrock\](https://docs.livekit.io/agents/integrations/llm/aws.md): How to use the Amazon Bedrock LLM plugin for LiveKit Agents. - \[Baseten\](https://docs.livekit.io/agents/integrations/llm/baseten.md): How to use the Baseten LLM plugin for LiveKit Agents. - \[Cerebras\](https://docs.livekit.io/agents/integrations/llm/cerebras.md): How to use the Cerebras inference with LiveKit Agents. - \[DeepSeek\](https://docs.livekit.io/agents/integrations/llm/deepseek.md): How to use DeepSeek models with LiveKit Agents. - \[Fireworks\](https://docs.livekit.io/agents/integrations/llm/fireworks.md): How to use Fireworks AI Llama models with LiveKit Agents. - \[Google Gemini\](https://docs.livekit.io/agents/integrations/llm/gemini.md): A guide to using Google Gemini with LiveKit Agents. - \[Groq\](https://docs.livekit.io/agents/integrations/llm/groq.md): How to use the Groq LLM plugin for LiveKit Agents. - \[LangChain\](https://docs.livekit.io/agents/integrations/llm/langchain.md): How to use LangGraph workflows with LiveKit Agents. - \[Letta\](https://docs.livekit.io/agents/integrations/llm/letta.md): How to use a Letta agent for your LLM with LiveKit Agents. - \[Ollama\](https://docs.livekit.io/agents/integrations/llm/ollama.md): How to run models locally using Ollama with LiveKit Agents. - \[OpenAI\](https://docs.livekit.io/agents/integrations/llm/openai.md): How to use the OpenAI LLM plugin for LiveKit Agents. - \[Azure OpenAI\](https://docs.livekit.io/agents/integrations/llm/azure-openai.md): How to use the Azure OpenAI LLM plugin for LiveKit Agents. - \[Perplexity\](https://docs.livekit.io/agents/integrations/llm/perplexity.md): How to use Perplexity LLM with LiveKit Agents. - \[Telnyx\](https://docs.livekit.io/agents/integrations/llm/telnyx.md): How to use Telnyx inference with LiveKit Agents. - \[Together AI\](https://docs.livekit.io/agents/integrations/llm/together.md): How to use Together AI Llama models with LiveKit Agents. - \[xAI\](https://docs.livekit.io/agents/integrations/llm/xai.md): How to use xAI LLM with LiveKit Agents. #### Speech-to-text (STT) - \[Overview\](https://docs.livekit.io/agents/integrations/stt.md): Guides for adding STT integrations to your agents. - \[AssemblyAI\](https://docs.livekit.io/agents/integrations/stt/assemblyai.md): How to use the AssemblyAI STT plugin for LiveKit Agents. - \[Amazon Transcribe\](https://docs.livekit.io/agents/integrations/stt/aws.md): How to use the Amazon Transcribe STT plugin for LiveKit Agents. - \[Azure AI Speech\](https://docs.livekit.io/agents/integrations/stt/azure.md): How to use the Azure Speech STT plugin for LiveKit Agents. - \[Azure OpenAI\](https://docs.livekit.io/agents/integrations/stt/azure-openai.md): How to use the Azure OpenAI STT plugin for LiveKit Agents. - \[Baseten\](https://docs.livekit.io/agents/integrations/stt/baseten.md): How to use the Baseten STT plugin for LiveKit Agents. - \[Cartesia\](https://docs.livekit.io/agents/integrations/stt/cartesia.md): How to use the Cartesia STT plugin for LiveKit Agents. - \[Clova\](https://docs.livekit.io/agents/integrations/stt/clova.md): How to use the Clova STT plugin for LiveKit Agents. - \[Deepgram\](https://docs.livekit.io/agents/integrations/stt/deepgram.md): How to use the Deepgram STT plugin for LiveKit Agents. - \[fal\](https://docs.livekit.io/agents/integrations/stt/fal.md): How to use the fal STT plugin for LiveKit Agents. - \[Gladia\](https://docs.livekit.io/agents/integrations/stt/gladia.md): How to use the Gladia STT plugin for LiveKit Agents. - \[Google Cloud\](https://docs.livekit.io/agents/integrations/stt/google.md): How to use the Google Cloud STT plugin for LiveKit Agents. - \[Groq\](https://docs.livekit.io/agents/integrations/stt/groq.md): How to use the Groq STT plugin for LiveKit Agents. - \[OpenAI\](https://docs.livekit.io/agents/integrations/stt/openai.md): How to use the OpenAI STT plugin for LiveKit Agents. - \[Sarvam\](https://docs.livekit.io/agents/integrations/stt/sarvam.md): How to use the Sarvam STT plugin for LiveKit Agents. - \[Speechmatics\](https://docs.livekit.io/agents/integrations/stt/speechmatics.md): How to use the Speechmatics STT plugin for LiveKit Agents. - \[Spitch\](https://docs.livekit.io/agents/integrations/stt/spitch.md): How to use the Spitch STT plugin for LiveKit Agents. #### Text-to-speech (TTS) - \[Overview\](https://docs.livekit.io/agents/integrations/tts.md): Guides for adding TTS integrations to your agents. - \[Amazon Polly\](https://docs.livekit.io/agents/integrations/tts/aws.md): How to use the Amazon Polly TTS plugin for LiveKit Agents. - \[Azure AI Speech\](https://docs.livekit.io/agents/integrations/tts/azure.md): How to use the Azure Speech TTS plugin for LiveKit Agents. - \[Azure OpenAI\](https://docs.livekit.io/agents/integrations/tts/azure-openai.md): How to use the Azure OpenAI TTS plugin for LiveKit Agents. - \[Baseten\](https://docs.livekit.io/agents/integrations/tts/baseten.md): How to use the Baseten TTS plugin for LiveKit Agents. - \[Cartesia\](https://docs.livekit.io/agents/integrations/tts/cartesia.md): How to use the Cartesia TTS plugin for LiveKit Agents. - \[Deepgram\](https://docs.livekit.io/agents/integrations/tts/deepgram.md): How to use the Deepgram TTS plugin for LiveKit Agents. - \[ElevenLabs\](https://docs.livekit.io/agents/integrations/tts/elevenlabs.md): How to use the ElevenLabs TTS plugin for LiveKit Agents. - \[Google Cloud\](https://docs.livekit.io/agents/integrations/tts/google.md): How to use the Google Cloud TTS plugin for LiveKit Agents. - \[Groq\](https://docs.livekit.io/agents/integrations/tts/groq.md): How to use the Groq TTS plugin for LiveKit Agents. - \[Hume\](https://docs.livekit.io/agents/integrations/tts/hume.md): How to use the Hume TTS plugin for LiveKit Agents. - \[Inworld\](https://docs.livekit.io/agents/integrations/tts/inworld.md): How to use the Inworld TTS plugin for LiveKit Agents. - \[LMNT\](https://docs.livekit.io/agents/integrations/tts/lmnt.md): How to use the LMNT TTS plugin for LiveKit Agents. - \[Neuphonic\](https://docs.livekit.io/agents/integrations/tts/neuphonic.md): How to use the Neuphonic TTS plugin for LiveKit Agents. - \[OpenAI\](https://docs.livekit.io/agents/integrations/tts/openai.md): How to use the OpenAI TTS plugin for LiveKit Agents. - \[PlayHT\](https://docs.livekit.io/agents/integrations/tts/playai.md): How to use the PlayHT TTS plugin for LiveKit Agents. - \[Resemble AI\](https://docs.livekit.io/agents/integrations/tts/resemble.md): How to use the Resemble AI TTS plugin for LiveKit Agents. - \[Rime\](https://docs.livekit.io/agents/integrations/tts/rime.md): How to use the Rime TTS plugin for LiveKit Agents. - \[Sarvam\](https://docs.livekit.io/agents/integrations/tts/sarvam.md): How to use the Sarvam TTS plugin for LiveKit Agents. - \[Speechify\](https://docs.livekit.io/agents/integrations/tts/speechify.md): How to use the Speechify TTS plugin for LiveKit Agents. - \[Spitch\](https://docs.livekit.io/agents/integrations/tts/spitch.md): How to use the Spitch TTS plugin for LiveKit Agents. #### Virtual avatars - \[Overview\](https://docs.livekit.io/agents/integrations/avatar.md): Guides for adding virtual avatars to your agents. - \[Anam\](https://docs.livekit.io/agents/integrations/avatar/anam.md): How to use the Anam virtual avatar plugin for LiveKit Agents. - \[Beyond Presence\](https://docs.livekit.io/agents/integrations/avatar/bey.md): How to use the Beyond Presence virtual avatar plugin for LiveKit Agents. - \[bitHuman\](https://docs.livekit.io/agents/integrations/avatar/bithuman.md): How to use the bitHuman virtual avatar plugin for LiveKit Agents. - \[Hedra\](https://docs.livekit.io/agents/integrations/avatar/hedra.md): How to use the Hedra virtual avatar plugin for LiveKit Agents. - \[Simli\](https://docs.livekit.io/agents/integrations/avatar/simli.md): How to use the Simli virtual avatar plugin for LiveKit Agents. - \[Tavus\](https://docs.livekit.io/agents/integrations/avatar/tavus.md): How to use the Tavus virtual avatar plugin for LiveKit Agents. ## Telephony ### Getting started - \[Overview\](https://docs.livekit.io/sip.md): Connect LiveKit to a telephone system using Session Initiation Protocol (SIP). - \[Cloud\](https://docs.livekit.io/sip/cloud.md): Overview of SIP cloud global and regional endpoints. - \[SIP trunk setup\](https://docs.livekit.io/sip/quickstarts/configuring-sip-trunk.md): Guide to setting up SIP trunks for inbound and outbound calls with LiveKit. ### Provider-specific guides - \[Twilio\](https://docs.livekit.io/sip/quickstarts/configuring-twilio-trunk.md): Step-by-step instructions for creating inbound and outbound SIP trunks using Twilio. - \[Telnyx\](https://docs.livekit.io/sip/quickstarts/configuring-telnyx-trunk.md): Step-by-step instructions for creating inbound and outbound SIP trunks using Telnyx. - \[Plivo\](https://docs.livekit.io/sip/quickstarts/configuring-plivo-trunk.md): Step-by-step instructions for creating inbound and outbound SIP trunks using Plivo. ### Accepting calls - \[Workflow\](https://docs.livekit.io/sip/accepting-calls.md): Workflow and configuration guide for accepting inbound calls. - \[Inbound trunk\](https://docs.livekit.io/sip/trunk-inbound.md): How to create and configure an inbound trunk to accept incoming calls. - \[Dispatch rule\](https://docs.livekit.io/sip/dispatch-rule.md): How to create and configure a dispatch rule. - \[Inbound calls with Twilio Voice\](https://docs.livekit.io/sip/accepting-calls-twilio-voice.md): How to use LiveKit SIP with TwiML and Twilio conferencing. ### Making calls - \[Workflow\](https://docs.livekit.io/sip/making-calls.md): Workflow for making outbound calls. - \[Outbound trunk\](https://docs.livekit.io/sip/trunk-outbound.md): How to create and configure a outbound trunk to make outgoing calls. - \[Make outbound calls\](https://docs.livekit.io/sip/outbound-calls.md): Create a LiveKit SIP participant to make outbound calls. ### Features - \[DTMF\](https://docs.livekit.io/sip/dtmf.md): Sending and receiving DTMF tones. - \[Cold transfer\](https://docs.livekit.io/sip/transfer-cold.md): Using the TransferSIPParticipant API for cold transfers. - \[HD voice\](https://docs.livekit.io/sip/hd-voice.md): LiveKit SIP supports high fidelity calls by enabling HD voice. ### Reference - \[SIP participant\](https://docs.livekit.io/sip/sip-participant.md): Mapping a caller to a SIP participant. - \[SIP API\](https://docs.livekit.io/sip/api.md): Use LiveKit's built-in SIP APIs to manage your SIP-based apps. ## Recipes - \*\*\[Voice AI quickstart\](https://docs.livekit.io/agents/start/voice-ai.md)\*\*: Create a voice AI agent in less than 10 minutes. - \*\*\[SwiftUI Voice Agent\](https://github.com/livekit-examples/agent-starter-swift)\*\*: A native iOS, macOS, and visionOS voice AI assistant built in SwiftUI. - \*\*\[Next.js Voice Agent\](https://github.com/livekit-examples/agent-starter-react)\*\*: A web voice AI assistant built with React and Next.js. - \*\*\[Flutter Voice Agent\](https://github.com/livekit-examples/agent-starter-flutter)\*\*: A cross-platform voice AI assistant app built with Flutter. - \*\*\[React Native Voice Agent\](https://github.com/livekit-examples/agent-starter-react-native)\*\*: A native voice AI assistant app built with React Native and Expo. - \*\*\[Android Voice Agent\](https://github.com/livekit-examples/agent-starter-android)\*\*: A native Android voice AI assistant app built with Kotlin and Jetpack Compose. - \*\*\[Web Embed Voice Agent\](https://github.com/livekit-examples/agent-starter-embed)\*\*: A voice AI agent that can be embedded in any web page. - \*\*\[Medical Office Triage\](https://github.com/livekit-examples/python-agents-examples/tree/main/complex-agents/medical\_office\_triage)\*\*: Agent that triages patients based on symptoms and medical history. - \*\*\[Personal Shopper\](https://github.com/livekit-examples/python-agents-examples/tree/main/complex-agents/personal\_shopper)\*\*: AI shopping assistant that helps find products based on user preferences. - \*\*\[Restaurant Agent\](https://github.com/livekit/agents/blob/main/examples/voice\_agents/restaurant\_agent.py)\*\*: A restaurant front-of-house agent that can take orders, add items to a shared cart, and checkout. - \*\*\[LivePaint\](https://github.com/livekit-examples/livepaint)\*\*: A realtime drawing game where players compete to complete a drawing prompt while being judged by a realtime AI agent. - \*\*\[Push-to-Talk Agent\](https://github.com/livekit/agents/blob/main/examples/voice\_agents/push\_to\_talk.py)\*\*: A voice AI agent that uses push-to-talk for controlled multi-participant conversations, only enabling audio input when explicitly triggered. - \*\*\[Background Audio\](https://github.com/livekit/agents/blob/main/examples/voice\_agents/background\_audio.py)\*\*: A voice AI agent with background audio for thinking states and ambiance. - \*\*\[Uninterruptable Agent\](https://github.com/livekit-examples/python-agents-examples/tree/main/basics/uninterruptable.py)\*\*: An agent that continues speaking without being interrupted. - \*\*\[Change Language\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-tts/elevenlabs\_change\_language.py)\*\*: Agent that can switch between different languages during conversation. - \*\*\[TTS Comparison\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-tts/tts\_comparison.py)\*\*: Compare different text-to-speech providers side by side. - \*\*\[Transcriber\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-stt/transcriber.py)\*\*: Real-time speech transcription with high accuracy. - \*\*\[Keyword Detection\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-stt/keyword\_detection.py)\*\*: Detect specific keywords in speech in real-time. - \*\*\[Using Twilio Voice\](https://docs.livekit.io/sip/accepting-calls-twilio-voice.md)\*\*: Use TwiML to accept incoming calls and bridge Twilio conferencing to LiveKit via SIP. - \*\*\[IVR Agent\](https://docs.livekit.io/recipes/ivr-navigator.md)\*\*: Build a voice agent that can call external voice lines and respond to IVR flows using DTMF tones. - \*\*\[Company Directory\](https://docs.livekit.io/recipes/company-directory.md)\*\*: Build a AI company directory agent. The agent can respond to DTMF tones and voice prompts, then redirect callers. - \*\*\[Phone Caller\](https://github.com/livekit-examples/python-agents-examples/tree/main/telephony/make\_call)\*\*: Agent that can make outbound phone calls and handle conversations. - \*\*\[SIP Warm Handoff\](https://github.com/livekit-examples/python-agents-examples/tree/main/telephony/warm\_handoff.py)\*\*: Transfer calls from an AI agent to a human operator seamlessly. - \*\*\[SIP Lifecycle\](https://github.com/livekit-examples/python-agents-examples/tree/main/telephony/sip\_lifecycle.py)\*\*: Complete lifecycle management for SIP calls. - \*\*\[Answer Incoming Calls\](https://github.com/livekit-examples/python-agents-examples/tree/main/telephony/answer\_call.py)\*\*: Set up an agent to answer incoming SIP calls. - \*\*\[Survey Caller\](https://github.com/livekit-examples/python-agents-examples/tree/main/telephony/survey\_caller/)\*\*: Automated survey calling system. - \*\*\[Chain-of-thought agent\](https://docs.livekit.io/recipes/chain-of-thought.md)\*\*: Build an agent for chain-of-thought reasoning using the \`llm\_node\` to clean the text before TTS. - \*\*\[LlamaIndex RAG\](https://github.com/livekit/agents/tree/main/examples/voice\_agents/llamaindex-rag)\*\*: A voice AI agent that uses LlamaIndex for RAG to answer questions from a knowledge base. - \*\*\[LiveKit Docs RAG\](https://github.com/livekit-examples/python-agents-examples/tree/main/rag)\*\*: An agent that can answer questions about LiveKit with lookups against the docs website. - \*\*\[Moviefone\](https://docs.livekit.io/recipes/moviefone.md)\*\*: This agent uses function calling and the OpenAI API to search for movies and give you realtime information about showtimes. - \*\*\[Context Variables\](https://github.com/livekit-examples/python-agents-examples/tree/main/basics/context\_variables.py)\*\*: Maintain conversation context across interactions. - \*\*\[Interrupt User\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-llm/interrupt\_user.py)\*\*: Example of how to implement user interruption in conversations. - \*\*\[LLM Content Filter\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-llm/llm\_powered\_content\_filter.py)\*\*: Implement content filtering in the \`llm\_node\`. - \*\*\[Simple Content Filter\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-llm/simple\_content\_filter.py)\*\*: Basic content filtering implementation. - \*\*\[Replacing LLM Output\](https://github.com/livekit-examples/python-agents-examples/tree/main/pipeline-llm/replacing\_llm\_output.py)\*\*: Example of modifying LLM output before processing. - \*\*\[Vision Assistant\](https://github.com/livekit-examples/vision-demo)\*\*: A voice AI agent with video input powered by Gemini Live. - \*\*\[Raspberry Pi Transcriber\](https://github.com/livekit-examples/python-agents-examples/tree/main/hardware/pi\_zero\_transcriber.py)\*\*: Run transcription on Raspberry Pi hardware. - \*\*\[Pipeline Translator\](https://github.com/livekit-examples/python-agents-examples/tree/main/translators/pipeline\_translator.py)\*\*: Implement translation in the processing pipeline. - \*\*\[TTS Translator\](https://github.com/livekit-examples/python-agents-examples/tree/main/translators/tts\_translator.py)\*\*: Translation with text-to-speech capabilities. - \*\*\[LLM Metrics\](https://github.com/livekit-examples/python-agents-examples/tree/main/metrics/metrics\_llm.py)\*\*: Track and analyze LLM performance metrics. - \*\*\[STT Metrics\](https://github.com/livekit-examples/python-agents-examples/tree/main/metrics/metrics\_stt.py)\*\*: Track and analyze speech-to-text performance metrics. - \*\*\[TTS Metrics\](https://github.com/livekit-examples/python-agents-examples/tree/main/metrics/metrics\_tts.py)\*\*: Track and analyze text-to-speech performance metrics. - \*\*\[VAD Metrics\](https://github.com/livekit-examples/python-agents-examples/tree/main/metrics/metrics\_vad.py)\*\*: Track and analyze voice activity detection metrics. - \*\*\[Playing Audio\](https://github.com/livekit-examples/python-agents-examples/tree/main/basics/playing\_audio.py)\*\*: Play audio files during agent interactions. - \*\*\[Sound Repeater\](https://github.com/livekit-examples/python-agents-examples/tree/main/basics/repeater.py)\*\*: Simple sound repeating demo for testing audio pipelines. - \*\*\[MCP Agent\](https://github.com/livekit-examples/python-agents-examples/tree/main/mcp)\*\*: A voice AI agent with an integrated Model Context Protocol (MCP) server for the LiveKit API. - \*\*\[Speedup Output Audio\](https://github.com/livekit/agents/blob/main/examples/voice\_agents/speedup\_output\_audio.py)\*\*: Speed up the audio output of an agent. - \*\*\[Structured Output\](https://github.com/livekit/agents/blob/main/examples/voice\_agents/structured\_output.py)\*\*: Handle structured output from the LLM by overriding the \`llm\_node\` and \`tts\_node\`. - \*\*\[RPC + State Agent\](https://github.com/livekit-examples/python-agents-examples/blob/main/rpc/rpc\_agent.py)\*\*: Voice agent with a state database updated through tool calling and queryable from the frontend with RPC. - \*\*\[Tavus Avatar Agent\](https://github.com/livekit-examples/python-agents-examples/blob/main/avatars/tavus)\*\*: An educational AI agent that uses Tavus to create an interactive study partner. - \*\*\[Rover Teleop\](https://github.com/livekit-examples/rover-teleop)\*\*: Build a high performance robot tele-op system using LiveKit. - \*\*\[VR Spatial Video\](https://github.com/livekit-examples/spatial-video)\*\*: Stream spatial video from a stereoscopic camera to a Meta Quest using LiveKit. - \*\*\[Echo Agent\](https://github.com/livekit/agents/blob/main/examples/primitives/echo-agent.py)\*\*: Echo user audio back to them. - \*\*\[Sync TTS Transcription\](https://github.com/livekit/agents/blob/main/examples/other/text-to-speech/sync\_tts\_transcription.py)\*\*: Uses manual subscription, transcription forwarding, and manually publishes audio output. - \*\*\[Drive-thru agent\](https://github.com/livekit/agents/blob/main/examples/drive-thru)\*\*: A complex food ordering agent with tasks, tools, and a complete evaluation suite. - \*\*\[Front-desk agent\](https://github.com/livekit/agents/blob/main/examples/frontdesk)\*\*: A calendar booking agent with tasks, tools, and evaluations. - \*\*\[Python Voice Agent\](https://github.com/livekit-examples/agent-starter-python)\*\*: A complete sample project for a voice AI agent built with Python. - \*\*\[Node.js Voice Agent\](https://github.com/livekit-examples/agent-starter-node)\*\*: A complete sample project for a voice AI agent built with Node.js. --- This document was rendered at 2025-08-08T18:04:54.017Z. For the latest version of this document, see \[https://docs.livekit.io/llms.txt\](https://docs.livekit.io/llms.txt). --- # Audio rendering with React Components | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/concepts/rendering-audio/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Render all audio tracks within the room](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#render-all-audio-tracks-within-the-room) [Full control and ownership of the audio rendering process](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#full-control-and-ownership-of-the-audio-rendering-process) Copy pageSee more page options There are two primary methods for rendering (making audio tracks audible) audio with React Components, each offering distinct benefits and suited for different use cases. Render all audio tracks within the room[](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#render-all-audio-tracks-within-the-room) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- The [`RoomAudioRenderer`](https://docs.livekit.io/reference/components/react/component/roomaudiorenderer/) component simplifies audio management in LiveKit Rooms by rendering all audio tracks together. It's a straightforward and often optimal solution. Just import `RoomAudioRenderer` and place it in your `LiveKitRoom` component for seamless audio integration. **Tip** Utilizing the `RoomAudioRenderer` ensures automatic benefits from future server side performance enhancements without requiring any modifications to your existing code. Full control and ownership of the audio rendering process[](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#full-control-and-ownership-of-the-audio-rendering-process) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- For complete control over individual audio Tracks, including muting and volume adjustments at the track level, you can craft a custom audio renderer using the [`useTracks`](https://docs.livekit.io/reference/components/react/hook/usetracks/) hook alongside the [``](https://docs.livekit.io/reference/components/react/component/audiotrack/) component. For example, this level of control can be used to create spatial audio applications where you may want to adjust each audio track based on the distance between participants. const tracks \= useTracks(\[\ \ Track.Source.Microphone,\ \ Track.Source.ScreenShareAudio,\ \ Track.Source.Unknown,\ \ \]).filter((ref) \=> !isLocal(ref.participant) && ref.publication.kind \=== Track.Kind.Audio); return (

{tracks.map((trackRef) \=> ( ))} ); Depending on your application it is possible that audio tracks have an unknown source. To render these as well, we include the `Track.Source.Unknown` in the array of sources passed to the `useTracks` hook, but then filter out the tracks that are not of kind `Audio`. On this page [Render all audio tracks within the room](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#render-all-audio-tracks-within-the-room) [Full control and ownership of the audio rendering process](https://docs.livekit.io/reference/components/react/concepts/rendering-audio/#full-control-and-ownership-of-the-audio-rendering-process) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/concepts/rendering-audio/) Search --- # RoomScope RoomScope ========= @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [RoomScope](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.local/-room-scope.html) (url: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ? = null, token: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ? = null, audio: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = false, video: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = false, connect: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = true, roomOptions: RoomOptions? = null, liveKitOverrides: LiveKitOverrides? = null, connectOptions: ConnectOptions? = null, onConnected: suspend CoroutineScope.(Room) -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ? = null, onDisconnected: suspend CoroutineScope.(Room) -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ? = null, onError: (Room, [Exception](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-exception/index.html) ?) -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ? = null, passedRoom: Room? = null, disconnectOnDispose: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = true, content: @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) (room: Room) -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ) Establishes a room scope which remembers a Room object which can be accessed through the [RoomLocal](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.local/-room-local.html) composition local. #### Parameters url the url of the livekit server to connect to. token the token to connect to livekit with. audio enable or disable audio. Defaults to false. video enable or disable video. Defaults to false. connect whether the room should connect to the server. Defaults to true. roomOptions options to pass to the Room. liveKitOverrides overrides to pass to the Room. Will not reflect changes beyond the initial creation. connectOptions options to use when connecting. Will not reflect changes if already connected. onConnected a listener to be called upon room connection. onDisconnected a listener to be called upon room disconnection. onError a listener to be called upon room error. passedRoom if a Room is provided, it will be used. If null, a new Room will be created instead. disconnectOnDispose by default, this composable handles the connection management and will disconnect the room when the composable goes out of scope. Setting this to false will disable this behavior. This is effective in combination with [passedRoom](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.local/-room-scope.html) , when you need to keep the Room object alive and connected separately from the UI (for example, with a background service). --- # ParticipantScope ParticipantScope ================ @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [ParticipantScope](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.local/-participant-scope.html) (participant: Participant, content: @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) () -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ) --- # livekit.plugins.nltk API documentation Classes ------- `class SentenceTokenizer (*, language: str = 'english', min_sentence_len: int = 20, stream_context_len: int = 10)` Expand source code class SentenceTokenizer(agents.tokenize.SentenceTokenizer): def __init__( self, *, language: str = "english", min_sentence_len: int = 20, stream_context_len: int = 10, ) -> None: super().__init__() self._config = _TokenizerOptions( language=language, min_sentence_len=min_sentence_len, stream_context_len=stream_context_len, ) def _sanitize_options(self, language: str | None = None) -> _TokenizerOptions: config = dataclasses.replace(self._config) if language: config.language = language return config def tokenize(self, text: str, *, language: str | None = None) -> list[str]: config = self._sanitize_options(language=language) sentences = nltk.tokenize.sent_tokenize(text, config.language) new_sentences = [] buff = "" for sentence in sentences: buff += sentence + " " if len(buff) - 1 >= config.min_sentence_len: new_sentences.append(buff.rstrip()) buff = "" if buff: new_sentences.append(buff.rstrip()) return new_sentences def stream(self, *, language: str | None = None) -> agents.tokenize.SentenceStream: config = self._sanitize_options(language=language) return agents.tokenize.BufferedSentenceStream( tokenizer=functools.partial( nltk.tokenize.sent_tokenize, language=config.language ), min_token_len=self._config.min_sentence_len, min_ctx_len=self._config.stream_context_len, ) Helper class that provides a standard way to create an ABC using inheritance. ### Ancestors * [SentenceTokenizer](https://docs.livekit.io/reference/python/livekit/agents/tokenize/tokenizer.html#livekit.agents.tokenize.tokenizer.SentenceTokenizer "livekit.agents.tokenize.tokenizer.SentenceTokenizer") * abc.ABC ### Methods `def stream(self, *, language: str | None = None) ‑> [SentenceStream](https://docs.livekit.io/reference/python/livekit/agents/tokenize/tokenizer.html#livekit.agents.tokenize.tokenizer.SentenceStream "livekit.agents.tokenize.tokenizer.SentenceStream") ` Expand source code def stream(self, *, language: str | None = None) -> agents.tokenize.SentenceStream: config = self._sanitize_options(language=language) return agents.tokenize.BufferedSentenceStream( tokenizer=functools.partial( nltk.tokenize.sent_tokenize, language=config.language ), min_token_len=self._config.min_sentence_len, min_ctx_len=self._config.stream_context_len, ) `def tokenize(self, text: str, *, language: str | None = None) ‑> list[str]` Expand source code def tokenize(self, text: str, *, language: str | None = None) -> list[str]: config = self._sanitize_options(language=language) sentences = nltk.tokenize.sent_tokenize(text, config.language) new_sentences = [] buff = "" for sentence in sentences: buff += sentence + " " if len(buff) - 1 >= config.min_sentence_len: new_sentences.append(buff.rstrip()) buff = "" if buff: new_sentences.append(buff.rstrip()) return new_sentences --- # livekit.plugins.turn_detector API documentation Sub-modules ----------- `[livekit.plugins.turn_detector.base](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/base.html "livekit.plugins.turn_detector.base") ` `[livekit.plugins.turn_detector.english](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/english.html "livekit.plugins.turn_detector.english") ` `[livekit.plugins.turn_detector.log](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/log.html "livekit.plugins.turn_detector.log") ` `[livekit.plugins.turn_detector.models](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/models.html "livekit.plugins.turn_detector.models") ` `[livekit.plugins.turn_detector.multilingual](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/multilingual.html "livekit.plugins.turn_detector.multilingual") ` `[livekit.plugins.turn_detector.version](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/version.html "livekit.plugins.turn_detector.version") ` Classes ------- `class EOUModel` Expand source code class EnglishModel(EOUModelBase): def __init__(self): super().__init__(model_type="en") def _inference_method(self) -> str: return _EUORunnerEn.INFERENCE_METHOD Helper class that provides a standard way to create an ABC using inheritance. ### Ancestors * [EOUModelBase](https://docs.livekit.io/reference/python/livekit/plugins/turn_detector/base.html#livekit.plugins.turn_detector.base.EOUModelBase "livekit.plugins.turn_detector.base.EOUModelBase") * abc.ABC --- # livekit.plugins.silero API documentation Sub-modules ----------- `[livekit.plugins.silero.resources](https://docs.livekit.io/reference/python/livekit/plugins/silero/resources/index.html "livekit.plugins.silero.resources") ` Used by importlib.resources and setuptools Classes ------- `class VAD (*, session: onnxruntime.InferenceSession, opts: _VADOptions)` Expand source code class VAD(agents.vad.VAD): """ Silero Voice Activity Detection (VAD) class. This class provides functionality to detect speech segments within audio data using the Silero VAD model. """ @classmethod def load( cls, *, min_speech_duration: float = 0.05, min_silence_duration: float = 0.55, prefix_padding_duration: float = 0.5, max_buffered_speech: float = 60.0, activation_threshold: float = 0.5, sample_rate: Literal[8000, 16000] = 16000, force_cpu: bool = True, # deprecated padding_duration: float | None = None, ) -> "VAD": """ Load and initialize the Silero VAD model. This method loads the ONNX model and prepares it for inference. When options are not provided, sane defaults are used. **Note:** This method is blocking and may take time to load the model into memory. It is recommended to call this method inside your prewarm mechanism. **Example:** ```python def prewarm(proc: JobProcess): proc.userdata["vad"] = silero.VAD.load() async def entrypoint(ctx: JobContext): vad = (ctx.proc.userdata["vad"],) # your agent logic... if __name__ == "__main__": cli.run_app( WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm) ) ``` Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. sample_rate (Literal[8000, 16000]): Sample rate for the inference (only 8KHz and 16KHz are supported). force_cpu (bool): Force the use of CPU for inference. padding_duration (float | None): **Deprecated**. Use `prefix_padding_duration` instead. Returns: VAD: An instance of the VAD class ready for streaming. Raises: ValueError: If an unsupported sample rate is provided. """ if sample_rate not in onnx_model.SUPPORTED_SAMPLE_RATES: raise ValueError("Silero VAD only supports 8KHz and 16KHz sample rates") if padding_duration is not None: logger.warning( "padding_duration is deprecated and will be removed in 1.5.0, use prefix_padding_duration instead", ) prefix_padding_duration = padding_duration session = onnx_model.new_inference_session(force_cpu) opts = _VADOptions( min_speech_duration=min_speech_duration, min_silence_duration=min_silence_duration, prefix_padding_duration=prefix_padding_duration, max_buffered_speech=max_buffered_speech, activation_threshold=activation_threshold, sample_rate=sample_rate, ) return cls(session=session, opts=opts) def __init__( self, *, session: onnxruntime.InferenceSession, opts: _VADOptions, ) -> None: super().__init__(capabilities=agents.vad.VADCapabilities(update_interval=0.032)) self._onnx_session = session self._opts = opts self._streams = weakref.WeakSet[VADStream]() def stream(self) -> "VADStream": """ Create a new VADStream for processing audio data. Returns: VADStream: A stream object for processing audio input and detecting speech. """ stream = VADStream( self, self._opts, onnx_model.OnnxModel( onnx_session=self._onnx_session, sample_rate=self._opts.sample_rate ), ) self._streams.add(stream) return stream def update_options( self, *, min_speech_duration: float | None = None, min_silence_duration: float | None = None, prefix_padding_duration: float | None = None, max_buffered_speech: float | None = None, activation_threshold: float | None = None, ) -> None: """ Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. """ self._opts = _VADOptions( min_speech_duration=min_speech_duration or self._opts.min_speech_duration, min_silence_duration=min_silence_duration or self._opts.min_silence_duration, prefix_padding_duration=prefix_padding_duration or self._opts.prefix_padding_duration, max_buffered_speech=max_buffered_speech or self._opts.max_buffered_speech, activation_threshold=activation_threshold or self._opts.activation_threshold, sample_rate=self._opts.sample_rate, ) for stream in self._streams: stream.update_options( min_speech_duration=min_speech_duration, min_silence_duration=min_silence_duration, prefix_padding_duration=prefix_padding_duration, max_buffered_speech=max_buffered_speech, activation_threshold=activation_threshold, ) Silero Voice Activity Detection (VAD) class. This class provides functionality to detect speech segments within audio data using the Silero VAD model. ### Ancestors * [VAD](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VAD "livekit.agents.vad.VAD") * abc.ABC * [EventEmitter](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter "livekit.rtc.event_emitter.EventEmitter") * typing.Generic ### Static methods `def load(*, min_speech_duration: float = 0.05, min_silence_duration: float = 0.55, prefix_padding_duration: float = 0.5, max_buffered_speech: float = 60.0, activation_threshold: float = 0.5, sample_rate: Literal[8000, 16000] = 16000, force_cpu: bool = True, padding_duration: float | None = None) ‑> livekit.plugins.silero.vad.VAD` Load and initialize the Silero VAD model. This method loads the ONNX model and prepares it for inference. When options are not provided, sane defaults are used. **Note:** This method is blocking and may take time to load the model into memory. It is recommended to call this method inside your prewarm mechanism. **Example:** ```python def prewarm(proc: JobProcess): proc.userdata["vad"] = silero.VAD.load() async def entrypoint(ctx: JobContext): vad = (ctx.proc.userdata["vad"],) # your agent logic... if __name__ == "__main__": cli.run_app( WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm) ) ``` Args ---- **`min_speech_duration`** : `float` Minimum duration of speech to start a new speech chunk. **`min_silence_duration`** : `float` At the end of each speech, wait this duration before ending the speech. **`prefix_padding_duration`** : `float` Duration of padding to add to the beginning of each speech chunk. **`max_buffered_speech`** : `float` Maximum duration of speech to keep in the buffer (in seconds). **`activation_threshold`** : `float` Threshold to consider a frame as speech. **`sample_rate`** : `Literal[8000, 16000]` Sample rate for the inference (only 8KHz and 16KHz are supported). **`force_cpu`** : `bool` Force the use of CPU for inference. **`padding_duration`** : `float | None` **Deprecated**. Use `prefix_padding_duration` instead. Returns ------- `[VAD](https://docs.livekit.io/reference/python/livekit/plugins/silero/index.html#livekit.plugins.silero.VAD "livekit.plugins.silero.VAD") ` An instance of the VAD class ready for streaming. Raises ------ `ValueError` If an unsupported sample rate is provided. ### Methods `def stream(self) ‑> livekit.plugins.silero.vad.VADStream` Expand source code def stream(self) -> "VADStream": """ Create a new VADStream for processing audio data. Returns: VADStream: A stream object for processing audio input and detecting speech. """ stream = VADStream( self, self._opts, onnx_model.OnnxModel( onnx_session=self._onnx_session, sample_rate=self._opts.sample_rate ), ) self._streams.add(stream) return stream Create a new VADStream for processing audio data. Returns ------- `[VADStream](https://docs.livekit.io/reference/python/livekit/plugins/silero/index.html#livekit.plugins.silero.VADStream "livekit.plugins.silero.VADStream") ` A stream object for processing audio input and detecting speech. `def update_options(self, *, min_speech_duration: float | None = None, min_silence_duration: float | None = None, prefix_padding_duration: float | None = None, max_buffered_speech: float | None = None, activation_threshold: float | None = None) ‑> None` Expand source code def update_options( self, *, min_speech_duration: float | None = None, min_silence_duration: float | None = None, prefix_padding_duration: float | None = None, max_buffered_speech: float | None = None, activation_threshold: float | None = None, ) -> None: """ Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. """ self._opts = _VADOptions( min_speech_duration=min_speech_duration or self._opts.min_speech_duration, min_silence_duration=min_silence_duration or self._opts.min_silence_duration, prefix_padding_duration=prefix_padding_duration or self._opts.prefix_padding_duration, max_buffered_speech=max_buffered_speech or self._opts.max_buffered_speech, activation_threshold=activation_threshold or self._opts.activation_threshold, sample_rate=self._opts.sample_rate, ) for stream in self._streams: stream.update_options( min_speech_duration=min_speech_duration, min_silence_duration=min_silence_duration, prefix_padding_duration=prefix_padding_duration, max_buffered_speech=max_buffered_speech, activation_threshold=activation_threshold, ) Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args ---- **`min_speech_duration`** : `float` Minimum duration of speech to start a new speech chunk. **`min_silence_duration`** : `float` At the end of each speech, wait this duration before ending the speech. **`prefix_padding_duration`** : `float` Duration of padding to add to the beginning of each speech chunk. **`max_buffered_speech`** : `float` Maximum duration of speech to keep in the buffer (in seconds). **`activation_threshold`** : `float` Threshold to consider a frame as speech. ### Inherited members * `**[VAD](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VAD "livekit.agents.vad.VAD") **`: * `[emit](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.emit "livekit.agents.vad.VAD.emit") ` * `[off](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.off "livekit.agents.vad.VAD.off") ` * `[on](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.on "livekit.agents.vad.VAD.on") ` * `[once](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.once "livekit.agents.vad.VAD.once") ` `class VADStream (vad: [VAD](https://docs.livekit.io/reference/python/livekit/plugins/silero/index.html#livekit.plugins.silero.VAD "livekit.plugins.silero.VAD") , opts: _VADOptions, model: onnx_model.OnnxModel)` Expand source code class VADStream(agents.vad.VADStream): def __init__( self, vad: VAD, opts: _VADOptions, model: onnx_model.OnnxModel ) -> None: super().__init__(vad) self._opts, self._model = opts, model self._loop = asyncio.get_event_loop() self._executor = ThreadPoolExecutor(max_workers=1) self._task.add_done_callback(lambda _: self._executor.shutdown(wait=False)) self._exp_filter = utils.ExpFilter(alpha=0.35) self._input_sample_rate = 0 self._speech_buffer: np.ndarray | None = None self._speech_buffer_max_reached = False self._prefix_padding_samples = 0 # (input_sample_rate) def update_options( self, *, min_speech_duration: float | None = None, min_silence_duration: float | None = None, prefix_padding_duration: float | None = None, max_buffered_speech: float | None = None, activation_threshold: float | None = None, ) -> None: """ Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. """ old_max_buffered_speech = self._opts.max_buffered_speech self._opts = _VADOptions( min_speech_duration=min_speech_duration or self._opts.min_speech_duration, min_silence_duration=min_silence_duration or self._opts.min_silence_duration, prefix_padding_duration=prefix_padding_duration or self._opts.prefix_padding_duration, max_buffered_speech=max_buffered_speech or self._opts.max_buffered_speech, activation_threshold=activation_threshold or self._opts.activation_threshold, sample_rate=self._opts.sample_rate, ) if self._input_sample_rate: assert self._speech_buffer is not None self._prefix_padding_samples = int( self._opts.prefix_padding_duration * self._input_sample_rate ) self._speech_buffer.resize( int(self._opts.max_buffered_speech * self._input_sample_rate) + self._prefix_padding_samples ) if self._opts.max_buffered_speech > old_max_buffered_speech: self._speech_buffer_max_reached = False @agents.utils.log_exceptions(logger=logger) async def _main_task(self): inference_f32_data = np.empty(self._model.window_size_samples, dtype=np.float32) speech_buffer_index: int = 0 # "pub_" means public, these values are exposed to the users through events pub_speaking = False pub_speech_duration = 0.0 pub_silence_duration = 0.0 pub_current_sample = 0 pub_timestamp = 0.0 speech_threshold_duration = 0.0 silence_threshold_duration = 0.0 input_frames = [] inference_frames = [] resampler: rtc.AudioResampler | None = None # used to avoid drift when the sample_rate ratio is not an integer input_copy_remaining_fract = 0.0 extra_inference_time = 0.0 async for input_frame in self._input_ch: if not isinstance(input_frame, rtc.AudioFrame): continue # ignore flush sentinel for now if not self._input_sample_rate: self._input_sample_rate = input_frame.sample_rate # alloc the buffers now that we know the input sample rate self._prefix_padding_samples = int( self._opts.prefix_padding_duration * self._input_sample_rate ) self._speech_buffer = np.empty( int(self._opts.max_buffered_speech * self._input_sample_rate) + self._prefix_padding_samples, dtype=np.int16, ) if self._input_sample_rate != self._opts.sample_rate: # resampling needed: the input sample rate isn't the same as the model's # sample rate used for inference resampler = rtc.AudioResampler( input_rate=self._input_sample_rate, output_rate=self._opts.sample_rate, quality=rtc.AudioResamplerQuality.QUICK, # VAD doesn't need high quality ) elif self._input_sample_rate != input_frame.sample_rate: logger.error("a frame with another sample rate was already pushed") continue assert self._speech_buffer is not None input_frames.append(input_frame) if resampler is not None: # the resampler may have a bit of latency, but it is OK to ignore since it should be # negligible inference_frames.extend(resampler.push(input_frame)) else: inference_frames.append(input_frame) while True: start_time = time.perf_counter() available_inference_samples = sum( [frame.samples_per_channel for frame in inference_frames] ) if available_inference_samples < self._model.window_size_samples: break # not enough samples to run inference input_frame = utils.combine_frames(input_frames) inference_frame = utils.combine_frames(inference_frames) # convert data to f32 np.divide( inference_frame.data[: self._model.window_size_samples], np.iinfo(np.int16).max, out=inference_f32_data, dtype=np.float32, ) # run the inference p = await self._loop.run_in_executor( self._executor, self._model, inference_f32_data ) p = self._exp_filter.apply(exp=1.0, sample=p) window_duration = ( self._model.window_size_samples / self._opts.sample_rate ) pub_current_sample += self._model.window_size_samples pub_timestamp += window_duration resampling_ratio = self._input_sample_rate / self._model.sample_rate to_copy = ( self._model.window_size_samples * resampling_ratio + input_copy_remaining_fract ) to_copy_int = int(to_copy) input_copy_remaining_fract = to_copy - to_copy_int # copy the inference window to the speech buffer available_space = len(self._speech_buffer) - speech_buffer_index to_copy_buffer = min(to_copy_int, available_space) if to_copy_buffer > 0: self._speech_buffer[\ speech_buffer_index : speech_buffer_index + to_copy_buffer\ ] = input_frame.data[:to_copy_buffer] speech_buffer_index += to_copy_buffer elif not self._speech_buffer_max_reached: # reached self._opts.max_buffered_speech (padding is included) speech_buffer_max_reached = True logger.warning( "max_buffered_speech reached, ignoring further data for the current speech input" ) inference_duration = time.perf_counter() - start_time extra_inference_time = max( 0.0, extra_inference_time + inference_duration - window_duration, ) if inference_duration > SLOW_INFERENCE_THRESHOLD: logger.warning( "inference is slower than realtime", extra={"delay": extra_inference_time}, ) def _reset_write_cursor(): nonlocal speech_buffer_index, speech_buffer_max_reached assert self._speech_buffer is not None if speech_buffer_index <= self._prefix_padding_samples: return padding_data = self._speech_buffer[\ speech_buffer_index\ - self._prefix_padding_samples : speech_buffer_index\ ] self._speech_buffer_max_reached = False self._speech_buffer[: self._prefix_padding_samples] = padding_data speech_buffer_index = self._prefix_padding_samples def _copy_speech_buffer() -> rtc.AudioFrame: # copy the data from speech_buffer assert self._speech_buffer is not None speech_data = self._speech_buffer[:speech_buffer_index].tobytes() return rtc.AudioFrame( sample_rate=self._input_sample_rate, num_channels=1, samples_per_channel=speech_buffer_index, data=speech_data, ) if pub_speaking: pub_speech_duration += window_duration else: pub_silence_duration += window_duration self._event_ch.send_nowait( agents.vad.VADEvent( type=agents.vad.VADEventType.INFERENCE_DONE, samples_index=pub_current_sample, timestamp=pub_timestamp, silence_duration=pub_silence_duration, speech_duration=pub_speech_duration, probability=p, inference_duration=inference_duration, frames=[\ rtc.AudioFrame(\ data=input_frame.data[:to_copy_int].tobytes(),\ sample_rate=self._input_sample_rate,\ num_channels=1,\ samples_per_channel=to_copy_int,\ )\ ], speaking=pub_speaking, raw_accumulated_silence=silence_threshold_duration, raw_accumulated_speech=speech_threshold_duration, ) ) if p >= self._opts.activation_threshold: speech_threshold_duration += window_duration silence_threshold_duration = 0.0 if not pub_speaking: if speech_threshold_duration >= self._opts.min_speech_duration: pub_speaking = True pub_silence_duration = 0.0 pub_speech_duration = speech_threshold_duration self._event_ch.send_nowait( agents.vad.VADEvent( type=agents.vad.VADEventType.START_OF_SPEECH, samples_index=pub_current_sample, timestamp=pub_timestamp, silence_duration=pub_silence_duration, speech_duration=pub_speech_duration, frames=[_copy_speech_buffer()], speaking=True, ) ) else: silence_threshold_duration += window_duration speech_threshold_duration = 0.0 if not pub_speaking: _reset_write_cursor() if ( pub_speaking and silence_threshold_duration >= self._opts.min_silence_duration ): pub_speaking = False pub_speech_duration = 0.0 pub_silence_duration = silence_threshold_duration self._event_ch.send_nowait( agents.vad.VADEvent( type=agents.vad.VADEventType.END_OF_SPEECH, samples_index=pub_current_sample, timestamp=pub_timestamp, silence_duration=pub_silence_duration, speech_duration=pub_speech_duration, frames=[_copy_speech_buffer()], speaking=False, ) ) _reset_write_cursor() # remove the frames that were used for inference from the input and inference frames input_frames = [] inference_frames = [] # add the remaining data if len(input_frame.data) - to_copy_int > 0: data = input_frame.data[to_copy_int:].tobytes() input_frames.append( rtc.AudioFrame( data=data, sample_rate=self._input_sample_rate, num_channels=1, samples_per_channel=len(data) // 2, ) ) if len(inference_frame.data) - self._model.window_size_samples > 0: data = inference_frame.data[\ self._model.window_size_samples :\ ].tobytes() inference_frames.append( rtc.AudioFrame( data=data, sample_rate=self._opts.sample_rate, num_channels=1, samples_per_channel=len(data) // 2, ) ) Helper class that provides a standard way to create an ABC using inheritance. ### Ancestors * [VADStream](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VADStream "livekit.agents.vad.VADStream") * abc.ABC ### Methods `def update_options(self, *, min_speech_duration: float | None = None, min_silence_duration: float | None = None, prefix_padding_duration: float | None = None, max_buffered_speech: float | None = None, activation_threshold: float | None = None) ‑> None` Expand source code def update_options( self, *, min_speech_duration: float | None = None, min_silence_duration: float | None = None, prefix_padding_duration: float | None = None, max_buffered_speech: float | None = None, activation_threshold: float | None = None, ) -> None: """ Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args: min_speech_duration (float): Minimum duration of speech to start a new speech chunk. min_silence_duration (float): At the end of each speech, wait this duration before ending the speech. prefix_padding_duration (float): Duration of padding to add to the beginning of each speech chunk. max_buffered_speech (float): Maximum duration of speech to keep in the buffer (in seconds). activation_threshold (float): Threshold to consider a frame as speech. """ old_max_buffered_speech = self._opts.max_buffered_speech self._opts = _VADOptions( min_speech_duration=min_speech_duration or self._opts.min_speech_duration, min_silence_duration=min_silence_duration or self._opts.min_silence_duration, prefix_padding_duration=prefix_padding_duration or self._opts.prefix_padding_duration, max_buffered_speech=max_buffered_speech or self._opts.max_buffered_speech, activation_threshold=activation_threshold or self._opts.activation_threshold, sample_rate=self._opts.sample_rate, ) if self._input_sample_rate: assert self._speech_buffer is not None self._prefix_padding_samples = int( self._opts.prefix_padding_duration * self._input_sample_rate ) self._speech_buffer.resize( int(self._opts.max_buffered_speech * self._input_sample_rate) + self._prefix_padding_samples ) if self._opts.max_buffered_speech > old_max_buffered_speech: self._speech_buffer_max_reached = False Update the VAD options. This method allows you to update the VAD options after the VAD object has been created. Args ---- **`min_speech_duration`** : `float` Minimum duration of speech to start a new speech chunk. **`min_silence_duration`** : `float` At the end of each speech, wait this duration before ending the speech. **`prefix_padding_duration`** : `float` Duration of padding to add to the beginning of each speech chunk. **`max_buffered_speech`** : `float` Maximum duration of speech to keep in the buffer (in seconds). **`activation_threshold`** : `float` Threshold to consider a frame as speech. ### Inherited members * `**[VADStream](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VADStream "livekit.agents.vad.VADStream") **`: * `[aclose](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VADStream.aclose "livekit.agents.vad.VADStream.aclose") ` * `[end_input](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VADStream.end_input "livekit.agents.vad.VADStream.end_input") ` * `[flush](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VADStream.flush "livekit.agents.vad.VADStream.flush") ` * `[push_frame](https://docs.livekit.io/reference/python/livekit/agents/vad.html#livekit.agents.vad.VADStream.push_frame "livekit.agents.vad.VADStream.push_frame") ` --- # livekit.plugins.cartesia API documentation Classes ------- `class ChunkedStream (*, tts: [TTS](https://docs.livekit.io/reference/python/livekit/plugins/cartesia/#livekit.plugins.cartesia.TTS "livekit.plugins.cartesia.TTS") , input_text: str, opts: _TTSOptions, session: aiohttp.ClientSession, conn_options: Optional[APIConnectOptions] = None)` Expand source code class ChunkedStream(tts.ChunkedStream): """Synthesize chunked text using the bytes endpoint""" def __init__( self, *, tts: TTS, input_text: str, opts: _TTSOptions, session: aiohttp.ClientSession, conn_options: Optional[APIConnectOptions] = None, ) -> None: super().__init__(tts=tts, input_text=input_text, conn_options=conn_options) self._opts, self._session = opts, session async def _run(self) -> None: request_id = utils.shortuuid() bstream = utils.audio.AudioByteStream( sample_rate=self._opts.sample_rate, num_channels=NUM_CHANNELS ) json = _to_cartesia_options(self._opts) json["transcript"] = self._input_text headers = { API_AUTH_HEADER: self._opts.api_key, API_VERSION_HEADER: API_VERSION, } try: async with self._session.post( self._opts.get_http_url("/tts/bytes"), headers=headers, json=json, timeout=aiohttp.ClientTimeout( total=30, sock_connect=self._conn_options.timeout, ), ) as resp: resp.raise_for_status() emitter = tts.SynthesizedAudioEmitter( event_ch=self._event_ch, request_id=request_id, ) async for data, _ in resp.content.iter_chunks(): for frame in bstream.write(data): emitter.push(frame) for frame in bstream.flush(): emitter.push(frame) emitter.flush() except asyncio.TimeoutError as e: raise APITimeoutError() from e except aiohttp.ClientResponseError as e: raise APIStatusError( message=e.message, status_code=e.status, request_id=None, body=None, ) from e except Exception as e: raise APIConnectionError() from e Synthesize chunked text using the bytes endpoint ### Ancestors * [ChunkedStream](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.ChunkedStream "livekit.agents.tts.tts.ChunkedStream") * abc.ABC ### Inherited members * `**[ChunkedStream](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.ChunkedStream "livekit.agents.tts.tts.ChunkedStream") **`: * `[aclose](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.ChunkedStream.aclose "livekit.agents.tts.tts.ChunkedStream.aclose") ` * `[collect](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.ChunkedStream.collect "livekit.agents.tts.tts.ChunkedStream.collect") ` `class TTS (*, model: TTSModels | str = 'sonic-2', language: str = 'en', encoding: TTSEncoding = 'pcm_s16le', voice: str | list[float] = '794f9389-aac1-45b6-b726-9d9369183238', speed: TTSVoiceSpeed | float | None = None, emotion: list[TTSVoiceEmotion | str] | None = None, sample_rate: int = 24000, api_key: str | None = None, http_session: aiohttp.ClientSession | None = None, base_url: str = 'https://api.cartesia.ai')` Expand source code class TTS(tts.TTS): def __init__( self, *, model: TTSModels | str = "sonic-2", language: str = "en", encoding: TTSEncoding = "pcm_s16le", voice: str | list[float] = TTSDefaultVoiceId, speed: TTSVoiceSpeed | float | None = None, emotion: list[TTSVoiceEmotion | str] | None = None, sample_rate: int = 24000, api_key: str | None = None, http_session: aiohttp.ClientSession | None = None, base_url: str = "https://api.cartesia.ai", ) -> None: """ Create a new instance of Cartesia TTS. See https://docs.cartesia.ai/reference/web-socket/stream-speech/stream-speech for more details on the the Cartesia API. Args: model (TTSModels, optional): The Cartesia TTS model to use. Defaults to "sonic-2". language (str, optional): The language code for synthesis. Defaults to "en". encoding (TTSEncoding, optional): The audio encoding format. Defaults to "pcm_s16le". voice (str | list[float], optional): The voice ID or embedding array. speed (TTSVoiceSpeed | float, optional): Voice Control - Speed (https://docs.cartesia.ai/user-guides/voice-control) emotion (list[TTSVoiceEmotion], optional): Voice Control - Emotion (https://docs.cartesia.ai/user-guides/voice-control) sample_rate (int, optional): The audio sample rate in Hz. Defaults to 24000. api_key (str, optional): The Cartesia API key. If not provided, it will be read from the CARTESIA_API_KEY environment variable. http_session (aiohttp.ClientSession | None, optional): An existing aiohttp ClientSession to use. If not provided, a new session will be created. base_url (str, optional): The base URL for the Cartesia API. Defaults to "https://api.cartesia.ai". """ super().__init__( capabilities=tts.TTSCapabilities(streaming=True), sample_rate=sample_rate, num_channels=NUM_CHANNELS, ) api_key = api_key or os.environ.get("CARTESIA_API_KEY") if not api_key: raise ValueError("CARTESIA_API_KEY must be set") self._opts = _TTSOptions( model=model, language=language, encoding=encoding, sample_rate=sample_rate, voice=voice, speed=speed, emotion=emotion, api_key=api_key, base_url=base_url, ) self._session = http_session self._pool = utils.ConnectionPool[aiohttp.ClientWebSocketResponse]( connect_cb=self._connect_ws, close_cb=self._close_ws, max_session_duration=300, mark_refreshed_on_get=True, ) self._streams = weakref.WeakSet[SynthesizeStream]() async def _connect_ws(self) -> aiohttp.ClientWebSocketResponse: session = self._ensure_session() url = self._opts.get_ws_url( f"/tts/websocket?api_key={self._opts.api_key}&cartesia_version={API_VERSION}" ) return await asyncio.wait_for( session.ws_connect(url), self._conn_options.timeout ) async def _close_ws(self, ws: aiohttp.ClientWebSocketResponse): await ws.close() def _ensure_session(self) -> aiohttp.ClientSession: if not self._session: self._session = utils.http_context.http_session() return self._session def prewarm(self) -> None: self._pool.prewarm() def update_options( self, *, model: TTSModels | str | None = None, language: str | None = None, voice: str | list[float] | None = None, speed: TTSVoiceSpeed | float | None = None, emotion: list[TTSVoiceEmotion | str] | None = None, ) -> None: """ Update the Text-to-Speech (TTS) configuration options. This method allows updating the TTS settings, including model type, language, voice, speed, and emotion. If any parameter is not provided, the existing value will be retained. Args: model (TTSModels, optional): The Cartesia TTS model to use. Defaults to "sonic-2". language (str, optional): The language code for synthesis. Defaults to "en". voice (str | list[float], optional): The voice ID or embedding array. speed (TTSVoiceSpeed | float, optional): Voice Control - Speed (https://docs.cartesia.ai/user-guides/voice-control) emotion (list[TTSVoiceEmotion], optional): Voice Control - Emotion (https://docs.cartesia.ai/user-guides/voice-control) """ self._opts.model = model or self._opts.model self._opts.language = language or self._opts.language self._opts.voice = voice or self._opts.voice self._opts.speed = speed or self._opts.speed if emotion is not None: self._opts.emotion = emotion def synthesize( self, text: str, *, conn_options: Optional[APIConnectOptions] = None, ) -> ChunkedStream: return ChunkedStream( tts=self, input_text=text, conn_options=conn_options, opts=self._opts, session=self._ensure_session(), ) def stream( self, *, conn_options: Optional[APIConnectOptions] = None ) -> "SynthesizeStream": stream = SynthesizeStream( tts=self, pool=self._pool, opts=self._opts, ) self._streams.add(stream) return stream async def aclose(self) -> None: for stream in list(self._streams): await stream.aclose() self._streams.clear() await self._pool.aclose() await super().aclose() Helper class that provides a standard way to create an ABC using inheritance. Create a new instance of Cartesia TTS. See [https://docs.cartesia.ai/reference/web-socket/stream-speech/stream-speech](https://docs.cartesia.ai/reference/web-socket/stream-speech/stream-speech) for more details on the the Cartesia API. Args ---- **`model`** : `TTSModels`, optional The Cartesia TTS model to use. Defaults to "sonic-2". **`language`** : `str`, optional The language code for synthesis. Defaults to "en". **`encoding`** : `TTSEncoding`, optional The audio encoding format. Defaults to "pcm\_s16le". **`voice`** : `str | list[float]`, optional The voice ID or embedding array. **`speed`** : `TTSVoiceSpeed | float`, optional Voice Control - Speed ([https://docs.cartesia.ai/user-guides/voice-control](https://docs.cartesia.ai/user-guides/voice-control) ) **`emotion`** : `list[TTSVoiceEmotion]`, optional Voice Control - Emotion ([https://docs.cartesia.ai/user-guides/voice-control](https://docs.cartesia.ai/user-guides/voice-control) ) **`sample_rate`** : `int`, optional The audio sample rate in Hz. Defaults to 24000. **`api_key`** : `str`, optional The Cartesia API key. If not provided, it will be read from the CARTESIA\_API\_KEY environment variable. **`http_session`** : `aiohttp.ClientSession | None`, optional An existing aiohttp ClientSession to use. If not provided, a new session will be created. **`base_url`** : `str`, optional The base URL for the Cartesia API. Defaults to "https://api.cartesia.ai". ### Ancestors * [TTS](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.TTS "livekit.agents.tts.tts.TTS") * abc.ABC * [EventEmitter](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter "livekit.rtc.event_emitter.EventEmitter") * typing.Generic ### Methods `async def aclose(self) ‑> None` Expand source code async def aclose(self) -> None: for stream in list(self._streams): await stream.aclose() self._streams.clear() await self._pool.aclose() await super().aclose() `def stream(self, *, conn_options: Optional[APIConnectOptions] = None) ‑> livekit.plugins.cartesia.tts.SynthesizeStream` Expand source code def stream( self, *, conn_options: Optional[APIConnectOptions] = None ) -> "SynthesizeStream": stream = SynthesizeStream( tts=self, pool=self._pool, opts=self._opts, ) self._streams.add(stream) return stream `def synthesize(self, text: str, *, conn_options: Optional[APIConnectOptions] = None) ‑> livekit.plugins.cartesia.tts.ChunkedStream` Expand source code def synthesize( self, text: str, *, conn_options: Optional[APIConnectOptions] = None, ) -> ChunkedStream: return ChunkedStream( tts=self, input_text=text, conn_options=conn_options, opts=self._opts, session=self._ensure_session(), ) `def update_options(self, *, model: TTSModels | str | None = None, language: str | None = None, voice: str | list[float] | None = None, speed: TTSVoiceSpeed | float | None = None, emotion: list[TTSVoiceEmotion | str] | None = None) ‑> None` Expand source code def update_options( self, *, model: TTSModels | str | None = None, language: str | None = None, voice: str | list[float] | None = None, speed: TTSVoiceSpeed | float | None = None, emotion: list[TTSVoiceEmotion | str] | None = None, ) -> None: """ Update the Text-to-Speech (TTS) configuration options. This method allows updating the TTS settings, including model type, language, voice, speed, and emotion. If any parameter is not provided, the existing value will be retained. Args: model (TTSModels, optional): The Cartesia TTS model to use. Defaults to "sonic-2". language (str, optional): The language code for synthesis. Defaults to "en". voice (str | list[float], optional): The voice ID or embedding array. speed (TTSVoiceSpeed | float, optional): Voice Control - Speed (https://docs.cartesia.ai/user-guides/voice-control) emotion (list[TTSVoiceEmotion], optional): Voice Control - Emotion (https://docs.cartesia.ai/user-guides/voice-control) """ self._opts.model = model or self._opts.model self._opts.language = language or self._opts.language self._opts.voice = voice or self._opts.voice self._opts.speed = speed or self._opts.speed if emotion is not None: self._opts.emotion = emotion Update the Text-to-Speech (TTS) configuration options. This method allows updating the TTS settings, including model type, language, voice, speed, and emotion. If any parameter is not provided, the existing value will be retained. Args ---- **`model`** : `TTSModels`, optional The Cartesia TTS model to use. Defaults to "sonic-2". **`language`** : `str`, optional The language code for synthesis. Defaults to "en". **`voice`** : `str | list[float]`, optional The voice ID or embedding array. **`speed`** : `TTSVoiceSpeed | float`, optional Voice Control - Speed ([https://docs.cartesia.ai/user-guides/voice-control](https://docs.cartesia.ai/user-guides/voice-control) ) **`emotion`** : `list[TTSVoiceEmotion]`, optional Voice Control - Emotion ([https://docs.cartesia.ai/user-guides/voice-control](https://docs.cartesia.ai/user-guides/voice-control) ) ### Inherited members * `**[TTS](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.TTS "livekit.agents.tts.tts.TTS") **`: * `[emit](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.emit "livekit.agents.tts.tts.TTS.emit") ` * `[off](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.off "livekit.agents.tts.tts.TTS.off") ` * `[on](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.on "livekit.agents.tts.tts.TTS.on") ` * `[once](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.once "livekit.agents.tts.tts.TTS.once") ` * `[prewarm](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.TTS.prewarm "livekit.agents.tts.tts.TTS.prewarm") ` --- # livekit.plugins.rag API documentation Sub-modules ----------- `[livekit.plugins.rag.annoy](https://docs.livekit.io/reference/python/livekit/plugins/rag/annoy.html "livekit.plugins.rag.annoy") ` Classes ------- `class SentenceChunker (*, max_chunk_size: int = 120, chunk_overlap: int = 30, paragraph_tokenizer: Callable[[str], list[str]] = , sentence_tokenizer: [SentenceTokenizer](https://docs.livekit.io/reference/python/livekit/agents/tokenize/tokenizer.html#livekit.agents.tokenize.tokenizer.SentenceTokenizer "livekit.agents.tokenize.tokenizer.SentenceTokenizer")  = , word_tokenizer: [WordTokenizer](https://docs.livekit.io/reference/python/livekit/agents/tokenize/tokenizer.html#livekit.agents.tokenize.tokenizer.WordTokenizer "livekit.agents.tokenize.tokenizer.WordTokenizer")  = )` Expand source code class SentenceChunker: def __init__( self, *, max_chunk_size: int = 120, chunk_overlap: int = 30, paragraph_tokenizer: Callable[\ [str], list[str]\ ] = tokenize.basic.tokenize_paragraphs, sentence_tokenizer: tokenize.SentenceTokenizer = tokenize.basic.SentenceTokenizer(), word_tokenizer: tokenize.WordTokenizer = tokenize.basic.WordTokenizer( ignore_punctuation=False ), ) -> None: self._max_chunk_size = max_chunk_size self._chunk_overlap = chunk_overlap self._paragraph_tokenizer = paragraph_tokenizer self._sentence_tokenizer = sentence_tokenizer self._word_tokenizer = word_tokenizer def chunk(self, *, text: str) -> list[str]: chunks = [] buf_words: list[str] = [] for paragraph in self._paragraph_tokenizer(text): last_buf_words: list[str] = [] for sentence in self._sentence_tokenizer.tokenize(text=paragraph): for word in self._word_tokenizer.tokenize(text=sentence): reconstructed = self._word_tokenizer.format_words( buf_words + [word] ) if len(reconstructed) > self._max_chunk_size: while ( len(self._word_tokenizer.format_words(last_buf_words)) > self._chunk_overlap ): last_buf_words = last_buf_words[1:] new_chunk = self._word_tokenizer.format_words( last_buf_words + buf_words ) chunks.append(new_chunk) last_buf_words = buf_words buf_words = [] buf_words.append(word) if buf_words: while ( len(self._word_tokenizer.format_words(last_buf_words)) > self._chunk_overlap ): last_buf_words = last_buf_words[1:] new_chunk = self._word_tokenizer.format_words( last_buf_words + buf_words ) chunks.append(new_chunk) buf_words = [] return chunks ### Methods `def chunk(self, *, text: str) ‑> list[str]` Expand source code def chunk(self, *, text: str) -> list[str]: chunks = [] buf_words: list[str] = [] for paragraph in self._paragraph_tokenizer(text): last_buf_words: list[str] = [] for sentence in self._sentence_tokenizer.tokenize(text=paragraph): for word in self._word_tokenizer.tokenize(text=sentence): reconstructed = self._word_tokenizer.format_words( buf_words + [word] ) if len(reconstructed) > self._max_chunk_size: while ( len(self._word_tokenizer.format_words(last_buf_words)) > self._chunk_overlap ): last_buf_words = last_buf_words[1:] new_chunk = self._word_tokenizer.format_words( last_buf_words + buf_words ) chunks.append(new_chunk) last_buf_words = buf_words buf_words = [] buf_words.append(word) if buf_words: while ( len(self._word_tokenizer.format_words(last_buf_words)) > self._chunk_overlap ): last_buf_words = last_buf_words[1:] new_chunk = self._word_tokenizer.format_words( last_buf_words + buf_words ) chunks.append(new_chunk) buf_words = [] return chunks --- # livekit.plugins.playai API documentation Classes ------- `class TTS (*, api_key: str | None = None, user_id: str | None = None, voice: str = 's3://voice-cloning-zero-shot/d9ff78ba-d016-47f6-b0ef-dd630f59414e/female-cs/manifest.json', language: str = 'english', sample_rate: int = 24000, model: TTSModel | str = 'Play3.0-mini', word_tokenizer: tokenize.WordTokenizer = , **kwargs)` Expand source code class TTS(tts.TTS): def __init__( self, *, api_key: str | None = None, user_id: str | None = None, voice: str = "s3://voice-cloning-zero-shot/d9ff78ba-d016-47f6-b0ef-dd630f59414e/female-cs/manifest.json", language: str = "english", sample_rate: int = 24000, model: TTSModel | str = "Play3.0-mini", word_tokenizer: tokenize.WordTokenizer = tokenize.basic.WordTokenizer( ignore_punctuation=False ), **kwargs, ) -> None: """ Initialize the PlayAI TTS engine. Args: api_key (str): PlayAI API key. user_id (str): PlayAI user ID. voice (str): Voice manifest URL. model (TTSModel): TTS model, defaults to "Play3.0-mini". language (str): language, defaults to "english". sample_rate (int): sample rate (Hz), A number greater than or equal to 8000, and must be less than or equal to 48000 word_tokenizer (tokenize.WordTokenizer): Tokenizer for processing text. Defaults to basic WordTokenizer. **kwargs: Additional options. """ super().__init__( capabilities=tts.TTSCapabilities( streaming=True, ), sample_rate=sample_rate, num_channels=1, ) api_key = api_key or os.environ.get("PLAYHT_API_KEY") user_id = user_id or os.environ.get("PLAYHT_USER_ID") if not api_key or not user_id: raise ValueError( "PlayHT API key and user ID are required. Set environment variables PLAYHT_API_KEY and PLAYHT_USER_ID or pass them explicitly." ) _validate_kwargs(kwargs) self._config = TTSOptions( voice=voice, format=Format.FORMAT_OGG, # Using OGG format for AudioDecoder sample_rate=sample_rate, language=Language(language), **kwargs, ) self._opts = _Options( model=model, tts_options=self._config, word_tokenizer=word_tokenizer, ) self._client = PlayHTAsyncClient( user_id=user_id, api_key=api_key, ) self._streams = weakref.WeakSet[SynthesizeStream]() def update_options( self, *, voice: str | None = None, model: TTSModel | str | None = None, language: str | None = None, **kwargs, ) -> None: """ Update the TTS options. """ updates = {} if voice is not None: updates["voice"] = voice if language is not None: updates["language"] = Language(language) updates.update(kwargs) _validate_kwargs(updates) for key, value in updates.items(): if value is not None: setattr(self._config, key, value) if model is not None: self._opts.model = model def synthesize( self, text: str, *, conn_options: Optional[APIConnectOptions] = None, ) -> "ChunkedStream": return ChunkedStream( tts=self, input_text=text, conn_options=conn_options, opts=self._opts, ) def stream( self, *, conn_options: Optional[APIConnectOptions] = None ) -> "SynthesizeStream": stream = SynthesizeStream( tts=self, conn_options=conn_options, opts=self._opts, ) self._streams.add(stream) return stream Helper class that provides a standard way to create an ABC using inheritance. Initialize the PlayAI TTS engine. Args ---- **`api_key`** : `str` PlayAI API key. **`user_id`** : `str` PlayAI user ID. **`voice`** : `str` Voice manifest URL. **`model`** : `TTSModel` TTS model, defaults to "Play3.0-mini". **`language`** : `str` language, defaults to "english". **`sample_rate`** : `int` sample rate (Hz), A number greater than or equal to 8000, and must be less than or equal to 48000 **`word_tokenizer`** : `tokenize.WordTokenizer` Tokenizer for processing text. Defaults to basic WordTokenizer. **`**kwargs`** Additional options. ### Ancestors * [TTS](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.TTS "livekit.agents.tts.tts.TTS") * abc.ABC * [EventEmitter](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter "livekit.rtc.event_emitter.EventEmitter") * typing.Generic ### Methods `def stream(self, *, conn_options: Optional[APIConnectOptions] = None) ‑> livekit.plugins.playai.tts.SynthesizeStream` Expand source code def stream( self, *, conn_options: Optional[APIConnectOptions] = None ) -> "SynthesizeStream": stream = SynthesizeStream( tts=self, conn_options=conn_options, opts=self._opts, ) self._streams.add(stream) return stream `def synthesize(self, text: str, *, conn_options: Optional[APIConnectOptions] = None) ‑> livekit.plugins.playai.tts.ChunkedStream` Expand source code def synthesize( self, text: str, *, conn_options: Optional[APIConnectOptions] = None, ) -> "ChunkedStream": return ChunkedStream( tts=self, input_text=text, conn_options=conn_options, opts=self._opts, ) `def update_options(self, *, voice: str | None = None, model: TTSModel | str | None = None, language: str | None = None, **kwargs) ‑> None` Expand source code def update_options( self, *, voice: str | None = None, model: TTSModel | str | None = None, language: str | None = None, **kwargs, ) -> None: """ Update the TTS options. """ updates = {} if voice is not None: updates["voice"] = voice if language is not None: updates["language"] = Language(language) updates.update(kwargs) _validate_kwargs(updates) for key, value in updates.items(): if value is not None: setattr(self._config, key, value) if model is not None: self._opts.model = model Update the TTS options. ### Inherited members * `**[TTS](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.TTS "livekit.agents.tts.tts.TTS") **`: * `[emit](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.emit "livekit.agents.tts.tts.TTS.emit") ` * `[off](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.off "livekit.agents.tts.tts.TTS.off") ` * `[on](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.on "livekit.agents.tts.tts.TTS.on") ` * `[once](https://docs.livekit.io/reference/python/livekit/rtc/event_emitter.html#livekit.rtc.event_emitter.EventEmitter.once "livekit.agents.tts.tts.TTS.once") ` * `[prewarm](https://docs.livekit.io/reference/python/livekit/agents/tts/tts.html#livekit.agents.tts.tts.TTS.prewarm "livekit.agents.tts.tts.TTS.prewarm") ` --- # rememberParticipants rememberParticipants ==================== @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [rememberParticipants](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.state/remember-participants.html) (passedRoom: Room? = null): [List](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.collections/-list/index.html) Remembers the full list of participants, with the local participant included as the first item in the list. Updates automatically whenever the participant list changes. --- # CameraPreview CameraPreview ============= @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [CameraPreview](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-camera-preview.html) (cameraPosition: CameraPosition, modifier: [Modifier](https://developer.android.com/reference/kotlin/androidx/compose/ui/Modifier.html) = Modifier, mirror: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = false) A standalone camera preview composable that can be used without a Room object. Due to hardware limitations, this should not be used while any camera is in use, or it may fail. If using this outside of a RoomScope, ensure that LiveKit.init is called prior to use (e.g. in your Application's onCreate method). --- # rememberTrackMuted rememberTrackMuted ================== @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [rememberTrackMuted](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.state/remember-track-muted.html) (trackRef: [TrackReference](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.types/-track-reference/index.html) ): [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) #### Return true if the referenced track is muted or is a placeholder --- # LiveKit | Recipes and examples for voice AI and more. | LiveKit Docs [Skip to main content](https://docs.livekit.io/recipes/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/recipes/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/recipes/) Search --- # rememberTracks rememberTracks ============== @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [rememberTracks](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.state/remember-tracks.html) (sources: [List](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.collections/-list/index.html) = listOf( Track.Source.CAMERA, Track.Source.SCREEN\_SHARE ), usePlaceholders: [Set](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.collections/-set/index.html) = emptySet(), passedRoom: Room? = null, onlySubscribed: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = true): [List](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.collections/-list/index.html) <[TrackReference](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.types/-track-reference/index.html) \> Returns an array of TrackReferences depending the sources provided. #### Parameters sources The sources of the tracks to provide. Defaults to camera and screen share tracks. usePlaceholders A set of sources to provide placeholders for. A placeholder will provide a TrackReference for participants that don't yet have a track published for that source. Defaults to no placeholders. passedRoom The room to use on, or [RoomLocal](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.local/-room-local.html) if null/not passed. onlySubscribed If true, only return tracks that have been subscribed. Defaults to true. --- # OpenAI Realtime API and LiveKit | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/integrations/openai/realtime/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) ![OpenAI + LiveKit banner](https://docs.livekit.io/_next/image/?url=%2Fimages%2Fagents%2Foai-banner.png&w=3840&q=75)![OpenAI + LiveKit banner](https://docs.livekit.io/_next/image/?url=%2Fimages%2Fagents%2Foai-banner-mobile.png&w=2048&q=75) [Try the playground](https://playground.livekit.io/) On this page [Quickstarts](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#quickstarts) [OpenAI Realtime API and LiveKit](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#openai-realtime-api-and-livekit) [How it works](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#how-it-works) [The Agents framework](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#the-agents-framework) [LiveKit concepts](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#livekit-concepts) [MultimodalAgent](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#multimodal-agent) **Agents 1.0 available for Python** This documentation is for v0.x of the LiveKit Agents framework. See updated documentation here: [OpenAI Realtime API integration guide](https://docs.livekit.io/agents/integrations/realtime/openai/) . _v1.0 for Node.js is coming soon._ Quickstarts[](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#quickstarts) ------------------------------------------------------------------------------------------- [OpenAI Playground\ -----------------\ \ Experiment with OpenAI's Realtime API in the playground with personalities like the **Snarky Teenager** or **Opera Singer**.](https://playground.livekit.io/) [Speech to speech\ ----------------\ \ Use OpenAI's Realtime API in this quickstart guide to create a speech-to-speech agent.\ \ OpenAIRealtime](https://docs.livekit.io/agents/v0/quickstarts/s2s/) OpenAI Realtime API and LiveKit[](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#openai-realtime-api-and-livekit) ----------------------------------------------------------------------------------------------------------------------------------- OpenAI's Realtime API is a WebSocket interface for low-latency audio streaming, best suited for server-to-server use rather than direct consumption by end-user devices. LiveKit offers Python and Node.js integrations for the API, enabling developers to build realtime conversational AI applications using LiveKit's Agents framework. This framework integrates with LiveKit's SDKs and telephony solutions, allowing you to build applications for any platform. [How it works\ ------------](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#how-it-works) ![How the OpenAI Realtime API works with LiveKit Agents](https://docs.livekit.io/images/agents/agents-openai-realtime-api-how-it-works.svg)![How the OpenAI Realtime API works with LiveKit Agents](https://docs.livekit.io/images/agents/agents-openai-realtime-api-how-it-works.svg) WebSocket is not ideal for realtime audio and video over long distances or slower networks. LiveKit bridges this gap by converting the transport to WebRTC and routing data through our global edge network to minimize transmission latency. With the Agents framework, user audio is first transmitted to LiveKit's edge network via WebRTC and routed to your backend agent over low-latency connections. The agent then uses Agents framework integration to relay audio to OpenAI's model via WebSocket. Similarly, speech from OpenAI is streamed back through WebSocket to the agent and relayed to the user via WebRTC. The Agents framework[](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#the-agents-framework) ------------------------------------------------------------------------------------------------------------- The Agents framework provides everything needed to build conversational applications using OpenAI's Realtime API, including: * Support for [Python](https://github.com/livekit/agents) and [Node.js](https://github.com/livekit/agents-js) * SDKs for nearly every platform * Inbound and outbound calling (using SIP trunks) * WebRTC transport via LiveKit Cloud or self-host OSS * Worker load balancing and request distribution (see [Agent lifecycle](https://docs.livekit.io/agents/build/anatomy/#Agent-lifecycle) ) LiveKit concepts[](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#livekit-concepts) ----------------------------------------------------------------------------------------------------- The LiveKit Agents framework uses the following concepts: * **Room**: a realtime session with participants. The room acts as bridge between your end user and your agent. Each room has a name and is identified by a unique ID. * **Participant**: a user or process (i.e. agent) participating in a room. * **Agent**: a programmable AI participant in a room. * **Track**: audio, video, text, or data published by a user or agent, and subscribed to by other participants in the room. MultimodalAgent[](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#multimodal-agent) ---------------------------------------------------------------------------------------------------- The framework includes the [MultimodalAgent](https://docs.livekit.io/agents/voice-agent/multimodal-agent/) class for building speech-to-speech agents that use the OpenAI Realtime API. To learn more about the differences between speech-to-speech and voice pipeline agents, see [Voice agents comparison](https://docs.livekit.io/agents/voice-agent/#Multimodal-or-Voice-Pipeline) . [Try the playground](https://playground.livekit.io/) On this page [Quickstarts](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#quickstarts) [OpenAI Realtime API and LiveKit](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#openai-realtime-api-and-livekit) [How it works](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#how-it-works) [The Agents framework](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#the-agents-framework) [LiveKit concepts](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#livekit-concepts) [MultimodalAgent](https://docs.livekit.io/agents/v0/integrations/openai/realtime/#multimodal-agent) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/v0/integrations/openai/realtime/) Search --- # VAD | LiveKit Agents * [LiveKit Agents](https://docs.livekit.io/reference/agents-js/index.html) * [plugins/agents-plugin-silero](https://docs.livekit.io/reference/agents-js/modules/plugins_agents_plugin_silero.html) * [VAD](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html) Class VAD ========= #### Hierarchy ([view full](https://docs.livekit.io/reference/agents-js/hierarchy.html#plugins/agents-plugin-silero.VAD) ) * [VAD](https://docs.livekit.io/reference/agents-js/classes/agents.VAD.html) * VAD ##### Index ### Constructors [constructor](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#constructor) ### Properties [label](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#label) ### Accessors [capabilities](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#capabilities) ### Methods [stream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#stream) [load](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#load) Constructors ------------ ### constructor[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#constructor) * new VAD(session, opts): [VAD](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html) [](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#constructor.new_VAD) * #### Parameters * session: InferenceSession * opts: VADOptions #### Returns [VAD](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html) Properties ---------- ### label[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#label) label: string = 'silero.VAD' Accessors --------- ### capabilities[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#capabilities) * get capabilities(): [VADCapabilities](https://docs.livekit.io/reference/agents-js/interfaces/agents.VADCapabilities.html) * #### Returns [VADCapabilities](https://docs.livekit.io/reference/agents-js/interfaces/agents.VADCapabilities.html) Methods ------- ### stream[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#stream) * stream(): [VADStream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VADStream.html) [](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#stream.stream-1) * Returns a [VADStream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VADStream.html) that can be used to push audio frames and receive VAD events. #### Returns [VADStream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VADStream.html) ### `Static` load[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#load) * load(opts?): Promise<[VAD](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html) \>[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#load.load-1) * Load and initialize the Silero VAD model. This method loads the ONNX model and prepares it for inference. When options are not provided, sane defaults are used. #### Parameters * opts: Partial = {} #### Returns Promise<[VAD](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html) \> Promise<[VAD](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html) \>: An instance of the VAD class ready for streaming. #### Remarks This method may take time to load the model into memory. It is recommended to call this method inside your prewarm mechanism. #### Example export default defineAgent({ prewarm: async (proc: JobProcess) => { proc.userData.vad = await VAD.load(); }, entry: async (ctx: JobContext) => { const vad = ctx.proc.userData.vad! as VAD; // the rest of your agent logic },}); ### Settings #### Member Visibility * Inherited #### Theme OSLightDark ### On This Page [constructor](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#constructor) [label](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#label) [capabilities](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#capabilities) [stream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#stream) [load](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_silero.VAD.html#load) --- # VideoTrackView VideoTrackView ============== @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [VideoTrackView](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-video-track-view.html) (trackReference: [TrackReference](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.types/-track-reference/index.html) ?, modifier: [Modifier](https://developer.android.com/reference/kotlin/androidx/compose/ui/Modifier.html) = Modifier, room: Room? = null, mirror: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = false, scaleType: [ScaleType](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-scale-type/index.html) = ScaleType.Fill, rendererType: [RendererType](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-renderer-type/index.html) = RendererType.Texture, onFirstFrameRendered: () -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) = {}) @[Composable](https://developer.android.com/reference/kotlin/androidx/compose/runtime/Composable.html) fun [VideoTrackView](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-video-track-view.html) (videoTrack: VideoTrack?, modifier: [Modifier](https://developer.android.com/reference/kotlin/androidx/compose/ui/Modifier.html) = Modifier, passedRoom: Room? = null, mirror: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) = false, scaleType: [ScaleType](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-scale-type/index.html) = ScaleType.Fill, rendererType: [RendererType](https://docs.livekit.io/reference/components-android/livekit-compose-components/io.livekit.android.compose.ui/-renderer-type/index.html) = RendererType.Texture, onFirstFrameRendered: () -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ? = null) Widget for displaying a VideoTrack. Handles the Compose <-> AndroidView interop needed to use TextureViewRenderer. --- # Log collection | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/ops/deployment/cloud/logs/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#overview) [Log types](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-types) [Follow runtime logs](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#follow-runtime-logs) [View build logs](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#view-build-logs) [Forward runtime logs](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#forward-runtime-logs) [Datadog integration](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#datadog) [Log levels](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-levels) [Log retention](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-retention) [Additional resources](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#additional-resources) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#overview) ------------------------------------------------------------------------------- LiveKit Cloud provides realtime logging for your deployed agents, helping you monitor performance, debug issues, and understand your agent's behavior in production. Logs are collected from all phases of your agent's lifecycle - from build to runtime - and can be forwarded to external monitoring services such as [Datadog](https://www.datadoghq.com/) . You can also view some logs with the LiveKit CLI. LiveKit Cloud does not store runtime logs. Log types[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-types) --------------------------------------------------------------------------------- LiveKit Cloud collects two types of logs for your agents: * **Runtime logs**: Your agent's application logs, including stdout, stderr, and any other [logging](https://docs.livekit.io/agents/build/metrics/) you implement. * **Build logs**: Output from the container build process, including Dockerfile execution and dependency installation. Follow runtime logs[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#follow-runtime-logs) ----------------------------------------------------------------------------------------------------- Use the LiveKit CLI to follow logs from your deployed agents in realtime. lk agent logs This command continuously streams logs from the latest running instance of your agent. It also includes a short snapshot of recent logs. **Single instance** The LiveKit CLI only shows logs from the newest worker instance of your agent, which can include multiple jobs. All logs from this worker are included, but it is not a comprehensive view of all logs from all instances for agents running at scale. To collect logs from all instances, use the [Datadog integration](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#datadog-integration) . View build logs[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#view-build-logs) --------------------------------------------------------------------------------------------- Use the LiveKit CLI to view the logs from the most recent build of your agent. lk agent logs --log-type\=build This command prints the logs to stdout, but does not perform a live tail. Forward runtime logs[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#forward-runtime-logs) ------------------------------------------------------------------------------------------------------- Forward your agent logs to external monitoring services for long-term storage, advanced analytics, and integration with your existing observability stack. Currently, the only supported external service is [Datadog](https://www.datadoghq.com/) . ### Datadog integration[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#datadog) Add a [Datadog](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/) client token as a [secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/) to automatically enable log forwarding. If your account is in a region other than `us1`, you can also set the region. All runtime logs are automatically forwarded to your Datadog account. lk agent update-secrets \--secrets "DATADOG\_TOKEN=your-client-token" **DATADOG\_TOKEN**`string``Required` [#](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#DATADOG_TOKEN) Your Datadog [client token](https://docs.datadoghq.com/account_management/api-app-keys/#client-tokens) . **DATADOG\_REGION**`string``Optional`Default: `us1` [#](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#DATADOG_REGION) Your Datadog region. Supported regions are `us1`, `us3`, `us5`, `us1-fed`, `eu`, and `ap1`. Log levels[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-levels) ----------------------------------------------------------------------------------- Your agent worker configuration determines the log levels that are collected and forwarded. The default log level is `INFO`. To use a different value, set the log level in your Dockerfile: CMD \["python", "agent.py", "start", "--log-level=DEBUG"\] For more information on log levels, see the [worker options](https://docs.livekit.io/agents/worker/options/#log-levels) page. Log retention[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-retention) ----------------------------------------------------------------------------------------- LiveKit Cloud does not store runtime logs. Build logs are stored indefinitely for the most recently deployed version. Additional resources[](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#additional-resources) ------------------------------------------------------------------------------------------------------- The following resources may be helpful to design a logging strategy for your agent: [Logs, metrics, and telemetry\ ----------------------------\ \ Guide to collecting logs, metrics, and telemetry data from your agent.](https://docs.livekit.io/agents/build/metrics/) [Worker options\ --------------\ \ Learn how to configure your agent worker.](https://docs.livekit.io/agents/worker/options/) [Secrets management\ ------------------\ \ Learn how to securely manage API keys for log forwarding.](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/) On this page [Overview](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#overview) [Log types](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-types) [Follow runtime logs](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#follow-runtime-logs) [View build logs](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#view-build-logs) [Forward runtime logs](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#forward-runtime-logs) [Datadog integration](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#datadog) [Log levels](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-levels) [Log retention](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#log-retention) [Additional resources](https://docs.livekit.io/agents/ops/deployment/cloud/logs/#additional-resources) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/ops/deployment/cloud/logs/) Search --- # Secrets management | LiveKit Docs [Skip to main content](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/ops/deployment/cloud/secrets/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#overview) [Managing secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#managing-secrets) [Secrets file](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secrets-file) [Using the secrets flag](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#using-the-secrets-flag) [Overwriting all secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#overwriting-all-secrets) [Listing secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#listing-secrets) [Limitations](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#limitations) [Secret names](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secret-names) [Secret values](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secret-values) [LiveKit secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#livekit-credentials) Copy pageSee more page options Overview[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#overview) ---------------------------------------------------------------------------------- Secrets are secure variables that can store sensitive information like API keys, database credentials, and authentication tokens. LiveKit Cloud encrypts, stores, and securely injects these values into your agent containers at runtime, as environment variables. **Keep secrets out of version control** Use a `.env.local` file to store secrets for your local development environment, and a tool such as [python-dotenv](https://github.com/theskumar/python-dotenv) to load them as environment variables. Add `.env` and `.env.*` files to your `.gitignore`, and ensure that all sensitive values are loaded from environment variables rather than included in source code. The starter projects for [Python](https://github.com/livekit-examples/agent-starter-python) and [Node.js](https://github.com/livekit-examples/agent-starter-node) both implement these best practices by default. Managing secrets[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#managing-secrets) -------------------------------------------------------------------------------------------------- Initial secrets are set when the `lk agent create` command is run. You can update secrets at any time with `lk agent update-secrets`. Updating secrets triggers a rolling restart of the agent, to ensure new sessions start with the updated secrets. ### Secrets file[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secrets-file) If you don't pass any arguments, the LiveKit CLI looks for an environment, and prompts you to load the secrets from that file to your agent. The CLI looks for the following environment files: * `.env` * `.env.local` * `.env.production` You can explicitly specify a secrets file with the `--secrets-file` option. The file must contain one secret per line, in `KEY=value` format. lk agent create --secrets-file\=path/to/secrets.env The CLI copies all values form the file, [except for LiveKit Cloud credentials](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#livekit-credentials) . ### Using the secrets flag[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#using-the-secrets-flag) You can provide each secret individually with the CLI using the `--secrets` flag. Pass the secret in `KEY=value` format. To pass multiple secrets, use multiple `--secrets` flags. lk agent update-secrets \--secrets "SECRET\_A=foo" \--secrets "SECRET\_B=bar" ### Overwriting all secrets[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#overwriting-all-secrets) By default, the CLI adds or updates the provided secrets, while leaving other existing secrets as-is. To delete all existing secrets and replace them with the provided secrets, use the `--overwrite` flag. lk agent update-secrets --secrets-file\=new-secrets.env \--overwrite ### Listing secrets[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#listing-secrets) To list all secrets for an agent, use `lk agent secrets`. You can see the names, creation date, and last updated date for each secret. The secret values, however, aren't displayed and can't be retrieved from the CLI. Limitations[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#limitations) ---------------------------------------------------------------------------------------- The following limitations apply to all secrets. ### Secret names[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secret-names) Secret names have the following restrictions: * Must contain only letters, numbers, and underscores. * Must not exceed 70 characters in length. * Are case sensitive. LiveKit recommends that you use only uppercase letters and underscores for secret names, but this is not required. ### Secret values[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secret-values) Secret values have a maximum size of 16KB. They are stored in encrypted form, and can't be retrieved from the CLI or dashboard. The values are provided at runtime to your agent as unencrypted environment variables. ### LiveKit secrets[](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#livekit-credentials) LiveKit Cloud provides the following environment variables automatically, to ensure your agent connects to its associated LiveKit Cloud project: * `LIVEKIT_URL` - Your LiveKit Cloud server URL * `LIVEKIT_API_KEY` - An API key for your project * `LIVEKIT_API_SECRET` - An API secret for your project These values are auto-generated by LiveKit Cloud and can't be set or modified as secrets. On this page [Overview](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#overview) [Managing secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#managing-secrets) [Secrets file](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secrets-file) [Using the secrets flag](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#using-the-secrets-flag) [Overwriting all secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#overwriting-all-secrets) [Listing secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#listing-secrets) [Limitations](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#limitations) [Secret names](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secret-names) [Secret values](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#secret-values) [LiveKit secrets](https://docs.livekit.io/agents/ops/deployment/cloud/secrets/#livekit-credentials) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/agents/ops/deployment/cloud/secrets/) Search --- # EgressClient | LiveKit JS Server SDK - v2.13.1 * [LiveKit JS Server SDK](https://docs.livekit.io/reference/server-sdk-js/modules.html) * [EgressClient](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html) Class EgressClient ================== Client to access Egress APIs #### Hierarchy * ServiceBase * EgressClient ##### Index ### Constructors [constructor](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#constructor) ### Methods [authHeader](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#authheader) [listEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#listegress) [startParticipantEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startparticipantegress) [startRoomCompositeEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startroomcompositeegress) [startTrackCompositeEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackcompositeegress) [startTrackEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackegress) [startWebEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startwebegress) [stopEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#stopegress) [updateLayout](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatelayout) [updateStream](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatestream) Constructors ------------ ### constructor[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#constructor) * new EgressClient(host: string, apiKey?: string, secret?: string): [EgressClient](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html) [](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#constructoregressclient) * #### Parameters * host: string hostname including protocol. i.e. 'https://.livekit.cloud' * `Optional`apiKey: string API Key, can be set in env var LIVEKIT\_API\_KEY * `Optional`secret: string API Secret, can be set in env var LIVEKIT\_API\_SECRET #### Returns [EgressClient](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html) Methods ------- ### authHeader[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#authheader) * authHeader(grant: [VideoGrant](https://docs.livekit.io/reference/server-sdk-js/interfaces/VideoGrant.html) , sip?: [SIPGrant](https://docs.livekit.io/reference/server-sdk-js/interfaces/SIPGrant.html) ): Promise\>[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#authheader-1) * #### Parameters * grant: [VideoGrant](https://docs.livekit.io/reference/server-sdk-js/interfaces/VideoGrant.html) * `Optional`sip: [SIPGrant](https://docs.livekit.io/reference/server-sdk-js/interfaces/SIPGrant.html) #### Returns Promise\> ### listEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#listegress) * listEgress(options?: [ListEgressOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/ListEgressOptions.html) ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#listegress-1) * #### Parameters * `Optional`options: [ListEgressOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/ListEgressOptions.html) options to filter listed Egresses, by default returns all Egress instances #### Returns Promise * listEgress(roomName?: string): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#listegress-2) * #### Parameters * `Optional`roomName: string list egress for one room only #### Returns Promise #### Deprecated[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#deprecated) use `listEgress(options?: ListEgressOptions)` instead ### startParticipantEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startparticipantegress) * startParticipantEgress(     roomName: string,     identity: string,     output: [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) ,     opts?: [ParticipantEgressOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/ParticipantEgressOptions.html) , ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startparticipantegress-1) * Export a participant's audio and video tracks, #### Parameters * roomName: string room name * identity: string * output: [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) one or more outputs * `Optional`opts: [ParticipantEgressOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/ParticipantEgressOptions.html) ParticipantEgressOptions #### Returns Promise ### startRoomCompositeEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startroomcompositeegress) * startRoomCompositeEgress(     roomName: string,     output:         | EncodedFileOutput         | StreamOutput         | SegmentedFileOutput         | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) ,     opts?: [RoomCompositeOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/RoomCompositeOptions.html) , ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startroomcompositeegress-1) * #### Parameters * roomName: string room name * output: EncodedFileOutput | StreamOutput | SegmentedFileOutput | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) file or stream output * `Optional`opts: [RoomCompositeOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/RoomCompositeOptions.html) RoomCompositeOptions #### Returns Promise * startRoomCompositeEgress(     roomName: string,     output:         | EncodedFileOutput         | StreamOutput         | SegmentedFileOutput         | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) ,     layout?: string,     options?: EncodingOptionsPreset | EncodingOptions,     audioOnly?: boolean,     videoOnly?: boolean,     customBaseUrl?: string,     audioMixing?: AudioMixing, ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startroomcompositeegress-2) * #### Parameters * roomName: string * output: EncodedFileOutput | StreamOutput | SegmentedFileOutput | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) * `Optional`layout: string * `Optional`options: EncodingOptionsPreset | EncodingOptions * `Optional`audioOnly: boolean * `Optional`videoOnly: boolean * `Optional`customBaseUrl: string * `Optional`audioMixing: AudioMixing #### Returns Promise #### Deprecated[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#deprecated-1) use RoomCompositeOptions instead ### startTrackCompositeEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackcompositeegress) * startTrackCompositeEgress(     roomName: string,     output:         | EncodedFileOutput         | StreamOutput         | SegmentedFileOutput         | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) ,     opts?: [TrackCompositeOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/TrackCompositeOptions.html) , ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackcompositeegress-1) * #### Parameters * roomName: string room name * output: EncodedFileOutput | StreamOutput | SegmentedFileOutput | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) file or stream output * `Optional`opts: [TrackCompositeOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/TrackCompositeOptions.html) TrackCompositeOptions #### Returns Promise * startTrackCompositeEgress(     roomName: string,     output:         | EncodedFileOutput         | StreamOutput         | SegmentedFileOutput         | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) ,     audioTrackId?: string,     videoTrackId?: string,     options?: EncodingOptionsPreset | EncodingOptions, ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackcompositeegress-2) * #### Parameters * roomName: string * output: EncodedFileOutput | StreamOutput | SegmentedFileOutput | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) * `Optional`audioTrackId: string * `Optional`videoTrackId: string * `Optional`options: EncodingOptionsPreset | EncodingOptions #### Returns Promise #### Deprecated[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#deprecated-2) use TrackCompositeOptions instead ### startTrackEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackegress) * startTrackEgress(     roomName: string,     output: string | DirectFileOutput,     trackId: string, ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackegress-1) * #### Parameters * roomName: string room name * output: string | DirectFileOutput file or websocket output * trackId: string track Id #### Returns Promise ### startWebEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startwebegress) * startWebEgress(     url: string,     output:         | EncodedFileOutput         | StreamOutput         | SegmentedFileOutput         | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) ,     opts?: [WebOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/WebOptions.html) , ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startwebegress-1) * #### Parameters * url: string url * output: EncodedFileOutput | StreamOutput | SegmentedFileOutput | [EncodedOutputs](https://docs.livekit.io/reference/server-sdk-js/interfaces/EncodedOutputs.html) file or stream output * `Optional`opts: [WebOptions](https://docs.livekit.io/reference/server-sdk-js/interfaces/WebOptions.html) WebOptions #### Returns Promise ### stopEgress[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#stopegress) * stopEgress(egressId: string): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#stopegress-1) * #### Parameters * egressId: string #### Returns Promise ### updateLayout[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatelayout) * updateLayout(egressId: string, layout: string): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatelayout-1) * #### Parameters * egressId: string * layout: string #### Returns Promise ### updateStream[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatestream) * updateStream(     egressId: string,     addOutputUrls?: string\[\],     removeOutputUrls?: string\[\], ): Promise[](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatestream-1) * #### Parameters * egressId: string * `Optional`addOutputUrls: string\[\] * `Optional`removeOutputUrls: string\[\] #### Returns Promise ### Settings Member Visibility * Inherited ThemeOSLightDark ### On This Page Constructors [constructor](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#constructor) Methods [authHeader](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#authheader) [listEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#listegress) [startParticipantEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startparticipantegress) [startRoomCompositeEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startroomcompositeegress) [startTrackCompositeEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackcompositeegress) [startTrackEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#starttrackegress) [startWebEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#startwebegress) [stopEgress](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#stopegress) [updateLayout](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatelayout) [updateStream](https://docs.livekit.io/reference/server-sdk-js/classes/EgressClient.html#updatestream) --- # Web egress | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/egress/web/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/web/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/home/egress/web/#overview) [Usage](https://docs.livekit.io/home/egress/web/#usage) [Supported Outputs](https://docs.livekit.io/home/egress/web/#supported-outputs) Copy pageSee more page options Overview[](https://docs.livekit.io/home/egress/web/#overview) -------------------------------------------------------------- Web egress allows you to record or stream any website. Similar to room composite egress, it uses headless Chromium to render output. Unlike room composite egress, you can supply any URL, and the lifecycle of web egress is not attached to a LiveKit room. Usage[](https://docs.livekit.io/home/egress/web/#usage) -------------------------------------------------------- JavaScriptGoRubyJavaLiveKit CLI const output \= new EncodedFileOutput({ fileType: EncodedFileType.MP4, filepath: 'livekit-demo/web-test.mp4', output: { case: 's3', value: new S3Upload({ accessKey: 'aws-access-key', secret: 'aws-access-secret', bucket: 'my-bucket', }), }, }); const info \= await egressClient.startWebEgress('https://docs.livekit.io/', { file: output }); const egressID \= info.egressId; ### Supported Outputs[](https://docs.livekit.io/home/egress/web/#supported-outputs) Web egress supports the same output methods as RoomComposite: RTMP, MP4, HLS and Images. Refer to [RoomComposite examples](https://docs.livekit.io/home/egress/room-composite/#starting-a-roomcomposite) for additional details. On this page [Overview](https://docs.livekit.io/home/egress/web/#overview) [Usage](https://docs.livekit.io/home/egress/web/#usage) [Supported Outputs](https://docs.livekit.io/home/egress/web/#supported-outputs) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/web/) Search --- # Track composite egress | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/egress/track-composite/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/track-composite/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/home/egress/track-composite/#overview) [Usage](https://docs.livekit.io/home/egress/track-composite/#usage) [Supported Outputs](https://docs.livekit.io/home/egress/track-composite/#supported-outputs) Copy pageSee more page options Overview[](https://docs.livekit.io/home/egress/track-composite/#overview) -------------------------------------------------------------------------- TrackComposite combines a video and an audio tracks together, outputting as a MP4, HLS, or RTMP stream. In order to resolve encoding differences from the WebRTC session, TrackComposite will transcode input streams to an encoding that's suitable for the output container. TrackComposite egress handles tricky scenarios such as synchronization, or if one of the tracks has been muted by the user. Currently, TrackComposite is the method to use in order to export a single participant's audio and video together. Usage[](https://docs.livekit.io/home/egress/track-composite/#usage) -------------------------------------------------------------------- In order to use a TrackComposite, you need to know the TrackID of an audio and video track. This can be obtained either from the client SDK, or the [TrackPublished webhook](https://docs.livekit.io/home/server/webhooks/#track-published) . JavaScriptGoRubyJavaLiveKit CLI const output \= new EncodedFileOutput({ fileType: EncodedFileType.MP4, filepath: 'livekit-demo/track-composite-test.mp4', output: { case: 's3', value: new S3Upload({ accessKey: 'aws-access-key', secret: 'aws-access-secret', bucket: 'my-bucket', }), }, }); const info \= await egressClient.startTrackCompositeEgress( 'my-room', { file: output }, { audioTrackId, videoTrackId, }, ); const egressID \= info.egressId; ### Supported Outputs[](https://docs.livekit.io/home/egress/track-composite/#supported-outputs) TrackComposite supports the same output methods as RoomComposite: RTMP, MP4, HLS and Images. Refer to [RoomComposite examples](https://docs.livekit.io/home/egress/room-composite/#starting-a-roomcomposite) for additional details. On this page [Overview](https://docs.livekit.io/home/egress/track-composite/#overview) [Usage](https://docs.livekit.io/home/egress/track-composite/#usage) [Supported Outputs](https://docs.livekit.io/home/egress/track-composite/#supported-outputs) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/track-composite/) Search --- # Room composite egress | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/egress/room-composite/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/room-composite/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Overview](https://docs.livekit.io/home/egress/room-composite/#overview) [Default layouts](https://docs.livekit.io/home/egress/room-composite/#default-layouts) [Starting a RoomComposite](https://docs.livekit.io/home/egress/room-composite/#starting-a-roomcomposite) [Stream to RTMP](https://docs.livekit.io/home/egress/room-composite/#stream-to-rtmp) [Stream to HLS](https://docs.livekit.io/home/egress/room-composite/#stream-to-hls) [Generate Image snapshots](https://docs.livekit.io/home/egress/room-composite/#generate-image-snapshots) [Recording in portrait](https://docs.livekit.io/home/egress/room-composite/#recording-in-portrait) [Audio-only composite](https://docs.livekit.io/home/egress/room-composite/#audio-only-composite) Copy pageSee more page options Overview[](https://docs.livekit.io/home/egress/room-composite/#overview) ------------------------------------------------------------------------- One common requirement when recording a room is to capture all of the participants and interactions that take place. This can be challenging in a multi-user application, where different users may be joining, leaving, turning their cameras on and off. It may also be desirable for the recording to look as close to the actual application experience as possible, capturing the richness and interactivity of your application. A `RoomComposite` egress uses a web app to create the composited view, rendering the output with an instance of headless Chromium. In most cases, your existing LiveKit application can be used as a compositing template with few modifications. Default layouts[](https://docs.livekit.io/home/egress/room-composite/#default-layouts) --------------------------------------------------------------------------------------- We provide a few default compositing layouts that works out of the box. They'll be used by default if a custom template URL is not passed in. These templates are deployed alongside and served by the egress service ([source](https://github.com/livekit/egress/tree/main/template-default) ). While it's a great starting point, you can easily [create your own layout](https://docs.livekit.io/home/egress/custom-template/) using standard web technologies that you are already familiar with. | Layout | Preview | | --- | --- | | **grid** | ![Grid layout](https://docs.livekit.io/images/egress/layout-grid.png)![Grid layout](https://docs.livekit.io/images/egress/layout-grid.png) | | **speaker** | ![Speaker layout](https://docs.livekit.io/images/egress/layout-speaker.png)![Speaker layout](https://docs.livekit.io/images/egress/layout-speaker.png) | | **single-speaker** | ![Single speaker layout](https://docs.livekit.io/images/egress/layout-single-speaker.png)![Single speaker layout](https://docs.livekit.io/images/egress/layout-single-speaker.png) | Additionally, you can use a `-light` suffix to change background color to white. i.e. `grid-light`. Starting a RoomComposite[](https://docs.livekit.io/home/egress/room-composite/#starting-a-roomcomposite) --------------------------------------------------------------------------------------------------------- The following example starts a recording for archival to S3. It uses an `EncodedFileOutput` for the egress. JavaScriptGoRubyJavaLiveKit CLI const egressClient \= new EgressClient( 'https://my-livekit-host', 'livekit-api-key', 'livekit-api-secret', ); const fileOutput \= new EncodedFileOutput({ filepath: 'livekit-demo/room-composite-test.mp4', output: { case: 's3', value: new S3Upload({ accessKey: 'aws-access-key', secret: 'aws-access-secret', region: 'aws-region', bucket: 'my-bucket', }), }, }); const info \= await egressClient.startRoomCompositeEgress( 'my-room', { file: fileOutput, }, { layout: 'speaker', // uncomment to use your own templates // customBaseUrl: 'https://my-template-url.com', }, ); const egressID \= info.egressId; * `fileRequest.Output.File.Output` can be left empty if one of `s3`, `azure`, or `gcp` is supplied with your config. * `fileRequest.Output.File.Filepath` can be left empty, and a unique filename will be generated based on the date and room name. ### Stream to RTMP[](https://docs.livekit.io/home/egress/room-composite/#stream-to-rtmp) To stream to RTMP, you would use a `StreamOutput`. When multiple RTMP URLs are specified, LiveKit would multi-cast to all of them at the same time. Stream URLs can be changed even after the egress has started with [UpdateStream](https://docs.livekit.io/home/egress/overview/#updatestream) JavaScriptGoRubyJavaLiveKit CLI const streamOutput \= new StreamOutput({ protocol: StreamProtocol.RTMP, urls: \['rtmp://youtube-url/stream', 'rtmps://twitch-url/path'\], }); const info \= await egressClient.startRoomCompositeEgress( 'my-room', { stream: streamOutput }, { layout: 'grid' }, ); ### Stream to HLS[](https://docs.livekit.io/home/egress/room-composite/#stream-to-hls) As an alternative to generating a single media file, it is possible to have the Egress service generate segments by using the `SegmentedFileOutput` output. The Egress service will split the output in media segments of equal duration (6s by default), and generate a manifest listing all the generated segments. Currently, only [HTTP Live Streaming](https://datatracker.ietf.org/doc/html/rfc8216) compatible segments (using the MPEG TS file format) and manifests are supported. If one of `s3`, `azure`, or `gcp` is supplied with the config or request, each segment will be uploaded with an updated manifest as soon as it is generated. This allows playback of the exported media while the export is still ongoing. JavaScriptGoRubyJavaLiveKit CLI const segmentedOutput \= new SegmentedFileOutput({ filenamePrefix: 'livekit-demo/room-composite-test-', playlistName: 'room-composite-test.m3u8', segmentDuration: 6, protocol: SegmentedFileProtocol.HLS\_PROTOCOL, output: { case: 's3', value: new S3Upload({ accessKey: 'aws-access-key', secret: 'aws-access-secret', region: 'aws-region', bucket: 'my-bucket', }), }, }); const info \= await egressClient.startRoomCompositeEgress('my-room', { segments: segmentedOutput }); * `segmentsRequest.Output.Segments.FilenamePrefix` and `segmentsRequest.Output.Segments.PlaylistName` can be left empty, and unique filenames will be generated based on the date and room name. * The playlist file will always be created in the same directory as the segments, as indicated in `segmentsRequest.Output.Segments.FilenamePrefix`, regardless of any potential directory prefix added to `segmentsRequest.Output.Segments.PlaylistName`. ### Generate Image snapshots[](https://docs.livekit.io/home/egress/room-composite/#generate-image-snapshots) It is possible to have the egress service generate image snaphots of the room composite at regular interval. This can be used for instance to generate thumbnails. A manifest is generated that lists the images with their filename and timestamps. If one of `s3`, `azure`, or `gcp` is supplied with the config or request, each image will be uploaded with an updated manifest as soon as it is generated. JavaScriptGoRubyJavaLiveKit CLI const imageOutput \= new ImageOutput({ captureInterval: 10, filenamePrefix: 'livekit-demo/room-composite-test-', width: 640, height: 480, output: { case: 's3', value: new S3Upload({ accessKey: 'aws-access-key', secret: 'aws-access-secret', region: 'aws-region', bucket: 'my-bucket', }), }, }); const info \= await egressClient.startRoomCompositeEgress('my-room', { images: imageOutput }); ### Recording in portrait[](https://docs.livekit.io/home/egress/room-composite/#recording-in-portrait) Portrait orientation can be specified by either setting a preset or advanced options. Egress will resize the Chrome compositor to your specified resolution. However, keep in mind: * Chrome has a minimum browser width limit of 500px. * Your application should maintain a portrait layout, even when the browser reports a width larger than typical mobile phones. (e.g., 720px width or higher). JavaScriptGoRubyJavaLiveKit CLI const segmentedOutput \= new SegmentedFileOutput({ filenamePrefix: 'livekit-demo/room-composite-test-', playlistName: 'room-composite-test.m3u8', segmentDuration: 6, protocol: SegmentedFileProtocol.HLS\_PROTOCOL, preset: EncodingOptionsPreset.PORTRAIT\_H264\_720P\_30, output: { ... }, }); const info \= await egressClient.startRoomCompositeEgress('my-room', {segments: segmentedOutput}); ### Audio-only composite[](https://docs.livekit.io/home/egress/room-composite/#audio-only-composite) If your application is audio-only, you can export a mixed audio file containing audio from all participants in the room. To start an audio-only composite, pass `audio_only=true` when starting an Egress. JavaScriptGoRubyJavaLiveKit CLI const info \= await egressClient.startRoomCompositeEgress('my-room', output, { audioOnly: true }); On this page [Overview](https://docs.livekit.io/home/egress/room-composite/#overview) [Default layouts](https://docs.livekit.io/home/egress/room-composite/#default-layouts) [Starting a RoomComposite](https://docs.livekit.io/home/egress/room-composite/#starting-a-roomcomposite) [Stream to RTMP](https://docs.livekit.io/home/egress/room-composite/#stream-to-rtmp) [Stream to HLS](https://docs.livekit.io/home/egress/room-composite/#stream-to-hls) [Generate Image snapshots](https://docs.livekit.io/home/egress/room-composite/#generate-image-snapshots) [Recording in portrait](https://docs.livekit.io/home/egress/room-composite/#recording-in-portrait) [Audio-only composite](https://docs.livekit.io/home/egress/room-composite/#audio-only-composite) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/egress/room-composite/) Search --- # STT | LiveKit Agents * [LiveKit Agents](https://docs.livekit.io/reference/agents-js/index.html) * [plugins/agents-plugin-deepgram](https://docs.livekit.io/reference/agents-js/modules/plugins_agents_plugin_deepgram.html) * [STT](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html) Class STT ========= An instance of a speech-to-text adapter. #### Remarks This class is abstract, and as such cannot be used directly. Instead, use a provider plugin that exports its own child STT class, which inherits this class's methods. #### Hierarchy ([view full](https://docs.livekit.io/reference/agents-js/hierarchy.html#plugins/agents-plugin-deepgram.STT) ) * [stt](https://docs.livekit.io/reference/agents-js/modules/agents.stt.html) .[STT](https://docs.livekit.io/reference/agents-js/classes/agents.stt.STT.html) * STT ##### Index ### Constructors [constructor](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#constructor) ### Properties [label](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#label) ### Accessors [capabilities](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#capabilities) ### Methods [\_recognize](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#_recognize) [recognize](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#recognize) [stream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#stream) [updateOptions](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#updateOptions) Constructors ------------ ### constructor[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#constructor) * new STT(opts?): [STT](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html) [](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#constructor.new_STT) * #### Parameters * opts: Partial<[STTOptions](https://docs.livekit.io/reference/agents-js/interfaces/plugins_agents_plugin_deepgram.STTOptions.html) \> = defaultSTTOptions #### Returns [STT](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html) Properties ---------- ### label[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#label) label: string = 'deepgram.STT' Accessors --------- ### capabilities[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#capabilities) * get capabilities(): [STTCapabilities](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.STTCapabilities.html) * Returns this STT's capabilities #### Returns [STTCapabilities](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.STTCapabilities.html) Methods ------- ### \_recognize[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#_recognize) * \_recognize(\_): Promise<[SpeechEvent](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.SpeechEvent.html) \>[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#_recognize._recognize-1) * #### Parameters * \_: [AudioBuffer](https://docs.livekit.io/reference/agents-js/types/agents.AudioBuffer.html) #### Returns Promise<[SpeechEvent](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.SpeechEvent.html) \> ### recognize[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#recognize) * recognize(frame): Promise<[SpeechEvent](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.SpeechEvent.html) \>[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#recognize.recognize-1) * Receives an audio buffer and returns transcription in the form of a [SpeechEvent](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.SpeechEvent.html) #### Parameters * frame: [AudioBuffer](https://docs.livekit.io/reference/agents-js/types/agents.AudioBuffer.html) #### Returns Promise<[SpeechEvent](https://docs.livekit.io/reference/agents-js/interfaces/agents.stt.SpeechEvent.html) \> ### stream[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#stream) * stream(): [SpeechStream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.SpeechStream.html) [](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#stream.stream-1) * Returns a [SpeechStream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.SpeechStream.html) that can be used to push audio frames and receive transcriptions #### Returns [SpeechStream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.SpeechStream.html) ### updateOptions[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#updateOptions) * updateOptions(opts): void[](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#updateOptions.updateOptions-1) * #### Parameters * opts: Partial<[STTOptions](https://docs.livekit.io/reference/agents-js/interfaces/plugins_agents_plugin_deepgram.STTOptions.html) \> #### Returns void ### Settings #### Member Visibility * Inherited #### Theme OSLightDark ### On This Page [constructor](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#constructor) [label](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#label) [capabilities](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#capabilities) [\_recognize](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#_recognize) [recognize](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#recognize) [stream](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#stream) [updateOptions](https://docs.livekit.io/reference/agents-js/classes/plugins_agents_plugin_deepgram.STT.html#updateOptions) --- # Welcome to LiveKit | LiveKit Docs [Skip to main content](https://docs.livekit.io/home/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) Copy pageSee more page options [Voice AI quickstart\ -------------------\ \ Build your first voice AI agent in less than 10 minutes.](https://docs.livekit.io/agents/start/voice-ai/) [Intro to LiveKit\ ----------------\ \ An overview of the LiveKit ecosystem and advantages.](https://docs.livekit.io/home/get-started/intro-to-livekit/) [AI Agents\ ---------\ \ The LiveKit framework for building realtime multimodal AI agents.](https://docs.livekit.io/agents/) [Telephony\ ---------\ \ LiveKit SIP integration for telephony applications.](https://docs.livekit.io/sip/) [SDK Quickstarts\ ---------------\ \ Explore our collection of platform quickstarts to get up and running fast.](https://docs.livekit.io/home/quickstarts/) [LiveKit Cloud\ -------------\ \ The fully-managed, globally distributed LiveKit deployment option.](https://docs.livekit.io/home/cloud/) [Recipes\ -------\ \ A comprehensive collection of LiveKit examples, guides, and demos.](https://docs.livekit.io/recipes/) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/home/) Search --- # API reference | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) On this page [Agents framework](https://docs.livekit.io/reference/#agents-framework) [LiveKit SDKs](https://docs.livekit.io/reference/#livekit-sdks) [UI Components](https://docs.livekit.io/reference/#ui-components) [Server APIs](https://docs.livekit.io/reference/#server-apis) Copy pageSee more page options Agents framework[](https://docs.livekit.io/reference/#agents-framework) ------------------------------------------------------------------------ ![Node.js Agents](https://docs.livekit.io/images/sdks/nodejs.svg)![Node.js Agents](https://docs.livekit.io/images/sdks/nodejs.svg) ### Node.js Agents [API](https://docs.livekit.io/reference/agents-js/) [NPM](https://www.npmjs.com/package/@livekit/agents) [GitHub](https://github.com/livekit/agents-js) ![Python Agents](https://docs.livekit.io/images/sdks/python.svg)![Python Agents](https://docs.livekit.io/images/sdks/python.svg) ### Python Agents [API](https://docs.livekit.io/reference/python/v1/livekit/agents/index.html) [API v0](https://docs.livekit.io/reference/python/livekit/agents/index.html) [GitHub](https://github.com/livekit/agents) LiveKit SDKs[](https://docs.livekit.io/reference/#livekit-sdks) ---------------------------------------------------------------- ![JavaScript](https://docs.livekit.io/images/sdks/js.svg)![JavaScript](https://docs.livekit.io/images/sdks/js.svg) ### JavaScript [API](https://docs.livekit.io/reference/client-sdk-js/) [NPM](https://www.npmjs.com/package/livekit-client) [GitHub](https://github.com/livekit/client-sdk-js) ![Swift](https://docs.livekit.io/images/sdks/swift.svg)![Swift](https://docs.livekit.io/images/sdks/swift.svg) ### Swift iOS, macOS, visionOS, and tvOS [API](https://docs.livekit.io/reference/client-sdk-swift/documentation/livekit/) [GitHub](https://github.com/livekit/client-sdk-swift) ![Android](https://docs.livekit.io/images/sdks/android.svg)![Android](https://docs.livekit.io/images/sdks/android.svg) ### Android [API](https://docs.livekit.io/reference/client-sdk-android/index.html) [Android](https://central.sonatype.com/artifact/io.livekit/livekit-android) [GitHub](https://github.com/livekit/client-sdk-android) ![Flutter](https://docs.livekit.io/images/sdks/flutter.svg)![Flutter](https://docs.livekit.io/images/sdks/flutter.svg) ### Flutter [API](https://docs.livekit.io/reference/client-sdk-flutter/index.html) [Flutter](https://pub.dev/packages/livekit_client) [GitHub](https://github.com/livekit/client-sdk-flutter) ![React Native](https://docs.livekit.io/images/sdks/react.svg)![React Native](https://docs.livekit.io/images/sdks/react.svg) ### React Native [API](https://htmlpreview.github.io/?https://raw.githubusercontent.com/livekit/client-sdk-react-native/main/docs/modules.html) [NPM](https://www.npmjs.com/package/@livekit/react-native) [GitHub](https://github.com/livekit/client-sdk-react-native) ![Unity](https://docs.livekit.io/images/sdks/unity.svg)![Unity](https://docs.livekit.io/images/sdks/unity.svg) ### Unity [GitHub](https://github.com/livekit/client-sdk-unity) ![Unity WebGL](https://docs.livekit.io/images/sdks/unity.svg)![Unity WebGL](https://docs.livekit.io/images/sdks/unity.svg) ### Unity WebGL [API](https://livekit.github.io/client-sdk-unity-web/) [GitHub](https://github.com/livekit/client-sdk-unity-web) ![Node.js](https://docs.livekit.io/images/sdks/nodejs.svg)![Node.js](https://docs.livekit.io/images/sdks/nodejs.svg) ### Node.js [GitHub](https://github.com/livekit/node-sdks) ![Rust](https://docs.livekit.io/images/sdks/rust.svg)![Rust](https://docs.livekit.io/images/sdks/rust.svg) ### Rust [Crates.io](https://crates.io/crates/livekit) [GitHub](https://github.com/livekit/rust-sdks) ![Python](https://docs.livekit.io/images/sdks/python.svg)![Python](https://docs.livekit.io/images/sdks/python.svg) ### Python [API](https://docs.livekit.io/reference/python/v1/livekit/rtc/index.html) [GitHub](https://github.com/livekit/python-sdks) ![Go](https://docs.livekit.io/images/sdks/go.svg)![Go](https://docs.livekit.io/images/sdks/go.svg) ### Go [GitHub](https://github.com/livekit/server-sdk-go) UI Components[](https://docs.livekit.io/reference/#ui-components) ------------------------------------------------------------------ ![React Components](https://docs.livekit.io/images/sdks/react.svg)![React Components](https://docs.livekit.io/images/sdks/react.svg) ### React Components [API](https://docs.livekit.io/reference/components/react/) [NPM](https://www.npmjs.com/package/@livekit/components-react) [GitHub](https://github.com/livekit/components-js) ![Swift Components](https://docs.livekit.io/_next/image/?url=%2Fimages%2Fsdks%2Fswiftui.png&w=64&q=75)![Swift Components](https://docs.livekit.io/_next/image/?url=%2Fimages%2Fsdks%2Fswiftui.png&w=64&q=75) ### Swift Components iOS, macOS, visionOS, and tvOS [API](https://livekit.github.io/components-swift/documentation/livekitcomponents) [GitHub](https://github.com/livekit/components-swift) ![Android Components](https://docs.livekit.io/images/sdks/androidcompose.svg)![Android Components](https://docs.livekit.io/images/sdks/androidcompose.svg) ### Android Components [API](https://docs.livekit.io/reference/components/android/) [Android](https://central.sonatype.com/artifact/io.livekit/livekit-android-compose-components) [GitHub](https://github.com/livekit/components-android) ![Flutter Components](https://docs.livekit.io/images/sdks/flutter.svg)![Flutter Components](https://docs.livekit.io/images/sdks/flutter.svg) ### Flutter Components [Flutter](https://pub.dev/packages/livekit_components) [GitHub](https://github.com/livekit/components-flutter) Server APIs[](https://docs.livekit.io/reference/#server-apis) -------------------------------------------------------------- ![Go](https://docs.livekit.io/images/sdks/go.svg)![Go](https://docs.livekit.io/images/sdks/go.svg) ### Go [Go](https://pkg.go.dev/github.com/livekit/server-sdk-go/v2) [GitHub](https://github.com/livekit/server-sdk-go) ![Node.js](https://docs.livekit.io/images/sdks/nodejs.svg)![Node.js](https://docs.livekit.io/images/sdks/nodejs.svg) ### Node.js [API](https://docs.livekit.io/reference/server-sdk-js/index.html) [NPM](https://www.npmjs.com/package/livekit-server-sdk) [GitHub](https://github.com/livekit/server-sdk-js) ![Ruby](https://docs.livekit.io/images/sdks/ruby.svg)![Ruby](https://docs.livekit.io/images/sdks/ruby.svg) ### Ruby [GitHub](https://github.com/livekit/server-sdk-ruby) ![Kotlin/Java](https://docs.livekit.io/images/sdks/kotlin.svg)![Kotlin/Java](https://docs.livekit.io/images/sdks/kotlin.svg) ### Kotlin/Java [GitHub](https://github.com/livekit/server-sdk-kotlin) ![Python](https://docs.livekit.io/images/sdks/python.svg)![Python](https://docs.livekit.io/images/sdks/python.svg) ### Python RTC, API, and Agents Framework [API](https://docs.livekit.io/reference/python/v1/livekit/api/) [GitHub](https://github.com/livekit/python-sdks) ![PHP](https://docs.livekit.io/images/sdks/php.svg)![PHP](https://docs.livekit.io/images/sdks/php.svg) ### PHP [GitHub](https://github.com/agence104/livekit-server-sdk-php) On this page [Agents framework](https://docs.livekit.io/reference/#agents-framework) [LiveKit SDKs](https://docs.livekit.io/reference/#livekit-sdks) [UI Components](https://docs.livekit.io/reference/#ui-components) [Server APIs](https://docs.livekit.io/reference/#server-apis) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/) Search --- # livekit.api API documentation LiveKit Server APIs for Python `pip install livekit-api` Manage rooms, participants, egress, ingress, SIP, and Agent dispatch. Primary entry point is `[LiveKitAPI](https://docs.livekit.io/reference/python/livekit/api/index.html#livekit.api.LiveKitAPI "livekit.api.LiveKitAPI") `. See [https://docs.livekit.io/reference/server/server-apis](https://docs.livekit.io/reference/server/server-apis) for more information. Sub-modules ----------- `[livekit.api.access_token](https://docs.livekit.io/reference/python/livekit/api/access_token.html "livekit.api.access_token") ` `[livekit.api.agent_dispatch_service](https://docs.livekit.io/reference/python/livekit/api/agent_dispatch_service.html "livekit.api.agent_dispatch_service") ` `[livekit.api.egress_service](https://docs.livekit.io/reference/python/livekit/api/egress_service.html "livekit.api.egress_service") ` `[livekit.api.ingress_service](https://docs.livekit.io/reference/python/livekit/api/ingress_service.html "livekit.api.ingress_service") ` `[livekit.api.livekit_api](https://docs.livekit.io/reference/python/livekit/api/livekit_api.html "livekit.api.livekit_api") ` `[livekit.api.room_service](https://docs.livekit.io/reference/python/livekit/api/room_service.html "livekit.api.room_service") ` `[livekit.api.sip_service](https://docs.livekit.io/reference/python/livekit/api/sip_service.html "livekit.api.sip_service") ` `[livekit.api.twirp_client](https://docs.livekit.io/reference/python/livekit/api/twirp_client.html "livekit.api.twirp_client") ` `[livekit.api.version](https://docs.livekit.io/reference/python/livekit/api/version.html "livekit.api.version") ` `[livekit.api.webhook](https://docs.livekit.io/reference/python/livekit/api/webhook.html "livekit.api.webhook") ` Classes ------- `class AccessToken (api_key: str | None = None, api_secret: str | None = None)` Expand source code class AccessToken: ParticipantKind = Literal["standard", "egress", "ingress", "sip", "agent"] def __init__( self, api_key: Optional[str] = None, api_secret: Optional[str] = None, ) -> None: api_key = api_key or os.getenv("LIVEKIT_API_KEY") api_secret = api_secret or os.getenv("LIVEKIT_API_SECRET") if not api_key or not api_secret: raise ValueError("api_key and api_secret must be set") self.api_key = api_key # iss self.api_secret = api_secret self.claims = Claims() # default jwt claims self.identity = "" # sub self.ttl = DEFAULT_TTL # exp def with_ttl(self, ttl: datetime.timedelta) -> "AccessToken": self.ttl = ttl return self def with_grants(self, grants: VideoGrants) -> "AccessToken": self.claims.video = grants return self def with_sip_grants(self, grants: SIPGrants) -> "AccessToken": self.claims.sip = grants return self def with_identity(self, identity: str) -> "AccessToken": self.identity = identity return self def with_kind(self, kind: ParticipantKind) -> "AccessToken": self.claims.kind = kind return self def with_name(self, name: str) -> "AccessToken": self.claims.name = name return self def with_metadata(self, metadata: str) -> "AccessToken": self.claims.metadata = metadata return self def with_attributes(self, attributes: dict[str, str]) -> "AccessToken": self.claims.attributes = attributes return self def with_sha256(self, sha256: str) -> "AccessToken": self.claims.sha256 = sha256 return self def with_room_preset(self, preset: str) -> "AccessToken": self.claims.room_preset = preset return self def with_room_config(self, config: RoomConfiguration) -> "AccessToken": self.claims.room_config = config return self def to_jwt(self) -> str: video = self.claims.video if video and video.room_join and (not self.identity or not video.room): raise ValueError("identity and room must be set when joining a room") # we want to exclude None values from the token jwt_claims = self.claims.asdict() jwt_claims.update( { "sub": self.identity, "iss": self.api_key, "nbf": calendar.timegm(datetime.datetime.now(datetime.timezone.utc).utctimetuple()), "exp": calendar.timegm( (datetime.datetime.now(datetime.timezone.utc) + self.ttl).utctimetuple() ), } ) return jwt.encode(jwt_claims, self.api_secret, algorithm="HS256") ### Class variables `var ParticipantKind` ### Methods `def to_jwt(self) ‑> str` Expand source code def to_jwt(self) -> str: video = self.claims.video if video and video.room_join and (not self.identity or not video.room): raise ValueError("identity and room must be set when joining a room") # we want to exclude None values from the token jwt_claims = self.claims.asdict() jwt_claims.update( { "sub": self.identity, "iss": self.api_key, "nbf": calendar.timegm(datetime.datetime.now(datetime.timezone.utc).utctimetuple()), "exp": calendar.timegm( (datetime.datetime.now(datetime.timezone.utc) + self.ttl).utctimetuple() ), } ) return jwt.encode(jwt_claims, self.api_secret, algorithm="HS256") `def with_attributes(self, attributes: dict[str, str]) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_attributes(self, attributes: dict[str, str]) -> "AccessToken": self.claims.attributes = attributes return self `def with_grants(self, grants: [VideoGrants](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.VideoGrants "livekit.api.access_token.VideoGrants") ) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_grants(self, grants: VideoGrants) -> "AccessToken": self.claims.video = grants return self `def with_identity(self, identity: str) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_identity(self, identity: str) -> "AccessToken": self.identity = identity return self `def with_kind(self, kind: Literal['standard', 'egress', 'ingress', 'sip', 'agent']) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_kind(self, kind: ParticipantKind) -> "AccessToken": self.claims.kind = kind return self `def with_metadata(self, metadata: str) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_metadata(self, metadata: str) -> "AccessToken": self.claims.metadata = metadata return self `def with_name(self, name: str) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_name(self, name: str) -> "AccessToken": self.claims.name = name return self `def with_room_config(self, config: room.RoomConfiguration) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_room_config(self, config: RoomConfiguration) -> "AccessToken": self.claims.room_config = config return self `def with_room_preset(self, preset: str) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_room_preset(self, preset: str) -> "AccessToken": self.claims.room_preset = preset return self `def with_sha256(self, sha256: str) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_sha256(self, sha256: str) -> "AccessToken": self.claims.sha256 = sha256 return self `def with_sip_grants(self, grants: [SIPGrants](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.SIPGrants "livekit.api.access_token.SIPGrants") ) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_sip_grants(self, grants: SIPGrants) -> "AccessToken": self.claims.sip = grants return self `def with_ttl(self, ttl: datetime.timedelta) ‑> [AccessToken](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.AccessToken "livekit.api.access_token.AccessToken") ` Expand source code def with_ttl(self, ttl: datetime.timedelta) -> "AccessToken": self.ttl = ttl return self `class LiveKitAPI (url: str | None = None, api_key: str | None = None, api_secret: str | None = None, *, timeout: aiohttp.client.ClientTimeout | None = None, session: aiohttp.client.ClientSession | None = None)` Expand source code class LiveKitAPI: """LiveKit Server API Client This class is the main entrypoint, which exposes all services. Usage: ```python from livekit import api lkapi = api.LiveKitAPI() rooms = await lkapi.room.list_rooms(api.proto_room.ListRoomsRequest(names=['test-room'])) ``` """ def __init__( self, url: Optional[str] = None, api_key: Optional[str] = None, api_secret: Optional[str] = None, *, timeout: Optional[aiohttp.ClientTimeout] = None, session: Optional[aiohttp.ClientSession] = None, ): """Create a new LiveKitAPI instance. Args: url: LiveKit server URL (read from `LIVEKIT_URL` environment variable if not provided) api_key: API key (read from `LIVEKIT_API_KEY` environment variable if not provided) api_secret: API secret (read from `LIVEKIT_API_SECRET` environment variable if not provided) timeout: Request timeout (default: 60 seconds) session: aiohttp.ClientSession instance to use for requests, if not provided, a new one will be created """ url = url or os.getenv("LIVEKIT_URL") api_key = api_key or os.getenv("LIVEKIT_API_KEY") api_secret = api_secret or os.getenv("LIVEKIT_API_SECRET") if not url: raise ValueError("url must be set") if not api_key or not api_secret: raise ValueError("api_key and api_secret must be set") self._custom_session = True self._session = session if not self._session: self._custom_session = False if not timeout: timeout = aiohttp.ClientTimeout(total=60) self._session = aiohttp.ClientSession(timeout=timeout) self._room = RoomService(self._session, url, api_key, api_secret) self._ingress = IngressService(self._session, url, api_key, api_secret) self._egress = EgressService(self._session, url, api_key, api_secret) self._sip = SipService(self._session, url, api_key, api_secret) self._agent_dispatch = AgentDispatchService(self._session, url, api_key, api_secret) @property def agent_dispatch(self) -> AgentDispatchService: """Instance of the AgentDispatchService""" return self._agent_dispatch @property def room(self) -> RoomService: """Instance of the RoomService""" return self._room @property def ingress(self) -> IngressService: """Instance of the IngressService""" return self._ingress @property def egress(self) -> EgressService: """Instance of the EgressService""" return self._egress @property def sip(self) -> SipService: """Instance of the SipService""" return self._sip async def aclose(self): """Close the API client Call this before your application exits or when the API client is no longer needed.""" # we do not close custom sessions, that's up to the caller if not self._custom_session: await self._session.close() async def __aenter__(self): """@private Support for `async with`""" return self async def __aexit__(self, exc_type, exc_val, exc_tb): """@private Support for `async with`""" await self.aclose() LiveKit Server API Client This class is the main entrypoint, which exposes all services. Usage: from livekit import api lkapi = api.LiveKitAPI() rooms = await lkapi.room.list_rooms(api.proto_room.ListRoomsRequest(names=['test-room'])) Create a new LiveKitAPI instance. Args ---- **`url`** LiveKit server URL (read from `LIVEKIT_URL` environment variable if not provided) **`api_key`** API key (read from `LIVEKIT_API_KEY` environment variable if not provided) **`api_secret`** API secret (read from `LIVEKIT_API_SECRET` environment variable if not provided) **`timeout`** Request timeout (default: 60 seconds) **`session`** aiohttp.ClientSession instance to use for requests, if not provided, a new one will be created ### Instance variables `prop agent_dispatch : [AgentDispatchService](https://docs.livekit.io/reference/python/livekit/api/agent_dispatch_service.html#livekit.api.agent_dispatch_service.AgentDispatchService "livekit.api.agent_dispatch_service.AgentDispatchService") ` Expand source code @property def agent_dispatch(self) -> AgentDispatchService: """Instance of the AgentDispatchService""" return self._agent_dispatch Instance of the AgentDispatchService `prop egress : [EgressService](https://docs.livekit.io/reference/python/livekit/api/egress_service.html#livekit.api.egress_service.EgressService "livekit.api.egress_service.EgressService") ` Expand source code @property def egress(self) -> EgressService: """Instance of the EgressService""" return self._egress Instance of the EgressService `prop ingress : [IngressService](https://docs.livekit.io/reference/python/livekit/api/ingress_service.html#livekit.api.ingress_service.IngressService "livekit.api.ingress_service.IngressService") ` Expand source code @property def ingress(self) -> IngressService: """Instance of the IngressService""" return self._ingress Instance of the IngressService `prop room : [RoomService](https://docs.livekit.io/reference/python/livekit/api/room_service.html#livekit.api.room_service.RoomService "livekit.api.room_service.RoomService") ` Expand source code @property def room(self) -> RoomService: """Instance of the RoomService""" return self._room Instance of the RoomService `prop sip : [SipService](https://docs.livekit.io/reference/python/livekit/api/sip_service.html#livekit.api.sip_service.SipService "livekit.api.sip_service.SipService") ` Expand source code @property def sip(self) -> SipService: """Instance of the SipService""" return self._sip Instance of the SipService ### Methods `async def aclose(self)` Expand source code async def aclose(self): """Close the API client Call this before your application exits or when the API client is no longer needed.""" # we do not close custom sessions, that's up to the caller if not self._custom_session: await self._session.close() Close the API client Call this before your application exits or when the API client is no longer needed. `class SIPGrants (admin: bool = False, call: bool = False)` Expand source code @dataclasses.dataclass class SIPGrants: # manage sip resources admin: bool = False # make outbound calls call: bool = False SIPGrants(admin: bool = False, call: bool = False) ### Instance variables `var admin : bool` `var call : bool` `class TokenVerifier (api_key: str | None = None, api_secret: str | None = None, *, leeway: datetime.timedelta = datetime.timedelta(seconds=60))` Expand source code class TokenVerifier: def __init__( self, api_key: Optional[str] = None, api_secret: Optional[str] = None, *, leeway: datetime.timedelta = DEFAULT_LEEWAY, ) -> None: api_key = api_key or os.getenv("LIVEKIT_API_KEY") api_secret = api_secret or os.getenv("LIVEKIT_API_SECRET") self.api_key = api_key self.api_secret = api_secret self._leeway = leeway def verify(self, token: str, *, verify_signature: bool = True) -> Claims: if verify_signature and (not self.api_key or not self.api_secret): raise ValueError("api_key and api_secret must be set") claims = jwt.decode( token, key=self.api_secret or "", issuer=self.api_key or "", algorithms=["HS256"], leeway=self._leeway.total_seconds(), options={"verify_signature": verify_signature}, ) video_dict = claims.get("video", dict()) video_dict = {camel_to_snake(k): v for k, v in video_dict.items()} video_dict = {k: v for k, v in video_dict.items() if k in VideoGrants.__dataclass_fields__} video = VideoGrants(**video_dict) sip_dict = claims.get("sip", dict()) sip_dict = {camel_to_snake(k): v for k, v in sip_dict.items()} sip_dict = {k: v for k, v in sip_dict.items() if k in SIPGrants.__dataclass_fields__} sip = SIPGrants(**sip_dict) grant_claims = Claims( identity=claims.get("sub", ""), name=claims.get("name", ""), video=video, sip=sip, attributes=claims.get("attributes", {}), metadata=claims.get("metadata", ""), sha256=claims.get("sha256", ""), ) if claims.get("roomPreset"): grant_claims.room_preset = claims.get("roomPreset") if claims.get("roomConfig"): grant_claims.room_config = ParseDict( claims.get("roomConfig"), RoomConfiguration(), ignore_unknown_fields=True, ) return grant_claims ### Methods `def verify(self, token: str, *, verify_signature: bool = True) ‑> [Claims](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.Claims "livekit.api.access_token.Claims") ` Expand source code def verify(self, token: str, *, verify_signature: bool = True) -> Claims: if verify_signature and (not self.api_key or not self.api_secret): raise ValueError("api_key and api_secret must be set") claims = jwt.decode( token, key=self.api_secret or "", issuer=self.api_key or "", algorithms=["HS256"], leeway=self._leeway.total_seconds(), options={"verify_signature": verify_signature}, ) video_dict = claims.get("video", dict()) video_dict = {camel_to_snake(k): v for k, v in video_dict.items()} video_dict = {k: v for k, v in video_dict.items() if k in VideoGrants.__dataclass_fields__} video = VideoGrants(**video_dict) sip_dict = claims.get("sip", dict()) sip_dict = {camel_to_snake(k): v for k, v in sip_dict.items()} sip_dict = {k: v for k, v in sip_dict.items() if k in SIPGrants.__dataclass_fields__} sip = SIPGrants(**sip_dict) grant_claims = Claims( identity=claims.get("sub", ""), name=claims.get("name", ""), video=video, sip=sip, attributes=claims.get("attributes", {}), metadata=claims.get("metadata", ""), sha256=claims.get("sha256", ""), ) if claims.get("roomPreset"): grant_claims.room_preset = claims.get("roomPreset") if claims.get("roomConfig"): grant_claims.room_config = ParseDict( claims.get("roomConfig"), RoomConfiguration(), ignore_unknown_fields=True, ) return grant_claims `class TwirpError (code: str, msg: str, *, status: int, metadata: Dict[str, str] | None = None)` Expand source code class TwirpError(Exception): def __init__( self, code: str, msg: str, *, status: int, metadata: Optional[Dict[str, str]] = None, ) -> None: self._code = code self._msg = msg self._status = status self._metadata = metadata or {} @property def code(self) -> str: return self._code @property def message(self) -> str: return self._msg @property def status(self) -> int: """HTTP status code""" return self._status @property def metadata(self) -> Dict[str, str]: """Twirp metadata""" return self._metadata def __str__(self) -> str: result = f"TwirpError(code={self.code}, message={self.message}, status={self.status}" if self.metadata: result += f", metadata={self.metadata}" result += ")" return result Common base class for all non-exit exceptions. ### Ancestors * builtins.Exception * builtins.BaseException ### Instance variables `prop code : str` Expand source code @property def code(self) -> str: return self._code `prop message : str` Expand source code @property def message(self) -> str: return self._msg `prop metadata : Dict[str, str]` Expand source code @property def metadata(self) -> Dict[str, str]: """Twirp metadata""" return self._metadata Twirp metadata `prop status : int` Expand source code @property def status(self) -> int: """HTTP status code""" return self._status HTTP status code `class TwirpErrorCode` Expand source code class TwirpErrorCode: CANCELED = "canceled" UNKNOWN = "unknown" INVALID_ARGUMENT = "invalid_argument" MALFORMED = "malformed" DEADLINE_EXCEEDED = "deadline_exceeded" NOT_FOUND = "not_found" BAD_ROUTE = "bad_route" ALREADY_EXISTS = "already_exists" PERMISSION_DENIED = "permission_denied" UNAUTHENTICATED = "unauthenticated" RESOURCE_EXHAUSTED = "resource_exhausted" FAILED_PRECONDITION = "failed_precondition" ABORTED = "aborted" OUT_OF_RANGE = "out_of_range" UNIMPLEMENTED = "unimplemented" INTERNAL = "internal" UNAVAILABLE = "unavailable" DATA_LOSS = "dataloss" ### Class variables `var ABORTED` `var ALREADY_EXISTS` `var BAD_ROUTE` `var CANCELED` `var DATA_LOSS` `var DEADLINE_EXCEEDED` `var FAILED_PRECONDITION` `var INTERNAL` `var INVALID_ARGUMENT` `var MALFORMED` `var NOT_FOUND` `var OUT_OF_RANGE` `var PERMISSION_DENIED` `var RESOURCE_EXHAUSTED` `var UNAUTHENTICATED` `var UNAVAILABLE` `var UNIMPLEMENTED` `var UNKNOWN` `class VideoGrants (room_create: bool | None = None, room_list: bool | None = None, room_record: bool | None = None, room_admin: bool | None = None, room_join: bool | None = None, room: str = '', destination_room: str | None = None, can_publish: bool = True, can_subscribe: bool = True, can_publish_data: bool = True, can_publish_sources: List[str] | None = None, can_update_own_metadata: bool | None = None, ingress_admin: bool | None = None, hidden: bool | None = None, recorder: bool | None = None, agent: bool | None = None)` Expand source code @dataclasses.dataclass class VideoGrants: # actions on rooms room_create: Optional[bool] = None room_list: Optional[bool] = None room_record: Optional[bool] = None # actions on a particular room room_admin: Optional[bool] = None room_join: Optional[bool] = None room: str = "" # allows forwarding participant to room destination_room: Optional[str] = None # permissions within a room can_publish: bool = True can_subscribe: bool = True can_publish_data: bool = True # TrackSource types that a participant may publish. # When set, it supersedes CanPublish. Only sources explicitly set here can be # published can_publish_sources: Optional[List[str]] = None # by default, a participant is not allowed to update its own metadata can_update_own_metadata: Optional[bool] = None # actions on ingresses ingress_admin: Optional[bool] = None # applies to all ingress # participant is not visible to other participants (useful when making bots) hidden: Optional[bool] = None # [deprecated] indicates to the room that current participant is a recorder recorder: Optional[bool] = None # indicates that the holder can register as an Agent framework worker agent: Optional[bool] = None VideoGrants(room\_create: Optional\[bool\] = None, room\_list: Optional\[bool\] = None, room\_record: Optional\[bool\] = None, room\_admin: Optional\[bool\] = None, room\_join: Optional\[bool\] = None, room: str = '', destination\_room: Optional\[str\] = None, can\_publish: bool = True, can\_subscribe: bool = True, can\_publish\_data: bool = True, can\_publish\_sources: Optional\[List\[str\]\] = None, can\_update\_own\_metadata: Optional\[bool\] = None, ingress\_admin: Optional\[bool\] = None, hidden: Optional\[bool\] = None, recorder: Optional\[bool\] = None, agent: Optional\[bool\] = None) ### Instance variables `var agent : bool | None` `var can_publish : bool` `var can_publish_data : bool` `var can_publish_sources : List[str] | None` `var can_subscribe : bool` `var can_update_own_metadata : bool | None` `var destination_room : str | None` `var hidden : bool | None` `var ingress_admin : bool | None` `var recorder : bool | None` `var room : str` `var room_admin : bool | None` `var room_create : bool | None` `var room_join : bool | None` `var room_list : bool | None` `var room_record : bool | None` `class WebhookReceiver (token_verifier: [TokenVerifier](https://docs.livekit.io/reference/python/livekit/api/access_token.html#livekit.api.access_token.TokenVerifier "livekit.api.access_token.TokenVerifier") )` Expand source code class WebhookReceiver: def __init__(self, token_verifier: TokenVerifier): self._verifier = token_verifier def receive(self, body: str, auth_token: str) -> WebhookEvent: claims = self._verifier.verify(auth_token) if claims.sha256 is None: raise Exception("sha256 was not found in the token") body_hash = hashlib.sha256(body.encode()).digest() claims_hash = base64.b64decode(claims.sha256) if body_hash != claims_hash: raise Exception("hash mismatch") return Parse(body, WebhookEvent(), ignore_unknown_fields=True) ### Methods `def receive(self, body: str, auth_token: str) ‑> [WebhookEvent](https://docs.livekit.io/reference/python/livekit/api/webhook.html#livekit.api.webhook.WebhookEvent "livekit.api.webhook.WebhookEvent") ` Expand source code def receive(self, body: str, auth_token: str) -> WebhookEvent: claims = self._verifier.verify(auth_token) if claims.sha256 is None: raise Exception("sha256 was not found in the token") body_hash = hashlib.sha256(body.encode()).digest() claims_hash = base64.b64decode(claims.sha256) if body_hash != claims_hash: raise Exception("hash mismatch") return Parse(body, WebhookEvent(), ignore_unknown_fields=True) --- # Room Room ==== class [Room](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html)  @AssistedInject constructor(@Assisted context: [Context](https://developer.android.com/reference/kotlin/android/content/Context.html) , engine: RTCEngine, eglBase: EglBase, localParticipantFactory: [LocalParticipant.Factory](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-local-participant/-factory/index.html) , defaultsManager: DefaultsManager, @Named(value = "dispatcher\_default") defaultDispatcher: CoroutineDispatcher, @Named(value = "dispatcher\_io") ioDispatcher: CoroutineDispatcher, val audioHandler: [AudioHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-handler/index.html) , closeableManager: CloseableManager, e2EEManagerFactory: [E2EEManager.Factory](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.e2ee/-e2-e-e-manager/-factory/index.html) , communicationWorkaround: [CommunicationWorkaround](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-communication-workaround/index.html) , val audioProcessingController: [AudioProcessingController](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-processing-controller/index.html) , val lkObjects: [LKObjects](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.provisions/-l-k-objects/index.html) , networkCallbackManagerFactory: [NetworkCallbackManagerFactory](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.network/-network-callback-manager-factory/index.html) , audioDeviceModule: AudioDeviceModule, regionUrlProviderFactory: RegionUrlProvider.Factory, connectionWarmer: ConnectionWarmer, audioRecordPrewarmer: AudioRecordPrewarmer, incomingDataStreamManager: [IncomingDataStreamManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/index.html) ) : RTCEngine.Listener, ParticipantListener, [RpcManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.rpc/-rpc-manager/index.html) , [IncomingDataStreamManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/index.html) MembersMembers & Extensions Constructors ------------ [Room](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-room.html) Link copied to clipboard @AssistedInject constructor(@Assisted context: [Context](https://developer.android.com/reference/kotlin/android/content/Context.html) , engine: RTCEngine, eglBase: EglBase, localParticipantFactory: [LocalParticipant.Factory](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-local-participant/-factory/index.html) , defaultsManager: DefaultsManager, @Named(value = "dispatcher\_default") defaultDispatcher: CoroutineDispatcher, @Named(value = "dispatcher\_io") ioDispatcher: CoroutineDispatcher, audioHandler: [AudioHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-handler/index.html) , closeableManager: CloseableManager, e2EEManagerFactory: [E2EEManager.Factory](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.e2ee/-e2-e-e-manager/-factory/index.html) , communicationWorkaround: [CommunicationWorkaround](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-communication-workaround/index.html) , audioProcessingController: [AudioProcessingController](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-processing-controller/index.html) , lkObjects: [LKObjects](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.provisions/-l-k-objects/index.html) , networkCallbackManagerFactory: [NetworkCallbackManagerFactory](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.network/-network-callback-manager-factory/index.html) , audioDeviceModule: AudioDeviceModule, regionUrlProviderFactory: RegionUrlProvider.Factory, connectionWarmer: ConnectionWarmer, audioRecordPrewarmer: AudioRecordPrewarmer, incomingDataStreamManager: [IncomingDataStreamManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/index.html) ) Types ----- [Sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-sid/index.html) Link copied to clipboard @Serializable @[JvmInline](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.jvm/-jvm-inline/index.html) value class [Sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-sid/index.html) (val sid: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ) [State](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-state/index.html) Link copied to clipboard enum [State](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-state/index.html) : [Enum](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-enum/index.html) <[Room.State](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-state/index.html) \> Properties ---------- [activeSpeakers](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/active-speakers.html) Link copied to clipboard @FlowObservable @get:FlowObservable val [activeSpeakers](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/active-speakers.html) : [List](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.collections/-list/index.html) <[Participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/index.html) \> [adaptiveStream](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/adaptive-stream.html) Link copied to clipboard var [adaptiveStream](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/adaptive-stream.html) : [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) Automatically manage quality of subscribed video tracks, subscribe to the an appropriate resolution based on the size of the video elements that tracks are attached to. [audioHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-handler.html) Link copied to clipboard val [audioHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-handler.html) : [AudioHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-handler/index.html) The [AudioHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-handler/index.html) for setting up the audio as need. [audioProcessingController](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-processing-controller.html) Link copied to clipboard val [audioProcessingController](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-processing-controller.html) : [AudioProcessingController](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-processing-controller/index.html) [audioProcessorIsEnabled](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-processor-is-enabled.html) Link copied to clipboard var [audioProcessorIsEnabled](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-processor-is-enabled.html) : [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) audio processing is enabled [audioSwitchHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-switch-handler.html) Link copied to clipboard val [audioSwitchHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-switch-handler.html) : [AudioSwitchHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-switch-handler/index.html) ? A convenience getter for the audio handler as a [AudioSwitchHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/-audio-switch-handler/index.html) . [audioTrackCaptureDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-track-capture-defaults.html) Link copied to clipboard var [audioTrackCaptureDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-track-capture-defaults.html) : [LocalAudioTrackOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track/-local-audio-track-options/index.html) Default options to use when creating an audio track. [audioTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-track-publish-defaults.html) Link copied to clipboard var [audioTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/audio-track-publish-defaults.html) : [AudioTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-audio-track-publish-defaults/index.html) Default options to use when publishing an audio track. [dynacast](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/dynacast.html) Link copied to clipboard var [dynacast](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/dynacast.html) : [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) Dynamically pauses video layers that are not being consumed by any subscribers, significantly reducing publishing CPU and bandwidth usage. [e2eeManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/e2ee-manager.html) Link copied to clipboard var [e2eeManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/e2ee-manager.html) : [E2EEManager](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.e2ee/-e2-e-e-manager/index.html) ? end-to-end encryption manager [e2eeOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/e2ee-options.html) Link copied to clipboard var [e2eeOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/e2ee-options.html) : [E2EEOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.e2ee/-e2-e-e-options/index.html) ? Options for end-to-end encryption. Must be setup prior to [connect](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/connect.html) . [events](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/events.html) Link copied to clipboard val [events](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/events.html) : [EventListenable](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.events/-event-listenable/index.html) <[RoomEvent](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.events/-room-event/index.html) \> [isRecording](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/is-recording.html) Link copied to clipboard @FlowObservable @get:FlowObservable var [isRecording](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/is-recording.html) : [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) [lkObjects](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/lk-objects.html) Link copied to clipboard val [lkObjects](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/lk-objects.html) : [LKObjects](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.provisions/-l-k-objects/index.html) A holder for objects that are used internally within LiveKit. [localParticipant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/local-participant.html) Link copied to clipboard val [localParticipant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/local-participant.html) : [LocalParticipant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-local-participant/index.html) [metadata](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/metadata.html) Link copied to clipboard @FlowObservable @get:FlowObservable var [metadata](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/metadata.html) : [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ? [name](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/name.html) Link copied to clipboard @FlowObservable @get:FlowObservable var [name](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/name.html) : [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ? [remoteParticipants](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/remote-participants.html) Link copied to clipboard @FlowObservable @get:FlowObservable val [remoteParticipants](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/remote-participants.html) : [Map](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.collections/-map/index.html) <[Participant.Identity](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/-identity/index.html) , [RemoteParticipant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-remote-participant/index.html) \> [screenShareTrackCaptureDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/screen-share-track-capture-defaults.html) Link copied to clipboard var [screenShareTrackCaptureDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/screen-share-track-capture-defaults.html) : [LocalVideoTrackOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track/-local-video-track-options/index.html) Default options to use when creating a screen share track. [screenShareTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/screen-share-track-publish-defaults.html) Link copied to clipboard var [screenShareTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/screen-share-track-publish-defaults.html) : [VideoTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-video-track-publish-defaults/index.html) Default options to use when publishing a screen share track. [sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/sid.html) Link copied to clipboard @FlowObservable @get:FlowObservable var [sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/sid.html) : [Room.Sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-sid/index.html) ? The session id of the room. [state](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/state.html) Link copied to clipboard @FlowObservable @get:FlowObservable var [state](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/state.html) : [Room.State](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-state/index.html) [videoTrackCaptureDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/video-track-capture-defaults.html) Link copied to clipboard var [videoTrackCaptureDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/video-track-capture-defaults.html) : [LocalVideoTrackOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track/-local-video-track-options/index.html) Default options to use when creating a video track. [videoTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/video-track-publish-defaults.html) Link copied to clipboard var [videoTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/video-track-publish-defaults.html) : [VideoTrackPublishDefaults](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-video-track-publish-defaults/index.html) Default options to use when publishing a video track. Functions --------- [connect](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/connect.html) Link copied to clipboard suspend fun [connect](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/connect.html) (url: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , token: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , options: [ConnectOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android/-connect-options/index.html) = ConnectOptions()): [Nothing](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-nothing/index.html) ? Connect to a LiveKit Room. [disconnect](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/disconnect.html) Link copied to clipboard fun [disconnect](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/disconnect.html) () Disconnect from the room. [getParticipantByIdentity](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-participant-by-identity.html) Link copied to clipboard fun [getParticipantByIdentity](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-participant-by-identity.html) (identity: [Participant.Identity](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/-identity/index.html) ): [Participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/index.html) ? fun [getParticipantByIdentity](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-participant-by-identity.html) (identity: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ): [Participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/index.html) ? [getParticipantBySid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-participant-by-sid.html) Link copied to clipboard fun [getParticipantBySid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-participant-by-sid.html) (sid: [Participant.Sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/-sid/index.html) ): [Participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/index.html) ? fun [getParticipantBySid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-participant-by-sid.html) (sid: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ): [Participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/index.html) ? [getPublisherRTCStats](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-publisher-r-t-c-stats.html) Link copied to clipboard fun [getPublisherRTCStats](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-publisher-r-t-c-stats.html) (callback: RTCStatsCollectorCallback) Get stats for the publisher peer connection. [getSid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-sid.html) Link copied to clipboard suspend fun [getSid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-sid.html) (): [Room.Sid](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/-sid/index.html) Gets the sid of the room. [getSubscriberRTCStats](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-subscriber-r-t-c-stats.html) Link copied to clipboard fun [getSubscriberRTCStats](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/get-subscriber-r-t-c-stats.html) (callback: RTCStatsCollectorCallback) Get stats for the subscriber peer connection. [initVideoRenderer](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/init-video-renderer.html) Link copied to clipboard fun [initVideoRenderer](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/init-video-renderer.html) (viewRenderer: [TextureViewRenderer](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.renderer/-texture-view-renderer/index.html) ) Initialize a [TextureViewRenderer](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.renderer/-texture-view-renderer/index.html) for rendering a video from this room. fun [initVideoRenderer](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/init-video-renderer.html) (viewRenderer: SurfaceViewRenderer) Initialize a SurfaceViewRenderer for rendering a video from this room. [onDataReceived](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#1974421468%2FFunctions%2F-885714121) Link copied to clipboard open fun [onDataReceived](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#1974421468%2FFunctions%2F-885714121) (data: [ByteArray](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-byte-array/index.html) , participant: [RemoteParticipant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-remote-participant/index.html) ) Received data published by another participant [onEngineResumed](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#1717341102%2FFunctions%2F-885714121) Link copied to clipboard open fun [onEngineResumed](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#1717341102%2FFunctions%2F-885714121) () [onEngineResuming](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#-1395724181%2FFunctions%2F-885714121) Link copied to clipboard open fun [onEngineResuming](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#-1395724181%2FFunctions%2F-885714121) () [onRpcPacketReceived](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/on-rpc-packet-received.html) Link copied to clipboard open override fun [onRpcPacketReceived](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/on-rpc-packet-received.html) (dp: ) [onSpeakingChanged](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#1068231150%2FFunctions%2F-885714121) Link copied to clipboard open fun [onSpeakingChanged](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#1068231150%2FFunctions%2F-885714121) (participant: [Participant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/index.html) ) Fired when the current participant's isSpeaking property changes. (including LocalParticipant) [onTrackPublished](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#570354678%2FFunctions%2F-885714121) Link copied to clipboard open fun [onTrackPublished](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html#570354678%2FFunctions%2F-885714121) (publication: [RemoteTrackPublication](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.track/-remote-track-publication/index.html) , participant: [RemoteParticipant](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-remote-participant/index.html) ) When a new track is published to room after the local participant has joined. [performRpc](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/perform-rpc.html) Link copied to clipboard open suspend override fun [performRpc](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/perform-rpc.html) (destinationIdentity: [Participant.Identity](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-participant/-identity/index.html) , method: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , payload: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , responseTimeout: [Duration](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.time/-duration/index.html) ): [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) Initiate an RPC call to a remote participant [prepareConnection](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/prepare-connection.html) Link copied to clipboard suspend fun [prepareConnection](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/prepare-connection.html) (url: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , token: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ? = null) prepareConnection should be called as soon as the page is loaded, in order to speed up the connection attempt. This function will [registerByteStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/register-byte-stream-handler.html) Link copied to clipboard open override fun [registerByteStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/register-byte-stream-handler.html) (topic: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , handler: [ByteStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-byte-stream-handler/index.html) ) Registers a byte stream handler for topic. Only one handler can be set for a particular topic at a time. [registerRpcMethod](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/register-rpc-method.html) Link copied to clipboard open suspend override fun [registerRpcMethod](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/register-rpc-method.html) (method: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , handler: [RpcHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.participant/-rpc-handler/index.html) ) Establishes the participant as a receiver for calls of the specified RPC method. Will overwrite any existing callback for the same method. [registerTextStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/register-text-stream-handler.html) Link copied to clipboard open override fun [registerTextStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/register-text-stream-handler.html) (topic: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) , handler: [TextStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-text-stream-handler/index.html) ) Registers a text stream handler for topic. Only one handler can be set for a particular topic at a time. [release](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/release.html) Link copied to clipboard fun [release](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/release.html) () Release all resources held by this object. [setMicrophoneMute](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/set-microphone-mute.html) Link copied to clipboard fun [setMicrophoneMute](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/set-microphone-mute.html) (muted: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) ) Control muting/unmuting the audio input. [setRoomOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/set-room-options.html) Link copied to clipboard fun [setRoomOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/set-room-options.html) (options: [RoomOptions](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android/-room-options/index.html) ) Copies all the options to the Room object. [setSpeakerMute](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/set-speaker-mute.html) Link copied to clipboard fun [setSpeakerMute](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/set-speaker-mute.html) (muted: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-boolean/index.html) ) Control muting/unmuting all audio output. [unregisterByteStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/unregister-byte-stream-handler.html) Link copied to clipboard open override fun [unregisterByteStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/unregister-byte-stream-handler.html) (topic: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ) Unregisters a previously registered byte handler for topic. [unregisterRpcMethod](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/unregister-rpc-method.html) Link copied to clipboard open override fun [unregisterRpcMethod](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/unregister-rpc-method.html) (method: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ) Unregisters a previously registered RPC method. [unregisterTextStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/unregister-text-stream-handler.html) Link copied to clipboard open override fun [unregisterTextStreamHandler](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room.datastream.incoming/-incoming-data-stream-manager/unregister-text-stream-handler.html) (topic: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) ) Unregisters a previously registered text handler for topic. [withPreconnectAudio](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/with-preconnect-audio.html) Link copied to clipboard suspend fun <[T](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/with-preconnect-audio.html) \> [Room](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.room/-room/index.html) .[withPreconnectAudio](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/with-preconnect-audio.html) (timeout: [Duration](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin.time/-duration/index.html) = TIMEOUT, topic: [String](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-string/index.html) = DEFAULT\_TOPIC, onError: ([Exception](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-exception/index.html) ) -> [Unit](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin-stdlib/kotlin/-unit/index.html) ? = null, operation: suspend () -> [T](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/with-preconnect-audio.html) ): [T](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/with-preconnect-audio.html) Starts a pre-connect audio recording that will be sent to any agents that connect within the [timeout](https://docs.livekit.io/reference/client-sdk-android/livekit-android-sdk/io.livekit.android.audio/with-preconnect-audio.html) . This speeds up preceived connection times, as the user can start speaking prior to actual connection with the agent. --- # useRemoteParticipants | React Components | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) [GitHub\ \ Source](https://github.com/livekit/components-js/blob/main/packages/react/src/hooks/useRemoteParticipants.ts) useRemoteParticipants[](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#useremoteparticipants) ============================================================================================================================== The `useRemoteParticipants` hook returns all remote participants (without the local) of the current room. Import[](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#import) ------------------------------------------------------------------------------------------------ import { useRemoteParticipants } from "@livekit/components-react"; Remarks[](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#remarks) -------------------------------------------------------------------------------------------------- To optimize performance, you can use the `updateOnlyOn` property to decide on what `RoomEvents` the hook updates. Usage[](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#usage) ---------------------------------------------------------------------------------------------- const participants \= useRemoteParticipants(); ; Properties[](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#properties) -------------------------------------------------------------------------------------------------------- **options.room**`Room``Optional` [#](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#options.room) The room to use. If not provided, the hook will use the room from the context. **options.updateOnlyOn**`RoomEvent[]``Optional` [#](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#options.updateOnlyOn) To optimize performance, you can use the `updateOnlyOn` property to decide on what RoomEvents the hook updates. By default it updates on all relevant RoomEvents to keep the returned participants array up to date. The minimal set of non-overwriteable `RoomEvents` is: `[RoomEvent.ParticipantConnected, RoomEvent.ParticipantDisconnected, RoomEvent.ConnectionStateChanged]` Returns[](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#returns) -------------------------------------------------------------------------------------------------- RemoteParticipant\[\] On this page [Import](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#import) [Remarks](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#remarks) [Usage](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#usage) [Properties](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#properties) [Returns](https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/#returns) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/hook/useremoteparticipants/) Search --- # useRoomContext | React Components | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/components/react/hook/useroomcontext/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/hook/useroomcontext/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) [GitHub\ \ Source](https://github.com/livekit/components-js/blob/main/packages/react/src/context/room-context.ts) useRoomContext[](https://docs.livekit.io/reference/components/react/hook/useroomcontext/#useroomcontext) ========================================================================================================= Ensures that a room is provided via context. If no room is provided, an error is thrown. Import[](https://docs.livekit.io/reference/components/react/hook/useroomcontext/#import) ----------------------------------------------------------------------------------------- import { useRoomContext } from "@livekit/components-react"; Returns[](https://docs.livekit.io/reference/components/react/hook/useroomcontext/#returns) ------------------------------------------------------------------------------------------- Room; On this page [Import](https://docs.livekit.io/reference/components/react/hook/useroomcontext/#import) [Returns](https://docs.livekit.io/reference/components/react/hook/useroomcontext/#returns) HomeAI AgentsTelephonyRecipesReference [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) [Sign in](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/hook/useroomcontext/) Search --- # useChat | React Components | LiveKit Docs [Skip to main content](https://docs.livekit.io/reference/components/react/hook/usechat/#main-content) [Docs](https://docs.livekit.io/home/) [GitHub](https://github.com/livekit/livekit) [Slack](https://livekit.io/join-slack) Search [Sign in with Cloud](https://cloud.livekit.io/login?r=/login_success?redirect_to=https://docs.livekit.io/reference/components/react/hook/usechat/) [Home](https://docs.livekit.io/home/) [AI Agents](https://docs.livekit.io/agents/) [Telephony](https://docs.livekit.io/sip/) [Recipes](https://docs.livekit.io/recipes/) [Reference](https://docs.livekit.io/reference/) [GitHub\ \ Source](https://github.com/livekit/components-js/blob/main/packages/react/src/hooks/useChat.ts) useChat[](https://docs.livekit.io/reference/components/react/hook/usechat/#usechat) ==================================================================================== The `useChat` hook provides chat functionality for a LiveKit room. Import[](https://docs.livekit.io/reference/components/react/hook/usechat/#import) ---------------------------------------------------------------------------------- import { useChat } from "@livekit/components-react"; Remarks[](https://docs.livekit.io/reference/components/react/hook/usechat/#remarks) ------------------------------------------------------------------------------------ Message history is not persisted and will be lost if the component is refreshed. You may want to persist message history in the browser, a cache or a database. Usage[](https://docs.livekit.io/reference/components/react/hook/usechat/#usage) -------------------------------------------------------------------------------- function ChatComponent() { const { chatMessages, send, isSending } \= useChat(); return ( {chatMessages.map((msg) \=> (
{msg.from?.identity}: {msg.message} ))}