# Table of Contents - [xAI Grok: What It Is and How To Use It [Tutorial]](#xai-grok-what-it-is-and-how-to-use-it-tutorial-) - [Build Chat and Voice AI Agents Without Code | Voiceflow](#build-chat-and-voice-ai-agents-without-code-voiceflow) - [About Us | Voiceflow](#about-us-voiceflow) - [Agent CMS | Voiceflow](#agent-cms-voiceflow) - [Pathways | A publication by Voiceflow](#pathways-a-publication-by-voiceflow) - [Documentation | Voiceflow](#documentation-voiceflow) - [Demo | Voiceflow](#demo-voiceflow) - [Customer Experience | Voiceflow](#customer-experience-voiceflow) - [Customer Stories | Voiceflow](#customer-stories-voiceflow) - [Conversation Design | Voiceflow](#conversation-design-voiceflow) - [Data Processing Addendum | Voiceflow](#data-processing-addendum-voiceflow) - [Customer Support Automation | Voiceflow](#customer-support-automation-voiceflow) - [Functions | Voiceflow](#functions-voiceflow) - [GDPR Compliance | Voiceflow](#gdpr-compliance-voiceflow) - [Privacy | Voiceflow](#privacy-voiceflow) - [Voiceflow Marketplace | Collections](#voiceflow-marketplace-collections) - [Build advanced AI agents faster than Langchain](#build-advanced-ai-agents-faster-than-langchain) - [Take your Zendesk chatbot to the next level](#take-your-zendesk-chatbot-to-the-next-level) - [Internal Operations | Voiceflow](#internal-operations-voiceflow) - [Knowledge Base & Generative AI | Voiceflow](#knowledge-base-generative-ai-voiceflow) - [Voiceflow Community](#voiceflow-community) - [Platform Overview | Voiceflow](#platform-overview-voiceflow) - [Affiliate Terms](#affiliate-terms) - [Hire an expert](#hire-an-expert) - [Terms | Voiceflow](#terms-voiceflow) - [Revenue Protection | Voiceflow](#revenue-protection-voiceflow) - [Automate contact centers with Voice AI | Voiceflow](#automate-contact-centers-with-voice-ai-voiceflow) - [Tico Demo](#tico-demo) - [Workflows | Voiceflow](#workflows-voiceflow) - [Blog | Voiceflow](#blog-voiceflow) - [Events | Voiceflow](#events-voiceflow) - [Security | Voiceflow](#security-voiceflow) - [6 effective ways to design more impactful conversations | Pathways](#6-effective-ways-to-design-more-impactful-conversations-pathways) - [CXD in 2023: reflections and projections | Pathways](#cxd-in-2023-reflections-and-projections-pathways) - [4 tips for great conversational AI product management | Pathways](#4-tips-for-great-conversational-ai-product-management-pathways) - [15 KPIs for Measuring and Scaling a Generative AI Strategy | Pathways](#15-kpis-for-measuring-and-scaling-a-generative-ai-strategy-pathways) - [4 things you can do with Airtable and conversation design | Pathways](#4-things-you-can-do-with-airtable-and-conversation-design-pathways) - [Find a Voiceflow Expert](#find-a-voiceflow-expert) - [Resources | Voiceflow](#resources-voiceflow) - [AI Agent Boosted a Toronto Clinic’s Monthly Revenue by $50K | Pathways](#ai-agent-boosted-a-toronto-clinic-s-monthly-revenue-by-50k-pathways) - [Become a Voiceflow Expert](#become-a-voiceflow-expert) - [5 principles for good natural language understanding (NLU) design | Pathways](#5-principles-for-good-natural-language-understanding-nlu-design-pathways) - [5 User Personas to Consider in Conversation Design | Pathways](#5-user-personas-to-consider-in-conversation-design-pathways) - [5 tips for better multimodal design | Pathways](#5-tips-for-better-multimodal-design-pathways) - [The 6 major differences between mobile apps and voice apps | Pathways](#the-6-major-differences-between-mobile-apps-and-voice-apps-pathways) - [10 things every voice app should do | Pathways](#10-things-every-voice-app-should-do-pathways) - [A bit of entity advice: Delight your overfillers | Pathways](#a-bit-of-entity-advice-delight-your-overfillers-pathways) - [Announcing Voiceflow's 2020 Community Award winners! | Pathways](#announcing-voiceflow-s-2020-community-award-winners-pathways) - [5 Strategies to Elevate AI-Powered Customer Experiences | Pathways](#5-strategies-to-elevate-ai-powered-customer-experiences-pathways) - [Introducing #VFV2: The next phase of Voiceflow | Pathways](#introducing-vfv2-the-next-phase-of-voiceflow-pathways) - [A quick guide on integrating Alexa with Microsoft LUIS | Pathways](#a-quick-guide-on-integrating-alexa-with-microsoft-luis-pathways) - [AI Assist: LLMs are now in Voiceflow | Pathways](#ai-assist-llms-are-now-in-voiceflow-pathways) - [Enterprise | Build AI Agents for the Enterprise](#enterprise-build-ai-agents-for-the-enterprise) - [6 ways to build context into your conversation designs | Pathways](#6-ways-to-build-context-into-your-conversation-designs-pathways) - [Automation design: The evolution of conversation design | Pathways](#automation-design-the-evolution-of-conversation-design-pathways) - [Anthropic’s Claude is now in Voiceflow | Pathways](#anthropic-s-claude-is-now-in-voiceflow-pathways) - [AI will kill apps and I can’t wait | Pathways](#ai-will-kill-apps-and-i-can-t-wait-pathways) - [How to create an Alexa skill with GPT-4 and Voiceflow | Pathways](#how-to-create-an-alexa-skill-with-gpt-4-and-voiceflow-pathways) - [5 powerful templates for AI automation agencies | Pathways](#5-powerful-templates-for-ai-automation-agencies-pathways) - [Pricing | Voiceflow](#pricing-voiceflow) - [The art of debugging | Pathways](#the-art-of-debugging-pathways) - [AI Agents for Every Industry](#ai-agents-for-every-industry) - [Voiceflow Apps](#voiceflow-apps) - [5 ways to approach an AI project strategy | Pathways](#5-ways-to-approach-an-ai-project-strategy-pathways) - [5 tips to optimize your LLM intent classification prompts | Pathways](#5-tips-to-optimize-your-llm-intent-classification-prompts-pathways) - [Voiceflow Integrations](#voiceflow-integrations) - [AI Customer Support Software | Build Smart Support with Voiceflow](#ai-customer-support-software-build-smart-support-with-voiceflow) - [AI Call Center Solutions | Voiceflow AI Call Center](#ai-call-center-solutions-voiceflow-ai-call-center) - [AI Agents for Business | Voiceflow Use Cases & Builds](#ai-agents-for-business-voiceflow-use-cases-builds) - [AI Agent Builder | Create Intelligent AI Agents | Voiceflow](#ai-agent-builder-create-intelligent-ai-agents-voiceflow) - [AI Lead Generation Tools | Voiceflow AI Lead Generation](#ai-lead-generation-tools-voiceflow-ai-lead-generation) - [AI Chatbot Builder | Voiceflow AI Chatbot Platform](#ai-chatbot-builder-voiceflow-ai-chatbot-platform) - [AI Voice Agent Platform | Build Conversational Voice AI](#ai-voice-agent-platform-build-conversational-voice-ai) - [Cathy Pearl on all things conversation design | Pathways](#cathy-pearl-on-all-things-conversation-design-pathways) - [Voiceflow Templates](#voiceflow-templates) - [The evolution and outlook of voice user interfaces | Pathways](#the-evolution-and-outlook-of-voice-user-interfaces-pathways) - [Contributors Content](#contributors-content) - [AI agent builder for agencies](#ai-agent-builder-for-agencies) - [AI Phone Answering System | Voiceflow AI Phone Answering](#ai-phone-answering-system-voiceflow-ai-phone-answering) - [Automating 70% of customer support for 90+ products: How AI automation transformed Trilogy | Pathways](#automating-70-of-customer-support-for-90-products-how-ai-automation-transformed-trilogy-pathways) - [Building a knowledge base with OpenAI, LangChain, OpenSearch, and Unstructured | Pathways](#building-a-knowledge-base-with-openai-langchain-opensearch-and-unstructured-pathways) - [Want to try Claude in your Voiceflow assistant? Here's how | Pathways](#want-to-try-claude-in-your-voiceflow-assistant-here-s-how-pathways) - [Architecting the Future of AI Agents: 5 Flexible Conversation Frameworks You Need | Pathways](#architecting-the-future-of-ai-agents-5-flexible-conversation-frameworks-you-need-pathways) - [Instacart's Ayesha Saleem shares her 5 best conversational AI practices | Pathways](#instacart-s-ayesha-saleem-shares-her-5-best-conversational-ai-practices-pathways) - [Building a virtual agent from scratch? Start here. | Pathways](#building-a-virtual-agent-from-scratch-start-here-pathways) - [Breaking and rebuilding Nike's chatbot | Pathways](#breaking-and-rebuilding-nike-s-chatbot-pathways) - [How we're building Voiceflow’s machine learning platform from scratch | Pathways](#how-we-re-building-voiceflow-s-machine-learning-platform-from-scratch-pathways) - [AI automation on a budget: Getting started with high ROI use cases | Pathways](#ai-automation-on-a-budget-getting-started-with-high-roi-use-cases-pathways) - [Curiosity and conversation beat any best practice | Pathways](#curiosity-and-conversation-beat-any-best-practice-pathways) - [Confessions of a CxD killjoy: Why I think your AI assistant shouldn’t be funny | Pathways](#confessions-of-a-cxd-killjoy-why-i-think-your-ai-assistant-shouldn-t-be-funny-pathways) - [Creating the Voiceflow NLU | Pathways](#creating-the-voiceflow-nlu-pathways) - [How a few PayPal’ers tackled CXD’s most impossible divide | Pathways](#how-a-few-paypal-ers-tackled-cxd-s-most-impossible-divide-pathways) - [From NLU Intents to RAG with Embeddings: Enhancing User Experience | Pathways](#from-nlu-intents-to-rag-with-embeddings-enhancing-user-experience-pathways) - [Conversation design needs a universal language | Pathways](#conversation-design-needs-a-universal-language-pathways) - [3 reasons everyone hates your AI agent (and what you can do about it) | Pathways](#3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it-pathways) - [Conversational AI has a massive, UX-shaped hole | Pathways](#conversational-ai-has-a-massive-ux-shaped-hole-pathways) - [Behind the hype: How Klarna's customer support agent falls short | Pathways](#behind-the-hype-how-klarna-s-customer-support-agent-falls-short-pathways) - [Conversation design best practices | Pathways](#conversation-design-best-practices-pathways) - [Design conversations to be helpful, not human | Pathways](#design-conversations-to-be-helpful-not-human-pathways) - [Designing conversational bots that fix themselves | Pathways](#designing-conversational-bots-that-fix-themselves-pathways) - [From Hobbits to bots—lessons from Simon Mitchell-Wolf’s journey to CxD | Pathways](#from-hobbits-to-bots-lessons-from-simon-mitchell-wolf-s-journey-to-cxd-pathways) - [Design and prototype production-ready conversational experiences with user personas | Pathways](#design-and-prototype-production-ready-conversational-experiences-with-user-personas-pathways) - [Why the dominant discourse surrounding LLMs needs to change | Pathways](#why-the-dominant-discourse-surrounding-llms-needs-to-change-pathways) - [How eSnipe automated 70% of their help center tickets with an AI-powered search agent | Pathways](#how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent-pathways) - [Your CxD center of excellence (and why you don’t need one right now) | Pathways](#your-cxd-center-of-excellence-and-why-you-don-t-need-one-right-now-pathways) - [Experts share 3 expectations about LLMs to set with business stakeholders | Pathways](#experts-share-3-expectations-about-llms-to-set-with-business-stakeholders-pathways) - [Benchmarking hybrid LLM classification systems | Pathways](#benchmarking-hybrid-llm-classification-systems-pathways) - [How to design intelligent conversational interfaces | Pathways](#how-to-design-intelligent-conversational-interfaces-pathways) - [Building your AI agents like products: A blueprint from POC purgatory to production | Pathways](#building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production-pathways) - [Designing for voice assistants vs. chatbots | Pathways](#designing-for-voice-assistants-vs-chatbots-pathways) - [Flowcharts to Voiceflow: How WooliesX Reinvented Their Conversation Design Workflow | Pathways](#flowcharts-to-voiceflow-how-wooliesx-reinvented-their-conversation-design-workflow-pathways) - [Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate | Pathways](#dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate-pathways) - [How To build a discord bot with Voiceflow | Pathways](#how-to-build-a-discord-bot-with-voiceflow-pathways) - [Crawl, walk, run: 28+ tactics for evolving your AI agent | Pathways](#crawl-walk-run-28-tactics-for-evolving-your-ai-agent-pathways) - [From microbiology to mental health: In conversation with Ebere Wilson | Pathways](#from-microbiology-to-mental-health-in-conversation-with-ebere-wilson-pathways) - [Voiceflow brings visual prototyping & enhanced test navigation to tool | Pathways](#voiceflow-brings-visual-prototyping-enhanced-test-navigation-to-tool-pathways) - [How Rocket Mortgage Accelerated their Team’s Conversation Design Workflow | Pathways](#how-rocket-mortgage-accelerated-their-team-s-conversation-design-workflow-pathways) - [The Five Stages of GenAI Maturity (and How Not to Stall at Stage Two) | Pathways](#the-five-stages-of-genai-maturity-and-how-not-to-stall-at-stage-two-pathways) - [The future of conversational AI | Pathways](#the-future-of-conversational-ai-pathways) - [How I used conversation design to get a job | Pathways](#how-i-used-conversation-design-to-get-a-job-pathways) - [How much do ChatGPT versions affect real world performance? | Pathways](#how-much-do-chatgpt-versions-affect-real-world-performance-pathways) - [Monorepos can help improve your productivity. Here's how | Pathways](#monorepos-can-help-improve-your-productivity-here-s-how-pathways) - [Every app will be a voice app | Pathways](#every-app-will-be-a-voice-app-pathways) - [How to build a Microsoft Teams bot with Voiceflow | Pathways](#how-to-build-a-microsoft-teams-bot-with-voiceflow-pathways) - [How to prove the ROI of conversational AI through buyer journey mapping: framework | Pathways](#how-to-prove-the-roi-of-conversational-ai-through-buyer-journey-mapping-framework-pathways) - [How to create a simple Telegram bot with Voiceflow | Pathways](#how-to-create-a-simple-telegram-bot-with-voiceflow-pathways) - [From Intercom to Voiceflow: How Roam saved 30 hours a week in customer support hours | Pathways](#from-intercom-to-voiceflow-how-roam-saved-30-hours-a-week-in-customer-support-hours-pathways) - [Introducing the Agent step in Voiceflow | Pathways](#introducing-the-agent-step-in-voiceflow-pathways) --- # xAI Grok: What It Is and How To Use It [Tutorial] [](https://www.voiceflow.com/) xAI Grok: What It Is and How To Use It \[Tutorial\] =================================================== Last Updated: October 28, 2025 **Expert written and reviewed**  [\ \ Written by\ \ Voiceflow Team](https://www.voiceflow.com/contributors/voiceflow-team) [\ \ Reviewed by](https://www.voiceflow.com/blog/grok#)  Reviewed by Voiceflow team If you’ve been active on Twitter (now X), you might’ve heard of Grok and wondered if it’s just another AI chatbot. Elon Musk describes the latest innovation from xAI as the “[ultimate AI](https://www.voiceflow.com/blog/ultimate-ai) assistant, capable of transforming how we interact with technology” and one that can help you “discover the answers to life, the universe, everything.” **So does Grok live up to this bold vision?** In this article, we’ll explore its features, unpack its potential, and understand why it has everyone talking. What is Grok? ------------- **Grok is a chatbot created by Elon Musk’s startup, xAI.** The conversational AI assistant is designed for “serious-and-not-so-serious discussions” and enhances user interactions on Twitter (X).  Grok's answer to "Who is Taylor Swift dating?" xAI has released three versions of Grok so far in 2024: | | | | --- | --- | | Version | Release Date | | Grok-1 | March 17, 2024 | | Grok-1.5 | March 28, 2024 | | Grok-1.5 Vision | April 12, 2024 | Grok AI’s Features ------------------ Grok-1.5V is the first multimodal model that excels in text and visual data processing. Here’s a quick look at what Grok can do. ### Smart conversations with the most up-to-date information Grok’s most unique feature is its access to real-time data from X. This means that Grok can provide the most up-to-date responses by pulling from the latest headlines. ### Grok’s multimodal capabilities Grok-1.5V is the first multimodal model from xAI. It’s capable of processing text and visual information like documents, diagrams, and photographs. It excels in real-world spatial understanding and outperforms other models on the new RealWorldQA benchmark, which tests spatial understanding through various real-life images and contexts. ### Code generation from images Grok can translate visual diagrams into functional code.  ### Great for “not-so-serious” topics and discussions As Elon Musk describes, Grok has “a bit of wit” and a “rebellious streak”. This means that the chatbot can engage in conversations that other chatbots like ChatGPT might avoid, making it more relatable and entertaining. For example, according to TechCrunch, when asked about the best time to listen to Christmas music, Grok cheekily responds, “Whenever the hell you want.” Get the latest AI agent news Join Voiceflow CEO, Braden Ream, as he explores the future of agentic tech in business on the Humans Talking Agents podcast. [Listen Now](https://www.voiceflow.com/podcasts/humans-talking-agents?p=podcastcta)  Grok vs. ChatGPT: Key Differences, Strengths, and Weaknesses ------------------------------------------------------------  Grok vs. Competitors The RealWorldQA benchmark tests how well question-answering (QA) systems handle real-world questions. It includes everyday, often vague, questions that need complex reasoning. Here’s how Grok-1.5V compares to GPT-4: * **MMMU (Multidisciplinary Reasoning)**: GPT-4 performs slightly better, indicating a stronger ability in diverse subject areas. * **Mathvista (Math)**: Grok-1.5V excels in mathematical reasoning and problem-solving, scoring higher than GPT-4. * **AI2D (Diagrams)**: Grok-1.5V shows superior performance in understanding and interpreting diagrams. * **TextVQA (Text Reading)**: Both models are almost identical in text reading tasks. * **ChartQA (Charts)**: GPT-4V is slightly better at interpreting charts. * **DocVQA (Documents)**: GPT-4V has a higher capability in processing document-based questions. * **RealWorldQA (Real-world Understanding)**: Grok-1.5V excels in real-world spatial reasoning and comprehension. These results show that while GPT-4 has a slight edge in overall multidisciplinary and document tasks, Grok-1.5V is particularly strong in diagram interpretation and real-world understanding, making it highly effective in practical, real-world applications. Grok’s Fun Mode ---------------  Grok has two modes, including a Regular Mode and a Fun Mode. Fun Mode adds humor and personality (and sometimes sarcasm) to the AI chatbot’s answers, making them less boring and more lively than other chatbots like ChatGPT. To use Fun Mode, simply activate it in Grok’s settings. How to Get Started with Grok ----------------------------  Grok is available on X **Grok is available for all of X’s premium subscribers.** Premium subscriptions are paid on a monthly or annual basis on web, iOS, or Android. X’s Pricing in the US is as follows: * Basic: Starts at $3/month or $32/year on web * Premium: Starts at $8/month or $84/year on web * Premium+: Starts at $16/month or $168/year on web How to Access Grok’s API ------------------------ To create an API key for Grok, follow these steps: 1. Open PromptIDE and sign in with your X account.  2. Click your username in the top right corner and select “API Keys.”  3. Click “Create API Key,” set necessary access controls (ACLs), and save.  4. Find your new API key in the list, click the copy button, and use it with the Python SDK.  Create a Custom AI Agent In Less Than 10 Minutes [Join Now—It's Free](https://creator.voiceflow.com/signup?utm_content=grok) [Get started, it’s free](https://creator.voiceflow.com/signup)  ### How to Get Developer Access to Grok? To apply for developer access to Grok’s PromptIDE and API, follow these steps: 1. Go to x.ai and sign in with your X (Twitter) account. 2. After signing in, navigate to your profile settings. 3. Locate and fill out the “Request Access” form within the profile settings. This form will require details about your intended use case and other relevant information. Once your request has been submitted, it will be reviewed. You will receive an email notification regarding the status of your application. Create Your Custom AI Chatbot Today ----------------------------------- Did you know that you can create a custom AI-powered chatbot in less than 10 minutes? **Voiceflow makes it easy to design, prototype, and launch conversational AI chatbots without needing advanced technical (coding) skills. Whether you want to build a chatbot for your personal website or e-commerce shop, we’ve got you covered. Get started today! It’s free.** Frequently Asked Questions -------------------------- ### What future developments can we expect for Grok? Future developments for Grok include enhanced multimodal capabilities, integrating more advanced image and text processing features. We can also anticipate improvements in real-time data processing and interaction efficiency. ### What are the main features of the Grok chatbot? Grok chatbot features multimodal interaction, allowing users to input text and images for sophisticated responses. It also offers real-time access to public posts on X (formerly Twitter), enhancing its conversational relevance. ### Can Grok access real-time information? Yes, Grok can access real-time information, particularly from public posts on X (formerly Twitter). This enables it to provide up-to-date responses based on current events and trends. ### What are the technical specifications of Grok-1? Grok-1 is trained on a diverse range of internet text data up to Q3 2023, with additional datasets curated by human reviewers. [Apply Now](https://creator.voiceflow.com/signup?utm_content=grok) [Create an AI Chatbot Today](https://creator.voiceflow.com/signup?utm_content=grok) Contributor  Content reviewed by Voiceflow  [Voiceflow Team](https://www.voiceflow.com/contributors/voiceflow-team) [https://www.voiceflow.com/](https://www.voiceflow.com/) Create a Custom AI Agent In Less Than 10 Minutes [Join Now—It's Free](https://creator.voiceflow.com/signup?utm_content=grok) [Get started, it’s free](https://creator.voiceflow.com/signup)  This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. Table of Contents [What is Grok?](https://www.voiceflow.com/blog/grok#what-is-grok) [Grok AI’s Features](https://www.voiceflow.com/blog/grok#grok-ais-features) [Grok vs. ChatGPT: Key Differences, Strengths, and Weaknesses](https://www.voiceflow.com/blog/grok#grok-vs-chatgpt-key-differences-strengths-and-weaknesses) [Grok’s Fun Mode](https://www.voiceflow.com/blog/grok#groks-fun-mode) [How to Get Started with Grok](https://www.voiceflow.com/blog/grok#how-to-get-started-with-grok) [How to Access Grok’s API](https://www.voiceflow.com/blog/grok#how-to-access-groks-api) [Create Your Custom AI Chatbot Today](https://www.voiceflow.com/blog/grok#create-your-custom-ai-chatbot-today) [Frequently Asked Questions](https://www.voiceflow.com/blog/grok#frequently-asked-questions) Human-like AI across all channels Use Voiceflow to build human-like agents for chat, voice, and support. Deliver seamless, personalized customer experiences at scale. [Request a demo](https://www.voiceflow.com/demo) Keep Reading ------------ [See all](https://www.voiceflow.com/blog) [AI Platforms\ \ Read Time: \ \ 6\ \ mins\ \ #### Voiceflow: The No.1 AI Chatbot Builder for Conversational Experiences\ \ This article will answer your questions and introduce you to the basics of AI chatbots. You’ll learn how they can benefit your business and discover the best no-code AI chatbo...\ \ All\ \ ](https://www.voiceflow.com/blog/ai-chat) [Chatbots\ \ Read Time: \ \ 6\ \ mins\ \ #### Build AI Chatbot Using The #1 Free Chatbot API \[2025\]\ \ In this article, we’ll guide you through everything you need to know about chatbot APIs and show you how fast it is to get a powerful AI-driven chatbot running on your website...\ \ All\ \ ](https://www.voiceflow.com/blog/chatbot-api) Get the latest AI agent news ---------------------------- Join Voiceflow CEO, Braden Ream, as he explores the future of agentic tech in business on the Humans Talking Agents podcast. Subscribe You're subscribed. Look out for the next episode in your inbox. Oops! Something went wrong while submitting the form.  --- # Build Chat and Voice AI Agents Without Code | Voiceflow [](https://www.voiceflow.com/) Trusted by 4k+ customers building AI agents  4.8/5 rating  200k+ users %20(1).png) %20(1).png)     %20(1).png)  AI Agents that talk, type, and help customers like a human ---------------------------------------------------------- Voiceflow is a purpose-built platform for ambitious Product teams to build AI Agents with speed, control, and observability. %20(1).webp) ### Voice Agents for human-like phone calls Design, test and deploy AI-powered voice agents that are fast, human-like, and scalable. .avif)  ### Chat Agents with deep customization Build Chat Agents with deep capabilities and interface customization. %201%20(1).webp) Use cases Start with customer support, then scale across use cases ---------------------------------------------------------- Confidently start with support and scale quickly.  ### Resolved 70% of support tickets saving $425,000 in 90 days after launch The Trilogy team built a custom AI agent to automate customer support across 90 product lines, automating 70% of Level 1 tickets. [Read case study](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  ### Sanlam shipped an AI financial copilot 3x faster. It converts leads at 80%. Sanlam built custom guardrails in a regulated industry to make financial services more accessible, all while shipping 3x faster. [Continue Reading](https://www.voiceflow.com/blog/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) .avif) .avif)  ### Teams are building AI agents for highly specific voice and chat workflows Learn from other ambitious product teams that are automating interactions across e-commerce, travel, finance, and consumer technology. [Continue Reading](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  Customers Built for teams big and small to build AI Agents together --------------------------------------------------------- Hear it directly from our customers. [Book a demo](https://www.voiceflow.com/demo)  "Voiceflow takes the pain out of building powerful AI Agents. Building Agents is very much a "team sport" and Voiceflow makes it fun to work together and build customer facing use-cases and internal AI applications."  Victor Algaze Software Engineer  “Voiceflow is a very versatile tool, combining visual flows with strong API capabilities and also providing a code editor and functions, for developers.”  Ciprian Nastase L3 Support AI Engineer  "Voiceflow provides us with massive acceleration, enabling us to experiment without fear... Voiceflow allowed us to focus on value-adding activities like orchestration and building a robust, generative conversational architecture."  Andre Fredericks Chief Operating Officer Integrations Integrate Agents with every app in your stack --------------------------------------------- Extend what your agents can do with developer APIs for data, knowledge & interfaces. [See integrations](https://www.voiceflow.com/integrations)   {Analytics} Send Analytics Event to Segment Track agent interactions and send to Segment for analysis. [Go to Function](https://www.voiceflow.com/integrations/segment)  {CRM} Dynamic Carousel using Zendesk Help Center Articles Ship a dynamic carousel with Zendesk Help Center articles. [Get started](https://www.voiceflow.com/integrations/zendesk) .webp) {CRM} Connect with Salesforce Connect your Salesforce to power your agent responses, logic, and analytics. [Get started](https://www.voiceflow.com/integrations/salesforce)  {ecommerce} Integrate with Shopify Plus Connect your AI agent with a Shopify Plus account to power agent responses and actions. [Get started](https://www.voiceflow.com/integrations/shopify)  {Data Warehouse} Send data to Snowflake Connect your Snowflake data warehouse to power your agent responses, logic, and analytics. [Get started](https://www.voiceflow.com/integrations/snowflake) .webp) {Analytics} Analyse performance in Sigma Send agent events to Sigma for visualizing agent performance. [Get started](https://www.voiceflow.com/integrations/sigma) Security Enable every team to build AI Agents, securely ---------------------------------------------- Product teams securely build and deploy advanced AI agents using their approved LLMs alongside customer data and tech stack integrations. [Book a demo](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Control & customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business %20(1).png) %20(1).png)     %20(1).png)  Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)   --- # About Us | Voiceflow [](https://www.voiceflow.com/)  How we started Our story --------- We started building AI agents in 2018 and quickly grew frustrated with the tools to build chatbots. We were stuck choosing between legacy conversational AI platforms, or overly simple customer support chatbot solutions. We believed AI agents could be more than just a means to an end for support automation. AI agents could be a platform for customer interaction at scale the same as websites or mobile apps. We built Voiceflow to provide the best user experience when building advanced AI agents, powered by a flexible developer platform. How it's going Today we serve over 250,000 users and teams ------------------------------------------- Including many of the world's largest companies. .webp) .webp) The right community makes work better. Conversation designers and developers are shaping the future of communication. They do their best work when they can work together. This is what really drives us: connecting people — in the platform, across teams, within organizations, and most of all, throughout our industry. .svg) 2019 Year founded $35m Investment raised 60 Team members 250,000+ Voiceflow users .webp) Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Investors Backed by the best investors in Silicon Valley -------------------------------------------------  Felicis Ventures Aydin Senkut, Founder  Craft Ventures Mike Marg, Partner  Google Google Assistant Fund  True Ventures Tony Conrad, Partner  Ripple Ventures Matt Cohen, Founder  Clark Valberg Founder of Invision  Vlad Magdalin Founder of Webflow  OpenView Partners Blake Bartlett, Partner  Ryan Hoover Founder of Product Hunt  Amazon Amazon Alexa Fund  Kevin Hartz Founder of Eventbrite  Dylan Field Founder of Figma Show More Our mission Empower ambitious teams to build the world's best AI agents. ------------------------------------------------------------ We look for people who embody our core values: .svg) Put customers first, always. Empathy and curiosity help us find — and act on — opportunities to make our customers' work amazing. Make extreme quality the default. We exist to create products that consistently “wow” and deliver business value. Focus to go fast. Find out what matters. Ruthlessly prioritize. Decide with data. Put outcome over effort. Hard work is important but we expect results. We solve root problems, not symptoms. Invest in what matters. Spend less, and invest more in the things we care about — company, community, and technology. Job openings Openings at Voiceflow --------------------- Don't see a role for you? [Reach out anyways](mailto:braden@voiceflow.com) .  No openings available at this time. Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Agent CMS | Voiceflow [](https://www.voiceflow.com/) How does it work The Agent content manager ------------------------- Consolidate large datasets of agent content to easily access, manage, and scale across teams and agents. Centralized integrations Easily group your agent content into folders to handle large datasets effectively with each new use case. Reusable component library Centralize and update agent data from across the business, including Zendesk, sitemaps, and internal docs. NLU model management Create and manage variables, including presets and default values, used across all production agents. Response repository (soon) Classify, train and optimize all Natural Language Understanding (NLU) content to power deterministic flows. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .avif) Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .avif) Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .avif) Features Consolidate large datasets of Agent content across teams -------------------------------------------------------- The Agent CMS ensures that every agent input is organized and accessible. #### LLM Prompts Document polished prompts for reusability and standardization. #### Knowledge Centralize and update agent data from across the business. #### Variables Create and manage presets and default values used across variables in production agents. #### Natural Language Train and optimize all Intents & Entities to power deterministic flows. #### Functions House all developer-built integrations in a central location for easy reusability across agents. #### Events Trigger custom workflows and actions based on third-party events (coming soon). Developer Toolkit Extend agent content with our API library ----------------------------------------- Dynamically query and deploy content from the Agent CMS using our library of APIs. [Go to docs](https://www.voiceflow.com/features/agent-cms#)   Learn more about Voiceflow Explore the platform -------------------- Teams use Voiceflow to collaboratively design, develop, and launch impactful AI agents using any models and integrations. [#### Workflow Builder\ \ Visually create AI agents using logic, custom code, models, and business content.](https://www.voiceflow.com/features/workflows) [#### Knowledge Base\ \ Train AI agents on your data to power responses and perform tasks using any LLM model.](https://www.voiceflow.com/features/knowledge-base-generative-ai) [#### Developer Toolkit\ \ Build custom integrations and extend to any tailored interfaces using a powerful API library.](https://developer.voiceflow.com/docs/get-started) [#### Agent Content Manager\ \ Organize, control and scale all of your agent and team content in a single system.](https://www.voiceflow.com/features/agent-cms) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Pathways | A publication by Voiceflow [\ \ .svg)\ \ Pathways\ ========](https://www.voiceflow.com/pathways/no-impact-to-voiceflow-customers-in-supply-chain-security-incident-sha1-hulud) [Voiceflow news\ \ .svg)](https://www.voiceflow.com/pathways-categories/voiceflow-news) [No impact to Voiceflow customers in supply-chain security incident (sha1-hulud)\ -------------------------------------------------------------------------------](https://www.voiceflow.com/pathways/no-impact-to-voiceflow-customers-in-supply-chain-security-incident-sha1-hulud) [Read the latest article](https://www.voiceflow.com/pathways/no-impact-to-voiceflow-customers-in-supply-chain-security-incident-sha1-hulud) [](https://www.voiceflow.com/pathways/turbocharging-voice-agents) [Voiceflow news](https://www.voiceflow.com/pathways-categories/voiceflow-news) .svg) [### Turbocharging Voice Agents: Speed, Quality, and Powerful New Features](https://www.voiceflow.com/pathways/turbocharging-voice-agents) [](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) [Conversation design](https://www.voiceflow.com/pathways-categories/conversation-design) .svg) [### From decision trees to AI: How Turo built a global, multilingual chatbot in two months](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) [](https://www.voiceflow.com/pathways/introducing-voiceflow-credits) [Voiceflow news](https://www.voiceflow.com/pathways-categories/voiceflow-news) .svg) [### Introducing Voiceflow Credits: a simpler, more transparent billing approach](https://www.voiceflow.com/pathways/introducing-voiceflow-credits) [](https://www.voiceflow.com/pathways/introducing-the-agent-step) [Voiceflow news](https://www.voiceflow.com/pathways-categories/voiceflow-news) .svg) [### Introducing the Agent step in Voiceflow](https://www.voiceflow.com/pathways/introducing-the-agent-step) Our latest webinars and tutorials --------------------------------- No items found. [More videos](https://www.youtube.com/@Voiceflow)   [The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education\ \ The tutorials you need, the resources you deserve. Explore Voiceflow Education](https://www.voiceflow.com/docs) The tutorials you need, the resources you deserve. Explore Voiceflow Education [](https://www.voiceflow.com/pathways/stubhub-internationals-90-day-journey) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### The fast track to AI: StubHub International's 90-day journey from concept to customer service innovation](https://www.voiceflow.com/pathways/stubhub-internationals-90-day-journey) [](https://www.voiceflow.com/pathways/ai-agent-boosted-a-toronto-clinics-monthly-revenue-by-50k) [.svg)](https://www.voiceflow.com/pathways#) [### AI agent boosted a Toronto clinic’s monthly revenue by $50K](https://www.voiceflow.com/pathways/ai-agent-boosted-a-toronto-clinics-monthly-revenue-by-50k) [](https://www.voiceflow.com/pathways/5-ways-to-approach-an-ai-project-strategy) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### 5 ways to approach an AI project strategy](https://www.voiceflow.com/pathways/5-ways-to-approach-an-ai-project-strategy) [](https://www.voiceflow.com/pathways/5-strategies-to-transform-ai-powered-customer-experiences) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### 5 strategies product leaders use to transform AI-powered customer experiences \[Checklist\]](https://www.voiceflow.com/pathways/5-strategies-to-transform-ai-powered-customer-experiences) [](https://www.voiceflow.com/pathways/runtime-performance-improvements) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### Runtime performance improvements](https://www.voiceflow.com/pathways/runtime-performance-improvements) [](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### Money talks: How Sanlam Studios’ AI coach drives leads and financial literacy](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) [](https://www.voiceflow.com/pathways/voiceflow-integrates-with-anthropic) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### Anthropic Claude model family now in Voiceflow](https://www.voiceflow.com/pathways/voiceflow-integrates-with-anthropic) [](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts) [Applied research\ \ .svg)](https://www.voiceflow.com/pathways-categories/research) [### 5 tips to optimize your LLM intent classification prompts](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts) [](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### Building your AI agents like products: A blueprint from POC purgatory to production](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### AI automation on a budget: Getting started with high ROI use cases](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [](https://www.voiceflow.com/pathways/behind-the-hype-how-klarnas-customer-support-agent-falls-short) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### Behind the hype: How Klarna's customer support agent falls short](https://www.voiceflow.com/pathways/behind-the-hype-how-klarnas-customer-support-agent-falls-short) [](https://www.voiceflow.com/pathways/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy) [Industry\ \ .svg)](https://www.voiceflow.com/pathways-categories/industry) [### 15 KPIs for Measuring and Scaling a Generative AI Strategy](https://www.voiceflow.com/pathways/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy) [Load more](https://www.voiceflow.com/pathways?0c55a8b2_page=2) RECOMMENDED RESOURCES No items found. [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Documentation | Voiceflow [](https://www.voiceflow.com/) Documentation Voiceflow Docs ============== Powerful documentation for designers and developers to learn Voiceflow. View the documentation or learn using the resources below. [View Voiceflow Docs](https://docs.voiceflow.com/) [\ \ Video Tutorials\ \ Learn to use Voiceflow with helpful video tutorials.](https://www.youtube.com/c/Voiceflow) [\ \ Templates\ \ Start designing your agent by choosing a pre-built template.](https://www.voiceflow.com/templates) [\ \ Discord Community\ \ Meet and learn from other designers and developers.](https://link.voiceflow.com/community) [\ \ Github\ \ Explore pre-built code examples and repos.](https://github.com/voiceflow)  --- # Demo | Voiceflow [](https://www.voiceflow.com/) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Customer Experience | Voiceflow [](https://www.voiceflow.com/) Customer experience Why teams build AI copilots with Voiceflow -------------------------------------------- Refuse to compromise on control or speed. #### Copilot trained on your data Train an AI agent on your product catalog, developer docs or any knowledge source. #### Automate multi-step tasks Layer business logic to execute complex customer actions, including purchases or onboarding. #### Integrate with any APIs Connect business and user data from any source via API to complete tasks and trigger personalized experiences. #### Maintain copilot control Manage multi-LLM prompting, system prompts, and business logic in a secure cloud environment. Overview How it works ------------ Teams use Voiceflow as a central collaborative platform to build, launch, and scale AI Copilots. Upload data to automate FAQs and questions Import data and documents to the Knowledge Base to allow Agents to respond using curated data. Add workflows to automate complex tasks and questions Build multi-step tasks for Agents to complete using Voiceflow's Workflow Builder. Integrate with data sources and internal systems Integrate Agents with 3rd party services and APIs using Voiceflow Functions and API blocks. Embed copilot in-app Launch Agents with Voiceflow's embeddable web chat UI or any custom interface using the Dialog API. Import data and documents to the Knowledge Base to allow Agents to respond using curated data.  Build multi-step tasks for Agents to complete using Voiceflow's Workflow Builder.  Integrate Agents with 3rd party services and APIs using Voiceflow Functions and API blocks.  Launch Agents with Voiceflow's embeddable web chat UI or any custom interface using the Dialog API. .webp) Case studies AI copilots that do more than just answer questions ----------------------------------------------------- Teams use Voiceflow to design, test, and launch chat or voice AI agents — together, faster, at scale.  ### Trilogy automates 60% of their L1 and L2 support tickets. Their AI agent is powered by a curated knowledge base, integrates with Zendesk, and is embedded in the help centers of 90+ products. [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) ### Roam Auto saves 30 hours per week with Voiceflow. Nick and team went from rigid out-of-the-box chatbot to a customized AI agent reduced total tickets and delivered a better customer experience. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### We reduced ticket costs from $31 to $2. A multi-knowledge base approach paired with a Zendesk integration resolves complex user questions, including code-first API questions from technical builders. [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) .webp) [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Learn more about use cases Automate support, then everything else ---------------------------------------- Teams are automating and improving business critical use cases with AI agents. [#### Customer support\ \ Automate customer support tickets, enable live agents, and solve complex user questions.](https://www.voiceflow.com/solutions/customer-support) [#### In-app copilots\ \ Improve employee onboarding, human resources FAQs, and team activation.](https://www.voiceflow.com/solutions/customer-experience) [#### Conversation design\ \ Accelerate manual revenue collection tasks and protect revenue using custom AI agents.](https://www.voiceflow.com/solutions/conversation-design) [#### Custom AI agents\ \ Guide users through tools, recommend products, or authenticate customers.](https://www.voiceflow.com/docs) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Customer Stories | Voiceflow [](https://www.voiceflow.com/) Customer Stories ================  [\ \ Customer Stories\ \ ### From decision trees to AI: How Turo built a global, multilingual chatbot in two months\ \ This case study explores how a global peer-to-peer car sharing marketplace successfully navigated the transition to AI-powered customer service, offering valuable insights for other organizations considering similar transformations.](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months)  Braden Ream [\ \ Customer Stories\ \ ### The fast track to AI: StubHub International's 90-day journey from concept to customer service innovation\ \ Here's how a small, non-technical team built and deployed an advanced AI agent that exceeded all expectations.](https://www.voiceflow.com/pathways/stubhub-internationals-90-day-journey)  Lena Prickett [\ \ Customer Stories\ \ ### Money talks: How Sanlam Studios’ AI coach drives leads and financial literacy\ \ They’re building an AI financial coach, powered by Voiceflow.](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy)  Sam Burns [\ \ Customer Stories\ \ ### How eSnipe automated 70% of their help center tickets with an AI-powered search agent\ \ The company's more than 2,000,000 customers and 100s of product features meant a substantial number of user questions, resulting in a high ticket volume (average: 9,000 tickets/month).](https://www.voiceflow.com/pathways/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent)  Sam Burns [\ \ Customer Stories\ \ ### Flowcharts to Voiceflow: How WooliesX Reinvented Their Conversation Design Workflow\ \ Flowcharts to Voiceflow: How WooliesX Reinvented Their Conversation Design Workflow](https://www.voiceflow.com/pathways/flowcharts-to-voiceflow-how-wooliesx-reinvented-their-conversation-design-workflow)  Sam Burns [\ \ Customer Stories\ \ ### How Rocket Mortgage Accelerated their Team’s Conversation Design Workflow\ \ How Rocket Mortgage Accelerated their Team’s Conversation Design Workflow](https://www.voiceflow.com/pathways/how-rocket-mortgage-accelerated-their-teams-conversation-design-workflow)  Joanne Szeto [\ \ Customer Stories\ \ ### How a Global Tier 1 Investment Bank Uses Voiceflow to Run More Efficient, Inclusive Conversational AI Design Reviews\ \ Using Voiceflow to Run More Efficient, Inclusive AI Design Reviews](https://www.voiceflow.com/pathways/using-voiceflow-to-run-more-efficient-inclusive-ai-design-reviews)  Sam Burns [\ \ Customer Stories\ \ ### How Trilogy automated 70% of their customer support\ \ Trilogy's journey to 70% AI ticket resolution with Voiceflow.](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  Sam Burns [\ \ Customer Stories\ \ ### Introducing Tico: The AI customer support agent resolving 97% of tickets\ \ How the Voiceflow team reshaped customer support using AI automation.](https://www.voiceflow.com/pathways/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets)  Tahsim Ahmed [\ \ Customer Stories\ \ ### How Roam saved 30 hours a week in customer support hours\ \ Learn how Nick Coatsworth - Head of CS at Roam saved 30 hours a week in customer support hours with an AI agent](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours)  Kim Lu No results found... Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. Submit a creation ----------------- Want to explore how Voiceflow can be a valuable resource for you? Let's talk. [Share to marketplace](https://airtable.com/appCYnIC5ir7nKu8W/shra7TXU4NmWOhKZK) [Developer community](https://link.voiceflow.com/community)  --- # Conversation Design | Voiceflow [](https://www.voiceflow.com/) Conversation design Design complex conversations ---------------------------- Design complex conversations using advanced features built specifically for the design of voice and chat agents. #### Real-time collaboration Collaborate with up to 100 team members (we've tested) in real-time on the same design. #### Advanced logic & components Create and share reusable components and templates for your team to leverage. #### User permissions Secure and manage work across you organization with SSO and advanced user permissions. #### Single source of truth Maintain a true single source of truth, unifying your design files and team workflows in one place. Use cases Learn how these teams are designing conversations across teams -------------------------------------------------------------- Teams use Voiceflow to design, test, and launch chat or voice AI agents — together, faster, at scale.  ### Trilogy automates 60% of their L1 and L2 support tickets. Their AI agent is powered by a curated knowledge base, integrates with Zendesk, and is embedded in the help centers of 90+ products. [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) ### Roam Auto saves 30 hours per week with Voiceflow. Nick and team went from rigid out-of-the-box chatbot to a customized AI agent reduced total tickets and delivered a better customer experience. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### We reduced ticket costs from $31 to $2. A multi-knowledge base approach paired with a Zendesk integration resolves complex user questions, including code-first API questions from technical builders. [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) .webp) [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Learn more about use cases Automate support, then everything else ---------------------------------------- Teams are automating and improving business critical use cases with AI agents. [#### Customer support\ \ Automate customer support tickets, enable live agents, and solve complex user questions.](https://www.voiceflow.com/solutions/customer-support) [#### In-app copilots\ \ Improve employee onboarding, human resources FAQs, and team activation.](https://www.voiceflow.com/solutions/customer-experience) [#### Conversation design\ \ Accelerate manual revenue collection tasks and protect revenue using custom AI agents.](https://www.voiceflow.com/solutions/conversation-design) [#### Custom AI agents\ \ Guide users through tools, recommend products, or authenticate customers.](https://www.voiceflow.com/docs) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Data Processing Addendum | Voiceflow [](https://www.voiceflow.com/) Data Processing Addendum ======================== **_This Data Processing Addendum (“DPA”) is hereby incorporated into and governed by the terms of the_** [**_Terms of Service_**](https://www.voiceflow.com/legal/terms) **_(the “Agreement”) between Voiceflow (the "Company") and its end users (the "Customer") and applies to the extent that Company processes Personal Data (as defined herein) on behalf of Customer in the performance of Services thereunder. _**_ Because the Company’s Agreement already incorporates this DPA, you do not need to sign a separate copy. This DPA (and the applicable Standard Contractual Clauses, as defined below) contain legal terms that apply to personal information that may be contained in Customer input. This DPA supplements the Agreement for the provision of Company’s services as set out in Annex I of Appendix A (the “_**_Company Offering_**_” or “_**_Offering_**_”). In the event of any conflict between the Agreement and this DPA, the terms and conditions of this DPA shall control. Except to the extent expressly superseded or modified in this DPA, the terms and conditions of the Agreement will apply to this DPA and remain in full force and effect. _**_Definitions_** _“_**_Controller to Processor SCC_**_” means the Module Two (transfer controller to processor) of the European Commission Implementing Decision (EU) 2021/914, as applied in accordance with Appendix A “_**_Personal Data_**_” means any information relating to an identified or identifiable individual transferred by Customer or its permitted agents to Company in performance of or pursuant to the Agreement or this DPA, and any information relating to an identified or identifiable derived or otherwise created by Company in connection therewith. ”_**_Process_**_” or “_**_Processing_**_” means any operation or set of operations which is performed on Personal Data or on sets of Personal Data, whether or not by automated means, such as, but not limited to, collection, use, modification, retrieval, disclosure, storage, anonymization, deletion, and/or management. “_**_Processor to Processor SCCs_**_” means the Module Three (transfer processor to processor) of the European Commission Implementing Decision (EU) 2021/914, as applied in accordance with Appendix A. “_**_Privacy Laws_**_” means all applicable laws and regulations governing the Processing or protection of Personal Data as amended, modified or replaced from time to time, including for example and without limitation the Personal Information Protection and Electronic Documents Act, SC 2000 c. 5 (“_**_PIPEDA_**_”), Regulation (EU) 2016/679 (“_**_GDPR_**_”) and Directive 2002/58/EC, the EU GDPR as saved into UK law by virtue of section 3 of the European Union (Withdrawal) Act 2018 (“_**_UK GDPR_**_”), Swiss Federal Data Protection Act of 19 June 1992 and its corresponding ordinances (“_**_Swiss FADP_**_”), and any implementing legislation or further particularising rules, orders or regulations.“ _**_Standard Contractual Clauses_**_” means where the GDPR applies, the standard contractual clauses annexed to the European Commission's Implementing Decision 2021/914 of June 4, 2021 on standard contractual clauses for the Transfer of personal data to third countries pursuant to Regulation (EU) 2016/679 of the European Parliament and of the European Council (available as of June 2021 here: https://eur-lex.europa.eu/eli/dec\_impl/2021/914/oj), (the "_**_EU SCCs_**_"). “_**_Swiss Amendments_**_” mean the Controller to Processor SCCs or the Processor to Processor SCCs (as applicable) with the following amendments: (i) “_**_FDPIC_**_” means the Swiss Federal Data Protection and Information Commissioner, (ii) “_**_Revised FADP_**_” means the revised version of the FADP of 25 September 2020, which is scheduled to come into force on 1 January 2023, (c) the term “_**_EU Member State_**_” must not be interpreted in such a way as to exclude data subjects in Switzerland from the possibility for suing their rights in their place of habitual residence (Switzerland) in accordance with Clause 18 (iii), (iv) the Controller to Processor SCCs also protect the data of legal entities until the entry into force of the Revised FADP, and (v) the FDPIC shall act as the “competent supervisory authority” insofar as the relevant data transfer is governed by the FADP. “_**_UK Addendum_**_” means the template Addendum B.1.0 issued by the UK's Information Commissioner's Office and laid before Parliament in accordance with s119A of the Data Protection Act 2018 of the UK on 2 February 2022, and in force from 21 March 2022, available here:_ [_https://ico.org.uk/media/for-organisations/documents/4019539/international-data-transfer-addendum.pdf_](https://ico.org.uk/media/for-organisations/documents/4019539/international-data-transfer-addendum.pdf) _as updated and/or replaced from time to time. For the purposes of the UK Addendum, (i) the information required for Table 1 is contained in Schedule 1 of this DPA, and the start date shall be the commencement of the Service; (ii) in relation to Table 2, the version of the EU Clauses to which the UK Approved Addendum applies is Module Two for Controller to Processor where Company is acting as your Processor and Module Three for Processor to Processor where Company is acting as your sub-processor; (iii) in relation to Table 3, the list of parties and description of the transfer are as set out in Schedule 1 of this DPA, Company’s technical and organizational measures are set out in Schedule 1 of this DPA, and the list of Company's sub-processors is as provided in Section 8 of this DPA; and (iv) in relation to Table 4, neither party will be entitled to terminate the UK Addendum in accordance with clause 19 of Part 2 of the UK Addendum. _**_Data Processing and Security Responsibilities_**_ Customer and Company shall each comply with all Privacy Laws that apply to it in relation to any Personal Data Processed in connection with this DPA, as set out in Annex I of Appendix A to this DPA.Customer agrees that it has:made and shall maintain all necessary registrations and notifications as required in order to permit Company to perform its obligations and exercise its rights under this DPA;obtained and shall continue to obtain all consents necessary, and provided all necessary notices and otherwise has and continues to have all necessary authority to permit Company to perform its obligations and exercise its rights under this DPA, and shall inform Company immediately if any such consents are withdrawn;ensured and shall continue to ensure that all Personal Data Processed by Company is adequate, relevant, accurate and up-to-date, and limited to what is necessary to permit Company to perform its obligations and exercise its rights under this DPA; ensured and shall continue to ensure that there are valid legal bases to enable Company to Process Customer's Personal Data; Processed and will continue to Process the Personal Data in accordance with all applicable Privacy Laws.In the course of Processing Personal Data on behalf of Customer in connection with the Company Offering as set out in Annex I of Appendix A to this DPA, Company shall:only Process Personal Data for the purposes of rendering the Company Offering and as otherwise instructed by Customer in writing from time to time, and not Process any Personal Data in any other manner without the express prior written authorization of Customer unless required to do so by applicable law;immediately inform the Customer if, in Company’s opinion, any instruction received from the Customer infringes any Privacy Laws; not disclose (and not allow any of its employees, or permitted agents or representatives to disclose) any Personal Data to any third party without the prior written authorization of Customer unless required to do so under applicable law;where any disclosure, transfer or other Processing of Personal Data is required by applicable law, promptly notify Customer in writing before complying with any such requirement (unless prohibited by applicable law, such as on important grounds of public interest); notify Customer in writing of any (i) enquiry received from individuals relating to the individual’s right to access, update, correct, rectify, erase or restrict the processing of Personal Data or to exercise their right of data portability or an objection in accordance with Privacy Laws, (ii) complaint or correspondence received by Company relating to the Processing of Personal Data, and (iii) order, demand, warrant or any other document purporting to compel the production of any Personal Data, and provide reasonable assistance at Customer’s cost to facilitate Customer’s compliance with Customer’s obligations under Privacy Laws; implement appropriate physical, technical, administrative and organizational measures appropriate to the processing of the Personal Data in connection with the Company Offering (as further described in Annex II of Appendix A) as would allow Company to ensure the ongoing confidentiality, integrity, availability and resilience of Processing systems and services and to provide reasonable assistance at Customer's cost to ensure compliance with Customer's obligations to implement such security measures; limit access to Personal Data only to those employees and authorized agents of Company who need to have access to the Personal Data and solely for the purposes of Company rendering the Company Offering; ensure or cause each of the employees and permitted contractors of Company to agree in writing to keep and to protect the confidentiality and security of the Personal Data in accordance with the terms of this DPA, and otherwise properly advise and train each of its employees and permitted subcontractor of the requirements of Company under this DPA and applicable Privacy Law; andprovide reasonable assistance, at Customer’s cost and request, to Customer in connection with Customer’s obligations under Privacy Laws to carry out a data protection impact assessment or to consult with the relevant supervisory authority in respect of any such data protection impact assessment. _**_Standard Contractual Clauses_**_ Company and Customer agree that any transfer of Personal Data from Customer (as “data exporter”) to Company (as “data importer”) requires that appropriate safeguards are put in place in accordance with Privacy laws, the parties will be subject to the Standard Contractual Clauses (as attached hereto as Appendix A), which will be deemed incorporated into and form a part of this DPA. _**_Audit Rights _**_Company shall provide and Customer agrees to accept Company’s most current third-party certifications as may be relevant and available in respect of the Company Offering. Company shall provide Customer (or its representatives) with access to information necessary to demonstrate Company’s compliance with this DPA and to the records, facilities and premises of Company during business hours and upon at least_ _30 days’ advance notice in writing, at most once per year, for the purposes of verifying Company’s compliance with this DPA. _**_Subcontracting _**_Customer acknowledges and agrees that Company shall use sub-processors (including Company affiliates) to provide the Offering as set out in Annex III of Appendix A. Company shall enter into a written contract with each such sub-processor that imposes obligations on the sub-processor that are substantially similar to those imposed on Company under this DPA. Company shall only retain sub-processors that Company can reasonably expect to appropriately protect the privacy, confidentiality and security of the Personal Data. Where such sub-processors fail to fulfil their data protection obligations, Company shall remain fully liable to the Customer for the performance of those sub-processor’s obligations. Prior to appointing any new sub-processor in addition to or in lieu of those listed in Annex III of Appendix A, Company shall notify Customer of such sub-processors, whereupon Customer shall have 10 days to object to such appointment by providing detailed reasons for such objection to Company. _**_Security Breach Notification _**_Company shall notify Customer without undue delay upon Company becoming aware of any accidental or unlawful destruction, loss, alteration, theft, unauthorized access to, use, or disclosure of Personal Data Processed (“_**_Privacy Breach_**_”). Company shall reasonably cooperate with Customer in notifying individuals affected by a Privacy Breach and other parties in accordance with applicable law. _**_Termination _**_Upon the termination of the Agreement or at such other times as instructed by Customer in writing, Company shall either return or, upon the written instruction of Customer, securely dispose of the Personal Data and all existing copies. In the event applicable law does not permit Company to comply with the delivery or destruction of the Personal Data, Company warrants that it shall ensure the confidentiality of the Personal Data in accordance with applicable law. _**_Governing Law and Jurisdiction _**_This DPA and any action related thereto will be governed by and construed in accordance with the laws of the Province of Ontario and the federal laws of Canada applicable therein, without regard to conflicts of law principles. _**_APPENDIX A STANDARD CONTRACTUAL CLAUSES_** _If there is a Restricted Transfer of personal data from Customer (as data exporter) to Company (as data importer), the parties will comply with the following requirements:_**_ If the Restricted Transfer is an EU Restricted Transfer, then the EU SCCs shall apply on the following basis:_**_ where Customer is a controller and Company is also a controller of the personal data transferred, Module One will apply; where Customer is a controller and Company is a processor of the personal data transferred, Module Two will apply; and where Customer is a processor and Company is also a processor of the personal data transferred (i.e. Customer processes the personal data on behalf of a third party controller), Module Three will apply; in Clause 7, the optional docking clause will not apply; for Modules Two and Three only, in Clause 9 (use of sub-processors), option 2 (general written authorisation) will apply, and the Company will: provide a current list of agreed sub-processors; and provide prior notice of any sub-processor changes in accordance with any notice period specified for sub-processor changes in the Agreement or, if no such period is specified, Company will provide seven (7) days' prior notice of any sub-processor changes; in Clause 11, the optional redress language will not apply; in Clause 17, Option 1 will apply (the law of an EU Member State that allows for third-party beneficiary rights). The parties select the laws of Ireland; in Clause 18(b), the parties select the courts of Ireland; in Annex I: Part A shall be completed with the parties names, contact details and activities set out or otherwise described in the Agreement and this Addendum (with Customer acting as the data exporter, and the Company acting as the data importer), and execution of this Addendum shall be deemed execution of the EU SCCs; Part B shall be completed with the relevant information set out in Annex I of this Addendum; and Part C shall be the supervisory authority determined in accordance with the criteria set out in Clause 13(a) of the EU SCCs; and Annex II: shall be deemed completed with the technical and organizational measures described in the Agreement. _**_If the Restricted Transfer is a UK Restricted Transfer, then the EU SCCs and UK Addendum shall apply on the following basis_**_: the EU SCCs, completed as set out above in section 1 above of this Appendix A apply between Customer and Company, and shall be modified by the UK Addendum completed as set out in sub-paragraphs (b) to (d) below; Tables 1 to 3 of the UK Addendum shall be deemed completed with relevant information from the EU SCCs, completed as set out in section 1 above of this Appendix A; Table 4 of the UK Addendum shall be deemed checked "Importer"; and the start date of the UK Addendum (as set out in Table 1) shall be the first date the Customer provides Personal Data to the Company. _**_If the Restricted Transfer is a Swiss Restricted Transfer, then the EU SCCs and the Swiss Amendments shall apply on the following basis:_**_ the EU SCCs, completed as set out section 1 above of this Appendix A apply between Customer and Company, and shall be modified as set out in sub-paragraphs b. to i. below; references to "Regulation (EU) 2016/679" shall be interpreted as references to the Swiss DPA; references to specific Articles of "Regulation (EU) 2016/679" shall be replaced with the equivalent article or section of the Swiss DPA; references to "EU", "Union", "Member State" and "Member State law" shall be replaced with references to "Switzerland" or "Swiss law" (as applicable); the term "member state" shall not be interpreted in such a way as to exclude data subjects in Switzerland from the possibility of suing for their rights in their place of habitual residence (i.e., Switzerland); Clause 13(a) and Part C of Annex I are not used and the "competent supervisory authority" is the Swiss Federal Data Protection and Information Commissioner; references to the "competent supervisory authority" and "competent courts" shall be replaced with references to the "Swiss Federal Data Protection and Information Commissioner" and "applicable courts of Switzerland"; in Clause 17, the EU SCCs shall be governed by the laws of Switzerland; and the EU SCCs also protect the data of legal entities until the entry into force of the revised Swiss Federal Data Protection Act. _**_If the Restricted Transfer is a non-Adequate Country Restricted Transfer, then the EU SCCs shall apply on the following basis_**_: the EU SCCs, completed as set out above in section 1.1 above apply between Customer and Company, and shall apply on a mutatis mutandis basis. In the event that any provision of this Addendum conflicts, directly or indirectly, with the New Standard Contractual Clauses, the New Standard Contractual Clauses shall prevail. For the purposes of this DPA, the “_**_New Standard Contractual Clauses_**_” means, where the GDPR applies, the new or revised standard contractual clauses officially published by the European Commission._ Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Customer Support Automation | Voiceflow [](https://www.voiceflow.com/) Customer support Why teams choose Voiceflow to automate support ------------------------------------------------ Leverage your customer support data across the user journey to automate level 1 FAQs and handle complex questions. #### Native Zendesk integration Manage support tickets, assist live agents, and power your knowledge base with help center articles via Zendesk. #### Automate multi-step tasks Create custom integrations with your existing customer tech stack using our developer toolkit to act on user data. #### Live agent hand-off Use our open source chat widget and custom action step to seamlessly handoff to a live chat platform. #### Maintain AI agent control Curate and manage a custom knowledge base of your customer support documentation to handle complex questions and actions. Overview How it works ------------ Teams use Voiceflow to build and deploy advanced customer support automation across voice and chat channels, Upload support docs to automate FAQs Import data and documents to the Knowledge Base to allow Agents to respond using curated data. Add Workflows to automate complex support tasks Build multi-step tasks for Agents to complete using Voiceflow's Workflow Builder. Integrate with live agent hand-off platforms Integrate Agents with 3rd party services and APIs using Voiceflow Functions and API blocks. Launch via web chat or API Launch Agents with Voiceflow's web chat UI or any custom interface using the Dialog API. Import data and documents to the Knowledge Base to allow Agents to respond using curated data.  Build multi-step tasks for Agents to complete using Voiceflow's Workflow Builder.  Integrate Agents with 3rd party services and APIs using Voiceflow Functions and API blocks.  Launch Agents with Voiceflow's web chat UI or any custom interface using the Dialog API. .webp) Customer support agents Improve resolution rate and containment with AI agents --------------------------------------------------------- Start by automating level 1 FAQs. Scale to complex level 3 queries and actions.  ### Trilogy automates 60% of their L1 and L2 support tickets. Their AI agent is powered by a curated knowledge base, integrates with Zendesk, and is embedded in the help centers of 90+ products. [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) ### Roam Auto saves 30 hours per week with Voiceflow. Nick and team went from rigid out-of-the-box chatbot to a customized AI agent reduced total tickets and delivered a better customer experience. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### We reduced ticket costs from $31 to $2. A multi-knowledge base approach paired with a Zendesk integration resolves complex user questions, including code-first API questions from technical builders. [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) .webp) [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Developer platform Build outside the box with Voiceflow's developer platform ----------------------------------------------------------- Use developer features to extend the platform to add custom features, integrations, interfaces, and more. [Go to docs](https://www.voiceflow.com/docs)   Learn more about use cases Automate support, then everything else ---------------------------------------- Teams are automating and improving business critical use cases with AI agents. [#### Customer support\ \ Automate customer support tickets, enable live agents, and solve complex user questions.](https://www.voiceflow.com/solutions/customer-support) [#### In-app copilots\ \ Improve employee onboarding, human resources FAQs, and team activation.](https://www.voiceflow.com/solutions/customer-experience) [#### Conversation design\ \ Accelerate manual revenue collection tasks and protect revenue using custom AI agents.](https://www.voiceflow.com/solutions/conversation-design) [#### Custom AI agents\ \ Guide users through tools, recommend products, or authenticate customers.](https://www.voiceflow.com/docs) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Functions | Voiceflow [](https://www.voiceflow.com/) Functions Build reusable integrations with Functions ========================================== Connect to your tools in your way with Javascript Functions in Voiceflow. Make your own or choose from our growing library. [.svg)\ \ Watch Video](https://www.voiceflow.com/functions#) [Explore Functions](https://www.voiceflow.com/templates) .svg)  How do Functions work? Built by developers, used by designers -------------------------------------- Functions allow your development team to create a custom integration that can be reused in the design by the designer without touching any code. Define input and output variables Your developer can define what data they need to input into the function and what comes out. Make Fetch Requests Functions supports fetch requests. Meaning you can send and receive data from tools in your techstack. Define Exit Behaviour Use Javascript to define different exit paths, output variables, and even render cards, carousels, and debug messages in Voiceflow. Handoff to Designer Your designer can access your function like a step on the canvas. They don’t need to touch the code. Just map the input and output variables. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Functions supports fetch requests. Meaning you can send and receive data from tools in your techstack.  Use Javascript to define different exit paths, output variables, and even render cards, carousels, and debug messages in Voiceflow.  Your designer can access your function like a step on the canvas. They don’t need to touch the code. Just map the input and output variables.  What can you build? **Create a Function for any use case** -------------------------------------- Integrations, automations, analytics, you name it.  #### Databases Pull customer information from a database like Snowflake  #### Support Teams Create tickets for your support team in Zendesk  #### CRMs Update prospect information in Salesforce  #### E-commerce Create an order in Shopify or your Ecommerce Solution  #### Bring your own LLM Send a question to your own AI model  #### Marketing Add a contact to a new marketing campaign in Hubspot Featured functions Explore our pre-built Functions ------------------------------- [Browse Functions](https://www.voiceflow.com/templates) No items found. Build your own Custom integrations for any use case. ------------------------------------- Use functions to create integrations with any tool in your tech stack. Build once, reuse as many times as you want. [Go to docs](https://developer.voiceflow.com/docs/function-step)   Share your Functions -------------------- Build and submit a Function to have it featured in the community. [Share your Function](https://airtable.com/app2eoFuKeL8jfQGQ/shrQEtmPIP0KNxTcX) [Developer Docs](https://developer.voiceflow.com/docs/function-step)  --- # GDPR Compliance | Voiceflow [](https://www.voiceflow.com/) GDPR Compliance =============== Voiceflow follows all processes and policies for GDPR compliance, and has ISO27001 compliance. Learn more about our Security practices [here](https://www.voiceflow.com/legal/security) .[](https://www.voiceflow.com/legal/gdpr#) [Learn More](https://www.voiceflow.com/legal/gdpr#Learn-more) [Contact sales](https://www.voiceflow.com/demo) Security and compliance trusted by the Fortune 500 across every industry.   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) **General Data Protection Regulation (GDPR) Compliance Statement** ------------------------------------------------------------------ ** Purpose **This GDPR Compliance Statement outlines how Voiceflow collects, processes, and protects personal data in accordance with the General Data Protection Regulation (GDPR). **Contact Information **For any inquiries related to data protection and GDPR compliance, you can reach out to security@voiceflow.com. **Commitment to GDPR **Voiceflow is committed to upholding the principles and requirements outlined in the GDPR. We ensure that personal data is processed lawfully, transparently, and for legitimate purposes. Data subjects' rights, including the rights to access, rectification, erasure, restriction, and data portability, are respected and supported by our data processing practices. **Information Collected and Processed **Voiceflow collects and processes various types of personal data for the purpose of delivering our voice and chat technology solutions effectively. The data we collect may include but is not limited to: Contact information (name and email address), User-generated content, Usage data (interactions with our technology). **Purposes for Processing Personal Data **Voiceflow processes personal data for the following purposes: providing and improving our services, responding to customer inquiries and support requests, sending product updates, promotional materials, and relevant communications, conducting analytical research to enhance our products and services, ensuring compliance with legal obligations. **Rights **As a Data Subject pursuant to the GDPR, you have certain rights. This GDPR Compliance Statement summarizes what these rights under the GDPR involve and how you can exercise these rights. More detail about each right, including exceptions and limitations, can be found in Articles 15-21 and 77 of the GDPR. The Right to Be Informed: You have the right to be informed about how your personal data is being processed. Voiceflow is committed to transparency and will provide you with clear and concise information about the purposes, legal basis, and recipients of your data. The Right of Access: You can request access to the personal data we hold about you. This enables you to verify the lawfulness of processing and ensure the accuracy of your information. The Right to Rectification: If your personal data is inaccurate or incomplete, you have the right to request its rectification. We will promptly update any data that is found to be incorrect. The Right to Erasure: Also known as the "right to be forgotten," you can request the deletion of your personal data. Voiceflow will comply unless there are legal reasons to retain the information. The Right to Restrict Processing: You can request the restriction of processing your personal data. This means we will store the data but not process it further, in certain circumstances. The Right to Data Portability: You have the right to receive the personal data you've provided to us in a structured, commonly used, and machine-readable format. This enables you to transfer it to another data controller. The Right to Object: You can object to the processing of your personal data for specific purposes, including direct marketing. Voiceflow will respect your objection unless there are compelling legitimate grounds for processing. The Right to Avoid Automated Decision-Making:You have the right not to be subject to decisions based solely on automated processing, including profiling, if it significantly affects you. Voiceflow ensures that human intervention is available in such processes. At Voiceflow, we are dedicated to upholding these rights and ensuring that your personal data is treated with the utmost care and respect. If you wish to exercise any of these rights, please contact our Director of Engineering or Head of Legal. **Safeguarding and Security Measures: **Voiceflow takes security seriously, and ensures that it incorporates industry standard security controls in all its products and services, including the following:1. Data Security Governance 2. Physical Access Control 3. Virtual Access Control 4. Data Access Control 5. Disclosure Control 6. Data Entry Control 7. Instructional Control 8. Availability Control 9. Separation Control **Privacy Policy **For detailed information on how Voiceflow collects, processes, stores, and shares personal data, as well as data subjects' rights and choices, please refer to our Privacy Policy. The Privacy Policy can be accessed at the following link: [Voiceflow Privacy Privacy](https://www.voiceflow.com/privacy) . **International Data Transfers **Voiceflow may transfer personal data to countries outside the European Economic Area (EEA) or to third parties located in such countries. In such cases, we ensure that appropriate safeguards, such as standard contractual clauses or security certifications, are in place to protect the data and ensure its secure transfer in compliance with GDPR requirements.Voiceflow is dedicated to safeguarding the privacy and rights of individuals with regard to their personal data. By adhering to the principles of transparency, accountability, and data protection, we strive to maintain the trust and confidence of our users and partners.This GDPR Compliance Statement is subject to updates as necessary to ensure ongoing compliance with changing regulations and standards. Contact ------- If you believe you've discovered a security-related issue or would like to learn more about Voiceflow's security practices, please contact us at [security@voiceflow.com.](mailto:security@voiceflow.com?subject=Contact%20Us%3A%20Security%20at%20Voiceflow) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Privacy | Voiceflow [](https://www.voiceflow.com/) Privacy Policy ============== **Privacy Policy** _Last Updated: 20 December 2023_ At Voiceflow, Inc., (hereinafter, “**Voiceflow**,” “**Company**,” “**us**,” or “**we**”), we respect and protect the privacy of visitors to our website, www.Voiceflow.com, and the other websites under the Voiceflow.com domain (collectively, the “**Sites**”), and our customers who use our SaaS product, web design software, tools, and related services (together with the Sites, the “**Service**”). This Privacy Policy (“**Policy**”) explains how we collect, use, disclose, and protect visitors’ and users’ information as part of the Service. Any discussion of your use of the Service in this Policy is meant to include your visits and other interactions with the Sites and Services, whether or not you are a user of Voiceflow’s SaaS product. By accessing and using our Service, you indicate that you understand, accept, and consent to the terms of this Policy. If you do not agree with or you are not comfortable with any aspect of this Policy, you should immediately discontinue access or use of our Service. Proper access and use of information provided on the website, including User Contributions, is governed by our terms of use found [here](https://www.voiceflow.com/legal/terms) (“**Terms of Service**”). In addition to the below Policy, Voiceflow follows and adheres to the industry practices to be compliant with the GDPR and data protection laws. For more information, check out our GDPR compliance page [here](https://www.voiceflow.com/legal/gdpr) . If applicable and as provided for under Section 10 (Privacy) of our Terms of Service, you may also be subject to Voiceflow’s DPA which can be found [here](https://www.voiceflow.com/legal/dpa) . **1\. What “Personal Information” means **In this Policy, “**Personal Information**” means information or an information set that identifies or could be used to identify an individual directly or indirectly. Except as described in this Policy, Voiceflow will not give, sell, rent, or loan any Personal Information to any third party. Personal Information does not include “**Usage Data**” which we define as encoded or anonymized information or aggregated data we collect about a group or category of services, features, or users which does not contain personally identifying information. Usage Data helps us understand trends in usage of the Service so that we can better consider new features or otherwise tailor the Service. In addition to collecting and using Usage Data ourselves, we may share Usage Data with third parties, including our customers, partners and service providers, for various purposes, including to help us better understand our customers’ needs and improve the Service as well as for advertising and marketing purposes. **2\. How we collect, use, and share your Personal Information and other information you provide to us **When you register for the Service, we may ask for Personal Information such as your name, email address, credit card or other billing information. You may also provide at your own discretion certain related information like your personal website name, social media websites, your employment information, the date you started using the Service, and a description of yourself. Additionally, as you employ the Sites to create voice apps, the software continuously saves changes made by you with the server and will document items such as when you are working and saving changes, when you last opened the Services, when voice apps are published, or when you perform certain tasks. We may also retain the contents of any messages you send to us or through the Service, and we may collect information you provide in user content that you post or upload to the Service (“**User Content**”). Voiceflow may use Personal Information provided to operate, improve upon, and personalize the Service, for billing identification and authentication, to contact and communicate with you about the Service and your use of the Service, to support use of the Service, to send marketing materials for research purposes, and to generally improve the content and functionality of the Service. We may also combine your Personal Information with third-party analytics information to build a broader profile of our individual users so that we can serve you better and provide custom, personalized content and information. Additionally, we may use the Personal Information we collect for identification and authentication and fraud prevention, to analyze Site usage and improve the Service, for internal market research, troubleshooting problems, and as otherwise set forth in this Policy. We will only use your Personal Information in accordance with this Policy unless otherwise required by applicable law. We take steps to ensure that the personal information that we collect about you is adequate, relevant, not excessive, and used for limited purposes. Note that we will never email you to ask for your account information. If you ever receive such an email, please forward it to [support@voiceflow.com](mailto:support@voiceflow.com) . We use financial information solely as authorized by you in accordance with the Service. While Voiceflow does not store your credit card information and will use commercially reasonable efforts to ensure the security of all credit card and all other Personal Information, we expressly disclaim any liability for any unauthorized access to or use of our secure servers and/or any and all personal and/or financial information stored therein, and you agree to hold Voiceflow harmless for any damages that may result therefrom. **b. Information we may receive from and share with third parties **We may now or in the future receive Personal Information about you from third parties. For example, if you access our Site or Service through a third-party connection or log-in or connect an application to Voiceflow, that third party you connected with may pass certain information about your use of its service to Voiceflow. This information could include, but is not limited to, the user ID associated with your account, an access token necessary to access that service, any information that you have permitted the third party to share with us, and any information you have made public in connection with that third-party service. You should always review, and if necessary, adjust your privacy settings on third-party websites and services before linking or connecting them to the Service. Voiceflow may also share Personal Information with its third-party service providers (such as its credit card processors and hosting partners) to provide the necessary hardware, software, networking, storage, and other services we use to operate the Service and maintain a high-quality user experience. We do not permit our service providers to use the Personal Information we share for their marketing purposes. **c. Application integration **To facilitate the exchange of data between third-party SaaS applications, we may need to store certain information (“**App Credentials**”) that helps us access these third-party SaaS application accounts on your behalf. We store your App Credentials in an encrypted form. When we access these third-party applications on your behalf, the third-party application provides us with access to certain data. We will use, store, and disclose this data in accordance with this Policy. You should note that Voiceflow shall have no liability or responsibility for the privacy practices or other actions of any third-party applications for which you provide us with App Credentials. **d. Other information sharing **We may be required to disclose Personal Information to respond to subpoenas, court orders, and law enforcement or governmental requests or investigations, or to establish or exercise our legal rights or defend against legal claims. We may also share Personal Information if we believe it is necessary in order to investigate, prevent, or take action regarding illegal activities, suspected fraud, situations involving potential threats to the physical safety of any person, violations or our Terms of Service, or as otherwise required by law. We use or may use information collected by cookies, log files, device identifiers, and clear GIFs information to: (a) remember information so that you will not have to re-enter it during your visit or the next time you visit the site, (b) provide custom, personalized content and information, (c) to provide and monitor the effectiveness of our Service, (d) monitor aggregate metrics such as total number of visitors, traffic, and demographic patterns, (e) diagnose or fix technology problems, (f) help you efficiently access your information after you sign in, (g) to provide advertising to your browser or device, and (h) automatically update the Voiceflow Service on your mobile devices. We may share Personal Information with any member of the Voiceflow team, which includes any entity that controls, is controlled by, or under common control with Voiceflow. **Use of Data from Google APIs.** Voiceflow’s use and transfer to any other app of information received from Google APIs will adhere to the [Google API Services User Data Policy](https://developers.google.com/workspace/workspace-api-user-data-developer-policy) , including the [Limited Use](https://developers.google.com/workspace/workspace-api-user-data-developer-policy#limited-use) requirements. **3\. Links to other websites **The Service contains links to websites and applications other than the Service, including websites and applications operated by affiliates and other third parties. This Policy applies only to information collected by the Service. Voiceflow does not endorse and is not responsible for the practices of third parties or their websites or applications. We do not determine and are not responsible for the privacy practices or the content of websites or applications operated by third parties. Your browsing and interaction on any third-party website or service, including those that have a link on our website, are subject to that third party’s own rules and policies. We are not responsible for and we do not control any third parties that you authorize to access your User Content. If you are using a third-party website or service and you allow such a third-party access to your User Content, you do so at your own risk. **4\. Public forums **Our Service offers publicly accessible blogs and community forums. If you elect to post something in a public area of our Service, any Personal Information or content that you voluntarily disclose for posting to the Service, becomes available to the public, as controlled by any applicable privacy settings. If you remove information that you posted to the Service, copies may remain viewable in cached and archived pages of the Service, or if other users have copied or saved that information. To request removal of your Personal Information from one of our blogs or community forums, contact us at [support@voiceflow.com](mailto:support@voiceflow.com) . In some cases, we may not be able to remove all your Personal Information. **5\. Customer testimonials / comments / reviews ** From time to time, we post customer testimonials on the Sites which may contain Personal Information. We always obtain the customers’ consent prior to posting their names along with their testimonials. **6\. Protection of information ** Voiceflow is committed to ensuring the security of your Personal Information. We use commercially reasonable technological, physical, and administrative security safeguards, such as firewalls and carefully developed security features, to protect the confidentiality and security of your Personal Information within the Service and the Sites. When you enter confidential information (such as login credentials or information submitted from within the Service) we encrypt the transmission of that information using secure socket layer technology (SSL). These technologies, procedures, and other measures are used in an effort to ensure that your data is safe, secure, and only available to you and to those you authorized to access your data. However, no internet, email, or other electronic transmission is ever fully secure or error-free, so you should take care in deciding what information you send to us in this way. Voiceflow is not responsible for the functionality or security measures of any third party. **7\. Hosting and data transfer ** We are based in Canada, but, unless we expressly agree otherwise, we may host, transfer, and process data, including Personal Information, in Canada and in other countries through Voiceflow and third parties that we use to operate and manage the Service. These countries may have data protection laws that are different from those of your country of residence. Voiceflow takes appropriate measures to ensure such transfers are in compliance with applicable laws. **8\. Customer Warranty** You represent and warrant to, and covenant with Voiceflow that your data will only contain personal information in respect of which you have provided all notices and disclosures, obtained all applicable third party consents and permissions and otherwise has all authority, in each case as required by applicable laws, to enable Voiceflow to provide its services, including with respect to the collection, storage, access, use, disclosure, processing and transmission of personal information, including by or to Voiceflow and to or from all applicable third parties. **9\. Choice ** We process and store information on behalf of our customers. You may decline to submit any Personal Information through the Service, in which case we may not be able to provide certain services to you. Please refer to your mobile device or browser’s technical information for instructions on how to delete and disable cookies, and other tracking/recording tools. Note that disabling cookies on your mobile device or browser may prevent us or our business partners from tracking your browser’s activities in relation to the Service. However, doing so may disable many of the features available through the Service. **10\. Correcting and updating your information ** You may update, delete, or change your Personal Information you have provided Voiceflow by logging in to the Service and providing such additional information or deleting such information where applicable. If you are not our customer and would like to gain access to, or request deletion of, information we have collected, please contact us at support@voiceflow.com. We will use commercially reasonable efforts to respond to such queries within a reasonable time. Voiceflow has no direct relationship with the individuals or companies (“**End Users**”) with whom you may interact using the Service. Any such End Users seeking access to, or who would like to correct, amend, or delete data which may be stored in the Service should direct their query to the applicable Voiceflow customer acting as the data controller for such information. **11\. Right to Withdraw Consent** Where you have provided your consent to the collection, use, and transfer of your personal information, you may withdraw consent at any time, subject to legal or contractual restrictions and reasonable notice. To withdraw your consent, if applicable, contact us at support@voiceflow.com. **12\. Data retention ** Voiceflow will retain Personal Information we process on behalf of our customers or collect directly from our customers for as long as needed to provide Service to our customers, subject to our compliance with this Policy, or as required or permitted under the applicable law. We may further retain and use this Personal Information as necessary to comply with our legal obligations; maintain accurate accounting, financial, and other operational records; resolve disputes; and enforce our agreements. We have established internal policies for the deletion of data from customer accounts following termination of a customer’s subscription to the Service. **13\. Children's Personal Information ** Voiceflow does not knowingly collect any Personal Information from children under the age of 13. If you are under the age of 13, please do not submit any Personal Information through our Service. We encourage parents and legal guardians to monitor their children’s internet usage and to help enforce our Policy by instructing their children never to provide Personal Information on our Service without their permission. If you have reason to believe that a child under the age of 13 has provided Personal Information to Voiceflow through our Service, please contact us, and we will use commercially reasonable efforts to delete that information from our databases. **14\. Changes to the Privacy Policy ** We may, in our sole discretion, modify or update this Policy from time to time. If we make any material changes to this Policy, we will endeavor to notify you by email or by posting a prominent notice on the Sites prior to the change becoming effective. We encourage you to periodically review this page for the latest information on our privacy practices. Your continued use of the Service constitutes your acceptance to such changes to this Policy. If you do not accept the terms of this Policy, you should immediately discontinue use of the Service. **15\. Business transactions ** Voiceflow may assign or transfer this Policy, and your user account and related Personal Information, to any person or entity that acquires all or substantially all of the business, stock or assets of, or is merged with Voiceflow. We may also transfer or assign such information in the course of corporate divestitures, mergers, acquisitions, bankruptcies, dissolutions, or similar transactions or proceedings, provided that the recipient of such information will continue to be bound by the terms of this Privacy Policy. **16\. Contact us ** If you have questions regarding this Policy or about the privacy practices of Voiceflow, please contact us by email at [support@voiceflow.com](mailto:support@voiceflow.com) . Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Voiceflow Marketplace | Collections [](https://www.voiceflow.com/) AllCustomer supportE-commerceLead generationTask automationAgencies [Top Resources for AI Agencies\ \ It’s no secret that the release of tools like ChatGPT has challenged how we see the role of AI across every industry. AI is growing at a rate that no other technology has before, with new models, platforms, and research being published daily. To keep up with the pace of change, AI agencies formed, helping businesses—big and small—carve out their stake in AI.](https://www.voiceflow.com/collections/top-resources-for-agencies) All Agencies [Top 7 Coolest Custom Interface Projects Built on Voiceflow (For Developers)\ \ The true power of Voiceflow is in our ability to let developers integrate to any platform, execute custom actions, and deploy to any interface—with speed. This is made possible using our APIs, custom webchat, built-in code steps, functions, and extensions. If you’re a developer curious to build an AI agent on a custom interface, here’s some art-of-possible inspiration for you.](https://www.voiceflow.com/collections/top-custom-interface-projects) All Task automation [Top 6 Customer Support Templates and Tutorials\ \ AI agents are revolutionizing the way the businesses interact with customers, offering 24/7 service, ability to handle multiple queries, and now even complete custom tasks. We’ve compiled templates and tutorials to help you go from zero to hero with your support AI agent. Stick to the end to checkout key case studies of teams who’ve successfully automated their customer support function with Voiceflow.](https://www.voiceflow.com/collections/customer-support-template-and-tutorial) All Customer support [Top 6 E-commerce Templates and Tutorials\ \ In today's AI wave, having a quality chatbot is the baseline for all businesses. It's crucial to enable 24/7 customer interaction and greatly improve your efficiency. What if I told you that your chatbot can do so much more? With these templates and tutorials level up your chatbot so it can manage product recommendations, collect customer data, track orders, and handle customer queries.](https://www.voiceflow.com/collections/top-ecomm-templates-and-tutorials) All E-commerce No results found... [](https://www.voiceflow.com/collections#) Have something to share? ------------------------ Share your creation with over 250,000 global Voiceflow users. [Share resource](https://airtable.com/appCYnIC5ir7nKu8W/shra7TXU4NmWOhKZK) [Join the Voiceflow Discord](https://link.voiceflow.com/community)  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. --- # Build advanced AI agents faster than Langchain [](https://www.voiceflow.com/) Build advanced AI agents faster than Langchain ---------------------------------------------- Customizable pre-built agent functionality on a visual canvas for you to build your own solutions with custom code. Connect to any interface with our APIs to save time and headache deploying AI agents. [Get started, it's free](https://creator.voiceflow.com/signup)  Create a Vector DB in minutes Instantly create a vector database to inform your AI agent. Simply upload your files into Voiceflow’s Knowledge Base and skip the hassle of using Pinecone. Pre-built front end for your AI agent Get your agent to market faster with Voiceflow’s out-of-the-box and open source front end solution. No more tedious custom-built front end so you can accelerate AI product team sprints. Eliminate LLM hallucinations Combine RAG with prompt-chaining techniques or mix LLMs to eliminate the risk of AI agent hallucination. Easily customize and reuse components across agents. [Try Template](https://creator.voiceflow.com/dashboard?import=6646bda54382243e6f082e3d) Connect to databases with ease Retrieve data from any database to integrate directly into your prompts. Perform custom actions using Voiceflow’s JavaScript Functions, creating a more engaging, effective agent. [Try Template](https://creator.voiceflow.com/dashboard?import=6646bdab9e2b4495737fbd4c) Learn more about Voiceflow Explore the platform -------------------- Teams use Voiceflow to collaboratively design, develop, and launch impactful AI agents using any models and integrations. [#### Developer Tutorials\ \ Get inspired with a curated playlist of developer tutorials with repositories included.](https://youtube.com/playlist?list=PLKYemGIohRgBo44Xr8RvgVUVQMW-uhjZl&feature=shared) [#### Developer Docs\ \ Build custom integrations and extend to any tailored interfaces using a powerful API library.](https://developer.voiceflow.com/docs/get-started) [#### Templates\ \ Start off of what others have made before expanding into your own use case.](https://www.voiceflow.com/marketplace-old) [#### Functions\ \ Build reusable integrations within Voiceflow that can be created using JavaScript and API calls.](https://www.voiceflow.com/functions) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Take your Zendesk chatbot to the next level [](https://www.voiceflow.com/) The Zendesk AI solution you're looking for ------------------------------------------ Create an AI agent with Voiceflow that connects to Zendesk and automates 60% customer requests, or more. Let your support team focus on the complex questions - AI can handle the rest. [Get started, it's free](https://creator.voiceflow.com/signup)  **Connect your Zendesk Help Docs instantly** Upload your Zendesk support documents in a few clicks and let AI handle your customer questions. **Pre-built flows with your team** Collaborate with your team to create pre-built flows to help customers debug known issues step by step. **Use any AI model in your flows** Optimize your AI agent with different AI models tailored to the problems you’re helping a customer solve. **Create tickets and connect to databases with ease** Retrieve data from any customer database using our API step or Javascript function that you can build into your flows. **Automate your customer support** ---------------------------------- “12 weeks since launching my Voiceflow Zendesk agent, it solved 40% of total chats, saved 97 hours, which translated to $710 of cost savings.”  ##### Bart Slodyczka Head of CS @ Joolca “Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint. Velit officia consequat duis enim velit mollit. Exercitation veniam consequat.”  ##### Author Name VP of Company “Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint. Velit officia consequat duis enim velit mollit. Exercitation veniam consequat.”  ##### Author Name VP of Company Start building AI Agents ------------------------ Sign up to receive Zendesk AI agent templates. [Get started, it’s free](https://creator.voiceflow.com/signup)  --- # Internal Operations | Voiceflow [](https://www.voiceflow.com/) Internal Automation Everything you need to build AI Agents for internal operations -------------------------------------------------------------- Teams use Voiceflow to design, test, and launch chat or voice AI agents — together, faster, at scale. #### Security & compliance SOC-2, ISO/IEC 27001 and GDPR compliance ensure that teams can trust agents to handle sensitive employee & company data. #### Tech stack integrations Create integrations via API with human resources and project management tools in your organization's tech stack. #### Slack & internal communication Embed agents in employee workflows, including Slack, HR platforms, or Microsoft Teams, via API. #### Knowledge base Curate and manage a custom knowledge base of internal documentation and databases. Use cases Learn how these teams shipped tailored AI agents for internal automation ------------------------------------------------------------------------ Teams use Voiceflow to design, test, and launch chat or voice AI agents — together, faster, at scale.  ### Trilogy automates 60% of their L1 and L2 support tickets. Their AI agent is powered by a curated knowledge base, integrates with Zendesk, and is embedded in the help centers of 90+ products. [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) ### Roam Auto saves 30 hours per week with Voiceflow. Nick and team went from rigid out-of-the-box chatbot to a customized AI agent reduced total tickets and delivered a better customer experience. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### We reduced ticket costs from $31 to $2. A multi-knowledge base approach paired with a Zendesk integration resolves complex user questions, including code-first API questions from technical builders. [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) .webp) [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Learn more about use cases Automate support, then everything else ---------------------------------------- Teams are automating and improving business critical use cases with AI agents. [#### Customer support\ \ Automate customer support tickets, enable live agents, and solve complex user questions.](https://www.voiceflow.com/solutions/customer-support) [#### In-app copilots\ \ Improve employee onboarding, human resources FAQs, and team activation.](https://www.voiceflow.com/solutions/customer-experience) [#### Conversation design\ \ Accelerate manual revenue collection tasks and protect revenue using custom AI agents.](https://www.voiceflow.com/solutions/conversation-design) [#### Custom AI agents\ \ Guide users through tools, recommend products, or authenticate customers.](https://www.voiceflow.com/docs) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Knowledge Base & Generative AI | Voiceflow [](https://www.voiceflow.com/) How does it work How the Knowledge Base works ---------------------------- Train your AI agent on business data to handle complex questions, perform advanced tasks, and tailor content to specific personas. Import data Power your agent with data from across your business, including Zendesk, sitemaps, and internal docs. Test responses Preview knowledge base responses in real-time as you build. Adjust temperature & system prompts. Tune model inputs Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. Manage and launch Organize and scale your agent's knowledge and data sources using the Knowledge Base CMS. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .webp) Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .webp) Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Features Resolve complex questions with a finely tuned knowledge base. ------------------------------------------------------------- Pair your business data with any LLM. #### Scalable vector database Scale your knowledge base to thousands of data sources with a scalable vector database. #### Multi-LLM support Power knowledge base responses with the latest LLM models. #### Tweak LLM settings Tweak your knowledge base responses by changing the LLM model, temperature, and more. #### Search and manage Manage large amounts of data sources with a knowledge base CMS #### Use data to perform actions Give your agent the ability to ask the knowledge base for data to perform advanced actions. #### Instant previews Preview and tweak your knowledge base answers instantly with 1:1 prototypes. Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Knowledge Base API Build on top of the Knowledge Base API -------------------------------------- Use the Knowledge Base API to query chunks, tag source information and serve responses on any custom interface. [Go to docs](https://developer.voiceflow.com/reference/post_knowledge-base-query)   Learn more about Voiceflow Explore the platform -------------------- Teams use Voiceflow to collaboratively design, develop, and launch impactful AI agents using any models and integrations. [#### Workflow Builder\ \ Visually create AI agents using logic, custom code, models, and business content.](https://www.voiceflow.com/features/workflows) [#### Knowledge Base\ \ Train AI agents on your data to power responses and perform tasks using any LLM model.](https://www.voiceflow.com/features/knowledge-base-generative-ai) [#### Developer Toolkit\ \ Build custom integrations and extend to any tailored interfaces using a powerful API library.](https://developer.voiceflow.com/docs/get-started) [#### Agent Content Manager\ \ Organize, control and scale all of your agent and team content in a single system.](https://www.voiceflow.com/features/agent-cms) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Voiceflow Community [](https://www.voiceflow.com/) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Platform Overview | Voiceflow [](https://www.voiceflow.com/) Overview How Voiceflow works ------------------- Teams use Voiceflow as a central collaborative platform to build, launch, and scale AI Agents for any use case. 1\. Build a Knowledge Base Import data and documents to the Knowledge Base to allow Agents to respond using curated data. 2\. Add Workflows Build multi-step tasks for Agents to complete using Voiceflow's Workflow Builder. 3\. Integrate Integrate Agents with 3rd party services and APIs using Voiceflow Functions and API blocks. 4\. Launch via web chat or API Launch Agents with Voiceflow's web chat UI or any custom interface using the Dialog API. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .webp) Voiceflow Platform The platform that meets your agent-building ambition. ------------------------------------------------------- Build, customize, ship and improve agents - at scale. #### Scalable knowledge base Build and segment a vector database with thousands of data sources. #### Multi-LLM support Use any AI vendor, including OpenAI and Anthropic, to power agent responses, logic, and actions. #### User control and collaboration Secure and manage work across you organization with SSO and advanced user permissions. #### Content management Curate agent content, components, and logic that scale across teams and agent use cases. #### Build in code Build custom logic, actions, and integrations with the full power of code, including API and JSON steps. #### High-fidelity prototyping Test agent builds 1:1 in real-time and share with stakeholders for feedback & approval. Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Developer platform Build outside the box with Voiceflow's developer platform ----------------------------------------------------------- Use developer features to extend the platform to add custom features, integrations, interfaces, and more. [Go to docs](https://www.voiceflow.com/docs)   Learn more about Voiceflow Explore the platform -------------------- Teams use Voiceflow to collaboratively design, develop, and launch impactful AI agents using any models and integrations. [#### Workflow Builder\ \ Visually create AI agents using logic, custom code, models, and business content.](https://www.voiceflow.com/features/workflows) [#### Knowledge Base\ \ Train AI agents on your data to power responses and perform tasks using any LLM model.](https://www.voiceflow.com/features/knowledge-base-generative-ai) [#### Developer Toolkit\ \ Build custom integrations and extend to any tailored interfaces using a powerful API library.](https://developer.voiceflow.com/docs/get-started) [#### Agent Content Manager\ \ Organize, control and scale all of your agent and team content in a single system.](https://www.voiceflow.com/features/agent-cms) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Affiliate Terms [](https://www.voiceflow.com/) Voiceflow, Inc. Master Affiliate Marketing Agreement ------------------------------------ Build AI Agents for customer support and beyondReady to explore how Voiceflow can help your team? Let’s talk.Sign up free --- # Hire an expert [](https://www.voiceflow.com/) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Terms | Voiceflow [](https://www.voiceflow.com/) Terms of Service Agreement ========================== __**1\. Introduction** _Last Updated: 20 December 2023 _Voiceflow, Inc., (“**Voiceflow**,” “**Company**,” “**us**,” or “**we**”), provides and the other websites under the Voiceflow.com domain (collectively, the “**Sites**”), and our SaaS product, web design software, tools, and related services (together with the Sites, the “**Service**”) subject to your compliance with all the terms, conditions, and notices contained or referenced herein (the “**Terms of Service**” or the “**Agreemen**t”). This Agreement explains our obligations to you, and your obligations to us. Except as noted otherwise below, this Agreement is the entire Agreement between us. By using the Sites and the Service in any way, you are agreeing to comply with these Terms of Service, our privacy policy found [here](https://www.voiceflow.com/privacy) (“**Privacy Policy**”), and any other legal notices or conditions or guidelines posted on the Sites. If you have questions regarding this Agreement or about Voiceflow, please contact us by email at [support@voiceflow.com](mailto:support@voiceflow.com) . **2\. Acknowledgement and acceptance of terms ** In addition, when using the Service, you shall be subject to any posted guidelines or rules applicable to the Service that may contain terms and conditions in addition to those in the Terms of Service. By using the Service or accessing the Sites, you agree to the Terms of Service, the Privacy Policy, which is incorporated herein by reference. If you do not agree, do not use the Service. Our Service may integrate with other services on a number of platforms provided by third parties. Please be aware that your activities on each of these platforms may be subject to additional terms of service for that particular platform. Certain parts of the Service may be subject to additional terms and conditions specified by us from time to time; your use of such Service is subject to those additional terms and conditions, which are incorporated into these Terms of Service by this reference. Prices of the Service are subject to change upon 30 days’ notice from us. Such notice may be provided at any time by posting the changes to the Voiceflow Sites or the Service itself. Voiceflow shall not be liable to you or to any third party for any modification, price change, suspension, or discontinuance of the Service. ** 3\. Eligibility ** The Service is available only for individuals aged 13 years or older. If you are age 13 or older but under the age of 18, or the legal age of majority where you reside if that jurisdiction has an older age of majority, then you agree to review these Terms of Service with your parent or guardian to make sure that both you and your parent or guardian understand and agree to these Terms of Service. You agree to have your parent or guardian review and accept these Terms of Service on your behalf. If you are a parent or guardian agreeing to these Terms of Service for the benefit of a child over the age of 13, then you agree to and accept full responsibility for that child’s use of the Service, including all financial charges and legal liability that he or she may incur. We may, in our sole discretion, refuse to offer the Service to any person or entity and change the eligibility criteria for using the Service at any time. The right to access the Service is revoked where these Terms of Service or use of the Service is prohibited or to the extent offering, sale, or provision of the Service conflicts with any applicable law, rule or regulation. If you are entering into these Terms of Service on behalf of a company or other legal entity, you represent that you have the authority to bind such entity, its affiliates, and all users who access the Service through your account to these Terms of Service, in which case the terms “**you**” or “**your**” shall refer to such entity, its affiliates, and users associated with it. If you do not have such authority, or if you do not agree with these Terms of Service, you must not accept these Terms of Service and you may not use the Service. You further agree that you assume all responsibility and liability in connection with your use of the Service on behalf of such a company or other legal entity, and you shall be solely responsible for all disputes, if any, that arise due to your use of the Service on behalf of such a company or other legal entity. ** 4\. Account information Registering for an account.** To use the Service, you will need to register and create an account (including your full name, password, and email address). If you upgrade your account, you will need to provide payment information as well. Voiceflow accounts will give you access to the Service and functionality that we may establish and maintain from time to time and at our sole discretion. We may maintain different types of accounts for different types of users. If you open a Voiceflow account on behalf of a company, organization, or other entity, then (a) “**you**” includes you and that entity, and (b) you represent and warrant that you are an authorized representative of the entity with the authority to bind the entity to this Agreement, and that you agree to this Agreement on the entity’s behalf. By connecting to Voiceflow with a third-party service, you give us permission to access and use your information from that service as permitted by that service, and to store your login credentials for that service. For more information on the types of information we collect from these third-party services, please read our Privacy Policy. We reserve the right to reclaim usernames on behalf of businesses or individuals that hold legal claim or trademark on those usernames. **Your responsibility for your account.** You are responsible for taking all steps to ensure that no unauthorized person shall have access to your account or password. When creating your account, you must provide accurate and complete information. We encourage you to use “strong” passwords (passwords that use a combination of upper- and lowercase letters, numbers, and symbols) with your account. It is your sole responsibility to: (i) control the dissemination and use of your account and password, and (ii) promptly inform us of any need to deactivate an account or password. We are not liable for any harm related to the theft of passwords, or your disclosure of passwords. You shall immediately notify us of any unauthorized use of your account or password or any breach of confidentiality. You are solely responsible for any damage resulting from the use of your account and username. You may not use your account or password to breach security of another account. **How to control your account.** You may control your account and how you interact with the Service by changing your settings. By providing us with your email address, you consent to our using the email address to send you Service-related notices, including any notices required by law, in lieu of communication by postal mail. We may also use your email address to send you other messages, such as changes to features of the Service. You may not opt out of these messages unless you cancel your account. For more information, please read our Privacy Policy. **5\. What we own ** Except for User Content (as defined below), all materials on the Service and the Service itself and all Intellectual Property Rights (as defined below) contained therein or related thereto, including text, graphics, user and visual interfaces, photographs, trademarks, logos, sounds, music, artwork, applications, computer code and associated documentation (collectively, the “**Content**”), including but not limited to the design, structure, arrangement, and “look and feel” of such Content, is owned by or licensed to us and our licensors and is protected by copyright, trademark, and other Intellectual Property Rights and laws. Other than as provided herein, Voiceflow does not grant to users any license, express or implied, to the Intellectual Property Rights of Voiceflow or its licensors. Except as expressly provided in the Terms of Service, no part of the Service and no Content may be copied, reproduced, sold, republished, transmitted, displayed, reposted, or otherwise distributed for public or commercial purposes. Voiceflow or its licensors retain all rights, title and interest including all Intellectual Property Rights in and to: (i) the Services; (ii) anything used, developed or delivered by or on behalf of Voiceflow under this Agreement; and (iii) any modifications to the foregoing (i) and (ii). Subject to the terms and conditions of this Agreement, Voiceflow provides you with a non-exclusive, revocable license to use the Service, as expressly permitted by the features and functionality of the Service and the Terms of Service. Voiceflow may terminate this license at any time for any reason or no reason. For the purposes of this Agreement, “**Intellectual Property Rights**” means all patent rights, copyright rights, moral rights, rights of publicity, trademark, trade dress and service mark rights, goodwill, trade secret rights and other intellectual property rights as may now exist or hereafter come into existence, and all applications therefore and registrations, renewals, and extensions thereof, under the laws of any state, country, territory, or other jurisdiction. **6\. User Content**We may now or in the future allow you to post or publish content on or upload content to the Service, including but not limited to voice app templates or functionality created and contributed by you. Any content a user submits, posts, displays, or otherwise makes available on the Service, including all Intellectual Property Rights (defined below) therein, is referred to as “**User Content**.” You retain ownership of your User Content. We have no obligation to store, maintain, or provide you a copy of your User Content. You alone are responsible for any of your User Content that may be lost or unrecoverable through your use of the Service. Additionally, you may choose to or we may invite you to submit comments, ideas, or feedback about the Service, including without limitation about how to improve the Service or our products (“**Feedback**”). By submitting any Feedback, you agree that your disclosure is gratuitous, unsolicited, and without restriction and will not place Voiceflow under any fiduciary or other obligation, and that we are free to use the Feedback without any additional compensation to you, and/or to disclose the Feedback on a non-confidential basis or otherwise to anyone. You further acknowledge that, by acceptance of your submission, Voiceflow does not waive any rights to use similar or related Feedback previously known to Voiceflow, or developed by its employees, or obtained from sources other than you. Additionally, you acknowledge that all email and other correspondence that you submit to us shall become our sole and exclusive property. You own all of the User Content you post or publish (“**post**”) on the Service. You grant us a non-exclusive, royalty-free, perpetual, transferable, sub-licensable, worldwide license to use, modify, reproduce, display, and distribute your User Content on the Service for the purposes of operating and providing the Service to you and other users (“**User Content Licens**e”). You grant to Voiceflow a nonexclusive, worldwide, royalty-free, irrevocable,sublicensable, and fully paid-up right during the Term to access, collect, use, process, store,disclose and transmit your data to: (i) provide the Services; (ii) improve and enhance the Services and its other offerings; and (iii) produce data, information or other materials that are notidentified as relating to a particular individual or company (such data, information and materials,the “**Aggregated Data**”). Voiceflow may use, process, store, disclose and transmit the Aggregated Data for any purpose and without restriction or obligation to you of any kind. **7\. Rules ** **Your obligations to end users: ** **Roles** Voiceflow is a “service provider” to you, and you are a “customer” of our Services irrespective of whether you pay fees to obtain the Services. Any natural persons accessing or using your Voiceflow created voice apps and other software created by you on our Services are deemed your “end users.” Voiceflow does not have a direct relationship with any of your end users, therefore Voiceflow is not responsible for how you handle end user information. **Applicable laws** You may have additional obligations under local law other than those described in this Terms of Service. Such obligations may be more restrictive than this Terms of Service. Use of the Services does not ensure compliance with such laws, nor is Voiceflow responsible for your compliance with such laws. **Security safeguards** You will use all reasonable efforts to protect information collected from end users via your use of the Services, including any personal information obtained from unauthorized access or use. In the event you discover that the end user information collected by you has been breached, compromised, or inadvertently exposed to non-authorized third parties, you shall notify Voiceflow promptly of such a breach or exposure including the root cause, remediation steps, and compensating controls to ensure such a breach does not occur in the future. **Security notification **You are responsible for providing notification to your end users, third parties, or authorities under the applicable breach notification statutes and any other applicable data protection laws. You acknowledge that you are solely responsible for any personal injury or property damage arising from or relating to your use of any Services. **Privacy **You acknowledge and agree that you are solely responsible for providing any required notices and obtaining all required consents from end users in connection with any use of the Services. Your notice and consent must be compliant with all applicable data protection and security laws and regulations. Without limiting the foregoing, before collecting or using any end user information, you must provide adequate notice of the end user information you collect, how it will be used and/or shared, and obtain any necessary consents required under the applicable data protection laws and regulations. You agree to comply with all data protection laws and regulations (including those applying to personal information) in connection with your access and use of the Services. **Notice **You will provide and adhere to a privacy policy for your use of the Services that: (i) complies with all applicable laws, rules, and regulations, (ii) is conspicuously displayed to all end users, and (iii) clearly and accurately describes to end users what information you collect (such as login information, contact form data, cookies, etc.) and how you use and share such information (including for marketing and advertising) with Voiceflow and other third parties. **Privacy rights **You will respond to any end users wishing to exercise their privacy rights, under the applicable law, as it relates to any information collected via your use of the Services. For example, if end users located in certain jurisdictions may exercise a “right to be forgotten” (or “erasure”) to the information you have collected about them, you shall comply with their request as required by applicable law. For clarity, you are directly responsible (or, the “data controller”) for the information processed by your use of the Services, including any information processed via third-party applications available on the Services. Voiceflow is not a data controller to such information and is not responsible for responding to end users on your behalf. To the extent Voiceflow receives a request from an end user or authority about your use of the Services, Voiceflow will notify you as permitted by law. **User content** **Customer Warranty**You represent, warrant and covenant that any User Content that you post does not and will not violate any law or infringe any rights of any third party, including without limitation any Intellectual Property Rights, publicity rights, or rights of privacy. If you think any User Content infringes your rights, or if you want to challenge User Content that has been taken down from the site, please refer to Voiceflow’s Copyright Policy in Section 13 below. We reserve the right, but are not obligated, to reject and/or remove any User Content that we believe, in our sole discretion, violates the Agreement. **Restricted Input Content**You agree that you shall not upload, transmit, distribute, post, or otherwise publish any User Content that: may create a risk of harm, loss, damage, physical or mental injury, emotional distress, death, disability, disfigurement, or physical or mental illness to you, to any other person, to any animal or to any property; seeks to harm or exploit children by exposing them to inappropriate content (including child sexual exploitation and abuse); asking for personally identifiable details or otherwise may constitute or contribute to a crime or tort; contains any information or content that we deem to be unlawful, harmful, abusive, racially or ethnically offensive, defamatory, infringing, harassing, humiliating to other people (publicly or otherwise); libellous, threatening, profane, or otherwise objectionable contain software or other material that violates or invades the Intellectual Property Rights (or rights of privacy or publicity) of any third party; includes any information or content to which you have not obtained any necessary rights or permissions to use accordingly or to make available under any law or under contractual or fiduciary relationships; contains any information or content that is illegal (including, without limitation, the disclosure of insider information under securities law or of another party’s trade secrets, the sale of counterfeit products, or the sale of opioids);contains any information or content that you know is not correct and current, or is adult in nature, such as any nudity in a sexual context, exposed genitalia, or any content with adult themes;involves commercial activities and/or sales without the Company’s prior written consent, such as contests, sweepstakes, barter, advertising or pyramid schemes; Any violation of the foregoing may result in restrictions on your access to the Services and may be referred to law enforcement authorities. Voiceflow reserves the right to disclose information as necessary to satisfy any law, regulation or government request. Voiceflow and its affiliates shall have no liability for any action or inaction by them in respect of any conduct relating to materials submitted or posted by users. **Use of the Service** Your use of the Service is subject to all applicable laws and regulations. You shall not: use any “deep-link”, “page-scrape”, “robot”, “spider” or other automatic device, program, algorithm or methodology, or any manual process to access, acquire, copy, or monitor any portion of the Service or any Content or obtain or attempt to obtain any materials, documents or information through any means not purposely made available through the Service;reverse engineer, decompile, disassemble, or otherwise attempt to discover the source code or underlying ideas or algorithms of the Services;copy, modify, or create derivative works of the Services;access or use the Services for the purpose of building a similar or competitive product or service;sell, resell, license, rent, lease, distribute, or time-share the Services;attempt to gain unauthorized access to any portion of the Service or any Content, or any systems or networks connected to the Service, by hacking, password “mining”, or any other illegitimate means probe, scan, or test the vulnerability of the Service or any Content, or any system or network connected to the Service; reverse look-up or trace any information of any other user or visitor or otherwise use the Service for the purpose of obtaining information of any other user or visitor transmit spam, chain letters, or other unsolicited email take any action that imposes, or may impose at our sole discretion an unreasonable or disproportionately large load on our infrastructure upload invalid data, viruses, worms, or other software agents through the Service impersonate another person or otherwise misrepresent your affiliation with a person or entity, conduct fraud, hide, or attempt to hide your identity disclose any information that you do not have the consent to disclose (such as confidential information of others, including their personally identifiable information);use any device, software, or process to interfere with, or attempt to interfere with, the proper working of the Service or any Content, or any systems or networks connected to the Service, or with any other person’s use of the Service conduct, engage in or otherwise process end user information or data to analyze individuals or groups of natural persons for unlawful or discriminatory purposes; use the Service or any Content for any purpose that is unlawful or prohibited by the Terms of Service, or store or collect any personally identifiable information that is deemed sensitive or requires special protections under applicable laws. For example, Social Security numbers, passwords, and credit card information. Any unauthorized use of the Service or any Content may violate copyright, trademark, and other applicable laws and could result in criminal or civil penalties. **Use of software** Any software that is accessed via the Service shall be governed by the terms of the license agreement that accompanies the software or is posted with the software on the Sites where the software can be accessed; however, if no such license agreement is made available, we grant you a limited, revocable, worldwide, royalty-free, non-assignable, non-exclusive license to use the software for your personal, non-commercial use in accordance with the Terms of Service. In the event of any inconsistency between the Terms of Service and any license agreement, the license agreement shall govern in relation to the software. The software may automatically update from time to time. These updates are designed to improve, enhance, and further develop the software and may take the form of bug fixes, enhanced functions, new software modules, and completely new versions. You agree to receive such updates (and permit us to deliver these to you with or without your knowledge) as part of your use of the software. **8\. Third-party content ** **User Content** We are not responsible for and do not necessarily hold the opinions expressed by our users, content contributors, and third parties; such opinions and other statements are theirs alone, not opinions of Voiceflow. Content created by users or third parties is the sole responsibility of such users or third parties and its accuracy and completeness are not endorsed or guaranteed. You acknowledge that by providing you with the ability to view and post content through our Service, Voiceflow is not undertaking any obligation or liability relating to the content. Voiceflow and its affiliates, successors, assigns, employees, agents, directors, officers, and stockholders (i) do not undertake or assume any duty to monitor our site for inappropriate or unlawful content and (ii) assume no responsibility or liability which may arise from the content thereof, including, but not limited to, claims for defamation, libel, slander, infringement, invasion of privacy and publicity rights, obscenity, pornography, profanity, fraud, or misrepresentation. Notwithstanding the foregoing, Voiceflow reserves the right to block or remove communications, postings, or materials at any time in our sole discretion. Voiceflow takes no responsibility and assumes no liability for any User Content that you or any other user or third party posts or sends over the Service. Voiceflow assumes no liability for your interactions with other users, or for any user’s action or inaction. You are solely responsible for your User Content and the consequences of posting or publishing it, and you agree that we are only acting as a passive conduit for your distribution and publication of your User Content. You understand and agree that you may be exposed to User Content that is inaccurate, objectionable, inappropriate for children, or otherwise unsuited to your purpose, and you agree that Voiceflow shall not be liable for any damages you allege to incur as a result of User Content. Voiceflow reserves the right, but has no obligation, to monitor disputes between you and other users. **Links** We may provide links to third-party websites. We do not recommend or endorse the content of any third-party website. We are not responsible for the content of linked third-party websites and we do not make any representations or warranties regarding their content or accuracy. Your use of third-party websites is at your own risk and subject to the terms and conditions of use for such websites. You expressly release us from any and all liability arising from your use of any third-party website. **9\. Billing and payment ** Users of the Service may be required to provide their credit card details to Company or the payment service provider retained by Company (the “**PSP**”). Users will be responsible for paying agreed-upon fees for the Service. Users of the Service will be liable for any taxes required to be paid on the Service provided under the Agreement (other than taxes on the Company’s income). All paid plans must enter a valid payment account. Free accounts are not required to provide payment account information. An upgrade from the free plan to any paying plan will immediately bill you. All payment plans, without limitation, whether on a monthly or yearly basis, are paid in advance and are non-refundable. There will be no refunds or credits for partial months or years of service, upgrade/downgrade refunds, or refunds for months unused with an open account. All fees are exclusive of all taxes, levies, or duties imposed by taxing authorities, and you shall be responsible for payment of all such taxes, levies, or duties. **10\. Privacy ** Our Privacy Policy applies to the use of the Service and its terms are made a part of the Terms of Service by this reference. You understand that by using the Service you consent to the collection, use, and disclosure of your Personal Information and aggregate data as set forth in our Privacy Policy, and to have your Personal Information collected, used, transferred to, and processed. We cannot guarantee that unauthorized third parties will never be able to defeat our security measures. You acknowledge that you provide your information at your own risk. Our Terms of Service incorporate the Voiceflow Data Processing Addendum (“**DPA**”), when using Voiceflow Services to process personal information or Customer Data (as defined in the DPA), if applicable. By agreeing to these Terms of Service, you hereby agree to the DPA. When you agree to the Voiceflow Terms of Services, the DPA, as found [here](https://www.voiceflow.com/legal/dpa) , shall apply to any processing of personal information or Customer Data. **11\. Disclaimer of warranties and limitation of liability ** **Disclaimer** THE SERVICE AND ALL CONTENT ARE PROVIDED ON AN "AS IS" AND “AS AVAILABLE” BASIS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY, COMPLETENESS, CURRENTNESS, FREEDOM FROM INTERRUPTION, VIRUSES OR OTHER DEFECT, AND NON-INFRINGEMENT.YOUR SOLE REMEDY AGAINST US FOR DISSATISFACTION WITH THE SERVICE OR ANY CONTENT IS TO STOP USING THE SERVICE OR ANY SUCH CONTENT. IN NO EVENT SHALL WE OR OUR SUBSIDIARIES, OFFICERS, DIRECTORS, EMPLOYEES, AGENTS, LICENSORS, OR CONTENT PROVIDERS (OUR “**AFFILIATES**”) BE LIABLE FOR ANY DAMAGES, INCLUDING, WITHOUT LIMITATION, INDIRECT, CONSEQUENTIAL, SPECIAL, INCIDENTAL, OR PUNITIVE DAMAGES, RESULTING FROM THE USE OR INABILITY TO USE THE SERVICE OR THE CONTENT, WHETHER BASED ON WARRANTY, CONTRACT, TORT, OR ANY OTHER LEGAL THEORY, AND WHETHER OR NOT WE OR OUR AFFILIATES ARE ADVISED ON THE POSSIBILITY OF SUCH DAMAGES.TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, VOICEFLOW ASSUMES NO LIABILITY OR RESPONSIBILITY FOR ANY (I) ERRORS, MISTAKES, OR INACCURACIES OF CONTENT; (II) PERSONAL INJURY OR PROPERTY DAMAGE, OF ANY NATURE WHATSOEVER, RESULTING FROM YOUR ACCESS TO OR USE OF OUR SERVICE; (III) ANY UNAUTHORIZED ACCESS TO OR USE OF OUR SECURE SERVERS AND/OR ANY AND ALL INFORMATION, INCLUDING PERSONAL INFORMATION OR FINANCIAL INFORMATION, STORED THEREIN; (IV) ANY ERRORS, MISTAKES, LOSSES, DAMAGE OR UNAUTHORIZED ACCESS RESULTING FROM THE USE OF THIRD PARTY APPLICATIONS BY YOU, BY YOUR END USERS, OR BY VOICEFLOW ON YOUR BEHALF; (V) ANY INTERRUPTION OR CESSATION OF TRANSMISSION TO OR FROM THE SERVICE; (VI) ANY BUGS, VIRUSES, TROJAN HORSES, OR THE LIKE THAT MAY BE TRANSMITTED TO OR THROUGH OUR SERVICE BY ANY THIRD PARTY; (VII) ANY ERRORS OR OMISSIONS IN ANY CONTENT OR FOR ANY LOSS OR DAMAGE INCURRED AS A RESULT OF THE USE OF ANY CONTENT POSTED, EMAILED, TRANSMITTED, OR OTHERWISE MADE AVAILABLE THROUGH THE SERVICE; AND/OR (VIII) USER CONTENT OR THE DEFAMATORY, OFFENSIVE, OR ILLEGAL CONDUCT OF ANY THIRD PARTY. IN NO EVENT SHALL VOICEFLOW, ITS AFFILIATES, AGENTS, DIRECTORS, EMPLOYEES, SUPPLIERS, OR LICENSORS BE LIABLE TO YOU FOR ANY CLAIMS, PROCEEDINGS, LIABILITIES, OBLIGATIONS, DAMAGES, LOSSES, OR COSTS IN AN AMOUNT EXCEEDING ONE DOLLAR. THIS LIMITATION OF LIABILITY SECTION APPLIES WHETHER THE ALLEGED LIABILITY IS BASED ON CONTRACT, TORT, NEGLIGENCE, STRICT LIABILITY, OR ANY OTHER BASIS, EVEN IF VOICEFLOW HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE FOREGOING LIMITATION OF LIABILITY SHALL APPLY TO THE FULLEST EXTENT PERMITTED BY LAW IN THE APPLICABLE JURISDICTION.SOME STATES DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES OR THE EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THE ABOVE LIMITATIONS OR EXCLUSIONS MAY NOT APPLY TO YOU. THIS AGREEMENT GIVES YOU SPECIFIC LEGAL RIGHTS, AND YOU MAY ALSO HAVE OTHER RIGHTS WHICH VARY FROM STATE TO STATE. THE DISCLAIMERS, EXCLUSIONS, AND LIMITATIONS OF LIABILITY UNDER THIS AGREEMENT WILL NOT APPLY TO THE EXTENT PROHIBITED BY APPLICABLE LAW. **Artificial Intelligence Disclaimer**Voiceflow’s artificial intelligence technology and features, if enabled, automatically generates content (via OpenAI’s GPT or Anthropic’s Claude language model) and communicates such content to you and anyone who uses your bot (your “end user(s)”) upon specific events and interactions with your end users (i) on Voiceflow’s platform (without limitation, internal testing and prototyping) or (ii) in a production environment (without limitation, your organization’s, product’s, or website’s chatbot or conversation channel with customers) (the “**AI-Enabled Technology**”). You may not use AI-Enabled Technology or output (“**AI Content**”): (i) to develop foundation models or other large-scale models that compete with Voiceflow or the AI-Enabled Technology; (ii) to mislead any person that output from the Service was solely human-generated; (iii) in a manner that violates any technical documentation, usage guidelines, or parameters; (iv) to make automated decisions that may have a detrimental impact on individual rights without appropriate human supervision; or (v) in a manner that infringes, violates, or misappropriates any of our rights or the rights of any third party. You are solely responsible for your AI Content, including compliance with applicable laws and these Terms. Other users providing similar input to the Company’s AI-enabled Technology may receive the same or similar output. Responses received by other users are not your AI Content. Any reliance on the content generated by AI-Enabled Technology is solely at your own risk. The AI-Enabled Technology and all content provided by it are provided on an “as-is” and “as available” basis. Voiceflow does not give any warranties, whether express or implied, as to (i) the suitability or usability of the AI-Enabled technology or any of its content; or (ii) the accuracy, adequacy, validity, reliability, availability or completeness of any content generated by the AI-Enabled Technology. NOTWITHSTANDING ANYTHING TO THE CONTRARY, VOICEFLOW SHALL BEAR NO LIABILITY WHATSOEVER FOR (i) USE OF THE AI-ENABLED TECHNOLOGY; (ii) ITS CONTENT; AND (iii) ANY AND ALL INACCURACIES, ERRORS, OMISSIONS AND PROBLEMS OF THE CONTENT GENERATED BY THE AI-ENABLED TECHNOLOGY. **Contacting Emergency Services**Services cannot be utilized for contacting emergency services or obtaining immediate emergency assistance. In the event of an emergency, it is crucial to immediately contact the appropriate emergency services in your jurisdiction or seek direct assistance from local authorities. The Services are not to be relied upon as any means of alerting emergency responders or obtaining timely help during life-threatening situations. You acknowledge and agree that Voiceflow shall not be held liable for any consequences, damages, or losses arising from the inability to use the Services for emergency communication or for any reliance placed on the Services during emergency situations. **12\. Changes to the Service, termination, and indemnity ** Voiceflow is constantly innovating and finding ways to provide our users with new features and services. Therefore, we may, without prior notice, change the Service, add features, stop providing the Service or features of the Service to you or to users generally, or create usage limits for the Service. We may permanently or temporarily terminate, downgrade, suspend, or prohibit your access to the Service without prior notice and liability for any reason, including if, in our sole determination, you violate any provision of this Agreement, or for no reason. Upon termination of your use of the Service, you continue to be bound by this Agreement. You agree to defend, indemnify, and hold us and our Affiliates harmless from and against any demands, loss, liability, claims, or expenses (including attorneys’ fees) made against us by any third party due to or arising out of your User Content, use of the Service, violation of these Terms of Service, or violation of any rights of another. **13\. Copyright policy ** Voiceflow respects the intellectual property of others, and expects users to do the same. If you believe, in good faith, that any materials provided on or in connection with the Service infringe upon your copyright or other Intellectual Property Right, please send the following information to Voiceflow at: Voiceflow, Inc., 30 Duncan Street, Lower Level North Suite, Toronto, ON, Canada, M5V 2C3, or [support@voiceflow.com](mailto:support@voiceflow.com) . A description of the copyrighted work or other intellectual property that you claim has been infringed, including the voice app store URL (internet address) or other specific location on the Service where the material you claim is infringed is located. Include enough information to allow Voiceflow to locate the material, and explain why you think an infringement has taken place. A description of the location where the original or an authorized copy of the copyrighted work exists — for example, the URL (internet address) where it is posted or the name of the book in which it has been published Your address, telephone number, and email address A statement by you that you have a good faith belief that the disputed use is not authorized by the copyright or intellectual property owner, its agent, or the law A statement by you, made under penalty of perjury, that the information in your notice is accurate, and that you are the copyright owner or authorized to act on the copyright owner's behalf, and An electronic or physical signature of the owner of the copyright or the person authorized to act on behalf of the owner of the copyright interest ** Counter-notice** If you believe that the content that was removed (or to which access was disabled) is not infringing, or that you have the authorization from the copyright owner, the copyright owner's agent, or pursuant to the law, to post and use the content, you may send a written counter-notice containing the following information to the Copyright Agent: Your physical or electronic signature Identification of the content that has been removed or to which access has been disabled and the location at which the content appeared before it was removed or disabled A statement that you have a good faith belief that the content was removed or disabled as a result of mistake or a misidentification of the content, and Your name, address, telephone number, and email address If a counter-notice is received by the Copyright Agent, Voiceflow will send a copy of the counter-notice to the original complaining party informing that person that it may replace the removed content or cease disabling it in 10 business days. Unless the copyright owner files an action seeking a court order against the content provider, member, or user, the removed content may be replaced, or access to it restored, in 10 to 14 business days or more after receipt of the counter-notice, at our sole discretion. ** Repeat-infringer policy** In accordance with applicable laws, Voiceflow has adopted a policy of terminating, without notice, in appropriate circumstances and at Voiceflow’s sole discretion, the accounts or memberships of users who are deemed to be repeat infringers. Voiceflow may also at its sole discretion, without notice, limit access to the Service and/or terminate the memberships of any users who infringe any Intellectual Property Rights of others, whether or not there is any repeat infringement. ** 14\. Amendments, notification procedures, and changes to the Agreement ** We reserve the right, in our sole discretion, to modify, update, or change the Terms of Service ("**Updated Terms**") from time to time, so you should review this page periodically. When we change the Agreement in a material manner, we will update the “last modified” date at the top of this page. Your continued use of the Service after any such change constitutes your acceptance of the new Terms of Service. If you do not agree to any of these terms or any future Terms of Service, do not use or access (or continue to access) the Service. Voiceflow is not responsible for any automatic filtering you or your network provider may apply to email notifications we send to the email address you provide us. Unless we make a change for legal or administrative reasons, we will provide reasonable advance notice before the Updated Terms become effective. Voiceflow may provide such notifications, whether the notifications are required by law or are for marketing or other business-related purposes, to you via email notice, written or hard copy notice, or through posting of such notice on our website, as determined by Voiceflow in our sole discretion. Voiceflow reserves the right to determine the form and means of providing notifications to our users, provided that you may opt out of certain means of notification by contacting us at [support@voiceflow.com](mailto:support@voiceflow.com) . You agree that we may notify you of the Updated Terms by posting them on the Service, and that your use of the Service after the effective date of the Updated Terms (or engaging in such other conduct as we may reasonably specify) constitutes your agreement to the Updated Terms. Therefore, you should review these Terms of Service and any Updated Terms before using the Service. The Updated Terms will be effective as of the time of posting, or such later date as may be specified in the Updated Terms, and will apply to your use of the Service from that point forward. The Terms of Service will govern any disputes arising before the effective date of the Updated Terms. **15\. Dispute resolution ** INFORMAL NEGOTIATIONS. To expedite resolution and reduce the cost of any dispute, controversy or claim related to this Agreement (“**Dispute**”), you and Voiceflow agree to first attempt to negotiate any Dispute (except those Disputes expressly excluded below) informally for at least thirty (30) days before initiating any arbitration or court proceeding. Such informal negotiations will commence upon written notice. Your address for such notices is your billing address, with an email copy to the email address you have provided to Voiceflow. Voiceflow’s address for such notices is [legal@voiceflow.com](mailto:legal@voiceflow.com) , Attention: Legal Voiceflow. WAIVER OF RIGHT TO BE A PLAINTIFF OR CLASS MEMBER IN A PURPORTED CLASS ACTION OR REPRESENTATIVE PROCEEDING. You and Voiceflow agree that any arbitration will be limited to the Dispute between Voiceflow and you individually. YOU ACKNOWLEDGE AND AGREE THAT YOU AND Voiceflow ARE EACH WAIVING THE RIGHT TO PARTICIPATE AS A PLAINTIFF OR CLASS MEMBER IN ANY PURPORTED CLASS ACTION OR REPRESENTATIVE PROCEEDING. Further, unless both you and Voiceflow otherwise agree, the arbitrator may not consolidate more than one person’s claims, and may not otherwise preside over any form of any class or representative proceeding. If this specific paragraph is held unenforceable, then the entirety of this “**Dispute Resolution**” Section will be deemed null and void. **16\. Miscellaneous ** **Assignment** This Agreement, and any rights and licenses granted hereunder, may not be transferred or assigned by you, but may be assigned by Voiceflow without restriction. Any attempted transfer or assignment in violation hereof shall be null and void. **Governing law, international use, and users** The Terms of Service will be governed by and construed in accordance with the internal laws of the Province of Ontario. The Service is controlled and operated from its facilities in Canada. Although the Service may be accessible worldwide, we make no representation that the Service or the materials on the Service are appropriate or available for use in locations outside Canada, and accessing the Service from territories where its use is illegal is prohibited. Those who access or use the Service from other jurisdictions do so at their own volition and are entirely responsible for compliance with all applicable Canadian and local laws and regulations, including but not limited to export and import regulations. You may not use the Service if you are a resident of a country embargoed by Canada, or are a foreign person or entity blocked or denied by the Canadian government. **Entire agreement/severabilitym**The Terms of Service represent the entire agreement between you and us with respect to the subject matter addressed herein and supersedes all prior to contemporaneous agreements or understandings, written or oral. The Terms of Service shall be binding upon and inure to the benefit of the parties hereto and their respective successors and lawful assigns. **No waiver** Any failure by us to enforce or exercise any provisions of the Terms of Service shall not constitute a waiver of that right or provision. Our failure to act with respect to a breach by you or others does not waive our rights to act with respect to subsequent or similar breaches. **Attorneys’ and accountants’ fees, remedies **In any action to enforce the Terms of Service, the prevailing party shall be entitled to attorneys’ and accountants’ fees and costs. You agree that Voiceflow's remedy at law for any actual or threatened breach of this Agreement would be inadequate and that Voiceflow shall be entitled to specific performance or injunctive relief, or both, in addition to any damages that Voiceflow may be legally entitled to recover, together with reasonable expenses of any form of dispute resolution, including, without limitation, attorneys' fees. No right or remedy of Voiceflow shall be exclusive of any other, whether at law or in equity, including without limitation damages injunctive relief, attorneys' fees, and expenses.No instance of waiver by Voiceflow of its rights or remedies under these terms and conditions shall imply any obligation to grant any similar, future, or other waiver. Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Revenue Protection | Voiceflow [](https://www.voiceflow.com/) Revenue Protection Build efficient AI agents for revenue protection ------------------------------------------------ Teams use Voiceflow to automate collections and billing workflows. #### Data security SOC-2, ISO/IEC 27001 and GDPR compliance ensure that teams can trust agents to handle sensitive billing and revenue data. #### Billing platform integrations Create custom integrations via API with Stripe, Salesforce or other tools in your billing tech stack. #### Voice or chat interfaces Ship AI Agents for voice or chat interfaces using the web chat widget or Dialog API. #### Knowledge base Curate and mange a custom knowledge base of your billing support documentation to handle complex questions and actions. Use cases Learn how these teams shipped tailored AI agents to impact revenue ------------------------------------------------------------------ Teams use Voiceflow to design, test, and launch chat or voice AI agents — together, faster, at scale.  ### Trilogy automates 60% of their L1 and L2 support tickets. Their AI agent is powered by a curated knowledge base, integrates with Zendesk, and is embedded in the help centers of 90+ products. [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) ### Roam Auto saves 30 hours per week with Voiceflow. Nick and team went from rigid out-of-the-box chatbot to a customized AI agent reduced total tickets and delivered a better customer experience. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### We reduced ticket costs from $31 to $2. A multi-knowledge base approach paired with a Zendesk integration resolves complex user questions, including code-first API questions from technical builders. [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) .webp) [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Learn more about use cases Automate support, then everything else ---------------------------------------- Teams are automating and improving business critical use cases with AI agents. [#### Customer support\ \ Automate customer support tickets, enable live agents, and solve complex user questions.](https://www.voiceflow.com/solutions/customer-support) [#### In-app copilots\ \ Improve employee onboarding, human resources FAQs, and team activation.](https://www.voiceflow.com/solutions/customer-experience) [#### Conversation design\ \ Accelerate manual revenue collection tasks and protect revenue using custom AI agents.](https://www.voiceflow.com/solutions/conversation-design) [#### Custom AI agents\ \ Guide users through tools, recommend products, or authenticate customers.](https://www.voiceflow.com/docs) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Automate contact centers with Voice AI | Voiceflow [](https://www.voiceflow.com/) Build AI Agents with control ---------------------------- Sign up and build AI Agents for free. [Get started, it’s free](https://creator.voiceflow.com/signup) [Contact sales](https://www.voiceflow.com/demo)  --- # Tico Demo [](https://www.voiceflow.com/) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Workflows | Voiceflow [](https://www.voiceflow.com/) Overview Using the Workflow Builder -------------------------- Voiceflow's Workflow Builder is a collaborative drag-and-drop builder tool to create conversational logic for AI Agents. Build faster without code The Workflow Builder features an infinite no-code drag-and-drop building canvas to create Agents of scale. Collaborate in one place Centralize your team's workflow in one platform that brings together designers, developers, and other stakeholders. Test and share prototypes Test and share Agents in real-time across the build lifecycle, from first prototype to production launch. Create AI powered logic Create both hardcoded and AI powered logic to retain control whilst giving your Agent AI superpowers. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. .webp) Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Features Industry leading features to build incredible Agents ------------------------------------------------------ Teams use Voiceflow to design, test, and launch chat or voice AI agents — together, faster, at scale. #### Scalable Vectorbase Scale your knowledge base to thousands of data sources with a scalable vector database. #### Build with NLUs & LLMs Power knowledge base responses with the latest large language models. #### Tweak LLM settings Tweak your knowledge base responses by changing the LLM model, temperature, and more. #### Search and manage Manage large amounts of data sources with a knowledge base CMS. #### Use data to perform actions Give your agent the ability to ask the knowledge base for data to perform advanced actions. #### Instant previews Preview and tweak your knowledge base answers instantly with 1:1 prototypes. Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Workflows Extend Voiceflow with flexible APIs ----------------------------------- Use our library of APIs to build custom integrations with your tech stack, interfaces and data sources. [Go to docs](https://www.voiceflow.com/docs)   Learn more about Voiceflow Explore the platform -------------------- Teams use Voiceflow to collaboratively design, develop, and launch impactful AI agents using any models and integrations. [#### Workflow Builder\ \ Visually create AI agents using logic, custom code, models, and business content.](https://www.voiceflow.com/features/workflows) [#### Knowledge Base\ \ Train AI agents on your data to power responses and perform tasks using any LLM model.](https://www.voiceflow.com/features/knowledge-base-generative-ai) [#### Developer Toolkit\ \ Build custom integrations and extend to any tailored interfaces using a powerful API library.](https://developer.voiceflow.com/docs/get-started) [#### Agent Content Manager\ \ Organize, control and scale all of your agent and team content in a single system.](https://www.voiceflow.com/features/agent-cms) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # Blog | Voiceflow [](https://www.voiceflow.com/) Blog ==== Everything you need to know about conversational AI, at your fingertips. [AI Basics](https://www.voiceflow.com/blog-category/ai-basics) [AI Implementation](https://www.voiceflow.com/blog-category/ai-implementation) [AI Platforms](https://www.voiceflow.com/blog-category/ai-platforms) [Automation](https://www.voiceflow.com/blog-category/automation) [Business & Agencies](https://www.voiceflow.com/blog-category/business-agencies) [Chatbots](https://www.voiceflow.com/blog-category/chatbots) [Conversational Design](https://www.voiceflow.com/blog-category/conversational-design) [Sales & Marketing](https://www.voiceflow.com/blog-category/sales-marketing) [Vertical AI](https://www.voiceflow.com/blog-category/vertical-ai) [Voice AI](https://www.voiceflow.com/blog-category/voice-ai) Thank you! Your submission has been received! Oops! Something went wrong while submitting the form.  [Chatbots\ \ November 12, 2025\ \ October 30, 2025\ \ #### How to Build a Chatbot with Zapier (Step by Step)\ \ This step-by-step guide shows you how to build a Zapier AI chatbot using Voiceflow. Learn to design a conversation that captures leads and use Zapier to automatically send that data to Google Sheets, your CRM, or Slack.\ \ All\ \ ](https://www.voiceflow.com/blog/zapier-ai-chatbot) [Voice AI\ \ November 12, 2025\ \ October 30, 2025\ \ #### How to Build an AI IVR and Call Center \[2025\]\ \ This guide explains why traditional "press-button" IVR systems are costing you customers, and why your business should adopt an AI IVR today.\ \ All\ \ ](https://www.voiceflow.com/blog/ai-ivr) [Conversational Design\ \ November 12, 2025\ \ October 30, 2025\ \ #### Best Way to Use Conversational AI for E-Commerce \[2025\]\ \ This guide breaks down how modern conversational AI is no longer optional for e-commerce, driving 4x higher conversion rates and recovering up to 35% of abandoned carts.\ \ All\ \ ](https://www.voiceflow.com/blog/conversational-ai-for-e-commerce) [Voice AI\ \ November 12, 2025\ \ October 30, 2025\ \ #### How to Build an AI Call Agent for Restaurants \[2025\]\ \ Stop losing customers to your voicemail. With 83% of diners ordering elsewhere after one missed call, an AI call agent is a necessity. This guide shows how a "digital host" takes orders and answers common questions 24/7.\ \ All\ \ ](https://www.voiceflow.com/blog/ai-call-agent-for-restaurants) [AI Platforms\ \ November 12, 2025\ \ October 30, 2025\ \ #### 3 Best AI Scheduling Assistants In 2025 \[Tried & Tested\]\ \ This guide breaks down how AI scheduling assistants work and why they are crucial for protecting your focus. We review the 3 best AI scheduling tools for 2025 to help you find the right fit for your needs.\ \ All\ \ ](https://www.voiceflow.com/blog/ai-scheduling-assistant) [Voice AI\ \ November 12, 2025\ \ October 30, 2025\ \ #### Automate Phone Calls with the #1 AI Voice Agent \[2025\]\ \ Stop making customers wait on hold and paying human agents to answer the same questions. This guide explains how modern AI voice agents can automate your phone calls, providing 24/7, human-like support.\ \ All\ \ ](https://www.voiceflow.com/blog/ai-phone-call) ... [1](https://www.voiceflow.com/blog#) [Next](https://www.voiceflow.com/blog?8caee28d_page=2) Get the latest AI agent news ---------------------------- Join Voiceflow CEO, Braden Ream, as he explores the future of agentic tech in business on the Humans Talking Agents podcast. Subscribe You're subscribed. Look out for the next episode in your inbox. Oops! Something went wrong while submitting the form.  --- # Events | Voiceflow [](https://www.voiceflow.com/) Events ====== Join live and on-demand events with our team of expert agent builders. AI Strategy Agent Builds All Thank you! Your submission has been received! Oops! Something went wrong while submitting the form.  [AI Strategy\ \ October 16, 2025\ \ #### Live showcase: how e-commerce brands use AI agents to save time and costs\ \ We’re featuring 5+ live e-commerce AI agents that are helping brands automate "WISMO", reduce support costs, and improve upsell\ \ All\ \ .png)](https://www.voiceflow.com/events/live-showcase-how-e-commerce-brands-use-ai-agents-to-save-time-and-costs) [Agent Builds\ \ August 19, 2025\ \ #### How to supercharge your Gorgias AI customer support with Voiceflow\ \ Learn how to build powerful AI agents in Voiceflow that integrate directly into your Gorgias ticketing system\ \ All\ \ .webp)](https://www.voiceflow.com/events/gorgias-voiceflow) [Agent Builds\ \ July 16, 2025\ \ #### How to Build a Production-Ready Phone AI Agent for Lead Generation\ \ Learn how to connect Voiceflow to Zendesk via webhooks to create AI agents that understand user intent and trigger real actions\ \ All\ \ .avif)](https://www.voiceflow.com/events/voiceflow-roarkai) [Agent Builds\ \ June 11, 2025\ \ #### Build a Zendesk AI copilot with Voiceflow\ \ Learn how to connect Voiceflow to Zendesk via webhooks to create AI agents that understand user intent and trigger real actions\ \ All\ \ ](https://www.voiceflow.com/events/zendesk-voiceflow) [AI Strategy\ \ April 30, 2025\ \ #### Voiceflow Winter Release\ \ Join us for the Voiceflow Winter Release to see our latest product releases in action.\ \ All\ \ .png)](https://www.voiceflow.com/events/voiceflow-winter-release) [Agent Builds\ \ April 16, 2025\ \ #### Build a Custom Voiceflow Interface Inside Webflow\ \ Come out to learn how Zac embeds AI agents into Webflow with Voiceflow\ \ All\ \ ](https://www.voiceflow.com/events/build-a-custom-voiceflow-interface-inside-webflow) [AI Strategy\ \ March 27, 2025\ \ #### How to get Leads for your AI Agency\ \ Join Eugene and JC as they share their tips and tricks on getting leads for your AI Agency.\ \ All\ \ .avif)](https://www.voiceflow.com/events/get-leads-for-your-ai-agency) [AI Strategy\ \ February 26, 2025\ \ #### How to Build a Successful AI Agency\ \ Join our six of our Certified Voiceflow Agencies as they share their thoughts on the space and their journeys becoming agency founders.\ \ All\ \ ](https://www.voiceflow.com/events/how-to-build-a-successful-ai-agency) [Next](https://www.voiceflow.com/events?8caee28d_page=2) Start building AI Agents ------------------------ Want to explore how Voiceflow can be a valuable resource for you? Let's talk. [Get started, it’s free](https://creator.voiceflow.com/signup) [Contact sales](https://www.voiceflow.com/demo)  --- # Security | Voiceflow [](https://www.voiceflow.com/) Platform Security Secure and scalable agents -------------------------- Teams trust Voiceflow to build and deploy AI automation use cases securely and at scale.  Reliability ### Reliablity * 24x7x365 system server monitoring and on-call support from dedicated cloud-ops team * Infrastructure hosted on Amazon Web Services (AWS) and Google Cloud Platform (GCP) * Annually tested business continuity and disaster recovery (DR) plans * Separate production and testing environments * Multi-availability zone (AZ) compute instances * Infrastructure as Code (IaaC) management of cloud resources to ensure repeatable and reliable changes. Security ### Security Features * Voiceflow follows secure credential storage best practices by storing passwords using one-way hash encrypted passwords (BCRYPT) * Audit logging and event alerting * Regular updates rolled out to all customers, ensuring everyone has the latest application and security innovation * User-managed workspace access control to govern sharing privileges * Application audit log that includes security events such as user logins or configuration changes. * Project history tracking and rollback capability Data ### Data Security * Encryption-at-rest with AWS/GCP KMS customer-managed keys (AES-256) * Geographically distributed and encrypted offsite backups * Fully managed multi-AZ database instances with point-in-time-restore (PITR) Network ### Network Security * A CDN-based Web Application Firewall (WAF) and (D)DOS mitigation technologies * Encryption-in-transit using industry-standard TLS v1.2+ to ensure that all traffic between users and Voiceflow is secure. * All cloud-internal traffic is encrypted with mTLS with short-lived per-application certificates * Tiered, firewalled, and segmented network infrastructure to ensure that communication between Voiceflow services is strictly controlled. Launch ### Organizational Security * Employee background and reference checks in accordance with local laws. * Annual employee security awareness training covers topics such as data privacy, information security, and password security. * Principle-of-least-privilege implemented across the organization for both information and resource access. tested business continuity and disaster recovery (DR) plans * Audit logging of all cloud resources Analyze ### Application Security * Automated vulnerability analysis via network, host, and application scans. * Code assessment through both automated and manual review processes governed by Voiceflow's document Software Development Life Cycle (SDLC) policy. * Annual external penetration testing on primary public-facing endpoints. Single Sign-On (SSO) support for enterprise users       "With Voiceflow, users were no longer led through fixed linear flows. They were engaged with a real experience, using natural language, which revealed true-to-life results."  James Schneider Head of UX Design “Voiceflow has made creating conversation dialogue journeys easy and smooth. It has facilitated the collaboration and best practice sharing between people across different teams and countries.”  Illaria Di Donfrancesco Product Owner - AI Platforms  “Voiceflow has made creating conversation dialogue journeys easy and smooth. It has facilitated the collaboration and best practice sharing between people across different teams and countries.”  Illaria Di Donfrancesco Product Owner - AI Platforms  "With Voiceflow, users were no longer led through fixed linear flows. They were engaged with a real experience, using natural language, which revealed true-to-life results."  James Schneider Head of UX Design  Bug bounty program ------------------ If you discover a potential security vulnerability, please email security@voiceflow.com. Eligible submissions may qualify for our bug bounty program. Voiceflow values the contributions of the security research community in helping us maintain a safe and secure platform. We provide safe harbor for good-faith security research conducted in accordance with this Vulnerability Disclosure Policy. This means that if you comply with the guidelines set forth in this policy, Voiceflow will not initiate legal action against you under the Computer Fraud and Abuse Act (CFAA), the Digital Millennium Copyright Act (DMCA), or similar laws in other jurisdictions. We ask that you refrain from publicly disclosing any potential vulnerability until our security team has had the opportunity to review and address it. Enterprise Build agents using any AI vendor in an enterprise cloud. -------------------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover enterprise](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale and collaborate on AI products in one, central place. Get ideas to market fast Accelerate AI product team workflows to ship with speed and accuracy that can effeciently scale. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Community Ecosystem Get started with help from Voiceflow's global community. ---------------------------------------------------------- Connect with other builders, share agent functions, and learn how to create advanced AI automations. [Discord Community\ \ Join our fast-growing community of AI agent designers, developers, and builders.](https://discord.gg/9JRv5buT39) [YouTube Tutorials\ \ Learn from video tutorials created by the Voiceflow team.](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) [Templates\ \ Discover our library of agent templates to help you get started with your use case.](https://www.voiceflow.com/templates) [Marketplace\ \ Explore collections of agent-building resources created by the community.](https://www.voiceflow.com/resources) [Pathways Blog\ \ A publication for designers, developers, tinkerers, and trailblazers.](https://www.voiceflow.com/pathways) [Workshops\ \ Watch us make and break bots on the way to creating advanced AI agents.](https://youtube.com/playlist?list=PLKYemGIohRgBdoIl2cOWJQ3xSBUgMGW5j&feature=shared) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # 6 effective ways to design more impactful conversations | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 6 effective ways to design more impactful conversations ======================================================= [\ \ Emily Lonetto\ \ Head of Growth at](https://www.voiceflow.com/contributors/emily-lonetto) Conversation design is a fairly new concept, however it's one that holds the key to unlocking impactful voice experiences. Whether you're drafting meeting notes, a script for your new bot, or a brand new voice experience for your Alexa or Google Assistant, nailing the conversation is the very first step. To make things a little bit easier, we've broken it down into 6 best practices that will help you get started with your voice design journey.   > Made by our friends at Venngage — [The Infographic Maker](http://venngage.com/features/infographic-maker)  > Made by our friends at Venngage — [The Infographic Maker](http://venngage.com/features/infographic-maker)  RECOMMENDED [\ \ ### Expanding the definition of conversation design](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) PUBLISHED April 23, 2020 TOPICS IN THIS ARTICLE Elevating our work [\ \ Emily Lonetto\ \ Head of Growth at\ \ Emily is a Pathways contributor.](https://www.voiceflow.com/contributors/emily-lonetto) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to write an effective conversation designer job description \[+template\]](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you should A/B test your conversational experiences](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [](https://www.voiceflow.com/pathways/4-tips-for-great-conversational-ai-product-management) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 4 tips for great conversational AI product management](https://www.voiceflow.com/pathways/4-tips-for-great-conversational-ai-product-management) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/6-best-practices-to-design-more-impactful-conversations#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/6-best-practices-to-design-more-impactful-conversations#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # CXD in 2023: reflections and projections | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) CXD in 2023: reflections and projections ======================================== [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) We're entering the next phase of conversational AI, and it's incredibly exciting. Conversation design as a practice is maturing, and the introduction of large language models (LLMs) is pushing forward the underlying technology at a rapid pace. The LLM "ChatGPT" by OpenAI had [over a million users in less than a week](https://www.entrepreneur.com/en-in/news-and-trends/internet-sensation-chatgpt-crosses-1-million-users-in-5-days/440390) . This renewed excitement for the conversational AI space paired with the rapid growth of conversation design teams is the start of a new era for conversational AI—one that delivers not only on cost, but amazing experiences. These are exciting times. Here are my reflections on the year and projections for the next.  Large language models have entered the chat ------------------------------------------- I don’t believe GPT3 isn’t taking anyone’s job anytime soon because you and it do different things. It writes words. But your job was never about writing words—it’s always been about _creating experiences_. In an environment where there’s nobody to write for you, you probably tend to do a lot of writing. But if a machine can assist? That frees you to craft more and better experiences. I think large language models like this have the potential to make CXD roles a lot more data-driven. They’ll make the job more about curation, not creation. About ensuring the system meets user expectations more than crafting the things they interact with. It’s quite analogous to flying—pilots no longer land planes. But we have pilots. Their job is to get people from point A to point B. CXD will be similar. Overall I think it’s an exciting trend. Though, I think there’s a lot about the human experience it’s going to miss and it will forever need a human editor. Plus, when something suddenly becomes cheaper, a lot more of it happens. Just like how when banks launched ATMs, they actually [hired more human tellers](https://www.aei.org/economics/what-atms-bank-tellers-rise-robots-and-jobs/) ([though not at each branch](https://www.vox.com/2017/5/8/15584268/eric-schmidt-alphabet-automation-atm-bank-teller) ), I think this is going to create a lot more work for all. We’re already looking for ways to implement large language models in Voiceflow—more on that next year. CXD teams are maturing and hungry for standards ----------------------------------------------- “What’s the best practice?” I’ll bet you hear that a lot. We think it a lot. Many teams have reached a size and maturity where they’re looking to organize around well-defined standards. They’re creating template libraries, documenting brand voices, crafting persona guides, and trying to figure out how to govern assistants that span many languages, geographies, teams, and cases. At a certain size, these teams are learning that the process matters more than the output. And that individual “superhero” team members who have all the knowledge in their heads and frequently save others might actually be _contributing_ to the company’s lack of documentation. If your process prevents you from working with others—if your flows are spaghetti and notes indecipherable—you can’t produce the right outputs. As I already alluded, it’s exciting that nobody has defined best practices yet—which means you get to. If you have ideas to share, others want to hear them. Seriously. Right now, this profession feels like a high school dance where everyone is milling about the edges waiting for a brave person to step out. I promise you, you do not need to know how to dance. [Contribute to the Voiceflow blog](https://docs.google.com/document/d/1Gwfb-IO7MibsRfNduJMRb27YInKkSuVs1kzw9o5SDz8/edit?usp=sharing) . We can teach you. Some of our best contributors have been people who didn’t think they were ready. In fact, I urge you to make that a resolution for next year: share more, write more, and help us all legitimize this industry. A tailwind next year: greater investment in conversation design --------------------------------------------------------------- One silver lining to all that’s happening with the economy and layoffs is that every company is interested in automation, and doing more with less. So, while they are divesting in lots of areas, it probably means they're going to increase their investment in conversational AI. A headwind next year: difficulty calculating our impact ------------------------------------------------------- Designers have long struggled to justify their return on investment in tangible, immediate terms. Direct response marketers running digital ads have a much easier time. They can show the cost of acquisition immediately, and can turn it on and off like a tap. Whereas it’s much more difficult to define the benefits of a new brand identity, or the perceptual shift from full live agent support to partially conversational assistant-guided support. Next year, I bet a lot of companies are going to challenge conversation design projects by asking about the return, and lots of designers will struggle to respond. Their projects may lose out to surer short-term bets. Yet at the same time, I do hear lots of executives talking about [investing for the long-term despite the environment](https://hbr.org/2022/05/when-your-business-needs-a-second-growth-engine) . So, maybe the tailwind will overpower the headwind. Voiceflow surpassed 100,000 users --------------------------------- This year more than 100,000 people used Voiceflow. We doubled our team size and the number of customers and I am elated. Seriously, thank you for those votes of confidence. We will repay it in innovation. One thing I’d take with me: you and the entire community -------------------------------------------------------- As always, thank you for being a member of our growing Voiceflow community. We truly have the best customers and we feel incredibly fortunate to have users who are as passionate about what we're building as we are. Without your impromptu product feedback, bug reporting, and forum posting, we wouldn't be anywhere near where we are today. So again, thank you. Our mission at Voiceflow is to democratize the creation of AI Assistants and together, we will. See you in '23! Large language models have entered the chat ------------------------------------------- I don’t believe GPT3 isn’t taking anyone’s job anytime soon because you and it do different things. It writes words. But your job was never about writing words—it’s always been about _creating experiences_. In an environment where there’s nobody to write for you, you probably tend to do a lot of writing. But if a machine can assist? That frees you to craft more and better experiences. I think large language models like this have the potential to make CXD roles a lot more data-driven. They’ll make the job more about curation, not creation. About ensuring the system meets user expectations more than crafting the things they interact with. It’s quite analogous to flying—pilots no longer land planes. But we have pilots. Their job is to get people from point A to point B. CXD will be similar. Overall I think it’s an exciting trend. Though, I think there’s a lot about the human experience it’s going to miss and it will forever need a human editor. Plus, when something suddenly becomes cheaper, a lot more of it happens. Just like how when banks launched ATMs, they actually [hired more human tellers](https://www.aei.org/economics/what-atms-bank-tellers-rise-robots-and-jobs/) ([though not at each branch](https://www.vox.com/2017/5/8/15584268/eric-schmidt-alphabet-automation-atm-bank-teller) ), I think this is going to create a lot more work for all. We’re already looking for ways to implement large language models in Voiceflow—more on that next year. CXD teams are maturing and hungry for standards ----------------------------------------------- “What’s the best practice?” I’ll bet you hear that a lot. We think it a lot. Many teams have reached a size and maturity where they’re looking to organize around well-defined standards. They’re creating template libraries, documenting brand voices, crafting persona guides, and trying to figure out how to govern assistants that span many languages, geographies, teams, and cases. At a certain size, these teams are learning that the process matters more than the output. And that individual “superhero” team members who have all the knowledge in their heads and frequently save others might actually be _contributing_ to the company’s lack of documentation. If your process prevents you from working with others—if your flows are spaghetti and notes indecipherable—you can’t produce the right outputs. As I already alluded, it’s exciting that nobody has defined best practices yet—which means you get to. If you have ideas to share, others want to hear them. Seriously. Right now, this profession feels like a high school dance where everyone is milling about the edges waiting for a brave person to step out. I promise you, you do not need to know how to dance. [Contribute to the Voiceflow blog](https://docs.google.com/document/d/1Gwfb-IO7MibsRfNduJMRb27YInKkSuVs1kzw9o5SDz8/edit?usp=sharing) . We can teach you. Some of our best contributors have been people who didn’t think they were ready. In fact, I urge you to make that a resolution for next year: share more, write more, and help us all legitimize this industry. A tailwind next year: greater investment in conversation design --------------------------------------------------------------- One silver lining to all that’s happening with the economy and layoffs is that every company is interested in automation, and doing more with less. So, while they are divesting in lots of areas, it probably means they're going to increase their investment in conversational AI. A headwind next year: difficulty calculating our impact ------------------------------------------------------- Designers have long struggled to justify their return on investment in tangible, immediate terms. Direct response marketers running digital ads have a much easier time. They can show the cost of acquisition immediately, and can turn it on and off like a tap. Whereas it’s much more difficult to define the benefits of a new brand identity, or the perceptual shift from full live agent support to partially conversational assistant-guided support. Next year, I bet a lot of companies are going to challenge conversation design projects by asking about the return, and lots of designers will struggle to respond. Their projects may lose out to surer short-term bets. Yet at the same time, I do hear lots of executives talking about [investing for the long-term despite the environment](https://hbr.org/2022/05/when-your-business-needs-a-second-growth-engine) . So, maybe the tailwind will overpower the headwind. Voiceflow surpassed 100,000 users --------------------------------- This year more than 100,000 people used Voiceflow. We doubled our team size and the number of customers and I am elated. Seriously, thank you for those votes of confidence. We will repay it in innovation. One thing I’d take with me: you and the entire community -------------------------------------------------------- As always, thank you for being a member of our growing Voiceflow community. We truly have the best customers and we feel incredibly fortunate to have users who are as passionate about what we're building as we are. Without your impromptu product feedback, bug reporting, and forum posting, we wouldn't be anywhere near where we are today. So again, thank you. Our mission at Voiceflow is to democratize the creation of AI Assistants and together, we will. See you in '23!  RECOMMENDED [\ \ ### Introducing Voiceflow's GPT-powered AI Builder](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) PUBLISHED December 16, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### LLMs won’t replace NLUs. Here’s why](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) [](https://www.voiceflow.com/pathways/layers-ai-assistant-llms) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### The 4 layers every AI assistant needs (hint: more than just an LLM)](https://www.voiceflow.com/pathways/layers-ai-assistant-llms) [](https://www.voiceflow.com/pathways/ai-will-kill-apps) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### AI will kill apps and I can’t wait](https://www.voiceflow.com/pathways/ai-will-kill-apps) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/2023-cxd-reflections-projections#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/2023-cxd-reflections-projections#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 4 tips for great conversational AI product management | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 4 tips for great conversational AI product management ===================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Conversational AI teams and channels are growing rapidly, which means the demand for product managers in these areas is increasing too. As with any new product area, [conversational AI](https://www.voiceflow.com/blog/future-of-conversational-ai) product management comes with its own set of challenges. We couldn’t think of a better person to chat about this with than the exceptional Brian Smith, who is currently Senior Product Manager, Generative AI Experiences at Intuit. (When we originally published this article in October of 2022, Brian was Conversational AI Design Product Team Manager at Intuit.) Here’s Brian’s best advice on how to take conversational AI project management to the next level. 1\. Challenge designers and developers through rapid experimentation -------------------------------------------------------------------- The first thing every product manager should do is get the product in front of customers for real-time feedback. And they should continue to do so constantly. Design should be seen multiple times during the design process, not just during user testing. The micro-feedback received from customers during this process can be used to inform designers and [developers](https://www.voiceflow.com/blog-categories/developers) for more experimentation before moving on to the next subset of customers.  2\. Break bots. Then break them again. -------------------------------------- Typically, the goal is to put out MVP-ready work. But that status quo approach is safe and rarely enjoyable for the customer. A product manager needs to first push their team to roll out features that can and will break the existing bot. In this way, teams will learn from what breaks and what works. That is the only way to keep the team creative and the customers delighted. As a result, these welcomed failures will lead to a far better end experience. 3\. Communicate wins to the larger team --------------------------------------- Good product managers are grounded in the day-to-day and focused tactically on the team’s work. Great product managers also make sure to spread the word widely. This means celebrating quick wins across teams, showing prototypes to stakeholders, and even communicating ROI to leadership. 4\. Paint the journey with horizons ----------------------------------- Create a timeline for the entire conversational AI journey. Here, product managers map out capabilities or experiences within the assistant. Use [**Brian**](https://www.linkedin.com/in/mrbsmith?trk=public_post-text) 's horizons framework (below) to create a single snapshot of the customer journey. Plot out the three horizons on a timeline (informational, transactional, and conversational) and decide with your team what core functions the experience should be able to do at each using customer insights. Then, assign timelines to each horizon. For example: * Informational: answer bank account FAQs (we are here now) (Q3 2022) * Transactional: help a user transfer funds (Q1 2023) * Conversational: proactively prompt a user to transfer funds based on spending trends (Q4 2023) Finally, write down and revisit the scope of the digital assistant. This helps everyone design with the user goal in mind and makes any net new requests from other teams an easy decision. [**Brian**](https://www.linkedin.com/in/mrbsmith?trk=public_post_share-update_update-text) says this is very helpful for his teams at [**Intuit**](https://www.linkedin.com/company/intuit?trk=public_post_share-update_update-text) as it sets everyone—leadership, design, developers, data science—on the same page.  Product managers need to think of conversational AI experiences as products and not just features. Thinking ahead and pushing the boundaries of experiences are at the core of the product management role, and these tips can be used as a guide to efficiently carry out that role. _Editor's note: This was originally published in October 2022 and updated in June 2023._ 2\. Break bots. Then break them again. -------------------------------------- Typically, the goal is to put out MVP-ready work. But that status quo approach is safe and rarely enjoyable for the customer. A product manager needs to first push their team to roll out features that can and will break the existing bot. In this way, teams will learn from what breaks and what works. That is the only way to keep the team creative and the customers delighted. As a result, these welcomed failures will lead to a far better end experience. 3\. Communicate wins to the larger team --------------------------------------- Good product managers are grounded in the day-to-day and focused tactically on the team’s work. Great product managers also make sure to spread the word widely. This means celebrating quick wins across teams, showing prototypes to stakeholders, and even communicating ROI to leadership. 4\. Paint the journey with horizons ----------------------------------- Create a timeline for the entire conversational AI journey. Here, product managers map out capabilities or experiences within the assistant. Use [**Brian**](https://www.linkedin.com/in/mrbsmith?trk=public_post-text) 's horizons framework (below) to create a single snapshot of the customer journey. Plot out the three horizons on a timeline (informational, transactional, and conversational) and decide with your team what core functions the experience should be able to do at each using customer insights. Then, assign timelines to each horizon. For example: * Informational: answer bank account FAQs (we are here now) (Q3 2022) * Transactional: help a user transfer funds (Q1 2023) * Conversational: proactively prompt a user to transfer funds based on spending trends (Q4 2023) Finally, write down and revisit the scope of the digital assistant. This helps everyone design with the user goal in mind and makes any net new requests from other teams an easy decision. [**Brian**](https://www.linkedin.com/in/mrbsmith?trk=public_post_share-update_update-text) says this is very helpful for his teams at [**Intuit**](https://www.linkedin.com/company/intuit?trk=public_post_share-update_update-text) as it sets everyone—leadership, design, developers, data science—on the same page.  Product managers need to think of conversational AI experiences as products and not just features. Thinking ahead and pushing the boundaries of experiences are at the core of the product management role, and these tips can be used as a guide to efficiently carry out that role. _Editor's note: This was originally published in October 2022 and updated in June 2023._  RECOMMENDED [\ \ ### Breaking and rebuilding Nike's chatbot](https://www.voiceflow.com/pathways/breaking-nike-chatbot) PUBLISHED June 9, 2023 TOPICS IN THIS ARTICLE Exceptional people [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/why-you-need-to-consider-designing-for-multimodal-interfaces) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you need to consider designing for multimodal interfaces](https://www.voiceflow.com/pathways/why-you-need-to-consider-designing-for-multimodal-interfaces) [](https://www.voiceflow.com/pathways/how-to-use-open-vs-closed-prompts-in-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to use open vs. closed prompts in conversation design](https://www.voiceflow.com/pathways/how-to-use-open-vs-closed-prompts-in-conversation-design) [](https://www.voiceflow.com/pathways/voiceflow-frog-sxsw) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How a pair of interaction designers used Voiceflow to create an unforgettable experience at SXSW](https://www.voiceflow.com/pathways/voiceflow-frog-sxsw) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/4-tips-for-great-conversational-ai-product-management#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/4-tips-for-great-conversational-ai-product-management#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 15 KPIs for Measuring and Scaling a Generative AI Strategy | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) 15 KPIs for Measuring and Scaling a Generative AI Strategy ========================================================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) Expectations for AI transformation plans are immense, with 58% of CEOs expecting product improvements in the next 12 months. Balancing this short term POC pressure with a 5 year AI strategy is challenging. There are several paths to a first POC, and the leaders that prioritize a well-built first experience with a clear path to scaling to other use cases will create competitive advantage in this AI automation race. In this report, we’ll share key milestones, frameworks, and KPIs that will help you align teams to deliver AI solutions. This journey starts with an initial bespoke use-case leading to a mature iterative product development organization.  Initial AI journey and Proof of Concept ---------------------------------------  Your initial AI project should be about clearing the list of unknowns. It’s essential to iterate on ideas and demos to create a first implementation. This work falls into three buckets - team coordination, delivery speed and user iteration speed. As a leader, your core objective is to unblock teams and provide a space to iterate and improve AI products. Each of these objectives should come with their own key performance indicators to track and measure time until delivery. **Leaders need to structure a team** that’s set up for success. Many generative AI projects have been staffed exclusively within a ML or IT team, ignoring cross-functional needs to build strong customer experiences and corresponding business cases. This team should be capable of deep diving into a problem area and creating clarity in a fast paced space. Although not a strict list, successful POCs typically have inputs from a product owner, UX, developer and ML engineer. **Clearly defining a problem statement for your POC** will help teams deliver with focus and avoid getting lost in AI distractions. In many circles, AI has become a solution looking for a problem, which is great for initial momentum, yet returns in fury and creates lasting tech debt in years 2-3 of a 5 year AI strategy. **User interviews** often forgotten in the push to build. Curating a beta group is a highly valuable way to measure product improvements, whether these users are internal or external. These iterations should be quick, with notes shared across the entire team to understand key concerns and positive outcomes. Often times this step is skipped as engineering teams focus on the build and only at the last moment deliver a sharable experience. The most important item is **getting a cohesive demo prepared**, including a clear value proposition and budget for taking the project to the next step. The initial proof of concept is designed to de-risk a project, but success is measured by projects’ progress. Without progress many teams get stuck in proof of concept purgatory, where these early use cases keep getting built with no clear next step for production or past first launch. Presenting a cohesive narrative for moving the POC to production becomes essential.  Deploying to Production ----------------------- _KPIs:_ 1. _Time to connect to production data_ 2. _Time to move between product environments_ 3. _Time to build an evaluation suite_ 4. _Time to finalize UI_ 5. _Time to complete security and risk assessments_ 6. _Time to launch feature_ As clearer use cases are defined, more teams become involved in the process. With more teams, clear communication becomes essential for progress and alignment. To navigate this complexity, it’s important that feature and product launches are well managed and the tooling reflects this collaborative, scaled effort. **Connecting to production data** is essential at this step. In a proof of concept environment we often focus on mimicking production data or using a subset since integrating with existing systems can be challenging. When moving into production, this should be a key task for the engineering team to connect and test the POC with production data. Depending on the organization, this data may sit in a higher environment, requiring the team to deploy to an user acceptance testing environment (UAT). Next, teams should focus on **speed to move the features through each stage of the product environment**. Typically there are 2-4 environments depending on the stage of the company, but similar to software projects, fast progress times through environments leads to faster iteration cycles. On the engineering side of the house, the team should be measured on **creating strong testing metrics and brand aligned front-ends**. Testing is particularly challenging since generative AI models are non-deterministic, requiring tests to validate how users can interact with the models. This work will typically be done in conjunction by a user testing, ML and engineering team to define and deliver on thorough and relevant tests. When considering the front end, generative AI applications should feel like a natural extension of an existing application, rather than being bolted on as an afterthought. This is where design, UX and front end teams need to collaborate to create the integration. To speed up the delivery process, using pre-existing open source or hosted front end frameworks can be strongly beneficial.  Beyond product development often lies a negotiation with the security, compliance, and risk teams. Generative AI security and risk policies are still being defined or iterated on within many organizations, and navigating these policies can slow down projects. Ideally these discussions are started in the POC phase to fully align teams early. We discussed some of these approaches in a prior piece. Launching the feature live is the final milestone. Once the first application goes into production, there will likely be more projects starting, so the KPIs can apply across new projects across the organization. It’s important that leaders have set a clear product roadmap and teaming structure so that each new use case and iteration is not additional technical debt, rather an additive product lifecycle that becomes more efficient with each new launch.  Iterate and Scale ----------------- _KPIs:_ 1. _Time to make simple updates_ 2. _Time to launch subsequent version_ 3. _Time for next team to launch product_ 4. _Cost per additional project_ 5. _Total cost of ownership_ After deploying at least one use case to production, the challenge for an organization is to scale their generative AI applications across teams in repeatable and financially sustainable way. Competitive advantage is created when teams can build with an innovation mindset and not be stuck in maintenance mode. The KPIs for this phase focus on this agility and cost. It’s first important to measure the **ability to quickly update your AI applications**. Simple changes like minor prompt updates, copywriting, or API version bumps should not be complex processes. Similar to [rapid software release](https://engineering.fb.com/2017/08/31/web/rapid-release-at-massive-scale/) , making small changes to generative AI updates, especially after initial release, builds trust with customers and address minor oversights. What about **larger iterations**? Models, data and product requirements continue to evolve quickly, so being able to update features and capabilities will future proof your generative AI products from becoming legacy product lines (e.g. an AI chatbot on your homepage that was launched as an initial use case, yet quickly becomes ignored and outdated without a proper team, workflow, and iteration cycle).  Similarly, as adoption increases across an organization through new use cases, teams should be able to **onboard more quickly** after the initial use case. The total cost of ownership for each new team and use case should become more efficient as the level of innovation and automation across an organization ramps exponentially. **At the CIO level, this should be an essential metric when implementing and scaling your AI strategy.** The final two KPIs in this section focus on cost, both the marginal cost per application and the total cost across a strategy. These two metrics help organizations plan future projects and measure the ROI across their generative AI initiatives. They also help scope investments to be leaner and more deeply integrated within a technology initiative, rather than a separate line item with a large commitment.  Conclusion ---------- As teams build their generative AI strategies and adopt their core workflows, planning each phase of the process is essential. While generative AI feels like an immediate fire to address, projects and capabilities will continue to evolve and require a comprehensive 5 year strategy. The leaders and teams that execute a well-built first POC that’s anchored in a strategy for scaling to other use cases will create competitive advantage in this AI automation race across industries. #### Citations PWC, 2024: [https://www.pwc.com/us/en/library/ceo-survey.html](https://www.pwc.com/us/en/library/ceo-survey.html) Initial AI journey and Proof of Concept ---------------------------------------  Your initial AI project should be about clearing the list of unknowns. It’s essential to iterate on ideas and demos to create a first implementation. This work falls into three buckets - team coordination, delivery speed and user iteration speed. As a leader, your core objective is to unblock teams and provide a space to iterate and improve AI products. Each of these objectives should come with their own key performance indicators to track and measure time until delivery. **Leaders need to structure a team** that’s set up for success. Many generative AI projects have been staffed exclusively within a ML or IT team, ignoring cross-functional needs to build strong customer experiences and corresponding business cases. This team should be capable of deep diving into a problem area and creating clarity in a fast paced space. Although not a strict list, successful POCs typically have inputs from a product owner, UX, developer and ML engineer. **Clearly defining a problem statement for your POC** will help teams deliver with focus and avoid getting lost in AI distractions. In many circles, AI has become a solution looking for a problem, which is great for initial momentum, yet returns in fury and creates lasting tech debt in years 2-3 of a 5 year AI strategy. **User interviews** often forgotten in the push to build. Curating a beta group is a highly valuable way to measure product improvements, whether these users are internal or external. These iterations should be quick, with notes shared across the entire team to understand key concerns and positive outcomes. Often times this step is skipped as engineering teams focus on the build and only at the last moment deliver a sharable experience. The most important item is **getting a cohesive demo prepared**, including a clear value proposition and budget for taking the project to the next step. The initial proof of concept is designed to de-risk a project, but success is measured by projects’ progress. Without progress many teams get stuck in proof of concept purgatory, where these early use cases keep getting built with no clear next step for production or past first launch. Presenting a cohesive narrative for moving the POC to production becomes essential.  Deploying to Production ----------------------- _KPIs:_ 1. _Time to connect to production data_ 2. _Time to move between product environments_ 3. _Time to build an evaluation suite_ 4. _Time to finalize UI_ 5. _Time to complete security and risk assessments_ 6. _Time to launch feature_ As clearer use cases are defined, more teams become involved in the process. With more teams, clear communication becomes essential for progress and alignment. To navigate this complexity, it’s important that feature and product launches are well managed and the tooling reflects this collaborative, scaled effort. **Connecting to production data** is essential at this step. In a proof of concept environment we often focus on mimicking production data or using a subset since integrating with existing systems can be challenging. When moving into production, this should be a key task for the engineering team to connect and test the POC with production data. Depending on the organization, this data may sit in a higher environment, requiring the team to deploy to an user acceptance testing environment (UAT). Next, teams should focus on **speed to move the features through each stage of the product environment**. Typically there are 2-4 environments depending on the stage of the company, but similar to software projects, fast progress times through environments leads to faster iteration cycles. On the engineering side of the house, the team should be measured on **creating strong testing metrics and brand aligned front-ends**. Testing is particularly challenging since generative AI models are non-deterministic, requiring tests to validate how users can interact with the models. This work will typically be done in conjunction by a user testing, ML and engineering team to define and deliver on thorough and relevant tests. When considering the front end, generative AI applications should feel like a natural extension of an existing application, rather than being bolted on as an afterthought. This is where design, UX and front end teams need to collaborate to create the integration. To speed up the delivery process, using pre-existing open source or hosted front end frameworks can be strongly beneficial.  Beyond product development often lies a negotiation with the security, compliance, and risk teams. Generative AI security and risk policies are still being defined or iterated on within many organizations, and navigating these policies can slow down projects. Ideally these discussions are started in the POC phase to fully align teams early. We discussed some of these approaches in a prior piece. Launching the feature live is the final milestone. Once the first application goes into production, there will likely be more projects starting, so the KPIs can apply across new projects across the organization. It’s important that leaders have set a clear product roadmap and teaming structure so that each new use case and iteration is not additional technical debt, rather an additive product lifecycle that becomes more efficient with each new launch.  Iterate and Scale ----------------- _KPIs:_ 1. _Time to make simple updates_ 2. _Time to launch subsequent version_ 3. _Time for next team to launch product_ 4. _Cost per additional project_ 5. _Total cost of ownership_ After deploying at least one use case to production, the challenge for an organization is to scale their generative AI applications across teams in repeatable and financially sustainable way. Competitive advantage is created when teams can build with an innovation mindset and not be stuck in maintenance mode. The KPIs for this phase focus on this agility and cost. It’s first important to measure the **ability to quickly update your AI applications**. Simple changes like minor prompt updates, copywriting, or API version bumps should not be complex processes. Similar to [rapid software release](https://engineering.fb.com/2017/08/31/web/rapid-release-at-massive-scale/) , making small changes to generative AI updates, especially after initial release, builds trust with customers and address minor oversights. What about **larger iterations**? Models, data and product requirements continue to evolve quickly, so being able to update features and capabilities will future proof your generative AI products from becoming legacy product lines (e.g. an AI chatbot on your homepage that was launched as an initial use case, yet quickly becomes ignored and outdated without a proper team, workflow, and iteration cycle).  Similarly, as adoption increases across an organization through new use cases, teams should be able to **onboard more quickly** after the initial use case. The total cost of ownership for each new team and use case should become more efficient as the level of innovation and automation across an organization ramps exponentially. **At the CIO level, this should be an essential metric when implementing and scaling your AI strategy.** The final two KPIs in this section focus on cost, both the marginal cost per application and the total cost across a strategy. These two metrics help organizations plan future projects and measure the ROI across their generative AI initiatives. They also help scope investments to be leaner and more deeply integrated within a technology initiative, rather than a separate line item with a large commitment.  Conclusion ---------- As teams build their generative AI strategies and adopt their core workflows, planning each phase of the process is essential. While generative AI feels like an immediate fire to address, projects and capabilities will continue to evolve and require a comprehensive 5 year strategy. The leaders and teams that execute a well-built first POC that’s anchored in a strategy for scaling to other use cases will create competitive advantage in this AI automation race across industries. #### Citations PWC, 2024: [https://www.pwc.com/us/en/library/ceo-survey.html](https://www.pwc.com/us/en/library/ceo-survey.html)  RECOMMENDED [\ \ ### Crawl, walk, run: 28+ tactics for evolving your AI agent](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) PUBLISHED May 1, 2024 TOPICS IN THIS ARTICLE Learning and APIs Exceptional work [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.webp)](https://www.voiceflow.com/pathways/what-is-an-interaction-model-in-conversational-ai) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### What is an interaction model in conversational AI?](https://www.voiceflow.com/pathways/what-is-an-interaction-model-in-conversational-ai) [](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Building your AI agents like products: A blueprint from POC purgatory to production](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [](https://www.voiceflow.com/pathways/ai-will-kill-apps) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### AI will kill apps and I can’t wait](https://www.voiceflow.com/pathways/ai-will-kill-apps) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 4 things you can do with Airtable and conversation design | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 4 things you can do with Airtable and conversation design ========================================================= [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Data is essential to creating contextual, dynamic conversations for users. Whether saving or retrieving information, conversation designers can utilize database tools like [Airtable](https://airtable.com/invite/r/AELCPvWo) to customize any conversation on the Voiceflow canvas. Here we look at the ins and outs of integrating with an [Airtable](https://airtable.com/invite/r/AELCPvWo) base.  The API Step ------------ The API step on the Voiceflow canvas can be a designer's best friend. Integrating with an Airtable base means the database tool is directly tied to a project's core steps and variables. On the canvas, there are two popular API requests designers use with Airtable: GET and POST. ⬅️ **GET Request** _retrieves data from a connected base_. This API function accesses specific fields in an existing Airtable base and brings that information into the experience. ➡️ **POST Request** _saves data to a connected base_. Inversely, this API function takes user inputs from a conversational experience and records the information to fields in an existing Airtable base.  A look at the GET and POST request API steps on the Voiceflow canvas > We've created two step-by-step video tutorials for using the Airtable GET and POST requests in Voiceflow. > > Save these links to watch after reading: > ⬅️ [GET Request Tutorial \ > ➡️](https://youtu.be/C6-V35uWscM) > [POST Request Tutorial](https://youtu.be/NFE8wAFfTx4) ### What's Possible Before you start adding the API step to the canvas, let's first consider what's possible. Utilizing the GET and POST requests, conversation designers unlock access to information outside of the canvas. Here are three additions to the CxD tool belt: * **Pull or save information using an existing base** - A conversation can now reference information in an existing database. This means that any data point that lives in the Airtable base can be referenced and pulled into the project. Similarly, data collected by the assistant can be saved to an integrated base. Here we're adding a list of tasks to a base for our assistant to access. %25202.avif) Adding Task data to an Airtable base. * **Map information to any variable in your design** - Variables on the Voiceflow canvas allow for detailed, flexible conversational experiences. Now, Airtable data can be mapped to any variable in your project. Here's a glimpse at a designer mapping the "Task" field in Airtable to a created "recent\_task" variable in Voiceflow.  Mapping Airtable fields to variables in Voiceflow. * **Easily update and deliver content in realtime** - Whether sending or receiving data, API requests happen in real-time. The assistant and Airtable base work in tandem, allowing designers to complement the existing conversation using a powerful database tool. Here the user asks the assistant for their next task, and the Airtable information is seamlessly pushed to the experience and delivered by the assistant. In this case, it's time to go grocery shopping.  Prototype experience of assistant pulling data from Airtable. * **Create a single source of truth for scaling -** Most importantly, Airtable is a solid segue into a content management system. As experiences get inherently more customized or scaleable, it's important to have a data system that scales with it. Using an external database of any kind is a great way to ensure your experience scales as your data does. Allowing your experience to pull and push data to a single source of truth, and make it easier for team members to add/remove data from your experience over time. This provides superior design and management flexibility over hard-coding responses or data directly into a project. ### Use Cases Using a database with your conversation designs inherently makes your experience more malleable. Using a tool like Airtable, custom variables can be pulled, tracked, and/or communicated across an experience. And although the to-do or task list is a foundational example in conversational AI, **the use cases are endless.** **Here are a few examples of how you can use Voiceflow and Airtable in your next project:** **📚 Personal Tracker:** A personal tracker relies on daily, consistent information that can be recorded and accessed later by the virtual assistant. How do you feel today? What was my mood last week Monday? An Airtable base integration makes this functionality a breeze. 📖 **Consumer Product Catalogs**: SKUs are constantly changing based on season, inventory, and trends. An Airtable base can be updated as needed, while the assistant will always have access to accurate product listings. Most of all, data integrity exists as there is one source of truth. 📝 **Small Business Budgets:** Manually entering data or searching through spreadsheets for information is a time sink for small business owners - not to mention a poor user experience. Deploying a budget assistant linked to an Airtable base can make it painless to record and check finances. For conversation design teams, it's also important to note that adding **the API step in the design and prototype stages can vastly improve developer hand-off.** CxD teams are no longer reliant on developers to execute their API vision - and all the necessary code lives under the hood of the exported Voiceflow project file. No matter the use case, deploying the API step is another way that conversation designers can rethink experiences across any channel. From IVR to web chat, an integrated Airtable database adds conversational depth and context for the user. > Ready to give these Airtable integrations a go? Start from scratch on the canvas or use this [Mood Tracker template](https://creator.voiceflow.com/dashboard?import=6050cf8857f78f001c3e2af4) > project that features an Airtable database GET and POST request. > We've created two step-by-step video tutorials for using the Airtable GET and POST requests in Voiceflow. > ⬅️ [GET Request Tutorial \ > ➡️](https://youtu.be/C6-V35uWscM) > [POST Request Tutorial](https://youtu.be/NFE8wAFfTx4) The API Step ------------ The API step on the Voiceflow canvas can be a designer's best friend. Integrating with an Airtable base means the database tool is directly tied to a project's core steps and variables. On the canvas, there are two popular API requests designers use with Airtable: GET and POST. ⬅️ **GET Request** _retrieves data from a connected base_. This API function accesses specific fields in an existing Airtable base and brings that information into the experience. ➡️ **POST Request** _saves data to a connected base_. Inversely, this API function takes user inputs from a conversational experience and records the information to fields in an existing Airtable base.  A look at the GET and POST request API steps on the Voiceflow canvas > We've created two step-by-step video tutorials for using the Airtable GET and POST requests in Voiceflow. > > Save these links to watch after reading: > ⬅️ [GET Request Tutorial \ > ➡️](https://youtu.be/C6-V35uWscM) > [POST Request Tutorial](https://youtu.be/NFE8wAFfTx4) ### What's Possible Before you start adding the API step to the canvas, let's first consider what's possible. Utilizing the GET and POST requests, conversation designers unlock access to information outside of the canvas. Here are three additions to the CxD tool belt: * **Pull or save information using an existing base** - A conversation can now reference information in an existing database. This means that any data point that lives in the Airtable base can be referenced and pulled into the project. Similarly, data collected by the assistant can be saved to an integrated base. Here we're adding a list of tasks to a base for our assistant to access. %25202.avif) Adding Task data to an Airtable base. * **Map information to any variable in your design** - Variables on the Voiceflow canvas allow for detailed, flexible conversational experiences. Now, Airtable data can be mapped to any variable in your project. Here's a glimpse at a designer mapping the "Task" field in Airtable to a created "recent\_task" variable in Voiceflow.  Mapping Airtable fields to variables in Voiceflow. * **Easily update and deliver content in realtime** - Whether sending or receiving data, API requests happen in real-time. The assistant and Airtable base work in tandem, allowing designers to complement the existing conversation using a powerful database tool. Here the user asks the assistant for their next task, and the Airtable information is seamlessly pushed to the experience and delivered by the assistant. In this case, it's time to go grocery shopping.  Prototype experience of assistant pulling data from Airtable. * **Create a single source of truth for scaling -** Most importantly, Airtable is a solid segue into a content management system. As experiences get inherently more customized or scaleable, it's important to have a data system that scales with it. Using an external database of any kind is a great way to ensure your experience scales as your data does. Allowing your experience to pull and push data to a single source of truth, and make it easier for team members to add/remove data from your experience over time. This provides superior design and management flexibility over hard-coding responses or data directly into a project. ### Use Cases Using a database with your conversation designs inherently makes your experience more malleable. Using a tool like Airtable, custom variables can be pulled, tracked, and/or communicated across an experience. And although the to-do or task list is a foundational example in conversational AI, **the use cases are endless.** **Here are a few examples of how you can use Voiceflow and Airtable in your next project:** **📚 Personal Tracker:** A personal tracker relies on daily, consistent information that can be recorded and accessed later by the virtual assistant. How do you feel today? What was my mood last week Monday? An Airtable base integration makes this functionality a breeze. 📖 **Consumer Product Catalogs**: SKUs are constantly changing based on season, inventory, and trends. An Airtable base can be updated as needed, while the assistant will always have access to accurate product listings. Most of all, data integrity exists as there is one source of truth. 📝 **Small Business Budgets:** Manually entering data or searching through spreadsheets for information is a time sink for small business owners - not to mention a poor user experience. Deploying a budget assistant linked to an Airtable base can make it painless to record and check finances. For conversation design teams, it's also important to note that adding **the API step in the design and prototype stages can vastly improve developer hand-off.** CxD teams are no longer reliant on developers to execute their API vision - and all the necessary code lives under the hood of the exported Voiceflow project file. No matter the use case, deploying the API step is another way that conversation designers can rethink experiences across any channel. From IVR to web chat, an integrated Airtable database adds conversational depth and context for the user. > Ready to give these Airtable integrations a go? Start from scratch on the canvas or use this [Mood Tracker template](https://creator.voiceflow.com/dashboard?import=6050cf8857f78f001c3e2af4) > project that features an Airtable database GET and POST request. > We've created two step-by-step video tutorials for using the Airtable GET and POST requests in Voiceflow. > ⬅️ [GET Request Tutorial \ > ➡️](https://youtu.be/C6-V35uWscM) > [POST Request Tutorial](https://youtu.be/NFE8wAFfTx4)  RECOMMENDED [](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design#) PUBLISHED June 4, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/best-practices-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Curiosity and conversation beat any best practice](https://www.voiceflow.com/pathways/best-practices-conversation-design) [](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to write an effective conversation designer job description \[+template\]](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [](https://www.voiceflow.com/pathways/6-best-practices-to-design-more-impactful-conversations) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 6 effective ways to design more impactful conversations](https://www.voiceflow.com/pathways/6-best-practices-to-design-more-impactful-conversations) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Find a Voiceflow Expert [](https://www.voiceflow.com/) Hire a Voiceflow Expert ======================= Find a Voiceflow Expert to help you build and launch your AI assistant! Our list of experts can handle everything in Voiceflow from design, build, launch and the integrations in between. [Explore Experts](https://www.voiceflow.com/experts#Experts) [Become a Partner](https://www.voiceflow.com/become-a-partner) Experts Voiceflow Experts can do it all ------------------------------- Design, customize, deploy, and optimize your AI assistant with members of our community we recognize as Voiceflow Experts. .svg) Tier [Clear](https://www.voiceflow.com/experts#) Enterprise Expert Premium Expert Certified Expert Services Offered [Clear](https://www.voiceflow.com/experts#) Custom Code Design Hosting Integrations Optimization Voice Industry [Clear](https://www.voiceflow.com/experts#) Ecommerce SaaS Telecom Financial Services Small Business Real Estate Government Budget [Clear](https://www.voiceflow.com/experts#) Under $1000 $1000 - $5000 $5000 - $10,000 $10,000 - $20,000 $20,000 + Location [Clear](https://www.voiceflow.com/experts#) Canada United States Australia United Kingdom India Europe Pakistan Germany Sweden Italy Sort byName (A-Z)Name (Z-A) [\ \ Corentin Pinel\ \ Certified Expert\ \ At Evolution Agency, we combined a child’s curiosity with a grand vision to create something truly unique. For us, AI was the ultimate toy, and Voiceflow became the space where we could turn imagination into reality. We’ve mastered the basics like: simple and advanced FAQs, lead qualification, advanced dashboards, quotes on demand, product recommendations, order tracking and follow-ups, gamification, learning and training, ticketing systems (Zendesk, Slack, Notion…), closing and deal-making, performance tracking, multi-language support ... and plenty more. We’ve got that covered. You’re paying for expertise, not just a skill set. But our real passion lies in uncharted territory—the projects that make you wonder, 'Is this even possible?' That’s where we truly thrive. From gaming and energy to law, e-commerce, and market intelligence, we’ve tackled ambitious challenges and delivered outstanding results. So, what’s the wild idea you have in mind?\ \ ](https://www.voiceflow.com/contributors/corentin-pinel) Certified Expert $1000 - $5000 [\ \ Pavel Cermak\ \ Certified Expert\ \ HypeDigitaly is the first AI agency in the Czech Republic to deliver a municipal AI chatbot, specializing in B2G solutions with a strong focus on accuracy (95%+ in RAG) and custom automated data handling. Since early 2024, we've successfully completed several governmental projects and continue to drive innovation in AI chatbot development and web data automation for public institutions.\ \ .webp)](https://www.voiceflow.com/contributors/pavel-cermak) Certified Expert $5000 - $10,000 Europe [\ \ Juan Carlos Quintero\ \ Certified Expert\ \ We build bespoke Voiceflow agents for SMBs to boost conversion and save business owners time\ \ ](https://www.voiceflow.com/contributors/juan-carlos-quintero) Certified Expert $1000 - $5000 Europe [\ \ Jessica Heaton\ \ I’m Jess - Voiceflow AI/ automation expert. If you’ve got a vision, I’m confident we can build it. Let’s jump on a quick call and figure out what’ll bring the biggest return for your business.\ \ .png)](https://www.voiceflow.com/contributors/jessica-heaton) $1000 - $5000 United Kingdom [\ \ Conner Burton\ \ Certified Expert\ \ I build AI Agent chatbots for business websites. Sometimes rants on Youtube 📺\ \ ](https://www.voiceflow.com/contributors/conner-burton) Certified Expert $1000 - $5000 United States [\ \ Abdullah Yahya\ \ Certified Expert\ \ I’m Abdullah Yahya, a hands-on AI agent builder from the Netherlands. I specialize in building chatbots and automated workflows using Voiceflow and make.com—cutting support costs, speeding up sales, and making businesses more efficient. I don’t waste time on theory or hype. Every solution I deliver is designed to solve a real business problem and show results. If you want a partner who actually builds, tests, and improves real automations—let’s work.\ \ %20(1).webp)](https://www.voiceflow.com/contributors/abdullah-yahya) Certified Expert $1000 - $5000 Europe [\ \ Luuk de Block\ \ Certified Expert\ \ I'm from the Netherlands and I'm a tech enthusiast with a passion for creating advanced solutions. I enjoy exploring new features and pushing the limits of what's possible, always aiming to innovate and improve. Let's connect and share insights to make our AI agents even smarter and more effective.\ \ .webp)](https://www.voiceflow.com/contributors/luuk-de-block) Certified Expert $1000 - $5000 Europe [\ \ Jan Rieck-Blankenburg\ \ Certified Expert\ \ At Ambotiq, we help businesses turn their customer support from a cost center into a revenue-driving machine. Based in Germany with years of experience in workflow automation and AI development, we build scalable, sustainable solutions that boost efficiency and unlock growth.\ \ .avif)](https://www.voiceflow.com/contributors/jan-rieck-blankenburg) Certified Expert $1000 - $5000 Germany [.webp)\ \ Edwin Chen\ \ Certified Expert\ \ We help B2C Service Brands Delivers A Premium Customer Experience on Autopilot In 30 Days. Our expertise comes from building both Voiceflow and Custom coded Conversational Agents in sales, customer experience and customer retention use cases. Some of our current clients include one of the world's largest retreat booking sites, a $250M Restaurant Chain in North America and a premium dog grooming company etc.\ \ .png)](https://www.voiceflow.com/contributors/edwin-chen-legacy-ai) Certified Expert $5000 - $10,000 [](https://www.voiceflow.com/experts?55ae82a0_page=2) [1](https://www.voiceflow.com/experts#) [Voice](https://www.voiceflow.com/expert-services/voice) [Custom Code](https://www.voiceflow.com/expert-services/custom-code) [Optimization](https://www.voiceflow.com/expert-services/optimization) [Hosting](https://www.voiceflow.com/expert-services/hosting) [Integrations](https://www.voiceflow.com/expert-services/integration) [Design](https://www.voiceflow.com/expert-services/design) [Government](https://www.voiceflow.com/expert-industry/government) [Real Estate](https://www.voiceflow.com/expert-industry/real-estate) [Small Business](https://www.voiceflow.com/expert-industry/small-business) [Financial Services](https://www.voiceflow.com/expert-industry/financial-services) [Telecom](https://www.voiceflow.com/expert-industry/telecom) [SaaS](https://www.voiceflow.com/expert-industry/saas) [Ecommerce](https://www.voiceflow.com/expert-industry/ecommerce) No results found... Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. Become an expert ---------------- Learn about our partner program and apply to join our growing network of AI professionals. [Become an Expert](https://www.voiceflow.com/become-a-partner)  --- # Resources | Voiceflow [](https://www.voiceflow.com/) Voiceflow Resources =================== Discover our library of templates, apps, and integrations, to help you get started. Build an agent to automate just about anything. What agent are you building? Here are some of the top use cases: [Templates\ \ Community built templates.](https://www.voiceflow.com/templates) [Community apps\ \ Apps that enhance Voiceflow.](https://www.voiceflow.com/apps) [Integrations\ \ Connect your techstack.](https://www.voiceflow.com/integrations) [Templates\ \ Community built templates.](https://www.voiceflow.com/templates) [Community apps\ \ Apps that enhance Voiceflow.](https://www.voiceflow.com/apps) [Integrations\ \ Community built functions.](https://www.voiceflow.com/functions)  Featured templates Our favourite templates ----------------------- [Browse Templates](https://www.voiceflow.com/templates) [](https://www.voiceflow.com/templates/complete-support-chatbot) .webp) Certified Template 4927 ### Complete support chatbot template Download this template to build a complete support chatbot that answers FAQs, recommends products, captures user information, and more.  Daniel D'Souza Customer support FREE Voiceflow [](https://www.voiceflow.com/templates/ai-agent-with-knowledge-base-memory)  Certified Template 2176 ### AI Agent with Knowledge Base Memory Use this template to create an AI agent that handles follow ups with contextual memory  W. Williams Customer support AI character E-commerce Lead generation Product Copilot FREE Community [](https://www.voiceflow.com/templates/building-an-ai-powered-hotel-booking-assistant)  Certified Template 1473 ### Building an AI-Powered Hotel Booking Assistant Pete walks through how to build a personalized hotel booking assistant for Hotels.com by leveraging CRM data, large language models (LLMs), and Airtable integration.  Peter Isaacs Customer support Product Copilot FREE Voiceflow [](https://www.voiceflow.com/templates/complete-retail-chatbot)  Certified Template 1391 ### Complete retail chatbot template Download this template to build a complete retail chatbot that handles customer service, lets users shop products, and more.  Daniel D'Souza Customer support FREE Voiceflow [](https://www.voiceflow.com/templates/shopify-order-checker-ai-agent)  Certified Template 905 ### Shopify Order Checker AI Agent Build an order checker agent for your Shopify store that then displays the data in a carousel format.  Brendan Jowett E-commerce Certified Partners [](https://www.voiceflow.com/templates/ai-order-return-agent-for-new-balance)  Certified Template 453 ### AI Order Return Agent for New Balance how to build an embedded web agent that can track order status, process refunds, and provide product recommendations  Peter Isaacs E-commerce Voiceflow Featured collections Curated resources, for every use case ------------------------------------- [Browse collections](https://www.voiceflow.com/collections) [Top Resources for AI Agencies\ \ It’s no secret that the release of tools like ChatGPT has challenged how we see the role of AI across every industry. AI is growing at a rate that no other technology has before, with new models, platforms, and research being published daily. To keep up with the pace of change, AI agencies formed, helping businesses—big and small—carve out their stake in AI.](https://www.voiceflow.com/collections/top-resources-for-agencies) All Agencies [Top 7 Coolest Custom Interface Projects Built on Voiceflow (For Developers)\ \ The true power of Voiceflow is in our ability to let developers integrate to any platform, execute custom actions, and deploy to any interface—with speed. This is made possible using our APIs, custom webchat, built-in code steps, functions, and extensions. If you’re a developer curious to build an AI agent on a custom interface, here’s some art-of-possible inspiration for you.](https://www.voiceflow.com/collections/top-custom-interface-projects) All Task automation Featured apps Push Voiceflow further with community built apps ------------------------------------------------ [Browse community apps](https://www.voiceflow.com/apps) [](https://www.voiceflow.com/apps/custom-agency-dashboard)  Certified App ### Chatdash - Voiceflow analytics for AI agencies Custom analytics dashboard for agency owners!  REAL AI E-commerce Lead generation Product Copilot PAID Community [](https://www.voiceflow.com/apps/converso)  Certified App ### Converso - AI + human agent helpdesk for optimum productivity Deploy Voiceflow AI Agents across webchat, WhatsApp and SMS channels, and enable human handover to support or marketing teams.  Chris Barley AI character Customer support E-commerce Lead generation Product Copilot [](https://www.voiceflow.com/apps/flowbridge)  Certified App ### FlowBridge - Launch Voiceflow Agents to Instagram and Messenger FlowBridge lets AI agencies seamlessly integrate and deliver AI Agents to clients across social messaging platforms.  FlowBridge Product Copilot Lead generation E-commerce Customer support PAID Community Featured integrations Connect the platforms and tools you use to Voiceflow ---------------------------------------------------- [Browse integrations](https://www.voiceflow.com/integrations) [\ \ ### Freshdesk](https://www.voiceflow.com/integrations/freshdesk) [\ \ ### OpenAI](https://www.voiceflow.com/integrations/openai) [\ \ ### Microsoft Teams](https://www.voiceflow.com/integrations/microsoft-teams) [\ \ ### X](https://www.voiceflow.com/integrations/x) [\ \ ### Zoho CRM](https://www.voiceflow.com/integrations/zoho-crm) Have something to share? ------------------------ Share your creation with over 250,000 other global Voiceflow users. [Share your resource](https://airtable.com/appCYnIC5ir7nKu8W/shra7TXU4NmWOhKZK) [Join the Voiceflow Discord](https://link.voiceflow.com/community)  --- # AI Agent Boosted a Toronto Clinic’s Monthly Revenue by $50K | Pathways [](https://www.voiceflow.com/pathways/ai-agent-boosted-a-toronto-clinics-monthly-revenue-by-50k#) AI agent boosted a Toronto clinic’s monthly revenue by $50K =========================================================== [\ \ Voiceflow Team\ \ at](https://www.voiceflow.com/contributors/voiceflow-team) #### **Key Numbers** * **+$50,000**: Additional monthly revenue generated. * **400**: Conversations resolved by the AI agent monthly. * **30 hours**: Time saved by staff monthly. Bilal, the founder of [Helpline Hero](https://helpline-hero.com/) —AI Automation Agency—successfully pitched the idea of a Voiceflow-powered AI customer experience to a Toronto medical clinic that specializes in treatments for musculoskeletal issues. Bilal knew that this AI approach could alleviate administrative burdens, enhance patient interactions, and transform the clinic’s customer experience. The decision to implement the AI agent stemmed from Bilal’s own experience as a patient at the clinic. He explained, “They had two receptionists that were continually answering the same questions over and over again. I was like, ‘There’s definitely a way that we can use AI here to save these receptionists’ time.’” **The results were remarkable.** “We put the bot on the website and within the first three to four weeks, it engaged in 30 hours of conversations with specific clients. After analyzing the data, we found that the bot directly contributed to approximately $50,000 in additional monthly revenue.” Bilal explained.  3 most common inefficiencies for healthcare organizations ---------------------------------------------------------- As the clinic expanded its clientele, the demand for immediate support became critical. The clinic faced three main pain points: 1. **Need for Immediate, 24/7 Support**: Patients often sought assistance outside regular business hours. Bilal explained, “About 50% of the inquiries overall that were coming in were coming in outside of business hours… A lot of people were coming in either super early morning on weekends or super late at night. And those are people who would have not been serviced otherwise, right? Because nobody’s picking up the phone.” 2. **Inefficiencies in Appointment Booking**: The traditional appointment booking process was limited by staff availability. If all the receptionists are busy, then “nobody’s allowing them to… get into the pipeline to book appointments.” These inefficiencies create missed opportunities for patient care and revenue. 3. **Difficulty in Managing Reviews and Feedback**: Handling both positive and negative reviews can be challenging for clinics. Happy patients often forget to post reviews, whereas angry patients tend to leave long reviews. This disparity can hurt the clinic’s discoverability and online reputation. “If you have good reviews, you want to highlight them. If you have negative reviews, you want to be able to deal with them internally,” Bilal emphasized. How an AI-Powered Chatbot Transformed Patient Support ----------------------------------------------------- **To address these pain points, Helpline Hero implemented an AI-powered chatbot using Voiceflow.** The custom agent streamlined the clinic’s customer support and delivered revenue and CSAT results. ### 24/7 Support and Revenue Growth **The AI agent allowed the clinic to convert inquiries into appointments and revenue around the clock.** This constant availability meant that potential patients could receive immediate answers to their questions, such as, “Can PRP \[Platelet Rich Plasma Treatment\] help me?” and could be booked in immediately to “strike while the iron is hot.” As a result, the clinic experienced an additional $50,000 in monthly revenue, which was directly attributed to the chatbot’s ability to handle inquiries and facilitate bookings. ### Accurate Answers Powered By a Knowledge Base **Bilal used Voiceflow to incorporate a comprehensive knowledge base that draws frequently asked questions from the clinic’s website and internal documentation, enabling the chatbot to answer patients’ questions accurately.** For instance, when a patient asks, “I tore my MCL, can you help me?” the AI agent can respond immediately. Thanks to its contextual awareness and the use of Retrieval-Augmented Generation (RAG), the chatbot retrieves relevant information swiftly and accurately, ensuring that patients receive timely and precise responses to their inquiries. ### More Patient Feedback and Positive Reviews **A standout feature that Bilal developed for the AI chatbot is its ability to encourage patients to leave reviews on Google, which in turn enhances the clinic’s online reputation and boosts website traffic.** Firstly, Bilal implemented a systematic approach to solicit feedback from patients regarding their experiences. He noted, “If they rated it as one, two, three, or four stars, we would handle that feedback internally. But if it was five stars, we encouraged them to leave a public review.” Secondly, for those patients who reported positive experiences, the chatbot would prompt them to provide specific feedback. Bilal explained, “We actually use AI to generate the review for them. We ask for their name and one positive thing about their experience. The AI takes that information and creates a review.” Finally, the AI chatbot confirms with the patient, asking if the generated review looks good. If it does, they can simply click “yes,” which directs them straight to the Google review page where they can paste their feedback. This automated approach to encouraging reviews not only bolsters the clinic’s online reputation but also increases the clinic’s website traffic without incurring additional marketing costs. Bilal shared that when they started, the clinic was receiving about 2,500 website visitors a month, and now they’re seeing “4,500 to 5,000 a month,” with the increases “coming 100% from this chatbot.” How Does the Healthcare AI Agent Protect Patient Privacy? --------------------------------------------------------- One of the primary concerns when implementing AI solutions in healthcare is patient privacy. **Bilal addressed these concerns by ensuring that the AI agent would not collect or store any personal information from patients.** He emphasized, “We’re answering general questions using information that is already available on your website. We’re not storing any patient data.” This commitment to privacy was crucial, especially given the stringent regulations surrounding patient data. **By directing inquiries back to the website, where patients could find the necessary referral forms, Bilal streamlined the process while maintaining confidentiality.** He explained, “This clinic doesn’t allow you to just book in directly; you need to get a referral.” This approach not only eased the administrative burden on staff but also established a secure framework for patient interactions, allowing them to engage confidently with the AI agent without fear of compromising their privacy. Why Bilal Needed to Switch From Botpress to Voiceflow ----------------------------------------------------- Bilal initially explored various AI chatbot providers, including Botpress. However, he encountered complexities that made the process cumbersome, stating, “They required you to interact with some other middleware, and it was complicated.” After testing Voiceflow’s build experience and developer tooling, he decided to make the switch. Impressed by the developer flexibility that Voiceflow offered, Bilal recognized its support for both no-code and advanced integrations. He remarked, “Voiceflow makes it easy. I can send an API request to a webhook and then take care of it. Sometimes the clients are using Zapier, sometimes they’re using Make, sometimes they have their own APIs. You can interface with all of that directly. So that’s a really big plus.” **This clarity led him to conclude, “All right, I think Voiceflow is the better platform.”** Bilal summarized his experience using Voiceflow: “It’s a platform where you can easily build custom AI experiences with conversational agents.” This combination of user-friendliness and powerful developer features made Voiceflow the ideal choice for Bilal as he sought to enhance his AI agency’s capabilities. Generate revenue for your clients with a Voiceflow-powered agent. [Start building for free today.](https://voiceflow.com/signup) 3 most common inefficiencies for healthcare organizations ---------------------------------------------------------- As the clinic expanded its clientele, the demand for immediate support became critical. The clinic faced three main pain points: 1. **Need for Immediate, 24/7 Support**: Patients often sought assistance outside regular business hours. Bilal explained, “About 50% of the inquiries overall that were coming in were coming in outside of business hours… A lot of people were coming in either super early morning on weekends or super late at night. And those are people who would have not been serviced otherwise, right? Because nobody’s picking up the phone.” 2. **Inefficiencies in Appointment Booking**: The traditional appointment booking process was limited by staff availability. If all the receptionists are busy, then “nobody’s allowing them to… get into the pipeline to book appointments.” These inefficiencies create missed opportunities for patient care and revenue. 3. **Difficulty in Managing Reviews and Feedback**: Handling both positive and negative reviews can be challenging for clinics. Happy patients often forget to post reviews, whereas angry patients tend to leave long reviews. This disparity can hurt the clinic’s discoverability and online reputation. “If you have good reviews, you want to highlight them. If you have negative reviews, you want to be able to deal with them internally,” Bilal emphasized. How an AI-Powered Chatbot Transformed Patient Support ----------------------------------------------------- **To address these pain points, Helpline Hero implemented an AI-powered chatbot using Voiceflow.** The custom agent streamlined the clinic’s customer support and delivered revenue and CSAT results. ### 24/7 Support and Revenue Growth **The AI agent allowed the clinic to convert inquiries into appointments and revenue around the clock.** This constant availability meant that potential patients could receive immediate answers to their questions, such as, “Can PRP \[Platelet Rich Plasma Treatment\] help me?” and could be booked in immediately to “strike while the iron is hot.” As a result, the clinic experienced an additional $50,000 in monthly revenue, which was directly attributed to the chatbot’s ability to handle inquiries and facilitate bookings. ### Accurate Answers Powered By a Knowledge Base **Bilal used Voiceflow to incorporate a comprehensive knowledge base that draws frequently asked questions from the clinic’s website and internal documentation, enabling the chatbot to answer patients’ questions accurately.** For instance, when a patient asks, “I tore my MCL, can you help me?” the AI agent can respond immediately. Thanks to its contextual awareness and the use of Retrieval-Augmented Generation (RAG), the chatbot retrieves relevant information swiftly and accurately, ensuring that patients receive timely and precise responses to their inquiries. ### More Patient Feedback and Positive Reviews **A standout feature that Bilal developed for the AI chatbot is its ability to encourage patients to leave reviews on Google, which in turn enhances the clinic’s online reputation and boosts website traffic.** Firstly, Bilal implemented a systematic approach to solicit feedback from patients regarding their experiences. He noted, “If they rated it as one, two, three, or four stars, we would handle that feedback internally. But if it was five stars, we encouraged them to leave a public review.” Secondly, for those patients who reported positive experiences, the chatbot would prompt them to provide specific feedback. Bilal explained, “We actually use AI to generate the review for them. We ask for their name and one positive thing about their experience. The AI takes that information and creates a review.” Finally, the AI chatbot confirms with the patient, asking if the generated review looks good. If it does, they can simply click “yes,” which directs them straight to the Google review page where they can paste their feedback. This automated approach to encouraging reviews not only bolsters the clinic’s online reputation but also increases the clinic’s website traffic without incurring additional marketing costs. Bilal shared that when they started, the clinic was receiving about 2,500 website visitors a month, and now they’re seeing “4,500 to 5,000 a month,” with the increases “coming 100% from this chatbot.” How Does the Healthcare AI Agent Protect Patient Privacy? --------------------------------------------------------- One of the primary concerns when implementing AI solutions in healthcare is patient privacy. **Bilal addressed these concerns by ensuring that the AI agent would not collect or store any personal information from patients.** He emphasized, “We’re answering general questions using information that is already available on your website. We’re not storing any patient data.” This commitment to privacy was crucial, especially given the stringent regulations surrounding patient data. **By directing inquiries back to the website, where patients could find the necessary referral forms, Bilal streamlined the process while maintaining confidentiality.** He explained, “This clinic doesn’t allow you to just book in directly; you need to get a referral.” This approach not only eased the administrative burden on staff but also established a secure framework for patient interactions, allowing them to engage confidently with the AI agent without fear of compromising their privacy. Why Bilal Needed to Switch From Botpress to Voiceflow ----------------------------------------------------- Bilal initially explored various AI chatbot providers, including Botpress. However, he encountered complexities that made the process cumbersome, stating, “They required you to interact with some other middleware, and it was complicated.” After testing Voiceflow’s build experience and developer tooling, he decided to make the switch. Impressed by the developer flexibility that Voiceflow offered, Bilal recognized its support for both no-code and advanced integrations. He remarked, “Voiceflow makes it easy. I can send an API request to a webhook and then take care of it. Sometimes the clients are using Zapier, sometimes they’re using Make, sometimes they have their own APIs. You can interface with all of that directly. So that’s a really big plus.” **This clarity led him to conclude, “All right, I think Voiceflow is the better platform.”** Bilal summarized his experience using Voiceflow: “It’s a platform where you can easily build custom AI experiences with conversational agents.” This combination of user-friendliness and powerful developer features made Voiceflow the ideal choice for Bilal as he sought to enhance his AI agency’s capabilities. Generate revenue for your clients with a Voiceflow-powered agent. [Start building for free today.](https://voiceflow.com/signup)  RECOMMENDED [\ \ ### Money talks: How Sanlam Studios’ AI coach drives leads and financial literacy](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) PUBLISHED October 11, 2024 TOPICS IN THIS ARTICLE Exceptional work [\ \ Voiceflow Team\ \ at](https://www.voiceflow.com/contributors/voiceflow-team) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH No items found. [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/ai-agent-boosted-a-toronto-clinics-monthly-revenue-by-50k#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/ai-agent-boosted-a-toronto-clinics-monthly-revenue-by-50k#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Become a Voiceflow Expert [](https://www.voiceflow.com/) Partnership tiers ----------------- Certified Premium Revenue share Pro plans 20% on the first year of subscription 30% - on the first year of subscription Business plans 15% - on the first year of subscription 30% - on the first year of subscription Marketplace Experts portal listing  Priority Jobs board access   Community Private Discord channel   Features in our community newsletter  Priority Exclusive community events   Access to community bounties and competitions   Product and support Whitelabelling   Early beta-access to features   Monthly syncs and partner office hours .svg)  Priority support .svg)  Marketing Co-marketing .svg)  Co-hosting events   Voiceflow blog opportunities   Access to a digital badge kit   Partner Requirements General Qualification 1+ active paying customer **or** 1+ approved piece of content on our marketplace 3+ active paying customer **or** 3+ approved piece of content on our marketplace By interview Qualification to be added to the Experts Page .svg) At least level 5 in the Discord community By interview Workspace Plan Any plan Any plan Any plan Portfolio Website .svg)   How to Apply Fill out form Fill out form Fill out form ### Sandbox [Signup](https://creator.voiceflow.com/signup) ### Professional [Start trial](https://creator.voiceflow.com/signup) Build and collaborate Editors Single Editor Single Editor View-only collaborators Unlimited Unlimited Number of workspaces A workspace is the home for all of your assistants & added team members, and will allow you to organize your work in whatever way you choose.  1 1 Number of agents 2 20 Version History 7 days 30 days Exports None PDF, CSV, JSON Commenting   Real-time collaboration   Prototype and user test Agent Prototypes Voiceflow Branded Custom branding Test user personas 1 5 Password protected prototypes .svg) .svg) Prototype expiration 1 day Unlimited Launch and host Customizable chat embed   Dialog API   Analytics API .svg)  Transcripts API .svg)  Transcripts dashboard   Analytics dashboard   Transcript history 7 Days 30 Days Prototyping NLU VF NLU VF NLU Knowledge base sources per agent 50 200 NLU and LLMs Runtime NLU Configurable Configurable Large Language Models included ChatGPT All: OpenAI, Anthropic Bring your own LLM .svg) .svg) Security and compliance User permissions .svg)  Password Protected Prototype .svg)  SAML + Single Sign On (SSO) .svg) .svg) Private cloud hosting (GCP or AWS) .svg) .svg) Custom contracting and invoicing .svg) .svg) Custom SLA .svg) .svg) Support and services Community support forums   Voiceflow email and chat support .svg) .svg) Dedicated account manager .svg) .svg) Training .svg) .svg) Roadmap previews .svg) .svg) Migration support .svg) .svg) ### Teams [Contact Sales](https://creator.voiceflow.com/signup) ### Enterprise [Contact Sales](https://creator.voiceflow.com/signup) Build and collaborate Editors upto 3 editors Unlimited View-only collaborators Unlimited Add on Number of workspaces A workspace is the home for all of your assistants & added team members, and will allow you to organize your work in whatever way you choose.  20 Unlimited Number of agents Unlimited Unlimited Version History Unlimited Unlimited Exports PDF, CSV, JSON PDF, CSV, JSON Commenting   Real-time collaboration   Prototype and user test Agent Prototypes Custom branding Custom branding Test user personas Unlimited Unlimited Password protected prototypes   Prototype expiration Unlimited Unlimited Launch and host Customizable chat embed   Dialog API   Analytics API   Transcripts API   Transcripts dashboard   Analytics dashboard   Transcript history Unlimited Unlimited Prototyping NLU VF NLU Configurable Knowledge base sources per agent 1000 Unlimited NLU and LLMs Runtime NLU Configurable Configurable Large Language Models included All: OpenAI, Anthropic Any Model Bring your own LLM .svg)  Security and compliance User permissions   Password Protected Prototype   SAML + Single Sign On (SSO) .svg)  Private cloud hosting (GCP or AWS) .svg)  Custom contracting and invoicing .svg)  Custom SLA .svg)  Support and services Community support forums   Voiceflow email and chat support 12 hours or less 15 minutes or less Dedicated account manager Pooled support  Training .svg)  Roadmap previews .svg)  Migration support Paid add-on service Paid add-on service Partnership requirements ------------------------ To be accepted as a Certified Partner Apply [here](https://link.voiceflow.com/experts_apply) and submit: \- at least **3 paying customers \-** **1 client case study** \- Once approved - you'll **complete an interview** with the team To be accepted as a Premium Partner \- Successfully refer at least **5 paying clients** on Partnerstack \- Submit at least **5 case studies** on your profile \- Once approved - you'll **complete an interview** with the team Become a partner ---------------- Learn about our partner program and apply to join our growing network of AI professionals. [Apply Now](https://link.voiceflow.com/experts_apply)  --- # 5 principles for good natural language understanding (NLU) design | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) 5 principles for good natural language understanding (NLU) design ================================================================= [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) The technology behind NLU models is quite remarkable, but it’s not magic. Similar to building intuitive user experiences, or providing good onboarding to a person, a NLU requires clear communication and structure to be properly trained. To get started, you can use a few utterances off the top of your head, and that will typically be enough to run through simple prototypes. As you get ready to launch your conversational experience to your live audience, you need be specific and methodical. Your conversational assistant is an extension of the platform and brand it supports.  Bad NLU Design -------------- Let’s say we’re building a simple banking application, with two intents, check balance and manage credit card. They have the following utterances:  We can see a problem off the bat, both the check balance and manage credit card intent have a balance checker for the credit card! This will potentially confuse the NLU since we don’t have many examples. If we were thinking of it from UI perspective, imagine your bank app had two screens for checking your credit card balance. That might seem convenient at first, but what if you could only do an action from one of those screens! That would be annoying, the UX is no longer intuitive. Likewise in conversational design, activating a certain intent leads a user down a path, and if it’s the “wrong” path, it’s usually more cumbersome to navigate the a UI. We should be careful in our NLU designs, and while this spills into the the conversational design space, thinking about user behaviour is still fundamental to good NLU design. NLU Design Principles --------------------- In the past section we covered one example of bad NLU design of utterance overlap, and in this section we’ll discuss good NLU practices. Our end goal is to improve our conversational ai data quality. 1. Minimize utterance overlap 2. Intent Balance 3. Real world data 4. Setting confidence thresholds 5. Look for other patterns ### Minimize utterance overlap In the past section we started with a training dataset that had some overlap in two intents:  We want to solve two potential issues, confusing the NLU and confusing the user. We could do a couple of things to fix this language model. 1. Move the problematic utterance to a different intent 2. Rephrase your utterances to be more precise 3. Delete the utterance 4. Create a new intent Let’s try approach number one. We can move our _How much money do I owe on my credit card?_ utterance to the manage credit card intent.  This would reduce our confusion problem, but now potentially removes the purpose of our check balance intent. Perhaps we can try to redesign our intent. To do this we can make a list of actions we want to let the user do and what is the best way to reach there. We can divide our intents into reader and writer set. A reader intent would let the user retrieve information, but not take any action, and writer intent would let the user complete an action, but not see any information. With this structure our intents might look like this:  This looks cleaner now, but we have changed how are conversational assistant behaves! Sometimes when we notice that our NLU model is broken we have to change both the NLU model and the conversational design. It’s part of the iterative UX process. Our other two options, deleting and creating a new intent, give us more flexibility to re-arrange our data based on user needs. ### Intent balance To start this section off let’s take a gameshow example. Let’s say you’re invited to a gameshow where if you guess 9/10 questions right, you win a prize! This might sound challenging, but you’re told that that 95% of times the answer is A, what will your strategy be? To guess A all 10 times! You’d usually win games (91% of the time) with this strategy. Now if you train an NLU, and you give 9 example for intent A, and 1 for Intent B, how do you think your model will behave? It might just learn to guess Intent A, since it will be right 90% of the time!  This dataset distribution is known as a prior, and will affect how the NLU learns. Imbalanced datasets are a challenge for any machine learning model, with data scientists often going to great lengths to try to correct the challenge. So avoid this pain, use your prior understanding to balance your dataset. To measure the consequence of data unbalance we can use a measure called a F1 score. A F1 score provides a more holistic representation of how accuracy works. We won’t go into depth in this article but you can read more about it [here.](https://txt.cohere.ai/classification-eval-metrics/)  These scores are meant to illustrate how a simple NLU can get trapped with poor data quality. With better data balance, your NLU should be able to learn better patterns to recognize the differences between utterances. ### Real world data An important part of NLU training is making sure that your data reflects the context of where your conversational assistant is deployed. This might include the channel, demographic, region or social norm. Understanding your end user and analyzing live data will reveal key information that will help your assistant be more successful. If we are deploying a conversational assistant as part of a commercial bank, the tone of CA and audience will be much different than that of digital first bank app aimed for students. Likewise the language used in a Zara CA in Canada will be different than one in the UK. When testing your conversational assistant it’s important to monitor how the data changes and if your customer persona is well represented in your training data. Some factors to look out for include: 1. Tone 2. Formality 3. Grammer 4. Spelling/Typos 5. Slang and jargon 6. Bot domain 7. Comfort with technology You can make assumptions during initial stage, but after the conversational assistant goes live into beta and real world test, only then you’ll know how to compare performance. ### Setting Confidence thresholds When a conversational assistant is live, it will run into data it has never seen before. Even google sees 15% of it’s searches for the first time everyday! With new requests and utterances, the NLU may be less confident in its ability to classify intents, so setting confidence intervals will help you handle these situations.  A higher confidence interval will help you be more sure that a user says is what they mean. The downside is that the user might have to repeat themselves which leads to a frustrating experience. The alternative is to set a lower value and potentially direct the user down an unintended path. The complexity of your project will also affect what your confidence for an intent should be. If you have 100 intents and one scores 50% on confidence, that’s much better than intent that scores 50% when there are only two options!  ### Look for other patterns One of the magical properties of NLUs is their ability to pattern match and learn representations of things quickly and in a generalizable way. Whether you’re classifying apples and oranges or automotive intents, NLUs find a way to learn the task at hand. Sometimes, they learn patterns that weren’t what we expected. Let’s say we have two intents, yes and no with the utterances below.  Initially this data looks good, no overlap with the intents, no strange phrases. But there a couple hidden patterns here: 1. The no intent always starts with an “N” 2. The no intent only uses one word 3. The no intent has no punctuation With only a couple examples, the NLU might learn these patterns rather than the intended meaning! Depending on the NLU and the utterances used, you may run into this challenge. To address this challenge, you can create more robust examples, taking some of the patterns we noticed and mixing them in.  We now have more robust training dataset! While NLU choice is important, the data is being fed in will make or break your model. Conclusion ---------- In this section post we went through various techniques on how to improve the data for your conversational assistant. This process of NLU management is essential to train effective language models, and creating amazing customer experiences. > Ready to chat about your own NLU management workflow? [Chat with our team.](https://voiceflow.com/demo) ### NLU Management Terms **NLU**: Short for Natural Language Understanding. Commonly refers to a machine learning model that extracts intents and entities from a users phrase **NLUM**: Short for NLU Management. The process of managing the data and testing of your NLU. **Utterance Overlap/Conflict**: When utterances from different intents overlap in their meeting. This causes confusion for NLU models. **Intent Balance**: The ratio between the number of utterances per intent. Good intent balance means that each intent has a roughly equal distribution of member utterances. **NLU Accuracy**: The percentage of correct answers from a NLU. Often associated with intent classification. **F1 Score:** A more comprehensive metric to calculating NLU accuracy. Incorporates false negatives and false positives. A value between 0 to 1. **Confidence Thresholds:** A level that a NLU’s prediction needs to reach to be considered valid. A value between 0 to 1. Bad NLU Design -------------- Let’s say we’re building a simple banking application, with two intents, check balance and manage credit card. They have the following utterances:  We can see a problem off the bat, both the check balance and manage credit card intent have a balance checker for the credit card! This will potentially confuse the NLU since we don’t have many examples. If we were thinking of it from UI perspective, imagine your bank app had two screens for checking your credit card balance. That might seem convenient at first, but what if you could only do an action from one of those screens! That would be annoying, the UX is no longer intuitive. Likewise in conversational design, activating a certain intent leads a user down a path, and if it’s the “wrong” path, it’s usually more cumbersome to navigate the a UI. We should be careful in our NLU designs, and while this spills into the the conversational design space, thinking about user behaviour is still fundamental to good NLU design. NLU Design Principles --------------------- In the past section we covered one example of bad NLU design of utterance overlap, and in this section we’ll discuss good NLU practices. Our end goal is to improve our conversational ai data quality. 1. Minimize utterance overlap 2. Intent Balance 3. Real world data 4. Setting confidence thresholds 5. Look for other patterns ### Minimize utterance overlap In the past section we started with a training dataset that had some overlap in two intents:  We want to solve two potential issues, confusing the NLU and confusing the user. We could do a couple of things to fix this language model. 1. Move the problematic utterance to a different intent 2. Rephrase your utterances to be more precise 3. Delete the utterance 4. Create a new intent Let’s try approach number one. We can move our _How much money do I owe on my credit card?_ utterance to the manage credit card intent.  This would reduce our confusion problem, but now potentially removes the purpose of our check balance intent. Perhaps we can try to redesign our intent. To do this we can make a list of actions we want to let the user do and what is the best way to reach there. We can divide our intents into reader and writer set. A reader intent would let the user retrieve information, but not take any action, and writer intent would let the user complete an action, but not see any information. With this structure our intents might look like this:  This looks cleaner now, but we have changed how are conversational assistant behaves! Sometimes when we notice that our NLU model is broken we have to change both the NLU model and the conversational design. It’s part of the iterative UX process. Our other two options, deleting and creating a new intent, give us more flexibility to re-arrange our data based on user needs. ### Intent balance To start this section off let’s take a gameshow example. Let’s say you’re invited to a gameshow where if you guess 9/10 questions right, you win a prize! This might sound challenging, but you’re told that that 95% of times the answer is A, what will your strategy be? To guess A all 10 times! You’d usually win games (91% of the time) with this strategy. Now if you train an NLU, and you give 9 example for intent A, and 1 for Intent B, how do you think your model will behave? It might just learn to guess Intent A, since it will be right 90% of the time!  This dataset distribution is known as a prior, and will affect how the NLU learns. Imbalanced datasets are a challenge for any machine learning model, with data scientists often going to great lengths to try to correct the challenge. So avoid this pain, use your prior understanding to balance your dataset. To measure the consequence of data unbalance we can use a measure called a F1 score. A F1 score provides a more holistic representation of how accuracy works. We won’t go into depth in this article but you can read more about it [here.](https://txt.cohere.ai/classification-eval-metrics/)  These scores are meant to illustrate how a simple NLU can get trapped with poor data quality. With better data balance, your NLU should be able to learn better patterns to recognize the differences between utterances. ### Real world data An important part of NLU training is making sure that your data reflects the context of where your conversational assistant is deployed. This might include the channel, demographic, region or social norm. Understanding your end user and analyzing live data will reveal key information that will help your assistant be more successful. If we are deploying a conversational assistant as part of a commercial bank, the tone of CA and audience will be much different than that of digital first bank app aimed for students. Likewise the language used in a Zara CA in Canada will be different than one in the UK. When testing your conversational assistant it’s important to monitor how the data changes and if your customer persona is well represented in your training data. Some factors to look out for include: 1. Tone 2. Formality 3. Grammer 4. Spelling/Typos 5. Slang and jargon 6. Bot domain 7. Comfort with technology You can make assumptions during initial stage, but after the conversational assistant goes live into beta and real world test, only then you’ll know how to compare performance. ### Setting Confidence thresholds When a conversational assistant is live, it will run into data it has never seen before. Even google sees 15% of it’s searches for the first time everyday! With new requests and utterances, the NLU may be less confident in its ability to classify intents, so setting confidence intervals will help you handle these situations.  A higher confidence interval will help you be more sure that a user says is what they mean. The downside is that the user might have to repeat themselves which leads to a frustrating experience. The alternative is to set a lower value and potentially direct the user down an unintended path. The complexity of your project will also affect what your confidence for an intent should be. If you have 100 intents and one scores 50% on confidence, that’s much better than intent that scores 50% when there are only two options!  ### Look for other patterns One of the magical properties of NLUs is their ability to pattern match and learn representations of things quickly and in a generalizable way. Whether you’re classifying apples and oranges or automotive intents, NLUs find a way to learn the task at hand. Sometimes, they learn patterns that weren’t what we expected. Let’s say we have two intents, yes and no with the utterances below.  Initially this data looks good, no overlap with the intents, no strange phrases. But there a couple hidden patterns here: 1. The no intent always starts with an “N” 2. The no intent only uses one word 3. The no intent has no punctuation With only a couple examples, the NLU might learn these patterns rather than the intended meaning! Depending on the NLU and the utterances used, you may run into this challenge. To address this challenge, you can create more robust examples, taking some of the patterns we noticed and mixing them in.  We now have more robust training dataset! While NLU choice is important, the data is being fed in will make or break your model. Conclusion ---------- In this section post we went through various techniques on how to improve the data for your conversational assistant. This process of NLU management is essential to train effective language models, and creating amazing customer experiences. > Ready to chat about your own NLU management workflow? [Chat with our team.](https://voiceflow.com/demo) ### NLU Management Terms **NLU**: Short for Natural Language Understanding. Commonly refers to a machine learning model that extracts intents and entities from a users phrase **NLUM**: Short for NLU Management. The process of managing the data and testing of your NLU. **Utterance Overlap/Conflict**: When utterances from different intents overlap in their meeting. This causes confusion for NLU models. **Intent Balance**: The ratio between the number of utterances per intent. Good intent balance means that each intent has a roughly equal distribution of member utterances. **NLU Accuracy**: The percentage of correct answers from a NLU. Often associated with intent classification. **F1 Score:** A more comprehensive metric to calculating NLU accuracy. Incorporates false negatives and false positives. A value between 0 to 1. **Confidence Thresholds:** A level that a NLU’s prediction needs to reach to be considered valid. A value between 0 to 1.  RECOMMENDED [\ \ ### 17 prompts for building AI apps in Voiceflow](https://www.voiceflow.com/pathways/prompts-for-building-ai-apps) PUBLISHED June 28, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/dominant-discourse-llms) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Why the dominant discourse surrounding LLMs needs to change](https://www.voiceflow.com/pathways/dominant-discourse-llms) [](https://www.voiceflow.com/pathways/building-voiceflows-machine-learning-platform-from-scratch) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### How we're building Voiceflow’s machine learning platform from scratch](https://www.voiceflow.com/pathways/building-voiceflows-machine-learning-platform-from-scratch) [](https://www.voiceflow.com/pathways/voiceflow-api) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### What is the Voiceflow API and how do you use it?](https://www.voiceflow.com/pathways/voiceflow-api) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/5-principles-for-good-natural-language-understanding-nlu-design#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/5-principles-for-good-natural-language-understanding-nlu-design#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 5 User Personas to Consider in Conversation Design | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 5 user personas you should account for in your conversation designs =================================================================== [\ \ Sara Pion\ \ Senior Marketing Manager at](https://www.voiceflow.com/contributors/sara-pion) Recently, we launched [User Personas](https://www.voiceflow.com/blog/design-and-prototype-production-ready-conversational-experiences-with-user-personas) — the best way to design conversations for every user. Designing conversational experiences for multiple users can feel daunting, which is why we always recommend building your [happy path first,](https://www.voiceflow.com/glossary/happy-path) then building off of that. Building differentiated experiences for each persona ensures that your conversational assistant is able to contain as many interactions as possible, facilitate more personalized conversations, and create a better user experience when users are having conversations with your business. Let’s dive into some experiences you should be including in your conversation designs that account for different user personas.  User Personas To Design Conversational Experiences Around --------------------------------------------------------- Every business’ conversation designs are going to be specific to their business, the team leveraging conversational experiences, and the maturity of the organization when it comes to conversation design. However there are some common scenarios that can be accounted for in your conversation designs to make them more personalized. ### First Time vs Returning Users When you’re a regular at a coffee shop, the baristas start to remember who you are and what your order is, and create a personalized experience for you every time you return to their store. Your conversational assistant should work in the same way. Even a simple “Welcome back” greeting for users who have interacted with your conversational experience in the past can add a layer of customization to the experience.  ### Existing Customer vs Anonymous User If you are building an experience that has user de-anonymization, then creating custom experiences for existing customers versus new users is imperative. Identifying users through your internal systems, asking for credentials, or account information in the beginning of your conversational experience can ensure that you’re answering user questions with appropriate information and, if applicable, using customer data to facilitate the resolution of issues. Being able to thoroughly test how your assistant will create different experiences for future customers versus existing customers will make your conversational assistant more powerful and more focused.  ### Different Product SKU Experiences Regardless of if you’re designing a conversational experience that’s for answering FAQs, facilitating payments, or customer service, if your business offers different product SKUs, trying to create a conversational experience that answers the same question across product SKUs could get complex. Asking a user which product they’re interested in towards the beginning of the conversation is a great way to make sure you’re giving contextual and relevant information to users based on their intent and interest.  ### Online vs Offline Availability for Live Human handoff If your team staffs your conversational assistant, regardless of channel whether it be IVR, SMS, website chat, or a mobile experience, and that team is only available during certain times of day, ensuring that you have logic set up in your AI assistant to ensure that you’re setting the right expectations for those operating hours.  ### Business Logic Scenarios Although conversation designers create [happy paths](https://www.voiceflow.com/glossary/happy-path) with the best case scenarios for each interaction, your conversational assistants should be able to perform multiple different actions for users engaging with different intents. Creating individual happy paths for each path your users could go down will create a more robust user experience that increases containment of conversations, answers more questions, and creates more satisfied users.  An Experience For Every Persona ------------------------------- The design process for a well done conversational assistant isn’t linear and doesn’t happen over night. By focusing on different scenarios and personas that could interact with your assistant before or during the design process, companies will be adding [more context to their conversational bots](https://www.voiceflow.com/blog/6-ways-to-build-context-into-your-conversation-designs) , make user testing a lot more informative to your design process, and add more bots into the world that create better user experiences. To learn more about how to use User Personas by Voiceflow, [check out this blog post](https://www.voiceflow.com/blog/design-and-prototype-production-ready-conversational-experiences-with-user-personas) . User Personas To Design Conversational Experiences Around --------------------------------------------------------- Every business’ conversation designs are going to be specific to their business, the team leveraging conversational experiences, and the maturity of the organization when it comes to conversation design. However there are some common scenarios that can be accounted for in your conversation designs to make them more personalized. ### First Time vs Returning Users When you’re a regular at a coffee shop, the baristas start to remember who you are and what your order is, and create a personalized experience for you every time you return to their store. Your conversational assistant should work in the same way. Even a simple “Welcome back” greeting for users who have interacted with your conversational experience in the past can add a layer of customization to the experience.  ### Existing Customer vs Anonymous User If you are building an experience that has user de-anonymization, then creating custom experiences for existing customers versus new users is imperative. Identifying users through your internal systems, asking for credentials, or account information in the beginning of your conversational experience can ensure that you’re answering user questions with appropriate information and, if applicable, using customer data to facilitate the resolution of issues. Being able to thoroughly test how your assistant will create different experiences for future customers versus existing customers will make your conversational assistant more powerful and more focused.  ### Different Product SKU Experiences Regardless of if you’re designing a conversational experience that’s for answering FAQs, facilitating payments, or customer service, if your business offers different product SKUs, trying to create a conversational experience that answers the same question across product SKUs could get complex. Asking a user which product they’re interested in towards the beginning of the conversation is a great way to make sure you’re giving contextual and relevant information to users based on their intent and interest.  ### Online vs Offline Availability for Live Human handoff If your team staffs your conversational assistant, regardless of channel whether it be IVR, SMS, website chat, or a mobile experience, and that team is only available during certain times of day, ensuring that you have logic set up in your AI assistant to ensure that you’re setting the right expectations for those operating hours.  ### Business Logic Scenarios Although conversation designers create [happy paths](https://www.voiceflow.com/glossary/happy-path) with the best case scenarios for each interaction, your conversational assistants should be able to perform multiple different actions for users engaging with different intents. Creating individual happy paths for each path your users could go down will create a more robust user experience that increases containment of conversations, answers more questions, and creates more satisfied users.  An Experience For Every Persona ------------------------------- The design process for a well done conversational assistant isn’t linear and doesn’t happen over night. By focusing on different scenarios and personas that could interact with your assistant before or during the design process, companies will be adding [more context to their conversational bots](https://www.voiceflow.com/blog/6-ways-to-build-context-into-your-conversation-designs) , make user testing a lot more informative to your design process, and add more bots into the world that create better user experiences. To learn more about how to use User Personas by Voiceflow, [check out this blog post](https://www.voiceflow.com/blog/design-and-prototype-production-ready-conversational-experiences-with-user-personas) .  RECOMMENDED [](https://www.voiceflow.com/pathways/5-user-personas-you-should-account-for-in-your-conversation-designs#) PUBLISHED April 13, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Sara Pion\ \ Senior Marketing Manager at\ \ Senior Marketing Manager, Bot Builder, Middle Sister](https://www.voiceflow.com/contributors/sara-pion) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-to-get-started-as-a-cx-designer) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to get started as a CX Designer](https://www.voiceflow.com/pathways/how-to-get-started-as-a-cx-designer) [](https://www.voiceflow.com/pathways/cxd-center-of-excellence) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Your CxD center of excellence (and why you don’t need one right now)](https://www.voiceflow.com/pathways/cxd-center-of-excellence) [](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Expanding the definition of conversation design](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/5-user-personas-you-should-account-for-in-your-conversation-designs#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/5-user-personas-you-should-account-for-in-your-conversation-designs#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 5 tips for better multimodal design | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 5 tips for better multimodal design =================================== [\ \ Elaine Anzaldo\ \ Conversation Designer at NLX at](https://www.voiceflow.com/contributors/elaine-anzaldo) It’s a multimodal world out there. As humans, we absorb information in a variety of ways. Our 5 physical senses (i.e. sight, hearing, smell, taste, touch) all work together to build our understanding of the world and the realities we face on a daily basis. They are also pretty handy when it comes to helping us reinforce certain behaviors or discourage us from repeating mistakes. A classic example of this is associating an open flame— its shape, color, heat, & crackle— with the danger zone. Put your hand too close and you might burn yourself! The more we experience these combinations of inputs (our physical senses reporting on that open flame to our brain), the more real that experience becomes, and the less we try to set ourselves on fire. Most of our interactions follow that same pattern. We get information about our environment through different inputs, to then produce an output, or reaction, to that information. [Multimodal](https://www.voiceflow.com/glossary/multimodal-experience) design is applying that same essential principle of how people navigate their world to designing machine-to-human interactive experiences. It’s still UX design, but instead of designing for one modality, you have to juggle 2 or more, while also taking into consideration the pros and cons of each.  **What is multimodal conversation design?** ------------------------------------------- We know what single-modality interactions feel like. Appreciating a fine work of art or quickly scanning a new drive-thru menu for the tastiest picture both have in common that you’re generally only relying on one of the physical senses: sight. It doesn’t take away our life experience to only use one of our senses. It’s something we do all the time. But if there were an option to Instagram “like” something in real life, wouldn’t we want to tap that button? Multimodal design aims to make life easier by giving users natural shortcuts through an interaction via different modalities. The aim of a multimodal conversational experience is to present and receive information in an intuitive way. Basically, you design multimodal conversational flows like how they would happen in a real life conversation. This means switching across contexts, capabilities, or even devices. ### **💡 A clarification: multimodal vs. multichannel** Multimodal interactions may include features that allow a user to start an experience in one communication channel and continue in another. For example, smart TVs allow for easy viewing of photo galleries or videos from a conversation that may have started with a voice-only medium (e.g. smart speaker). The golden rule is that not all multimodal are multichannel, but multichannel experiences are multimodal. While there is no hard and fast method of designing multimodal experiences, the concept itself has been at least thought about in the tech industry since the 90’s. Intentionally designed experiences work, while others can immediately come off as distracting or overwhelming. **Here are 5 of my rules of thumb I use to design multimodal conversational experiences:** 1. When in doubt, draw it out 2. Context, context, context 3. Eliminate the competition 4. Stay consistent, everywhere 5. Prioritize accessibility **When in doubt, draw it out** ------------------------------ Multimodal conversational experiences have a lot of moving parts and it can be easy to lose sight of the journey as a whole. Writing prompts without taking the accompanying visuals into consideration can be kind of like buying a lottery ticket and leaving it unscratched: your design will have all the potential in the world, but no one’s gonna take it if you try to cash it in. An easy way to keep the conversational flow in perspective is by making flowcharts with the dialog and [wireframes](https://en.wikipedia.org/wiki/Website_wireframe) (mockups that show the general placement of content on a screen) or finalized visuals side-by-side. There are others in the industry who also like sketching storyboards (like how they do in Hollywood!). This method not only gives you insight into the timing of what a user might see or hear, but also lets you place the conversation within its context. ### **Context, context, context** [Context in multimodal design is essential](https://www.voiceflow.com/blog/6-ways-to-build-context-into-your-conversation-designs) . Knowing where your users are while they progress through different steps of the journey can reveal pain points or opportunities in your design, _especially_ if the journey requires switching between devices (for more information on this, I highly recommend [Cheryl Platz’s book](https://rosenfeldmedia.com/books/design-beyond-devices/) ). It can help you understand the advantages of revealing or taking in information through one modality over another. Generally, you don’t want your user to reveal sensitive information orally if they have the option to type it out into a form on a screen. But there’s more to it than that. Keeping yourself informed of your user’s context can even stop you from forcing features on your users that are simply not feasible— like how some ecommerce websites have a checkout timer that empties your cart if you don’t get your payment information in before the time runs out. Consider your user in their environment to keep the conversation relevant. ### **Eliminate the competition** When everything in a design competes for a user’s attention, nothing wins. Too many screaming elements in a journey can actually push the feel of the UX into gimmicky territory. As designers, we need to remember that very few things about our designed products actually live “rent free” in our users’ minds. Keep in mind that there is an order to how people absorb information. In visual design, UI elements are intentionally placed according to how the human eye would scan the page. For English speakers, that usually means content will be scanned from top to bottom, left to right. In multimodal design, where the audible, visible, and tactile compete for attention, it’s less straight-forward. While it is true that humans read more words faster than they can hear them, you may not always want to prioritize speed. Depending on the use case, you may choose to wield audio for branding to leave an impression and set the tone more clearly than repeatedly making users absorb the visual and written elements on a screen (this is one of the reasons why [sonic branding](https://en.wikipedia.org/wiki/Sound_trademark) is having its moment). Each modality has its advantages. The key lies in emphasizing one at a time. **Stay consistent, everywhere** Visuals usually come with text. Very rarely do you see a website out in the wild with NO text at all. That’s just bananas, and [super inaccessible](https://www.w3.org/WAI/fundamentals/accessibility-intro/) . All written copy on a screen should be purposefully written in a way that aligns with the journey objective. Say, for example, you design a bot personality to be youthful, fun, and curious. If you insert a microcopy into your design that is usually more professional in mood (think “Submit”, “Next”, or “Forgot password”), it would completely misalign with the personality you designed! Think of the copy like an extension of the conversation. Like when a memorable video comes up in conversation and you just _have to_ search for it on your phone to show to a friend. You don’t change your entire personality to share something with your friend. If you did, your friend might start considering hanging out with you less. Sloppy, inconsistent copy can breed a user’s distrust. Stay consistent, even in the little details. ### **Prioritize accessibility** The power of multimodal experiences is huge. Technology in this form can now reduce friction for more people and include more people in the conversation. However, as we all learned with Spider-Man, with great power comes great responsibility (I’m sure Uncle Ben had multimodal designers in mind when he said that). We need to start talking more about accessibility at the design stage. It sets the tone not only for how conversations are shaped, but also how easy they are to use. As the lead conversation designer for [Voice Compass](https://voicecompass.ai/) ®, it’s my responsibility to make sure we’re asking the right questions at the beginning of our design process, and to find out if we’re accidentally excluding users with our product. Part of it includes knowing how to write alt-tags for our images or realizing that visual loading screens don’t need an alt-tag, but would require an aural notification. The bigger picture of it is determining whether all end users of a multimodal journey can easily complete a given task from A to B without major roadblocks. Design can be delightful, yes, but it should also be usable for all kinds of people. **A multimodal future** ----------------------- I believe we’re heading toward a future where multimodal is the norm and no longer a buzzword. Beyond the smart home, I envision a smart city, with smart neighborhoods that let you opt in to automated experiences based on more inputs than the ones we see used today. Even now, It’s an exciting time to work in multimodal design. Designers may not have all the answers or agree on what makes **great** multimodal design, but there are now enough resources to get started. Voiceflow has allowed for multimodal [prototyping](https://youtu.be/NaTWweLlWfA) , and more industry professionals are sharing their learnings, like the new [Conversation with Things](https://rosenfeldmedia.com/books/conversations-with-things/) book by Diana Deibel and Rebecca Evanhoe. It’s a multimodal world out there and it finally seems like our technology is catching up. > **Elaine Anzaldo** is an avid member of the voice community and an expert conversation designer at NLX. You can connect with her via [Twitter](https://twitter.com/elaineinthebay) > , [LinkedIn](https://www.linkedin.com/in/Elaine%20Anzaldo/) > , or check out her [portfolio here](https://elaineinthebay.netlify.app/) > . **What is multimodal conversation design?** ------------------------------------------- We know what single-modality interactions feel like. Appreciating a fine work of art or quickly scanning a new drive-thru menu for the tastiest picture both have in common that you’re generally only relying on one of the physical senses: sight. It doesn’t take away our life experience to only use one of our senses. It’s something we do all the time. But if there were an option to Instagram “like” something in real life, wouldn’t we want to tap that button? Multimodal design aims to make life easier by giving users natural shortcuts through an interaction via different modalities. The aim of a multimodal conversational experience is to present and receive information in an intuitive way. Basically, you design multimodal conversational flows like how they would happen in a real life conversation. This means switching across contexts, capabilities, or even devices. ### **💡 A clarification: multimodal vs. multichannel** Multimodal interactions may include features that allow a user to start an experience in one communication channel and continue in another. For example, smart TVs allow for easy viewing of photo galleries or videos from a conversation that may have started with a voice-only medium (e.g. smart speaker). The golden rule is that not all multimodal are multichannel, but multichannel experiences are multimodal. While there is no hard and fast method of designing multimodal experiences, the concept itself has been at least thought about in the tech industry since the 90’s. Intentionally designed experiences work, while others can immediately come off as distracting or overwhelming. **Here are 5 of my rules of thumb I use to design multimodal conversational experiences:** 1. When in doubt, draw it out 2. Context, context, context 3. Eliminate the competition 4. Stay consistent, everywhere 5. Prioritize accessibility **When in doubt, draw it out** ------------------------------ Multimodal conversational experiences have a lot of moving parts and it can be easy to lose sight of the journey as a whole. Writing prompts without taking the accompanying visuals into consideration can be kind of like buying a lottery ticket and leaving it unscratched: your design will have all the potential in the world, but no one’s gonna take it if you try to cash it in. An easy way to keep the conversational flow in perspective is by making flowcharts with the dialog and [wireframes](https://en.wikipedia.org/wiki/Website_wireframe) (mockups that show the general placement of content on a screen) or finalized visuals side-by-side. There are others in the industry who also like sketching storyboards (like how they do in Hollywood!). This method not only gives you insight into the timing of what a user might see or hear, but also lets you place the conversation within its context. ### **Context, context, context** [Context in multimodal design is essential](https://www.voiceflow.com/blog/6-ways-to-build-context-into-your-conversation-designs) . Knowing where your users are while they progress through different steps of the journey can reveal pain points or opportunities in your design, _especially_ if the journey requires switching between devices (for more information on this, I highly recommend [Cheryl Platz’s book](https://rosenfeldmedia.com/books/design-beyond-devices/) ). It can help you understand the advantages of revealing or taking in information through one modality over another. Generally, you don’t want your user to reveal sensitive information orally if they have the option to type it out into a form on a screen. But there’s more to it than that. Keeping yourself informed of your user’s context can even stop you from forcing features on your users that are simply not feasible— like how some ecommerce websites have a checkout timer that empties your cart if you don’t get your payment information in before the time runs out. Consider your user in their environment to keep the conversation relevant. ### **Eliminate the competition** When everything in a design competes for a user’s attention, nothing wins. Too many screaming elements in a journey can actually push the feel of the UX into gimmicky territory. As designers, we need to remember that very few things about our designed products actually live “rent free” in our users’ minds. Keep in mind that there is an order to how people absorb information. In visual design, UI elements are intentionally placed according to how the human eye would scan the page. For English speakers, that usually means content will be scanned from top to bottom, left to right. In multimodal design, where the audible, visible, and tactile compete for attention, it’s less straight-forward. While it is true that humans read more words faster than they can hear them, you may not always want to prioritize speed. Depending on the use case, you may choose to wield audio for branding to leave an impression and set the tone more clearly than repeatedly making users absorb the visual and written elements on a screen (this is one of the reasons why [sonic branding](https://en.wikipedia.org/wiki/Sound_trademark) is having its moment). Each modality has its advantages. The key lies in emphasizing one at a time. **Stay consistent, everywhere** Visuals usually come with text. Very rarely do you see a website out in the wild with NO text at all. That’s just bananas, and [super inaccessible](https://www.w3.org/WAI/fundamentals/accessibility-intro/) . All written copy on a screen should be purposefully written in a way that aligns with the journey objective. Say, for example, you design a bot personality to be youthful, fun, and curious. If you insert a microcopy into your design that is usually more professional in mood (think “Submit”, “Next”, or “Forgot password”), it would completely misalign with the personality you designed! Think of the copy like an extension of the conversation. Like when a memorable video comes up in conversation and you just _have to_ search for it on your phone to show to a friend. You don’t change your entire personality to share something with your friend. If you did, your friend might start considering hanging out with you less. Sloppy, inconsistent copy can breed a user’s distrust. Stay consistent, even in the little details. ### **Prioritize accessibility** The power of multimodal experiences is huge. Technology in this form can now reduce friction for more people and include more people in the conversation. However, as we all learned with Spider-Man, with great power comes great responsibility (I’m sure Uncle Ben had multimodal designers in mind when he said that). We need to start talking more about accessibility at the design stage. It sets the tone not only for how conversations are shaped, but also how easy they are to use. As the lead conversation designer for [Voice Compass](https://voicecompass.ai/) ®, it’s my responsibility to make sure we’re asking the right questions at the beginning of our design process, and to find out if we’re accidentally excluding users with our product. Part of it includes knowing how to write alt-tags for our images or realizing that visual loading screens don’t need an alt-tag, but would require an aural notification. The bigger picture of it is determining whether all end users of a multimodal journey can easily complete a given task from A to B without major roadblocks. Design can be delightful, yes, but it should also be usable for all kinds of people. **A multimodal future** ----------------------- I believe we’re heading toward a future where multimodal is the norm and no longer a buzzword. Beyond the smart home, I envision a smart city, with smart neighborhoods that let you opt in to automated experiences based on more inputs than the ones we see used today. Even now, It’s an exciting time to work in multimodal design. Designers may not have all the answers or agree on what makes **great** multimodal design, but there are now enough resources to get started. Voiceflow has allowed for multimodal [prototyping](https://youtu.be/NaTWweLlWfA) , and more industry professionals are sharing their learnings, like the new [Conversation with Things](https://rosenfeldmedia.com/books/conversations-with-things/) book by Diana Deibel and Rebecca Evanhoe. It’s a multimodal world out there and it finally seems like our technology is catching up. > **Elaine Anzaldo** is an avid member of the voice community and an expert conversation designer at NLX. You can connect with her via [Twitter](https://twitter.com/elaineinthebay) > , [LinkedIn](https://www.linkedin.com/in/Elaine%20Anzaldo/) > , or check out her [portfolio here](https://elaineinthebay.netlify.app/) > .  RECOMMENDED [](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design#) PUBLISHED June 23, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Elaine Anzaldo\ \ Conversation Designer at NLX at\ \ Elaine is a Pathways contributor.](https://www.voiceflow.com/contributors/elaine-anzaldo) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-to-get-started-as-a-cx-designer) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to get started as a CX Designer](https://www.voiceflow.com/pathways/how-to-get-started-as-a-cx-designer) [](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Experts share 3 expectations about LLMs to set with business stakeholders](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders) [](https://www.voiceflow.com/pathways/interested-in-cxd-here-are-the-5-best-conversation-design-tools) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Interested in CxD? Here are the best conversation design tools](https://www.voiceflow.com/pathways/interested-in-cxd-here-are-the-5-best-conversation-design-tools) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # The 6 major differences between mobile apps and voice apps | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) The 6 major differences between mobile apps and voice apps ========================================================== [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) In 2019 there are over 4 million mobile apps split between two dominant ecosystems: iOS and Android. However, there’s a new platform shift happening right now. Similar to when mobile was introduced, it won’t replace the previous platform but instead add a new set of use cases to interact with technology. I’m of course talking about the AI assistants, also called voice assistants because they primarily use a voice interface. Voice is not what makes these assistants unique, but it is instead the enabling interface — similar to the touch screen and iPhone. When raising investment for Voiceflow we were consistently asked what the “killer app” for “voice” was (voice being synonymous for the assistant). It made me realize there’s a fundamental misunderstanding to what voice platforms, and voice apps, are and aren’t. My goal is to highlight the 8 key differences between mobile and voice apps. Neither platform is superior, the same as web and mobile, but they are certainly very, very different.  ### [](https://medium.com/@bradenream?source=post_page-----37c20ed810bf----------------------) 1\. Trained vs natural interface -------------------------------- #### **_Mobile apps_** Mobile interfaces are set by designers on behalf of machines and learned by humans. The first time you pick up a smartphone you learn how to navigate its interface and experience. Because of this the best mobile User Interfaces (UI) are uniform in general structure but add unique twists where they can. If your UI is too far from the norm, then people have a hard time learning how to use it. This leads to many mobile apps to follow the conventions and norms set out by the industry. Every now and then a new generally accepted design convention arises, but broadly speaking it’s all the same. #### **_Voice apps_** [Voice user interfaces (VUIs)](https://www.voiceflow.com/glossary/voice-user-interface-vui) are different because it’s all about teaching computers how to talk to humans, and that’s a huge shift. The best VUI should be so intuitive that anyone can start using it without knowing what the voice app even does. The way humans talk is topic driven and often spontaneous. Think about the last time you greeted someone you know — odds are it was very different each time, even if the topic was different. Because of this, VUIs must be anything but uniform. VUIs must be as spontaneous and free-flowing as humans whilst retaining their ability to converse over defined topics. In the UI world, this would be akin to changing the color and position of a login button every time the user returns to keep the “conversation” fresh. ### 2\. Complex vs Simple to build #### _Mobile apps_ Mobile apps are costly to design and costly to build well because they have a lot of complexity, and are relatively large projects. This complexity makes mobile apps very hard to build. #### Voice apps The lack of visuals and complex frontend makes voice apps very straightforward, and easy to build. The heavy lifting of natural language processing and understanding is all performed by the platforms. What people forget about when it comes to voice apps however is design is the most important element. It’s easy to build a conversation but it is very hard to design a good one. The industry is still figuring out a loosely-held set of voice interface standards. ### 3\. Device vs cloud-based #### **_Mobile apps_** Mobile apps are heavy. Every single mobile app requires you to download an app which takes up space on your phone, and time to download. Because of this, people have fewer apps on their phone. The average number of mobile apps used per month according to App Annie in 2017 was 40. Personally, I was shocked to find it was 40 as I legitimately use less than 10. #### **_Voice apps_** Voice apps are light because they are cloud-based. This means they don’t live on the device itself but are accessed whenever needed from the cloud. This approach dramatically reduces the amount of effort the user has to contemplate when deciding to use a service. In theory, a user can “have” as many apps as on their voice assistants as they have questions for the assistant. ### 4\. Deep vs fast engagement #### **_Mobile apps_** People in North America spend on average of 45-minutes per day on their smart phones. This is because apps on smartphones are meant to have deep engagement that can entertain or help the user for a long duration of time. Paired with the heavy nature of mobile apps (see point above) and we come to the conclusion that mobile apps are meant to be few in number on your phone, but highly engaging. #### **_Voice apps_** Voice apps are not meant to be engaging but highly functional and fast. When ordering a coffee from Starbucks, you don’t want to have a conversation — you just want a coffee. Because of this, and the fact that voice apps are cloud-based points to voice apps being vast in number and shallow in engagement. You want to be able to order from any coffee shop you see through your voice, but you don’t want to have a conversation with the said coffee shop. ### **5\. Explicit vs Implicit discovery** #### **_Mobile apps_** Mobile apps are discovered through stores like the iOS app store and the Android Google Play store. This marketplace approach makes sense for these platforms because the apps are highly engaging. Usually, mobile apps have to compete or market externally in order to get people to download them. This model works however because the app is highly engaging, and thus highly monetizable, once it has been downloaded. #### **_Voice apps_** Voice app discovery is broken right now (mid-2019). Voice apps are light and shallow in functionality but wide in number. The problem is the current discovery system is explicit the same as an app store — you have to know what you want and find it by name. This manifests within the Alexa Skill store and Google Action store. However, this approach is completely backward and has caused many critics to push back on voice harder than they realistically should. The future of voice app discovery is implicit discovery where you state your intention and the platform discovers a voice app to serve your need. An example would be asking to go to the airport and your voice assistant finding you a ridesharing app appropriate for your needs. When this shift from explicit to implicit happens, and it’s already happening slowly, then voice apps won’t be seen as broken mobile apps but instead as nodes of functionality to serve users within the voice controlled and personalized search engines that are Alexa and Google Assistant. ### 6\. SMBs don’t need vs (might) need one #### Mobile apps Every big consumer-facing brand today has a mobile app, and sometimes several. There was a time when every business was expected to have a voice app with small pizzerias being sold by agencies on the idea of “the new website”. However, that myth has been generally debunked thanks to the help of aggregator platforms like Yelp and Google — and time. Most small businesses need a landing page that makes it accessible by search (even less so with Google businesses today). However, given the explicit discovery mechanics of mobile where you have to know what you’re looking for, mobile apps didn’t serve as discovery, and most small businesses don’t have any reason to have a deeply engaging mobile app either. A landing page suffices to solve all these needs and more. In short, not everyone needs a mobile app. #### Voice apps More businesses will need voice apps in the short term to be accessible by voice search once it shifts to explicit. However, there are two factors that remain to impact the outcome: ### 1\. The role of aggregators on AI assistants It could likely be the case that aggregators like Yelp and DoorDash or the platforms themselves act as the conversational bridge between small businesses and consumers. For example, you could ask Alexa to order you a large pizza from the shop down the street and pending the shop has filled in their information on Alexa’s database, they would be able to fulfill the order. This would be a heavy lift for Alexa however and would likely fall into the hands of aggregation platforms like DoorDash and Yelp who already have existing relationships with these small businesses. There is a future where for some reason these aggregators do not fulfill this need. In that case, there is room in the market for a “Wix of voice” to help small businesses build voice apps, but I find that unlikely. ### 2\. How fast explicit discovery shifts to implicit discovery Imagine if today Google Assistant performed a search query and returned you the perfect third party voice app every time you asked for something you needed. This is the world of implicit discovery, however, it seems to be further away than many hope. Until implicit discovery is solved, there is no point in being a small business with a voice app if your goal is to attract customers. There are use cases for small businesses we’ve seen where it’s great to build a voice app to retain and engage existing customers. An example we’ve seen at Voiceflow is a meditation studio creating a series of take-home lessons for existing customers to practice at home on their voice assistants. That’s a fantastic use case and a great idea for a small business. Building highly verticalized functional use cases for voice is the only reason a small business should build-out a voice app in 2019 because it certainly won’t bring any additional customers because of the lack of implicit discovery. ### Outro Voice apps are often ridiculed on the benchmark of mobile apps. They have shallow engagement, poor discovery, and even poorer retention. However, this ridicule is silly because it’s too early to know what voice apps will eventually become. In the early days of mobile when smartphones were benchmarked against desktops, they were considered to be underpowered and difficult to use for anything past a simple email. We now know this to be a dated opinion. We think the same way about voice apps. It’s early days, but there are certainly shining examples of a platform shift just getting started. ### [](https://medium.com/@bradenream?source=post_page-----37c20ed810bf----------------------) 1\. Trained vs natural interface -------------------------------- #### **_Mobile apps_** Mobile interfaces are set by designers on behalf of machines and learned by humans. The first time you pick up a smartphone you learn how to navigate its interface and experience. Because of this the best mobile User Interfaces (UI) are uniform in general structure but add unique twists where they can. If your UI is too far from the norm, then people have a hard time learning how to use it. This leads to many mobile apps to follow the conventions and norms set out by the industry. Every now and then a new generally accepted design convention arises, but broadly speaking it’s all the same. #### **_Voice apps_** [Voice user interfaces (VUIs)](https://www.voiceflow.com/glossary/voice-user-interface-vui) are different because it’s all about teaching computers how to talk to humans, and that’s a huge shift. The best VUI should be so intuitive that anyone can start using it without knowing what the voice app even does. The way humans talk is topic driven and often spontaneous. Think about the last time you greeted someone you know — odds are it was very different each time, even if the topic was different. Because of this, VUIs must be anything but uniform. VUIs must be as spontaneous and free-flowing as humans whilst retaining their ability to converse over defined topics. In the UI world, this would be akin to changing the color and position of a login button every time the user returns to keep the “conversation” fresh. ### 2\. Complex vs Simple to build #### _Mobile apps_ Mobile apps are costly to design and costly to build well because they have a lot of complexity, and are relatively large projects. This complexity makes mobile apps very hard to build. #### Voice apps The lack of visuals and complex frontend makes voice apps very straightforward, and easy to build. The heavy lifting of natural language processing and understanding is all performed by the platforms. What people forget about when it comes to voice apps however is design is the most important element. It’s easy to build a conversation but it is very hard to design a good one. The industry is still figuring out a loosely-held set of voice interface standards. ### 3\. Device vs cloud-based #### **_Mobile apps_** Mobile apps are heavy. Every single mobile app requires you to download an app which takes up space on your phone, and time to download. Because of this, people have fewer apps on their phone. The average number of mobile apps used per month according to App Annie in 2017 was 40. Personally, I was shocked to find it was 40 as I legitimately use less than 10. #### **_Voice apps_** Voice apps are light because they are cloud-based. This means they don’t live on the device itself but are accessed whenever needed from the cloud. This approach dramatically reduces the amount of effort the user has to contemplate when deciding to use a service. In theory, a user can “have” as many apps as on their voice assistants as they have questions for the assistant. ### 4\. Deep vs fast engagement #### **_Mobile apps_** People in North America spend on average of 45-minutes per day on their smart phones. This is because apps on smartphones are meant to have deep engagement that can entertain or help the user for a long duration of time. Paired with the heavy nature of mobile apps (see point above) and we come to the conclusion that mobile apps are meant to be few in number on your phone, but highly engaging. #### **_Voice apps_** Voice apps are not meant to be engaging but highly functional and fast. When ordering a coffee from Starbucks, you don’t want to have a conversation — you just want a coffee. Because of this, and the fact that voice apps are cloud-based points to voice apps being vast in number and shallow in engagement. You want to be able to order from any coffee shop you see through your voice, but you don’t want to have a conversation with the said coffee shop. ### **5\. Explicit vs Implicit discovery** #### **_Mobile apps_** Mobile apps are discovered through stores like the iOS app store and the Android Google Play store. This marketplace approach makes sense for these platforms because the apps are highly engaging. Usually, mobile apps have to compete or market externally in order to get people to download them. This model works however because the app is highly engaging, and thus highly monetizable, once it has been downloaded. #### **_Voice apps_** Voice app discovery is broken right now (mid-2019). Voice apps are light and shallow in functionality but wide in number. The problem is the current discovery system is explicit the same as an app store — you have to know what you want and find it by name. This manifests within the Alexa Skill store and Google Action store. However, this approach is completely backward and has caused many critics to push back on voice harder than they realistically should. The future of voice app discovery is implicit discovery where you state your intention and the platform discovers a voice app to serve your need. An example would be asking to go to the airport and your voice assistant finding you a ridesharing app appropriate for your needs. When this shift from explicit to implicit happens, and it’s already happening slowly, then voice apps won’t be seen as broken mobile apps but instead as nodes of functionality to serve users within the voice controlled and personalized search engines that are Alexa and Google Assistant. ### 6\. SMBs don’t need vs (might) need one #### Mobile apps Every big consumer-facing brand today has a mobile app, and sometimes several. There was a time when every business was expected to have a voice app with small pizzerias being sold by agencies on the idea of “the new website”. However, that myth has been generally debunked thanks to the help of aggregator platforms like Yelp and Google — and time. Most small businesses need a landing page that makes it accessible by search (even less so with Google businesses today). However, given the explicit discovery mechanics of mobile where you have to know what you’re looking for, mobile apps didn’t serve as discovery, and most small businesses don’t have any reason to have a deeply engaging mobile app either. A landing page suffices to solve all these needs and more. In short, not everyone needs a mobile app. #### Voice apps More businesses will need voice apps in the short term to be accessible by voice search once it shifts to explicit. However, there are two factors that remain to impact the outcome: ### 1\. The role of aggregators on AI assistants It could likely be the case that aggregators like Yelp and DoorDash or the platforms themselves act as the conversational bridge between small businesses and consumers. For example, you could ask Alexa to order you a large pizza from the shop down the street and pending the shop has filled in their information on Alexa’s database, they would be able to fulfill the order. This would be a heavy lift for Alexa however and would likely fall into the hands of aggregation platforms like DoorDash and Yelp who already have existing relationships with these small businesses. There is a future where for some reason these aggregators do not fulfill this need. In that case, there is room in the market for a “Wix of voice” to help small businesses build voice apps, but I find that unlikely. ### 2\. How fast explicit discovery shifts to implicit discovery Imagine if today Google Assistant performed a search query and returned you the perfect third party voice app every time you asked for something you needed. This is the world of implicit discovery, however, it seems to be further away than many hope. Until implicit discovery is solved, there is no point in being a small business with a voice app if your goal is to attract customers. There are use cases for small businesses we’ve seen where it’s great to build a voice app to retain and engage existing customers. An example we’ve seen at Voiceflow is a meditation studio creating a series of take-home lessons for existing customers to practice at home on their voice assistants. That’s a fantastic use case and a great idea for a small business. Building highly verticalized functional use cases for voice is the only reason a small business should build-out a voice app in 2019 because it certainly won’t bring any additional customers because of the lack of implicit discovery. ### Outro Voice apps are often ridiculed on the benchmark of mobile apps. They have shallow engagement, poor discovery, and even poorer retention. However, this ridicule is silly because it’s too early to know what voice apps will eventually become. In the early days of mobile when smartphones were benchmarked against desktops, they were considered to be underpowered and difficult to use for anything past a simple email. We now know this to be a dated opinion. We think the same way about voice apps. It’s early days, but there are certainly shining examples of a platform shift just getting started.  RECOMMENDED [.avif)\ \ ### 10 things every voice app should do](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) PUBLISHED October 23, 2019 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.avif)](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 10 things every voice app should do](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) [](https://www.voiceflow.com/pathways/breaking-nike-chatbot) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Breaking and rebuilding Nike's chatbot](https://www.voiceflow.com/pathways/breaking-nike-chatbot) [](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/6-differences-between-mobile-apps-and-voice-apps#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/6-differences-between-mobile-apps-and-voice-apps#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 10 things every voice app should do | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 10 things every voice app should do =================================== [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at](https://www.voiceflow.com/contributors/mark-ammendolia) During a recent Voiceflow webinar, Alexa Evangelist [Jeff Blankenburg](https://www.linkedin.com/in/jeffblankenburg/) highlighted 10 essential things you should be thinking about any time you build a voice experience. This article neatly summarizes Jeff's thoughts to help creators, developers and teams alike simplify their designs, handle unexpected errors, and enhance user experience. ### More on Jeff Blankenburg: _Over the past 15 years, Jeff has enthusiastically applied his technical expertise to build industry-changing websites and marketing efforts fo mega brands including Victoria's Secret, Abercrombie & Fitch, Ford Motor Company, Sony, and several pharmaceutical companies. He’s especially proficient in user interface design, web standards and mobile application development. Today, he educates developers on the Alexa platform through technical workshops, hackathons, conferences, live streams, and meet-ups. His content - including blogs, tutorials, videos, and live-builds on twitch - have helped nurture a robust developer community that continues to rapidly accelerate and expand across the globe. (source: Linkedin)_ #### Full recording below: .avif) **What you'll learn by reading this:** 1\. Do one thing really well 2\. Make your name memorable 3\. Focus on intents, not commands 4\. Simplify choices 5\. Use the one-breath test 6\. Include a variety of responses 7\. Handle the unexpected gracefully 8\. Make enhancements based on data 9\. Provide contextual help 10\. Beta test with real users 1\. Focus on doing one thing really well ---------------------------------------- **Jeff**: An example of this that I like to use is a skill I built called _Games Back._ Obviously, we don't have a lot of sports right now, but with baseball, one of the things that I always care about each morning \[...\] is the standings and how my team is doing. My team happens to be the Cleveland Indians, and so I jump in and want to see how many games back they are. That's the column that I really care about more than anything else. If they won or lost last night - that's great and all - but what really matters is how many games back from first \[they are\]. That's the kind of thing that I'm looking for. So what I did is I built a skill that specifically gives me that one piece of information. So if you have an Alexa device at home, you could say, "_hey Alexa, ask Games Back about the Cleveland Indians," and she'll_ tell you what the standings were at the end of last season \[since\] the data hasn't reset yet \[and\] we don't have a season this year. The idea \[here\] is that I'm doing that one thing. I'm not telling you about wins and losses or when their next game is or how they did yesterday. It's just focused-in on that one simple task. You could continue to add things to it, but this is the core idea. We really want to do one thing really, really well. 2\. **Make your name memorable** -------------------------------- **Jeff:** This is kind of a fun story to tell. There's a great skill in the Alexa skill store called _The Magic Door_. The way they advertise their skill \[is they have\] a door as their icon, and they tell people to say, "open the magic door." \[When you do so\], you hear this creeky old door open, where you then enter into their adventure. The key to this is if you think about any app on your phone — there's an icon and a name. And as I'm flipping through my phone, I can easily find what I'm looking for because I have it in a list. With voice, we don't have that luxury. So we need to be specific and focused on the question, "how do people remember to use our stuff?" These are some example names \[which\] I made up. You can see that there are lots of different ways to refer to a skill. So I might say, "run" or "resume" or "load" or "launch." All of these words mean the same thing; it's just a matter of how you tie them together with whatever your name is to allow the user to find your skill easily.  Source: Jeff Blankenburg So, "launch Rocket Ship Stories" might make a whole lot of sense or "open Box Maker," - if that's the name of your skill. In each of these cases, you can play with this to find the launch word that matches up with what you're trying to accomplish. I \[created\] a skill that gives you three clues, all of which have something in common. Then you have to figure out what those things are. And so it's a trivia game \[that\] challenges your mind. And so I thought — the word 'trivia' has the letters t-r-i in front. Maybe I should use that \[in the name\]. So I thought about flipping the words and using 'Viatri' which \[happens to mean\] "by way of three" — which I thought was a really clever name for it. I realized very quickly that no one's going to remember this name. \[It's\] not memorable. It's not useful. It's not something that is going to stick. I need \[users\] to remember how to play my game, or they're not going to. And so ultimately, I came up with the idea to call it_Three Clues —_ which made a whole lot more sense and makes it very, very easy for people to remember what the \[app\] is. "_It's the game that gives you the three clues_!" It's really simple. So think carefully about your name. It's very tempting — especially when you have a startup or a small business — to have some crazy name. Most of the time, starting from scratch, having a memorable name can be hard. So you want to think deeply about that. 3\. Focus on _intents_, not _commands_. --------------------------------------- **Jeff:** Let me explain what I mean by that. I'm working on a skill right now called _Dev Tips,_ \[and\] it's meant for Alexa developers to be able to get answers to all their questions about Alexa's skill-building but in a meta-way. \[For example, you can\] hear the sound effects available in the library or \[find out\] all the speechcons you can use. I want you to be able to experience \[...\] all of the different kinds of sound effects and audible things inside an Alexa skill \[...\] without having to build something. And so I started with commands like, "ask Dev Tips about monetization," or "tell me about persistence." And what I realized quickly is that I have a database of all of these terms, but the users don't. They don't know that monetization is a thing they can ask about. They \[also\] don't know that persistence is in \[that\] list. And so while these things do work, I also had to enable things like, "teach me something new," or "play a speechcon for me," \[or\] "what should I learn next?" By giving those kinds of capabilities inside my skill, it makes it easy for somebody to explore. You can't assume that \[the user\] knows what to say or that they know what your content is. It's much easier to give commands like this, so that they can continue to move and grow inside your skill. 4. **Simplify choices** ----------------------- **Jeff:** The reason for this is because we don't often think about a conversation in a deep, meaningful way. And if you think about it, most conversations are a back and forth, bouncing between each other. Maybe \[there are\] interruptions. Sometimes there are questions. But when you \[do\] ask that question, there's a variety of answers you could give, right? So what are all the questions we can ask, and how should we think about asking those questions? The big one for me is we often tend to lead with open-ended questions. These \[examples\] are NOT those: _"Is there something else I can help you with?" "Do you have another questions?" "Would you like to know something else?" _The answers to all of those questions are yes or no — those are the two cases. If they say yes, you're going to end up saying one of these: _"What else would you like to know about?" "What else can I help you with?" "What topic can I assist you with?"_ You don't want to\] ask yes or no questions unless it absolutely makes sense. Instead, ask \[open-ended\] questions — like in the examples above — that get \[the user\] to the point where they know exactly what they want to do. Imagine I have a fruit stand, and I've built a skill for it where I say things like, "we have apples, bananas, oranges, lemons, grapes, kiwis, blackberries, strawberries, and mangoes." And then \[after\] I say, "which fruit do you want?" The challenge with asking a question this way is that we've given \[the user\] a very long list, and it's hard to figure out exactly which \[fruit\] I might want. By the time you say mangoes, I've already forgotten that bananas are in this list. In addition to that, I asked the question afterward, and because of this, \[the user\] doesn't know that you were asking \[them\] to pick one. \[Initially\], you were just telling \[them\] stuff and \[so\] they weren't paying as much attention. Let's limit our choices and ask the question first. So \[instead\] say, "what fruit do you want — apples, bananas, or oranges?" You could probably also tweak this to say apples, bananas, oranges, or something else. And if they say something else, or if they say 'grapes' — then great! \[You can then\] go execute what they want to do with \[there choice being\] grapes. 5\. Use the One-Breath Test --------------------------- **Jeff:** Here's a really good example of this: I built a Star Wars skill that has lots and lots of content about everything you could imagine in the Star Wars universe. \[When I added\] Luke Skywalker to my content, I found a really good description of \[him\]: _Luke Skywalker was a Tatoonie farmboy who rose from humble beginnings to become one of the great Jedi the galaxy has ever known. Along with his friends Leia and Han Solo, Luke battled the evil empire, discovered the truth of his parentage, and ended the tyranny of the Sith. A generation later, the location of the famed Jedi master was one of the galaxy's greatest mysteries. _\[This description\] is really long, and so a good rule of thumb is if you can't say it in one breath — it's too long. This description is probably two or three breaths. What I did instead as an example, is that if you ask the skill about Darth Vader, \[Alexa\] says, "Darth Vader is a bad, bad, bad, man." Very simple. Very to-the-point. A little humor in there. And so, that's the thing that we're really trying to accomplish. If people want more, you could offer that, but I wouldn't lead with tons of content. You often lose people's attention. 6\. **Include a variety of responses** -------------------------------------- **Jeff:** This is something that is really important to me, and something I do all over the place. When we build applications for mobile or the web, you often find specific rules that they want you to follow. \[For example, you have\] the iOS style guide \[and\] the Android style guide. They \[use concepts\] like predictable and consistent, and those things are 100 percent the opposite of what we want voice to be.  Source: Jeff Blankenburg With voice, it needs to be unpredictable. It needs to be something that you are engaged with and that you are actually listening to or choosing to play along with. And so what I like to do, is keep everything fresh and random. For every single thing that my skill can say, I like to have 5 to 7 versions of that thing. You can do this with everything. Vary the order \[or\] the content you're using by keeping people on their toes and not letting it be predictable. You'll find that they stay engaged and that they're participating more in conversation with their voice system. The other thing I like to do is randomize the order that you present things. For example, if you're building something for a travel agency, there are five things you need to allow the user to make a flight search: origin, destination, airline, departure date, and return date. _Where are you flying from?_ _Where are you flying to?_ _What airline do you want to use?_ _When are you flying?_ _When do you want to return? _Now you may always ask the questions in this order, but what's to say that the second time they come back \[and complete this\], we don't \[use the same order\]. It doesn't matter what the order of these things is. We know that we want all five of those values. So by mixing it up, it doesn't feel automated. It doesn't feel like I'm filling out a form with my voice. It feels like I'm having a conversation with someone, and I'm just answering their questions. So this is another really good example of how we want to think about randomizing the order that we do things, and not only the order that we say things. 7\. Handle the unexpected gracefully ------------------------------------ **Jeff:** One example of these is errors. Errors are going to happen from time to time. Maybe an API is down. Maybe you built something that didn't work properly. \[Although\] Alexa has default error messages \[such as\], "there was a problem with the requested skill's response," you should control and manage these to make it seem like something accidentally happened \[and that\] you're handling it. Here's an example of that: _ "It seems our trivia questions are better than our software developers. Something is broken, but we've alerted our team to the problem. Can I offer you a random trivia question instead?" _So, the trivia questions stuff is working, but whatever they were trying to do is broken, so let's instead think about how we redirect them - maybe adding a little humour to it \[as well\]. And then just carry \[the conversation\] back to where they were going. \[This\] is something really important to think about, but not something people pay a lot of attention to. There's always opportunities for things to go wrong, and having some fun with it makes a big difference. \[Another\] example of this is my baseball skill. If I came in and asked about the Cleveland Indians, the New York Yankees, or the San Francisco Giants — my skills were ready for all of that. But what if they say the New York Jets. What do I do then? I don't have the New York Jets in my database \[because it is a football team\]. I don't want to throw an error. I need to do something with this information. So what I do in my case is I say, "I'm sorry, the New York Jets aren't a baseball team. Is there a different team I can tell you about?" I'm redirecting them \[instead\] of saying, "I don't know what to do." So this is a really good example of how you can redirect people to the thing they should be doing. 8\. Make enhancements based on data -------------------------------------- **Jeff:** Publishing a skill is not the end. It's more or less the beginning. As you're building, you'll see that you can collect a whole bunch of data about what's going on inside your skill. What are people saying? What are people asking for? Anytime that I'm collecting data from a user, I always persist that to a database so that I can compare later and say, "what are people asking my skill for that I'm not accommodating." By parsing through that data and looking at it in a deep way, we can say, "oh, we should be fixing this!" Another good example is intents. If we think about my _Dev Tip_ skill, we have a _get news intent_, we have an _answer intent_, we have a _launch request_ \[which is where\] they start. And \[I\] can see that the _get news_ intent seems to be the most popular \[as appose\] to the _display template_ intent \[which is\] very infrequently used. So when I ask myself, what am I going to build next? It should be focused on what people use the most and how I can continue to provide value to my users. 9\. Provide contextual help --------------------------- **Jeff:** This is something that we all skip over, but it's always required. We have to provide help inside of our skill. We often do things like this: _"This skill lets you order from our big menu of pizzas, breadsticks, pasta, sandwiches, wings, and desserts. What would you like to do?"_ All I've done here is given \[the user\] another opportunity to ask for something. I haven't actually helped them. If they were in the middle of ordering a pizza and weren't sure how to get double sauce, the above example doesn't give them any of that assistance \[if they ask for help\]. What help should look like is this: _"It looks like you're trying to order a pizza. You can add or remove toppings by saying things like add pepperoni or remove anchovies. You can also add extras like sauces, extra cheese, or our different crusts. Just ask for them! Do you want to get back to your pizza, or do you need help with something else?" _It's possible that they didn't need help with the thing they were doing - and in that case - you could try to redirect them to the appropriate kind of help that they're looking for. Help is super important because people get lost. There's no UI. There are no visual clues to let them know that you're supposed to click this button or go over here. They're only using their voices and sometimes they just can't figure out exactly what that thing should be. 10\. Beta test with REAL users --------------------------------- **Jeff:** I can't emphasize this enough. If you're not getting your skill out in front of real people, then you're never going to feel like it's as successful as it could have been. This is because each of us has a \[different\] idea about how people speak. We need to get other people in front of us and talking to our skill. We need to help us understand what's going on. And what you really want to do is get people you know to use this stuff. Invite your friends and your family — people that will give you open and honest feedback. A lot of times, if you open it up to people you met online or a large mailing list, very few people will tell you anything. They may use your skill a couple of times, but they're probably not going to give you the kind of meaningful feedback you're looking for. If you're working in your offices on voice stuff, the thing I recommend is taking the_Wizard of Oz_ approach. There's two lessons to be learned here. The first is that if you're trying to build something that replicates a mobile app or a website, a good rule of thumb is that if it takes more than three clicks to get to, and is a place the user wants to go regularly, should think about building it for voice. So I always think about the three clicks on the Ruby Slippers (Wizard of Oz) as a way to remember that those are the things we should really be focusing on. Conversely, if they can get to it \[quickly, or in less than 3 clicks\] by opening your app, that's probably going to be faster. So it's something to keep in mind. As far as testing goes, you should have somebody behind the curtain — like the wizard \[in The Wizard of Oz\], who knows what the skill should do. Then, you have people come up and talk to the curtain so they can't see any facial expressions or body language. They ask the skill to do things, and the skill — or the person behind the curtain in this case — responds in the way that the skill is programmed to. You'll learn very quickly how people want to interact with this content and how they want to interact with this skill. **What you'll learn by reading this:** 1\. Do one thing really well 2\. Make your name memorable 3\. Focus on intents, not commands 4\. Simplify choices 5\. Use the one-breath test 6\. Include a variety of responses 7\. Handle the unexpected gracefully 8\. Make enhancements based on data 9\. Provide contextual help 10\. Beta test with real users 1\. Focus on doing one thing really well ---------------------------------------- **Jeff**: An example of this that I like to use is a skill I built called _Games Back._ Obviously, we don't have a lot of sports right now, but with baseball, one of the things that I always care about each morning \[...\] is the standings and how my team is doing. My team happens to be the Cleveland Indians, and so I jump in and want to see how many games back they are. That's the column that I really care about more than anything else. If they won or lost last night - that's great and all - but what really matters is how many games back from first \[they are\]. That's the kind of thing that I'm looking for. So what I did is I built a skill that specifically gives me that one piece of information. So if you have an Alexa device at home, you could say, "_hey Alexa, ask Games Back about the Cleveland Indians," and she'll_ tell you what the standings were at the end of last season \[since\] the data hasn't reset yet \[and\] we don't have a season this year. The idea \[here\] is that I'm doing that one thing. I'm not telling you about wins and losses or when their next game is or how they did yesterday. It's just focused-in on that one simple task. You could continue to add things to it, but this is the core idea. We really want to do one thing really, really well. 2\. **Make your name memorable** -------------------------------- **Jeff:** This is kind of a fun story to tell. There's a great skill in the Alexa skill store called _The Magic Door_. The way they advertise their skill \[is they have\] a door as their icon, and they tell people to say, "open the magic door." \[When you do so\], you hear this creeky old door open, where you then enter into their adventure. The key to this is if you think about any app on your phone — there's an icon and a name. And as I'm flipping through my phone, I can easily find what I'm looking for because I have it in a list. With voice, we don't have that luxury. So we need to be specific and focused on the question, "how do people remember to use our stuff?" These are some example names \[which\] I made up. You can see that there are lots of different ways to refer to a skill. So I might say, "run" or "resume" or "load" or "launch." All of these words mean the same thing; it's just a matter of how you tie them together with whatever your name is to allow the user to find your skill easily.  Source: Jeff Blankenburg So, "launch Rocket Ship Stories" might make a whole lot of sense or "open Box Maker," - if that's the name of your skill. In each of these cases, you can play with this to find the launch word that matches up with what you're trying to accomplish. I \[created\] a skill that gives you three clues, all of which have something in common. Then you have to figure out what those things are. And so it's a trivia game \[that\] challenges your mind. And so I thought — the word 'trivia' has the letters t-r-i in front. Maybe I should use that \[in the name\]. So I thought about flipping the words and using 'Viatri' which \[happens to mean\] "by way of three" — which I thought was a really clever name for it. I realized very quickly that no one's going to remember this name. \[It's\] not memorable. It's not useful. It's not something that is going to stick. I need \[users\] to remember how to play my game, or they're not going to. And so ultimately, I came up with the idea to call it_Three Clues —_ which made a whole lot more sense and makes it very, very easy for people to remember what the \[app\] is. "_It's the game that gives you the three clues_!" It's really simple. So think carefully about your name. It's very tempting — especially when you have a startup or a small business — to have some crazy name. Most of the time, starting from scratch, having a memorable name can be hard. So you want to think deeply about that. 3\. Focus on _intents_, not _commands_. --------------------------------------- **Jeff:** Let me explain what I mean by that. I'm working on a skill right now called _Dev Tips,_ \[and\] it's meant for Alexa developers to be able to get answers to all their questions about Alexa's skill-building but in a meta-way. \[For example, you can\] hear the sound effects available in the library or \[find out\] all the speechcons you can use. I want you to be able to experience \[...\] all of the different kinds of sound effects and audible things inside an Alexa skill \[...\] without having to build something. And so I started with commands like, "ask Dev Tips about monetization," or "tell me about persistence." And what I realized quickly is that I have a database of all of these terms, but the users don't. They don't know that monetization is a thing they can ask about. They \[also\] don't know that persistence is in \[that\] list. And so while these things do work, I also had to enable things like, "teach me something new," or "play a speechcon for me," \[or\] "what should I learn next?" By giving those kinds of capabilities inside my skill, it makes it easy for somebody to explore. You can't assume that \[the user\] knows what to say or that they know what your content is. It's much easier to give commands like this, so that they can continue to move and grow inside your skill. 4. **Simplify choices** ----------------------- **Jeff:** The reason for this is because we don't often think about a conversation in a deep, meaningful way. And if you think about it, most conversations are a back and forth, bouncing between each other. Maybe \[there are\] interruptions. Sometimes there are questions. But when you \[do\] ask that question, there's a variety of answers you could give, right? So what are all the questions we can ask, and how should we think about asking those questions? The big one for me is we often tend to lead with open-ended questions. These \[examples\] are NOT those: _"Is there something else I can help you with?" "Do you have another questions?" "Would you like to know something else?" _The answers to all of those questions are yes or no — those are the two cases. If they say yes, you're going to end up saying one of these: _"What else would you like to know about?" "What else can I help you with?" "What topic can I assist you with?"_ You don't want to\] ask yes or no questions unless it absolutely makes sense. Instead, ask \[open-ended\] questions — like in the examples above — that get \[the user\] to the point where they know exactly what they want to do. Imagine I have a fruit stand, and I've built a skill for it where I say things like, "we have apples, bananas, oranges, lemons, grapes, kiwis, blackberries, strawberries, and mangoes." And then \[after\] I say, "which fruit do you want?" The challenge with asking a question this way is that we've given \[the user\] a very long list, and it's hard to figure out exactly which \[fruit\] I might want. By the time you say mangoes, I've already forgotten that bananas are in this list. In addition to that, I asked the question afterward, and because of this, \[the user\] doesn't know that you were asking \[them\] to pick one. \[Initially\], you were just telling \[them\] stuff and \[so\] they weren't paying as much attention. Let's limit our choices and ask the question first. So \[instead\] say, "what fruit do you want — apples, bananas, or oranges?" You could probably also tweak this to say apples, bananas, oranges, or something else. And if they say something else, or if they say 'grapes' — then great! \[You can then\] go execute what they want to do with \[there choice being\] grapes. 5\. Use the One-Breath Test --------------------------- **Jeff:** Here's a really good example of this: I built a Star Wars skill that has lots and lots of content about everything you could imagine in the Star Wars universe. \[When I added\] Luke Skywalker to my content, I found a really good description of \[him\]: _Luke Skywalker was a Tatoonie farmboy who rose from humble beginnings to become one of the great Jedi the galaxy has ever known. Along with his friends Leia and Han Solo, Luke battled the evil empire, discovered the truth of his parentage, and ended the tyranny of the Sith. A generation later, the location of the famed Jedi master was one of the galaxy's greatest mysteries. _\[This description\] is really long, and so a good rule of thumb is if you can't say it in one breath — it's too long. This description is probably two or three breaths. What I did instead as an example, is that if you ask the skill about Darth Vader, \[Alexa\] says, "Darth Vader is a bad, bad, bad, man." Very simple. Very to-the-point. A little humor in there. And so, that's the thing that we're really trying to accomplish. If people want more, you could offer that, but I wouldn't lead with tons of content. You often lose people's attention. 6\. **Include a variety of responses** -------------------------------------- **Jeff:** This is something that is really important to me, and something I do all over the place. When we build applications for mobile or the web, you often find specific rules that they want you to follow. \[For example, you have\] the iOS style guide \[and\] the Android style guide. They \[use concepts\] like predictable and consistent, and those things are 100 percent the opposite of what we want voice to be.  Source: Jeff Blankenburg With voice, it needs to be unpredictable. It needs to be something that you are engaged with and that you are actually listening to or choosing to play along with. And so what I like to do, is keep everything fresh and random. For every single thing that my skill can say, I like to have 5 to 7 versions of that thing. You can do this with everything. Vary the order \[or\] the content you're using by keeping people on their toes and not letting it be predictable. You'll find that they stay engaged and that they're participating more in conversation with their voice system. The other thing I like to do is randomize the order that you present things. For example, if you're building something for a travel agency, there are five things you need to allow the user to make a flight search: origin, destination, airline, departure date, and return date. _Where are you flying from?_ _Where are you flying to?_ _What airline do you want to use?_ _When are you flying?_ _When do you want to return? _Now you may always ask the questions in this order, but what's to say that the second time they come back \[and complete this\], we don't \[use the same order\]. It doesn't matter what the order of these things is. We know that we want all five of those values. So by mixing it up, it doesn't feel automated. It doesn't feel like I'm filling out a form with my voice. It feels like I'm having a conversation with someone, and I'm just answering their questions. So this is another really good example of how we want to think about randomizing the order that we do things, and not only the order that we say things. 7\. Handle the unexpected gracefully ------------------------------------ **Jeff:** One example of these is errors. Errors are going to happen from time to time. Maybe an API is down. Maybe you built something that didn't work properly. \[Although\] Alexa has default error messages \[such as\], "there was a problem with the requested skill's response," you should control and manage these to make it seem like something accidentally happened \[and that\] you're handling it. Here's an example of that: _ "It seems our trivia questions are better than our software developers. Something is broken, but we've alerted our team to the problem. Can I offer you a random trivia question instead?" _So, the trivia questions stuff is working, but whatever they were trying to do is broken, so let's instead think about how we redirect them - maybe adding a little humour to it \[as well\]. And then just carry \[the conversation\] back to where they were going. \[This\] is something really important to think about, but not something people pay a lot of attention to. There's always opportunities for things to go wrong, and having some fun with it makes a big difference. \[Another\] example of this is my baseball skill. If I came in and asked about the Cleveland Indians, the New York Yankees, or the San Francisco Giants — my skills were ready for all of that. But what if they say the New York Jets. What do I do then? I don't have the New York Jets in my database \[because it is a football team\]. I don't want to throw an error. I need to do something with this information. So what I do in my case is I say, "I'm sorry, the New York Jets aren't a baseball team. Is there a different team I can tell you about?" I'm redirecting them \[instead\] of saying, "I don't know what to do." So this is a really good example of how you can redirect people to the thing they should be doing. 8\. Make enhancements based on data -------------------------------------- **Jeff:** Publishing a skill is not the end. It's more or less the beginning. As you're building, you'll see that you can collect a whole bunch of data about what's going on inside your skill. What are people saying? What are people asking for? Anytime that I'm collecting data from a user, I always persist that to a database so that I can compare later and say, "what are people asking my skill for that I'm not accommodating." By parsing through that data and looking at it in a deep way, we can say, "oh, we should be fixing this!" Another good example is intents. If we think about my _Dev Tip_ skill, we have a _get news intent_, we have an _answer intent_, we have a _launch request_ \[which is where\] they start. And \[I\] can see that the _get news_ intent seems to be the most popular \[as appose\] to the _display template_ intent \[which is\] very infrequently used. So when I ask myself, what am I going to build next? It should be focused on what people use the most and how I can continue to provide value to my users. 9\. Provide contextual help --------------------------- **Jeff:** This is something that we all skip over, but it's always required. We have to provide help inside of our skill. We often do things like this: _"This skill lets you order from our big menu of pizzas, breadsticks, pasta, sandwiches, wings, and desserts. What would you like to do?"_ All I've done here is given \[the user\] another opportunity to ask for something. I haven't actually helped them. If they were in the middle of ordering a pizza and weren't sure how to get double sauce, the above example doesn't give them any of that assistance \[if they ask for help\]. What help should look like is this: _"It looks like you're trying to order a pizza. You can add or remove toppings by saying things like add pepperoni or remove anchovies. You can also add extras like sauces, extra cheese, or our different crusts. Just ask for them! Do you want to get back to your pizza, or do you need help with something else?" _It's possible that they didn't need help with the thing they were doing - and in that case - you could try to redirect them to the appropriate kind of help that they're looking for. Help is super important because people get lost. There's no UI. There are no visual clues to let them know that you're supposed to click this button or go over here. They're only using their voices and sometimes they just can't figure out exactly what that thing should be. 10\. Beta test with REAL users --------------------------------- **Jeff:** I can't emphasize this enough. If you're not getting your skill out in front of real people, then you're never going to feel like it's as successful as it could have been. This is because each of us has a \[different\] idea about how people speak. We need to get other people in front of us and talking to our skill. We need to help us understand what's going on. And what you really want to do is get people you know to use this stuff. Invite your friends and your family — people that will give you open and honest feedback. A lot of times, if you open it up to people you met online or a large mailing list, very few people will tell you anything. They may use your skill a couple of times, but they're probably not going to give you the kind of meaningful feedback you're looking for. If you're working in your offices on voice stuff, the thing I recommend is taking the_Wizard of Oz_ approach. There's two lessons to be learned here. The first is that if you're trying to build something that replicates a mobile app or a website, a good rule of thumb is that if it takes more than three clicks to get to, and is a place the user wants to go regularly, should think about building it for voice. So I always think about the three clicks on the Ruby Slippers (Wizard of Oz) as a way to remember that those are the things we should really be focusing on. Conversely, if they can get to it \[quickly, or in less than 3 clicks\] by opening your app, that's probably going to be faster. So it's something to keep in mind. As far as testing goes, you should have somebody behind the curtain — like the wizard \[in The Wizard of Oz\], who knows what the skill should do. Then, you have people come up and talk to the curtain so they can't see any facial expressions or body language. They ask the skill to do things, and the skill — or the person behind the curtain in this case — responds in the way that the skill is programmed to. You'll learn very quickly how people want to interact with this content and how they want to interact with this skill.  RECOMMENDED [\ \ ### Designing for voice assistants vs. chatbots](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots) PUBLISHED October 2, 2020 TOPICS IN THIS ARTICLE No items found. [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at\ \ Mark is a Pathways contributor.](https://www.voiceflow.com/contributors/mark-ammendolia) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/the-complete-guide-to-better-cxd-workflows) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The Complete Guide to Better CxD Workflows](https://www.voiceflow.com/pathways/the-complete-guide-to-better-cxd-workflows) [](https://www.voiceflow.com/pathways/conversation-design-career-journey) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### From Hobbits to bots—lessons from Simon Mitchell-Wolf’s journey to CxD](https://www.voiceflow.com/pathways/conversation-design-career-journey) [](https://www.voiceflow.com/pathways/best-practices-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Curiosity and conversation beat any best practice](https://www.voiceflow.com/pathways/best-practices-conversation-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # A bit of entity advice: Delight your overfillers | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) A bit of entity advice: Delight your overfillers ================================================ [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) As the adage goes, give me six hours to chop down a tree and I’ll spend the first four sharpening the ax. For conversation designers, those first four hours should be spent designing their NLU models—a large part of which includes organizing entities. Teams that don’t take this time, and who don’t “go slow to go fast,” may start solving problems before they understand them. That commits them to marching up the wrong hills and producing assistants that ask redundant questions and seem, to users, not so smart. Today, I’ll share ideas for collecting and using entities so your assistant misses nothing. (And gets people’s orders right!)    Let’s grab a coffee ------------------- To explore entity collection and use, let’s take the theoretical case of a chat assistant for a big, nameless Seattle-based coffee chain. Users tend to enter the app with a pretty narrow goal, or two intents: order a coffee and check out. Simple. But getting them from first touch to mocha latte is less so. As you probably know, customers are going to buck the script. To paraphrase Churchill, users always do the right thing … but only after exhausting every other option. How frustrating or supportive they perceive your assistant to be as they stumble through depends on: * How flexibly it responds to their non-formulaic inputs * How well it manages context switching * How well it acts upon conditional logic * How deftly it follows up on incomplete utterances, inquiring about the specific missing information * And maybe most important, how well it preserves and recalls earlier inputs for later use Today, our coffee ordering app user could say anything from “black, cream, no sugar” to the much more mysterious “Vente [PSL](https://guiltyeats.com/2022/08/06/psl-drop-starbucks-2022/) ,” “Frapxchino,” or a billion other variants, acronyms, synonyms, or misspellings. Let’s explore how your assistant can and should handle those. Perceived intelligence is all in how you capture and store overfilled information --------------------------------------------------------------------------------- You can sort that user’s utterance into one of three categories: underfilled, filled, or overfilled (for that inquiry). That is, did they give you less than you need, what you need, or too much information for that stage? * **Underfilling**: When someone provides too little. New users don’t know how this interaction is supposed to go, and so may give you a partial answer, like, “I’d like a coffee.” Or no answer—like, “Wait, is this a person?” * **Filling**: When someone provides just enough. If they’re a repeat user, they probably know what you need and may reply as prompted. E.g. “A large coffee, no cream or sugar.” (The assistant can then elicit more information.) * **Overfilling**: When someone provides too much. An experienced user may know the flow so well, they try to skip steps. E.g. “I want a Grande coffee sugar and no Pike, and my name is Braden.” In this example, they gave too much relevant information for the step. The name is only relevant later. This is called a “one-shot”—a completely correct answer—and it’s enough to skip multiple trees. When a user underfills, the experience hinges on how deftly your assistant fishes for additional information—and just the information needed. If they said “coffee” but not which type, it should ask for the type, but not the entire order again. (Sounds obvious, but not obvious.) When a user overfills and you respond appropriately, it’s a real delight. The user gives a one-shot, and your assistant says, “Great, please confirm I got all that”—and it did. This type of interaction is rare and requires you to understand those potentially overfilled entities and store them for use later. (You can add those additional, later entities in the first query, but leave them hidden. So if a user knows how this goes and gives you all the entities needed to fulfill multiple intents, the assistant will know what to do with them.) This is extremely difficult and something you really only arrive at through extensive user testing.  The thing I’ve also learned about overfilling is that users actually tend to underfill intentionally because they’re nervous about confusing the assistant. But if you delight them by using overfilled information once, it’s a great experience—in one shot, they skip the flow—and it teaches them that your assistant is not like the others. Yours is a lot more valuable and adaptive than they thought. This opens their eyes, and little surprises like this are the future of conversation design. > Successfully ingesting overfilled information teaches users that your assistant is not like the others. The challenge? Too many intents and entities -------------------------------------------- In our Seattle-based coffee chain example, intents are inherently narrow—nobody goes to the app for their horoscope or to place bets. It’s (almost) always about ordering coffee. That means there are relatively few intents and entities involved. But for other assistants where the potential is wider, like a customer support assistant for a big (also Seattle-based) e-commerce giant, with sixty departments ranging from online video to groceries, the list grows longer and the complexity compounds. If you don’t have a good system in place for that entity collection, things can get out of hand and it’s difficult to clean up later without starting over. To get technical, it’s a factorial equation where two entities means four checks, three entities means nine checks, and nine entities means 81 checks. (Or, (A + B) (A) (B) (0).) The more complex that gets, the more important it is you have a conversation design tool that has functions built in for entity fulfillment. Or if not that, a really well-architected spreadsheet where you can plot out all the matrices with unique answers for each part of the grid. Otherwise, a flowchart won’t suffice. It will become way too big to manage. How you collect is how people feel ---------------------------------- This is all to say that the secret to delighting overfillers like return power users is being really thorough. And always spending four hours sharpening your ax. Poring over tactual transcripts, having more conversations, and running tabletop user tests. And then mapping entities to intents, reevaluating your categories, nesting entities within others, and always trying to use the user’s full utterance, even if it’s a one-shot. That’s how you get someone’s order right.    Let’s grab a coffee ------------------- To explore entity collection and use, let’s take the theoretical case of a chat assistant for a big, nameless Seattle-based coffee chain. Users tend to enter the app with a pretty narrow goal, or two intents: order a coffee and check out. Simple. But getting them from first touch to mocha latte is less so. As you probably know, customers are going to buck the script. To paraphrase Churchill, users always do the right thing … but only after exhausting every other option. How frustrating or supportive they perceive your assistant to be as they stumble through depends on: * How flexibly it responds to their non-formulaic inputs * How well it manages context switching * How well it acts upon conditional logic * How deftly it follows up on incomplete utterances, inquiring about the specific missing information * And maybe most important, how well it preserves and recalls earlier inputs for later use Today, our coffee ordering app user could say anything from “black, cream, no sugar” to the much more mysterious “Vente [PSL](https://guiltyeats.com/2022/08/06/psl-drop-starbucks-2022/) ,” “Frapxchino,” or a billion other variants, acronyms, synonyms, or misspellings. Let’s explore how your assistant can and should handle those. Perceived intelligence is all in how you capture and store overfilled information --------------------------------------------------------------------------------- You can sort that user’s utterance into one of three categories: underfilled, filled, or overfilled (for that inquiry). That is, did they give you less than you need, what you need, or too much information for that stage? * **Underfilling**: When someone provides too little. New users don’t know how this interaction is supposed to go, and so may give you a partial answer, like, “I’d like a coffee.” Or no answer—like, “Wait, is this a person?” * **Filling**: When someone provides just enough. If they’re a repeat user, they probably know what you need and may reply as prompted. E.g. “A large coffee, no cream or sugar.” (The assistant can then elicit more information.) * **Overfilling**: When someone provides too much. An experienced user may know the flow so well, they try to skip steps. E.g. “I want a Grande coffee sugar and no Pike, and my name is Braden.” In this example, they gave too much relevant information for the step. The name is only relevant later. This is called a “one-shot”—a completely correct answer—and it’s enough to skip multiple trees. When a user underfills, the experience hinges on how deftly your assistant fishes for additional information—and just the information needed. If they said “coffee” but not which type, it should ask for the type, but not the entire order again. (Sounds obvious, but not obvious.) When a user overfills and you respond appropriately, it’s a real delight. The user gives a one-shot, and your assistant says, “Great, please confirm I got all that”—and it did. This type of interaction is rare and requires you to understand those potentially overfilled entities and store them for use later. (You can add those additional, later entities in the first query, but leave them hidden. So if a user knows how this goes and gives you all the entities needed to fulfill multiple intents, the assistant will know what to do with them.) This is extremely difficult and something you really only arrive at through extensive user testing.  The thing I’ve also learned about overfilling is that users actually tend to underfill intentionally because they’re nervous about confusing the assistant. But if you delight them by using overfilled information once, it’s a great experience—in one shot, they skip the flow—and it teaches them that your assistant is not like the others. Yours is a lot more valuable and adaptive than they thought. This opens their eyes, and little surprises like this are the future of conversation design. > Successfully ingesting overfilled information teaches users that your assistant is not like the others. The challenge? Too many intents and entities -------------------------------------------- In our Seattle-based coffee chain example, intents are inherently narrow—nobody goes to the app for their horoscope or to place bets. It’s (almost) always about ordering coffee. That means there are relatively few intents and entities involved. But for other assistants where the potential is wider, like a customer support assistant for a big (also Seattle-based) e-commerce giant, with sixty departments ranging from online video to groceries, the list grows longer and the complexity compounds. If you don’t have a good system in place for that entity collection, things can get out of hand and it’s difficult to clean up later without starting over. To get technical, it’s a factorial equation where two entities means four checks, three entities means nine checks, and nine entities means 81 checks. (Or, (A + B) (A) (B) (0).) The more complex that gets, the more important it is you have a conversation design tool that has functions built in for entity fulfillment. Or if not that, a really well-architected spreadsheet where you can plot out all the matrices with unique answers for each part of the grid. Otherwise, a flowchart won’t suffice. It will become way too big to manage. How you collect is how people feel ---------------------------------- This is all to say that the secret to delighting overfillers like return power users is being really thorough. And always spending four hours sharpening your ax. Poring over tactual transcripts, having more conversations, and running tabletop user tests. And then mapping entities to intents, reevaluating your categories, nesting entities within others, and always trying to use the user’s full utterance, even if it’s a one-shot. That’s how you get someone’s order right.   RECOMMENDED [](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers#) PUBLISHED November 7, 2022 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.avif)](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 10 things every voice app should do](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) [](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Designing conversational bots that fix themselves](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves) [](https://www.voiceflow.com/pathways/where-is-the-opportunity-for-conversation-designers) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Where is the opportunity for conversation designers?](https://www.voiceflow.com/pathways/where-is-the-opportunity-for-conversation-designers) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Announcing Voiceflow's 2020 Community Award winners! | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Announcing Voiceflow's 2020 Community Award winners! ==================================================== [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at](https://www.voiceflow.com/contributors/mark-ammendolia) To everyone who participated in the 2020 Community Awards - thank you so much! We are thrilled with the number of nominations submitted this year, and as a result, have added multiple winners to each category voted on by our vibrant community. Those categories are: 🏅**VF Hero Award:** an individual who has gone above and beyond to help grow, educate, and build a better community. 🏅**VF Growth Award:** an individual who champions designing for voice and who has positively impacted community growth and development through outreach and events. 🏅**VF Helping Hands Award:** an individual who has provided exceptional support, feedback, and encouragement within our community. 🏅**VF Creative Design Award:** a voice experience that pushes the limits & shows off what's possible in conversation design.  The Voiceflow team is thrilled to announce the recipients of the 2nd annual Voiceflow Community Awards. Congratulations to the following individuals:  This year, the Voiceflow Team Awards were broken down into four separate categories, honoring excellence in voice design, innovation, and user experience among the industry's very best CxD teams using Voiceflow. The categories are: 🏅**VF Team Award: Most Innovative Voice Experience** 🏅**VF Team Award: Most Advanced Design** 🏅**VF Team Award: Best Custom Assistant Design** 🏅**VF Team Award: Best use of IoT in Conversation Design** Congratulations to the following CxD teams: .png) To all of this year's nominees: thank you for your outstanding contributions to the growth, development, and success of this incredible community. These awards allow us to celebrate extraordinary individuals and teams, all of whom share an unwavering commitment to voice and the advancement of this technology. We truly appreciate everything you do. \- The Voiceflow Team The Voiceflow team is thrilled to announce the recipients of the 2nd annual Voiceflow Community Awards. Congratulations to the following individuals:  This year, the Voiceflow Team Awards were broken down into four separate categories, honoring excellence in voice design, innovation, and user experience among the industry's very best CxD teams using Voiceflow. The categories are: 🏅**VF Team Award: Most Innovative Voice Experience** 🏅**VF Team Award: Most Advanced Design** 🏅**VF Team Award: Best Custom Assistant Design** 🏅**VF Team Award: Best use of IoT in Conversation Design** Congratulations to the following CxD teams: .png) To all of this year's nominees: thank you for your outstanding contributions to the growth, development, and success of this incredible community. These awards allow us to celebrate extraordinary individuals and teams, all of whom share an unwavering commitment to voice and the advancement of this technology. We truly appreciate everything you do. \- The Voiceflow Team  RECOMMENDED [](https://www.voiceflow.com/pathways/announcing-our-2020-voiceflow-community-award-winners#) PUBLISHED January 29, 2021 TOPICS IN THIS ARTICLE Exceptional work [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at\ \ Mark is a Pathways contributor.](https://www.voiceflow.com/contributors/mark-ammendolia) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/runtime-performance-improvements) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Runtime performance improvements](https://www.voiceflow.com/pathways/runtime-performance-improvements) [](https://www.voiceflow.com/pathways/voiceflow-integrates-with-anthropic) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Anthropic Claude model family now in Voiceflow](https://www.voiceflow.com/pathways/voiceflow-integrates-with-anthropic) [](https://www.voiceflow.com/pathways/anthropic-claude-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Anthropic’s Claude is now in Voiceflow](https://www.voiceflow.com/pathways/anthropic-claude-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/announcing-our-2020-voiceflow-community-award-winners#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/announcing-our-2020-voiceflow-community-award-winners#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 5 Strategies to Elevate AI-Powered Customer Experiences | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) 5 strategies product leaders use to transform AI-powered customer experiences \[Checklist\] =========================================================================================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) _Gartner predicts that by 2027, human-like AI agents will become the “primary customer service channel” for 25% of organizations, reducing contact center labor costs by $80 billion._ **Success with AI agents isn’t merely about adopting the technology; it’s about how companies leverage it effectively.** Voiceflow has provided a platform for more than 250,000 teams to build, deploy, and scale AI agents across various use cases. Over time, I’ve observed that the most successful teams using Voiceflow share five key characteristics. These traits have consistently proven to be instrumental in building and scaling AI agents effectively. Whether you are considering adopting AI or are already in the process of scaling your customer experience initiatives, take a moment to self-check and benchmark your own team against these 5 characteristics, which will be crucial for achieving the same level of success as these industry leaders.  What Does Success Look Like? ---------------------------- **The AI journey of _Sanlam_, _Trilogy_, and _Parkfield Commerce_ offers clear examples of what success with AI agents can look like.** Whether it’s increasing customer engagement, automating support, or driving revenue, these companies showcase how effective AI agent deployment leads to real business outcomes. * **Sanlam** used Voiceflow to build an AI financial coach that resulted in a 45% offer-to-lead conversion rate, engaging over 350 customers daily without any marketing spend. * **Trilogy** used AI to automate 70% of both L1 and L2 tickets, saving time and improving customer service quality. * **Parkfield Commerce** boosted customer traffic by 73% for a client by integrating AI solutions into its e-commerce platform. **Drake Waterfowl,** a client of Parkfield Commerce, is handling 50% of its customer support using AI without “any negative feedback from customers”. Top 5 Characteristics of Successful AI Teams -------------------------------------------- These 5 characteristics have been consistently observed in companies that not only adopt AI but also successfully _scale it_. ### Characteristic I: Clear Vision and Strategic Ambition Success with AI starts with a clear vision. Companies like Trilogy exhibit strategic ambition, aiming to scale their AI capabilities beyond the basics. For instance, Trilogy moved from handling simple L1 support tickets to more complex L2 tickets, targeting a 90% AI resolution rate. This vision helped them stretch the potential of AI and push the boundaries of automation. **Self-Check Question**: **Are you setting ambitious goals for how AI can transform customer experiences beyond a simple FAQ support chatbot?** ### Characteristic II: Precise Control Over Content and Building Experience **A critical success factor for companies like _Sanlam_ is the level of control they maintain over the build process.** With Voiceflow, _Sanlam_ tailored its AI financial coach to fit seamlessly into its existing tech infrastructure, which played a major role in its success. The financial coach achieved a 80% offer-to-lead conversion rate, meaning nearly half of the AI’s suggestions led to follow-up conversations with human financial advisors. Furthermore, the AI’s interaction with customers reduced follow-up questions to zero, underscoring the coach’s ability to deliver clear and comprehensive responses from a curated knowledge base of company information. For organizations considering AI adoption, the lesson here is clear: control over content and the building process is crucial for ensuring that AI aligns with business goals, enhances customer interactions, and drives tangible outcomes. Tailoring AI to your unique needs, as _Sanlam_ did, can create a more robust, customer-centric experience. **Self-Check Question**: **How much control do you want over your existing AI agent’s interactions and the user experience it provides?** ### Characteristic III: Commitment to Iterative Development Successful AI teams understand that development is an ongoing process of refinement. _Sanlam_ exemplifies this through its iterative approach to building its AI financial coach. Currently in its fifth version (Coach v5), Sanlam’s team has continuously improved its AI agent over time. After building and discarding earlier versions, they found that experimenting with different AI orchestration processes, integrating multiple LLMs, and focusing on proactive engagement was key to refining the coach’s performance. This approach has resulted in the coach achieving a 45% offer-to-lead conversion rate and consistently engaging with 350+ clients daily. **Self-Check Question**: **Does your team continuously iterate and refine AI agents based on performance and feedback?** ### Characteristic IV: Strong Industry Knowledge and Technical Expertise A team’s industry knowledge and technical expertise play a crucial role in the success of AI agents, and _Parkfield Commerce_ is a prime example of this. The team discussed how they gain valuable insights from customer reviews—what they refer to as “golden nuggets.” These insights often highlight how customers use products in unexpected ways, which help them better meet customer needs. This deep understanding of customer behavior, coupled with technical expertise in AI and sentiment analysis, helps them continuously improve their offerings and provide personalized experiences for their clients. Similarly, Trilogy demonstrates the significance of combining robust technical expertise with deep industry knowledge to scale AI effectively. As an AI-first company, Trilogy’s strategy focuses on leveraging AI to boost efficiency and scalability. In just 12 weeks, Trilogy successfully automated 60% of customer support inquiries across 90 of their products using AI. Colin, discussing the technical proficiency of their clientele, remarked, “We’re getting a lot of developers hitting us up and asking for like, ‘Hey, how do you do this, or could you add this to the system?’” **This illustrates how their clients—highly technical professionals—demand sophisticated AI solutions.** The fusion of industry expertise and advanced technical knowledge has been instrumental in delivering impactful and efficient AI-driven agents. **Self-Check Question**: **Does your team have the necessary industry expertise to understand and act on customer feedback?** ### Characteristic V: Autonomous and Agile Team Dynamics Autonomy and agility are essential characteristics of teams that successfully implement and scale AI solutions. This is especially evident in the approach taken by Parkfield Commerce, where their agile and autonomous team structure allows them to iterate rapidly, respond to feedback, and continuously improve their AI products. Richard Emmanuel, CEO of Parkfield Commerce, discussed how their small team can swiftly adapt to new challenges and opportunities. He highlighted the power of having a small, autonomous team: > _“We’ve got a small team of two, and that’s the cool thing about having small teams. You can just be like, ‘OK, we need to add this,’ and by the end of the day, it’s done”._ This flexibility has been a key factor in their ability to innovate quickly, resulting in faster time-to-market for their AI-driven solutions. Moreover, Parkfield Commerce uses insights from real-time customer feedback and product reviews to adapt its AI strategies. For instance, they developed a review sentiment analysis tool that helps extract valuable insights from customer feedback, such as whether a product is suitable for specific use cases like camping in extreme conditions. Richard Emmanuel explained how quickly the team implemented this: “We had access to their API, and by the end of the day, we created a review sentiment analysis—it was super cool.” This agile approach allows them to continuously refine their AI solutions based on real-world usage. The combination of autonomy—empowering teams to make real-time decisions—and agility—responding rapidly to feedback—has been crucial to Parkfield Commerce’s success in AI. **Self-Check Question**: **Does your team have the autonomy and agility to quickly adapt to new challenges and innovate?** How to Assess AI-Readiness In Your Organization ----------------------------------------------- Assessing your team’s readiness to scale AI can make or break your success. Use the checklist below to evaluate your organization across the five key characteristics mentioned: * **Vision:** Does your team have a clear strategic vision for AI? * **Control:** Do you have sufficient control over the build and user experience? * **Iterative Development:** Is there a commitment to continuous improvement? * **Expertise:** Does your team possess the industry knowledge and technical skills required? * **Agility:** Can your team adapt quickly to new challenges and opportunities? **To replicate the success seen by _Sanlam_, _Trilogy_, and _Parkfield Commerce_, it’s critical that your organization embodies these five characteristics.** Achieving the desired outcomes—whether in customer engagement, operational efficiency, or revenue growth—requires the right mix of vision, control, technical expertise, and agility. If you’re ready to assess your AI readiness and start building with Voiceflow, [**let's connect**](https://voiceflow.com/demo) to explore how Voiceflow can help you achieve your AI automation goals. What Does Success Look Like? ---------------------------- **The AI journey of _Sanlam_, _Trilogy_, and _Parkfield Commerce_ offers clear examples of what success with AI agents can look like.** Whether it’s increasing customer engagement, automating support, or driving revenue, these companies showcase how effective AI agent deployment leads to real business outcomes. * **Sanlam** used Voiceflow to build an AI financial coach that resulted in a 45% offer-to-lead conversion rate, engaging over 350 customers daily without any marketing spend. * **Trilogy** used AI to automate 70% of both L1 and L2 tickets, saving time and improving customer service quality. * **Parkfield Commerce** boosted customer traffic by 73% for a client by integrating AI solutions into its e-commerce platform. **Drake Waterfowl,** a client of Parkfield Commerce, is handling 50% of its customer support using AI without “any negative feedback from customers”. Top 5 Characteristics of Successful AI Teams -------------------------------------------- These 5 characteristics have been consistently observed in companies that not only adopt AI but also successfully _scale it_. ### Characteristic I: Clear Vision and Strategic Ambition Success with AI starts with a clear vision. Companies like Trilogy exhibit strategic ambition, aiming to scale their AI capabilities beyond the basics. For instance, Trilogy moved from handling simple L1 support tickets to more complex L2 tickets, targeting a 90% AI resolution rate. This vision helped them stretch the potential of AI and push the boundaries of automation. **Self-Check Question**: **Are you setting ambitious goals for how AI can transform customer experiences beyond a simple FAQ support chatbot?** ### Characteristic II: Precise Control Over Content and Building Experience **A critical success factor for companies like _Sanlam_ is the level of control they maintain over the build process.** With Voiceflow, _Sanlam_ tailored its AI financial coach to fit seamlessly into its existing tech infrastructure, which played a major role in its success. The financial coach achieved a 80% offer-to-lead conversion rate, meaning nearly half of the AI’s suggestions led to follow-up conversations with human financial advisors. Furthermore, the AI’s interaction with customers reduced follow-up questions to zero, underscoring the coach’s ability to deliver clear and comprehensive responses from a curated knowledge base of company information. For organizations considering AI adoption, the lesson here is clear: control over content and the building process is crucial for ensuring that AI aligns with business goals, enhances customer interactions, and drives tangible outcomes. Tailoring AI to your unique needs, as _Sanlam_ did, can create a more robust, customer-centric experience. **Self-Check Question**: **How much control do you want over your existing AI agent’s interactions and the user experience it provides?** ### Characteristic III: Commitment to Iterative Development Successful AI teams understand that development is an ongoing process of refinement. _Sanlam_ exemplifies this through its iterative approach to building its AI financial coach. Currently in its fifth version (Coach v5), Sanlam’s team has continuously improved its AI agent over time. After building and discarding earlier versions, they found that experimenting with different AI orchestration processes, integrating multiple LLMs, and focusing on proactive engagement was key to refining the coach’s performance. This approach has resulted in the coach achieving a 45% offer-to-lead conversion rate and consistently engaging with 350+ clients daily. **Self-Check Question**: **Does your team continuously iterate and refine AI agents based on performance and feedback?** ### Characteristic IV: Strong Industry Knowledge and Technical Expertise A team’s industry knowledge and technical expertise play a crucial role in the success of AI agents, and _Parkfield Commerce_ is a prime example of this. The team discussed how they gain valuable insights from customer reviews—what they refer to as “golden nuggets.” These insights often highlight how customers use products in unexpected ways, which help them better meet customer needs. This deep understanding of customer behavior, coupled with technical expertise in AI and sentiment analysis, helps them continuously improve their offerings and provide personalized experiences for their clients. Similarly, Trilogy demonstrates the significance of combining robust technical expertise with deep industry knowledge to scale AI effectively. As an AI-first company, Trilogy’s strategy focuses on leveraging AI to boost efficiency and scalability. In just 12 weeks, Trilogy successfully automated 60% of customer support inquiries across 90 of their products using AI. Colin, discussing the technical proficiency of their clientele, remarked, “We’re getting a lot of developers hitting us up and asking for like, ‘Hey, how do you do this, or could you add this to the system?’” **This illustrates how their clients—highly technical professionals—demand sophisticated AI solutions.** The fusion of industry expertise and advanced technical knowledge has been instrumental in delivering impactful and efficient AI-driven agents. **Self-Check Question**: **Does your team have the necessary industry expertise to understand and act on customer feedback?** ### Characteristic V: Autonomous and Agile Team Dynamics Autonomy and agility are essential characteristics of teams that successfully implement and scale AI solutions. This is especially evident in the approach taken by Parkfield Commerce, where their agile and autonomous team structure allows them to iterate rapidly, respond to feedback, and continuously improve their AI products. Richard Emmanuel, CEO of Parkfield Commerce, discussed how their small team can swiftly adapt to new challenges and opportunities. He highlighted the power of having a small, autonomous team: > _“We’ve got a small team of two, and that’s the cool thing about having small teams. You can just be like, ‘OK, we need to add this,’ and by the end of the day, it’s done”._ This flexibility has been a key factor in their ability to innovate quickly, resulting in faster time-to-market for their AI-driven solutions. Moreover, Parkfield Commerce uses insights from real-time customer feedback and product reviews to adapt its AI strategies. For instance, they developed a review sentiment analysis tool that helps extract valuable insights from customer feedback, such as whether a product is suitable for specific use cases like camping in extreme conditions. Richard Emmanuel explained how quickly the team implemented this: “We had access to their API, and by the end of the day, we created a review sentiment analysis—it was super cool.” This agile approach allows them to continuously refine their AI solutions based on real-world usage. The combination of autonomy—empowering teams to make real-time decisions—and agility—responding rapidly to feedback—has been crucial to Parkfield Commerce’s success in AI. **Self-Check Question**: **Does your team have the autonomy and agility to quickly adapt to new challenges and innovate?** How to Assess AI-Readiness In Your Organization ----------------------------------------------- Assessing your team’s readiness to scale AI can make or break your success. Use the checklist below to evaluate your organization across the five key characteristics mentioned: * **Vision:** Does your team have a clear strategic vision for AI? * **Control:** Do you have sufficient control over the build and user experience? * **Iterative Development:** Is there a commitment to continuous improvement? * **Expertise:** Does your team possess the industry knowledge and technical skills required? * **Agility:** Can your team adapt quickly to new challenges and opportunities? **To replicate the success seen by _Sanlam_, _Trilogy_, and _Parkfield Commerce_, it’s critical that your organization embodies these five characteristics.** Achieving the desired outcomes—whether in customer engagement, operational efficiency, or revenue growth—requires the right mix of vision, control, technical expertise, and agility. If you’re ready to assess your AI readiness and start building with Voiceflow, [**let's connect**](https://voiceflow.com/demo) to explore how Voiceflow can help you achieve your AI automation goals.  RECOMMENDED [\ \ ### Money talks: How Sanlam Studios’ AI coach drives leads and financial literacy](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) PUBLISHED September 18, 2024 TOPICS IN THIS ARTICLE Exceptional people Exceptional work [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Building your AI agents like products: A blueprint from POC purgatory to production](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [.webp)](https://www.voiceflow.com/pathways/what-is-an-interaction-model-in-conversational-ai) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### What is an interaction model in conversational AI?](https://www.voiceflow.com/pathways/what-is-an-interaction-model-in-conversational-ai) [](https://www.voiceflow.com/pathways/why-were-building-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Why we're building Voiceflow](https://www.voiceflow.com/pathways/why-were-building-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/5-strategies-to-transform-ai-powered-customer-experiences#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/5-strategies-to-transform-ai-powered-customer-experiences#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Introducing #VFV2: The next phase of Voiceflow | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Introducing #VFV2: The next phase of Voiceflow ============================================== [\ \ Rob Hayes\ \ Head of Product at](https://www.voiceflow.com/contributors/rob-hayes) We began our journey in voice just over two years ago. What started as a company bent on building the best Alexa skills for children soon expanded into something much larger – allowing anyone to design, prototype, and build voice apps. Flash forward to today – Voiceflow now empowers over 55,000+ designers, developers, and their teams to build conversational experiences. Community groups have formed worldwide, aimed at fostering connection and skill-sharing while making it easier than ever for people to dive into conversation design. And it's only just the beginning. Announcing #VFV2 ------------------- Today, we're excited to announce Voiceflow V2 (#VFV2), our annual product event, where we'll be announcing a series of major company updates alongside several important product launches, workflow enhancements, and a few surprises along the way. [#VFV2 takes place on 02/22/2021 at 2 pm EST.](https://voiceflow.com/v2) Participants will get a first glimpse at the next phase of Voiceflow, all while connecting with members of the Voiceflow community, team, and industry-alike. Following the event, we've partnered with [Dr. Teri Fisher](https://www.linkedin.com/in/drterifisher/) , host of the [Voice Den](https://drterifisher.com/the-voice-den/) , to put on a #VFV2 Social – where participants from all around the world can join in.  ### Curious about what we'll be announcing? See you at #VFV2 – The Voiceflow Team [](https://voiceflow.com/v2) ### Curious about what we'll be announcing? See you at #VFV2 – The Voiceflow Team [](https://voiceflow.com/v2)  RECOMMENDED [\ \ ### Introducing Voiceflow's GPT-powered AI Builder](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) PUBLISHED January 22, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Rob Hayes\ \ Head of Product at\ \ VP of Product at Voiceflow.](https://www.voiceflow.com/contributors/rob-hayes) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/announcing-vfv2) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Introducing #VFV2: The next phase of Voiceflow](https://www.voiceflow.com/pathways/announcing-vfv2) [](https://www.voiceflow.com/pathways/voiceflow-raises-20m-to-build-the-future-of-conversation-design-tooling) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow raises $20M to build the future of conversation design tooling](https://www.voiceflow.com/pathways/voiceflow-raises-20m-to-build-the-future-of-conversation-design-tooling) [](https://www.voiceflow.com/pathways/voiceflow-integrates-with-anthropic) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Anthropic Claude model family now in Voiceflow](https://www.voiceflow.com/pathways/voiceflow-integrates-with-anthropic) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/announcing-vfv2#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/announcing-vfv2#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # A quick guide on integrating Alexa with Microsoft LUIS | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) A quick guide on integrating Alexa with Microsoft LUIS ====================================================== [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) tl;dr Despite Alexa's excellent natural language processing engine, there are other NLP engines on the market with more capabilities that can be used instead. In this article, we show you how to integrate your Alexa skills with Microsoft Luis. Introduction ------------ Although the NLP in Alexa allows us to create skills that cover 99% of the most common use cases for users, Microsoft LUIS has grown to include a range of powerful functionalities that we cannot find in Alexa. This includes [prebuilt domains](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/howto-add-prebuilt-models) , prebuilt entities, sub-entities and [regular expression entities](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-how-to-add-entities) . **Requirements** ---------------- You will need the following technologies to complete this project: 1. Node.js v12.x 2. Visual Studio Code 3. Azure Account  **Setting up our Alexa Skill** ------------------------------ The first thing we need to do is set up our **interaction model**. First, we are going to create the {% c-line %}OrderIntent{% c-line-end %}. For this example, this intent will have only one utterance and one slot. This slot will have the {% c-line %}AMAZON.SearchQuery{% c-line-end %} type:  By definition, a {% c-line %}AMAZON.SearchQuery{% c-line-end %} slot is a bit different from the rest. It functions like a standard search engine, and with it we will be able to recognize less-predictable inputs. To use this slot, you will need to add a carrier phrase to your utterances. In this example, I used {% c-line %}I want{% c-line-end %}. Essentially, what we are sending to MS LUIS is everything that Alexa recognizes after we say {% c-line %}I want...{% c-line-end %}. **Creating Azure Cognitive Services** ------------------------------------- We need to create some Azure resources in order to interact with a Microsoft LUIS App from an Amazon Alexa Skill. First, we need to create a Natural Language Understanding Service within **Cognitive** **Services** on the [Azure Portal](https://portal.azure.com/) :  **NOTE:** make sure that you have checked the **prediction** and **authoring** services during the creation process. After this, ensure you have copied the endpoint. We are going to use this endpoint to interact with MS LUIS. You can find the endpoint after the creation within the **Keys and Endpoint** section, exemplified below.  It's important to make sure you have copied the {% c-line %}region{% c-line-end %} and the {% c-line %}Key 1{% c-line-end %}. This is the subscription we are going to use in next steps. **Creating MS LUIS App** ------------------------ Once you have created your Azure resources, you can create your **MS Luis App** on the [Luis Portal](https://www.luis.ai/) :  **NOTE:** make sure that you use the prediction endpoint you created in the previous step. Now that we have our MS LUIS app, let's add to the interaction model:  When you have built your **entities** and **intents**, you can train your model and finally publish your LUIS app to {% c-line %}Staging{% c-line-end %}. **Calling MS LUIS from Alexa Skill** ------------------------------------ Now that we have everything set up let's write our code! To interact with MS LUIS from the Alexa Skill Lambda written Node.JS, we are going to use the npm package called {% c-line %}@azure/cognitiveservices-luis-runtime{% c-line-end %}. You can find the full documentation of the package [here](https://www.npmjs.com/package/@azure/cognitiveservices-luis-runtime) . First, we have to create our {% c-line %}OrderIntentHandler{% c-line-end %} which is the handler that is going to manage all the requests from the {% c-line %}OrderIntent{% c-line-end %}:  As you can see in the code above, we are getting the value from our {% c-line %}AMAZON.SearchQuery{% c-line-end %} slot called {% c-line %}luisquery{% c-line-end %} and then, we are sending that value to MS LUIS using the {% c-line %}client{% c-line-end %} and the function {% c-line %}getSlotPrediction{% c-line-end %}. To build the client, we need _three_ properties: 1. **MS LUIS app id** - You can find this value in you LUIS APP on the LUIS Portal. 2. **MS Subscription id** - This MS Subscription id is the one that we have get on the previous step. 3. **MS LUIS Prediction endpoint** - This endpoint is the one that we have get on the previous step. With these properties, we can create our MS LUIS Client as follows:  The result that we are going to receive from MS LUIS will be managed by the {% c-line %}intentDispatcher{% c-line-end %}:  **Final Result** ---------------- And that's it! Now you have the full code running with Alexa using Microsoft LUIS as its NLP engine:  > **Resources:**[** \ > 1\. Official Alexa Skills Kit Node.js SDK**](https://www.npmjs.com/package/ask-sdk) > [** \ > 2\. Official Alexa Skills Kit Documentation**](https://developer.amazon.com/docs/ask-overviews/build-skills-with-the-alexa-skills-kit.html) > [** \ > 3\. Official Express Adapter Documentation**](https://developer.amazon.com/en-US/docs/alexa/alexa-skills-kit-sdk-for-nodejs/host-web-service.html) > [** \ > 4\. Official Microsoft Azure SDK Documentation**](https://github.com/Azure/azure-sdk-for-js) **Conclusion** -------------- You can see how easy it is to integrate other NLP engines into our Alexa Skills. While this is an experiment, you may wish to stick with Alexa's built-in natural language processing, since you can get unpredictable results using the AMAZON.SearchQuery. I really hope this example project is useful in helping you integrate external NLPs with your Alexa Skills. Happy coding! **Setting up our Alexa Skill** ------------------------------ The first thing we need to do is set up our **interaction model**. First, we are going to create the {% c-line %}OrderIntent{% c-line-end %}. For this example, this intent will have only one utterance and one slot. This slot will have the {% c-line %}AMAZON.SearchQuery{% c-line-end %} type:  By definition, a {% c-line %}AMAZON.SearchQuery{% c-line-end %} slot is a bit different from the rest. It functions like a standard search engine, and with it we will be able to recognize less-predictable inputs. To use this slot, you will need to add a carrier phrase to your utterances. In this example, I used {% c-line %}I want{% c-line-end %}. Essentially, what we are sending to MS LUIS is everything that Alexa recognizes after we say {% c-line %}I want...{% c-line-end %}. **Creating Azure Cognitive Services** ------------------------------------- We need to create some Azure resources in order to interact with a Microsoft LUIS App from an Amazon Alexa Skill. First, we need to create a Natural Language Understanding Service within **Cognitive** **Services** on the [Azure Portal](https://portal.azure.com/) :  **NOTE:** make sure that you have checked the **prediction** and **authoring** services during the creation process. After this, ensure you have copied the endpoint. We are going to use this endpoint to interact with MS LUIS. You can find the endpoint after the creation within the **Keys and Endpoint** section, exemplified below.  It's important to make sure you have copied the {% c-line %}region{% c-line-end %} and the {% c-line %}Key 1{% c-line-end %}. This is the subscription we are going to use in next steps. **Creating MS LUIS App** ------------------------ Once you have created your Azure resources, you can create your **MS Luis App** on the [Luis Portal](https://www.luis.ai/) :  **NOTE:** make sure that you use the prediction endpoint you created in the previous step. Now that we have our MS LUIS app, let's add to the interaction model:  When you have built your **entities** and **intents**, you can train your model and finally publish your LUIS app to {% c-line %}Staging{% c-line-end %}. **Calling MS LUIS from Alexa Skill** ------------------------------------ Now that we have everything set up let's write our code! To interact with MS LUIS from the Alexa Skill Lambda written Node.JS, we are going to use the npm package called {% c-line %}@azure/cognitiveservices-luis-runtime{% c-line-end %}. You can find the full documentation of the package [here](https://www.npmjs.com/package/@azure/cognitiveservices-luis-runtime) . First, we have to create our {% c-line %}OrderIntentHandler{% c-line-end %} which is the handler that is going to manage all the requests from the {% c-line %}OrderIntent{% c-line-end %}:  As you can see in the code above, we are getting the value from our {% c-line %}AMAZON.SearchQuery{% c-line-end %} slot called {% c-line %}luisquery{% c-line-end %} and then, we are sending that value to MS LUIS using the {% c-line %}client{% c-line-end %} and the function {% c-line %}getSlotPrediction{% c-line-end %}. To build the client, we need _three_ properties: 1. **MS LUIS app id** - You can find this value in you LUIS APP on the LUIS Portal. 2. **MS Subscription id** - This MS Subscription id is the one that we have get on the previous step. 3. **MS LUIS Prediction endpoint** - This endpoint is the one that we have get on the previous step. With these properties, we can create our MS LUIS Client as follows:  The result that we are going to receive from MS LUIS will be managed by the {% c-line %}intentDispatcher{% c-line-end %}:  **Final Result** ---------------- And that's it! Now you have the full code running with Alexa using Microsoft LUIS as its NLP engine:  > **Resources:**[** \ > 1\. Official Alexa Skills Kit Node.js SDK**](https://www.npmjs.com/package/ask-sdk) > [** \ > 2\. Official Alexa Skills Kit Documentation**](https://developer.amazon.com/docs/ask-overviews/build-skills-with-the-alexa-skills-kit.html) > [** \ > 3\. Official Express Adapter Documentation**](https://developer.amazon.com/en-US/docs/alexa/alexa-skills-kit-sdk-for-nodejs/host-web-service.html) > [** \ > 4\. Official Microsoft Azure SDK Documentation**](https://github.com/Azure/azure-sdk-for-js) **Conclusion** -------------- You can see how easy it is to integrate other NLP engines into our Alexa Skills. While this is an experiment, you may wish to stick with Alexa's built-in natural language processing, since you can get unpredictable results using the AMAZON.SearchQuery. I really hope this example project is useful in helping you integrate external NLPs with your Alexa Skills. Happy coding!  RECOMMENDED [\ \ ### Building a knowledge base with OpenAI, LangChain, OpenSearch, and Unstructured](https://www.voiceflow.com/pathways/building-knowledge-base-openai) PUBLISHED October 1, 2021 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/a-quick-guide-on-integrating-alexa-with-microsoft-luis) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### A quick guide on integrating Alexa with Microsoft LUIS](https://www.voiceflow.com/pathways/a-quick-guide-on-integrating-alexa-with-microsoft-luis) [](https://www.voiceflow.com/pathways/transforming-your-raspberry-pi-into-a-simple-voice-assistant) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Transforming your Raspberry PI into a simple voice assistant](https://www.voiceflow.com/pathways/transforming-your-raspberry-pi-into-a-simple-voice-assistant) [](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Want to try Claude in your Voiceflow assistant? Here's how](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/a-quick-guide-on-integrating-alexa-with-microsoft-luis#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/a-quick-guide-on-integrating-alexa-with-microsoft-luis#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # AI Assist: LLMs are now in Voiceflow | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) AI Assist: LLMs are now in Voiceflow ==================================== [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow](https://www.voiceflow.com/contributors/kim-lu) Whether it’s [acing](https://www.newsweek.com/chatgpt-can-write-better-essays-my-college-students-thats-good-thing-opinion-1769136) essays, [eliminating recipe sagas](https://twitter.com/stephsmithio/status/1598920887029628928) , or [pissing off Nick Cave](https://www.theguardian.com/music/2023/jan/17/this-song-sucks-nick-cave-responds-to-chatgpt-song-written-in-style-of-nick-cave) , ChatGPT’s been keeping the internet busy for the past several weeks. As more and more people try it out, there’s a growing debate on how to best use AI-generated content—or whether it’s even useful. And in the conversational AI space, we’ve heard a version of “I’m excited about LLMs (large language models), but I don’t see how it’d fit into my day-to-day work,” again and again. We listened. We put our heads down and went to work. And now we’re thrilled to introduce **AI Assist**, an AI-powered design assistant for conversational AI teams.  **Introducing Voiceflow’s AI Assist** -------------------------------------- **Accelerate your conversation design workflow** using **Generative Tasks** to generate response variations, utterances, entities and synonyms, and no-match/no reply responses.  **Prepare and uncover conversation edge cases** using **AI Freestyle** by having the assistant take over and [push the user back to their original intent](https://www.linkedin.com/posts/Braden%20Ream_following-customer-feedback-this-weekend-activity-7017917755706286080-zoEf?utm_source=share&utm_medium=member_desktop) when they’re off a happy path.  **What can AI Assist do?** -------------------------- **Supercharge your Assistant creation.** Save time _creating_ diverse utterances & entities and _iterating_ on your Assistant’s response content. **Handle unexpected customer interactions.** Let the assistant get users back on a happy path by answering their out-of-scope questions with dynamic generated responses. **Fill the production data gap.** Generate more data to create functional prototypes you can use to better communicate design decisions with stakeholders. How to get started ------------------ Both Generative Tasks and AI Freestyle features can be toggled on and off in the Project Settings. For Generative Tasks, you’ll notice a “Generate” button showing the option to create response variations, utterances, entities and synonyms, and no-match/no reply responses.  You can see the Freestyle feature when prototyping in Voiceflow or viewing Transcripts. Now in the test modal, if the user gives an Utterance that doesn’t match with an Intent in your Assistant project, AI Freestyle will generate a response to the user. Any AI-generated responses will be marked with the AI Assist icon, but users won’t see the AI Assist icon when your Assistant is live.  While this feature is powerful for handling those conversation edge cases, it’s still an experimental feature leveraging Large Language Models (LLMs). **It should not be used in production use cases for business critical applications because of its tendency to generate false information.** As you’re getting started, [follow along with our walkthrough](https://www.youtube.com/watch?v=_piC89vqDdU&ab_channel=Voiceflow) or [check out our documentation.](https://voiceflow.zendesk.com/hc/en-us/articles/11935569385997-What-is-AI-Assist-) How we’re already using this at Voiceflow ----------------------------------------- Our team is already experimenting with these features. Here's a live customer transcript from our internal Voiceflow Assistant, along with the insights we drew from it: _P.S. Have a moment? Try our_ [_Voiceflow Assistant_](https://www.voiceflow.com/) _and let us know what you think._ We’re proud to have built this first-of-its-kind, game-changing feature for you—and we can’t wait to see what you build with it. **Introducing Voiceflow’s AI Assist** -------------------------------------- **Accelerate your conversation design workflow** using **Generative Tasks** to generate response variations, utterances, entities and synonyms, and no-match/no reply responses.  **Prepare and uncover conversation edge cases** using **AI Freestyle** by having the assistant take over and [push the user back to their original intent](https://www.linkedin.com/posts/Braden%20Ream_following-customer-feedback-this-weekend-activity-7017917755706286080-zoEf?utm_source=share&utm_medium=member_desktop) when they’re off a happy path.  **What can AI Assist do?** -------------------------- **Supercharge your Assistant creation.** Save time _creating_ diverse utterances & entities and _iterating_ on your Assistant’s response content. **Handle unexpected customer interactions.** Let the assistant get users back on a happy path by answering their out-of-scope questions with dynamic generated responses. **Fill the production data gap.** Generate more data to create functional prototypes you can use to better communicate design decisions with stakeholders. How to get started ------------------ Both Generative Tasks and AI Freestyle features can be toggled on and off in the Project Settings. For Generative Tasks, you’ll notice a “Generate” button showing the option to create response variations, utterances, entities and synonyms, and no-match/no reply responses.  You can see the Freestyle feature when prototyping in Voiceflow or viewing Transcripts. Now in the test modal, if the user gives an Utterance that doesn’t match with an Intent in your Assistant project, AI Freestyle will generate a response to the user. Any AI-generated responses will be marked with the AI Assist icon, but users won’t see the AI Assist icon when your Assistant is live.  While this feature is powerful for handling those conversation edge cases, it’s still an experimental feature leveraging Large Language Models (LLMs). **It should not be used in production use cases for business critical applications because of its tendency to generate false information.** As you’re getting started, [follow along with our walkthrough](https://www.youtube.com/watch?v=_piC89vqDdU&ab_channel=Voiceflow) or [check out our documentation.](https://voiceflow.zendesk.com/hc/en-us/articles/11935569385997-What-is-AI-Assist-) How we’re already using this at Voiceflow ----------------------------------------- Our team is already experimenting with these features. Here's a live customer transcript from our internal Voiceflow Assistant, along with the insights we drew from it: _P.S. Have a moment? Try our_ [_Voiceflow Assistant_](https://www.voiceflow.com/) _and let us know what you think._ We’re proud to have built this first-of-its-kind, game-changing feature for you—and we can’t wait to see what you build with it.  RECOMMENDED [\ \ ### Introducing Voiceflow's GPT-powered AI Builder](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) PUBLISHED January 24, 2023 TOPICS IN THIS ARTICLE No items found. [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow\ \ A growth and product marketer at Voiceflow who loves to design, build, and grow things. Equipped with an endless curiosity and bad eyesight.](https://www.voiceflow.com/contributors/kim-lu) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/anthropic-claude-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Anthropic’s Claude is now in Voiceflow](https://www.voiceflow.com/pathways/anthropic-claude-voiceflow) [](https://www.voiceflow.com/pathways/voiceflow-raises-20m-to-build-the-future-of-conversation-design-tooling) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow raises $20M to build the future of conversation design tooling](https://www.voiceflow.com/pathways/voiceflow-raises-20m-to-build-the-future-of-conversation-design-tooling) [](https://www.voiceflow.com/pathways/ai-assist-llms-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### AI Assist: LLMs are now in Voiceflow](https://www.voiceflow.com/pathways/ai-assist-llms-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/ai-assist-llms-voiceflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/ai-assist-llms-voiceflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Enterprise | Build AI Agents for the Enterprise [](https://www.voiceflow.com/) Enterprise Cloud Build and manage AI Agents inside one secure platform ===================================================== Teams use Voiceflow to collaboratively design, develop, and launch impactful AI agents using any models and integrations. [Request a demo](https://creator.voiceflow.com/signup)    Trusted by leading companies building AI automation at scale         Voiceflow platform Flexible tools used by the world's best teams building AI Agents ---------------------------------------------------------------- Out-of-the-box chatbots and legacy tools won’t cut for scaling agents across every use case. [Platform overview](https://www.voiceflow.com/features/platform-overview) Knowledge Base (RAG) Import data and documents to the Knowledge Base to allow Agents to respond using curated data. Workflow Builder Build multi-step tasks for Agents to complete using the Workflow Builder. Data Management System Manage Agents at scale with centrally managed data across prompts, responses, intents, workflows and more. Developer Tools & APIs Build custom responses, interfaces, functions, integrations and more. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks. Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.  Train your AI agent on your data, giving it the ability to respond more accurately and perform tasks.   “Voiceflow has made creating conversation dialogue journeys easy and smooth. It has facilitated the collaboration and best practice sharing between people across different teams and countries.”  Illaria Di Donfrancesco Product Owner - AI Platforms  "With Voiceflow, users were no longer led through fixed linear flows. They were engaged with a real experience, using natural language, which revealed true-to-life results."  James Schneider Head of UX Design Developer platform Build outside the box with Voiceflow's developer platform ----------------------------------------------------------- Use developer features to extend the platform to add custom features, integrations, interfaces, and more. [Go to docs](https://www.voiceflow.com/docs)   Integrations Integrate Agents with every app in your stack --------------------------------------------- Extend what your agents can do with developer APIs for data, knowledge & interfaces. [See integrations](https://www.voiceflow.com/integrations)   {Analytics} Send Analytics Event to Segment Track agent interactions and send to Segment for analysis. [Go to Function](https://www.voiceflow.com/integrations/segment)  {CRM} Dynamic Carousel using Zendesk Help Center Articles Ship a dynamic carousel with Zendesk Help Center articles. [Get started](https://www.voiceflow.com/integrations/zendesk) .webp) {CRM} Connect with Salesforce Connect your Salesforce to power your agent responses, logic, and analytics. [Get started](https://www.voiceflow.com/integrations/salesforce)  {ecommerce} Integrate with Shopify Plus Connect your AI agent with a Shopify Plus account to power agent responses and actions. [Get started](https://www.voiceflow.com/integrations/shopify)  {Data Warehouse} Send data to Snowflake Connect your Snowflake data warehouse to power your agent responses, logic, and analytics. [Get started](https://www.voiceflow.com/integrations/snowflake) .webp) {Analytics} Analyse performance in Sigma Send agent events to Sigma for visualizing agent performance. [Get started](https://www.voiceflow.com/integrations/sigma) Use cases Automate customer support, then everything else ------------------------------------------------- Our customers are shipping impactful AI agents across their businesses.  ### Trilogy automates 60% of their L1 and L2 support tickets. Their AI agent is powered by a curated knowledge base, integrates with Zendesk, and is embedded in the help centers of 90+ products. [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)  [Read more](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) ### Roam Auto saves 30 hours per week with Voiceflow. Nick and team went from rigid out-of-the-box chatbot to a customized AI agent reduced total tickets and delivered a better customer experience. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours)  [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### We reduced ticket costs from $31 to $2. A multi-knowledge base approach paired with a Zendesk integration resolves complex user questions, including code-first API questions from technical builders. [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) .webp) [Read more](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) Enterprise Cloud Scale and security with Voiceflow Enterprise -------------------------------------------- Voiceflow helps enterprise product teams securely build, test, launch, and manage conversational AI agents at scale. [Discover Enterprise Cloud](https://www.voiceflow.com/demo)  Your AI platform of record Build, scale, and collaborate on AI products in a centralized, secure platform across every team. Get ideas to market fast Accelerate AI product team sprints to ship with speed and quality, while efficiently scaling to new use cases. Avoid vendor lock-in Bet on AI, not single vendors. Agnostically adapt to changing LLM and NLU technologies. Controlled customization Build endless API-first data and interface integrations alongside any LLM models with secure guardrails. Trusted by the best in the business   .svg)      .svg) .svg) .svg)    .svg)        .svg)  .svg)   .svg)   .svg) .svg) .svg)   .svg) .svg)    .svg) .svg) .svg)   .svg)    .svg) .svg)   .svg)   .svg) Customers Finally, a conversational AI platform your team will actually love -------------------------------------------------------------------- Hear it directly from our customers.  Illaria Di Donfrancesco Product Owner - AI Platforms “Voiceflow has made creating conversation dialogue journeys easy and smooth. It has facilitated the collaboration and best practice sharing between people across different teams and countries.”  James Schneider Head of UX Design "With Voiceflow, users were no longer led through fixed linear flows. They were engaged with a real experience, using natural language, which revealed true-to-life results."  Bartlomiej Slodyczka Chief Customer Officer "Voiceflow is the A-Z of learning and executing on AI Automation."  Rachel Whitehorn Conversational AI Designer "Voiceflow is incredibly user-friendly and has all the elements needed to make organized, collaborative bot designs. I use Voiceflow all day every day, and it's made my work life a thousand times easier."  Colin Guilfoyle Vice President of Customer Support “Of the 7,000 tickets in central support, 59% were solved completely by AI.”  Jason Bejot Director of Conversation AI Design “We chose Voiceflow because of its powerful features, ability to create interactive flows, integrated prototyping, NLU modelling, separate workspaces... the list goes on. With Voiceflow we're building interactive prototypes as we design. This is the dream. Communicating what we design – even if it's a wild, half-baked idea – is now faster, easier, and way more effective for everyone involved. The VF team is fantastic!”  Gina Riley VP UX & Conversation Design Lead “There’s only one real tool that gets the job done for me and that’s Voiceflow. If you’re looking to build a basic static design then sure, use other tools but as a conversation designer for 8+ years, the only tool that helps create flows, visualize your design in real-time (prototyping), collaborate with partners AND even publish designs to your preferred platform is Voiceflow. Beginner friendly too! No brainer right here. 😊”  Yasaswi Dandamudi Conversation Designer "With Voiceflow, it’s much easier to navigate our designs and test conversations. Our company wanted to invest in tools that would work best for our CUI designers and we’re so glad we did. We couldn’t have done it without the Voiceflow team who helped and answered any questions I had.”  Collin Carnahan Senior UX Designer & Conversational AI Strategis "Voiceflow is how we make everything accessible to designers and non-designers alike."  Ciprian Nastase Level 3 Technical Support Quality Assurance / AI Engineer "It is a very versatile tool, combining visual flows with strong API capabilities and also providing a code editor and functions, for the development-oriented people."  Victor Algaze Software Engineer "Voiceflow takes the pain out of building powerful conversational experiences in a group setting. Building conversation agents is very much a "team sport" and Voiceflow makes it pleasant (even fun) to build systems that add value to external-facing customer use-cases and also labor-saving internal conversation applications."  Daryl Thomas Virtual Demand Center Sales Technology Leader "Voiceflow is most helpful to devise and construct conversational chatbot and voice solutions in an easy and efficient way enabling us to promote and share with business stakeholders." [Show less](https://www.voiceflow.com/enterprise#) Community Get started with help from Voiceflow's global community --------------------------------------------------------- Connect with other builders, share agent functions, and learn how to create advanced AI automations. [Hire a Voicefow Expert\ \ Hire certified Voiceflow Experts to help you build your next project.](https://www.voiceflow.com/experts) [Discord Community\ \ Join our fast-growing community of AI agent designers, developers, and builders.](https://link.voiceflow.com/community) [YouTube Tutorials\ \ Learn from video tutorials created by the Voiceflow team.](https://www.youtube.com/results?search_query=voiceflow+tutorial) [Templates\ \ Discover our library of agent templates to help you get started with your use case.](https://www.voiceflow.com/templates-old) [Marketplace\ \ Explore collections of agent-building resources created by the community.](https://www.voiceflow.com/marketplace-old) [Pathways Blog\ \ A publication for designers, developers, tinkerers, and trailblazers.](https://www.voiceflow.com/pathways) Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # 6 ways to build context into your conversation designs | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) 6 ways to build context into your conversation designs ====================================================== [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) What is conversational context? ------------------------------- Conversational context allows us to interpret what has been said to us based on how new dialogs link with previous dialogs and environmental factors. In practice, if you ask someone a question and they say "yes," we humans know they are saying yes in response to our question because we in our brains contextually linked their response to our prompt. With computers, however, this isn't true. Computers are programmed around a system of inputs and outputs – meaning every new input isn't automatically associated with previous inputs unless you program it to do so. Without context, when responding "yes" to a computer-generated question, the computer may not understand what you mean. "Yes" may be the correct answer – however, without a set context, the computer may not understand which question you are answering and treat your response as a new conversation. Humans are naturally attuned to context – we're born with the ability to manage and understand it. This innate understanding, however, doesn't translate well to designing conversations for computers. Although we're designing with the end-user and humans in mind, it's essential not to forget the computers that process those designs, and how _they understand_ and process information. At Voiceflow, we think a lot about giving designers better tools to create and manage context. We believe it's one of the most critical factors in creating engaging, natural conversations. Context is often bucketed as one term that can describe the theme, but it is hard to communicate concrete applications in practice. When designing Voiceflow, we've found ourselves creating new terms to segment different approaches to context that makes it easier for our team to communicate our design choices. These 6 context terms are what we'll be explaining today, and how we enable their design in Voiceflow.  ---- 1\. Dialog context ------------------ Dialog context is the most common example of context. Dialog context is the memory of a conversation flow as it progresses through its dialog turns. _For example:_ if you are ordering a coffee and the barista asks "do you want sugar," and you say "yes," the barista knows your version of "yes" in the context of their question. This is the most common context used, and it allows for the creation of multi-turn dialogs — without it, every dialog turn would be a new conversation. In Voiceflow, dialog context is handled by "lines," which can be used to connect two dialog turns. You can create the same conversation with or without dialog context either by using an open-intent step, or a choice-intent step.  Cx Design showing the same conversation with, and without dialog context 2\. Context switching --------------------- Context switching is the act of switching the context of a conversation, and for most people, you can think of it like _changing the topic_. _For example:_ let's say you are ordering a coffee through a conversation with a barista, you are now in the "ordering coffee" context. Suddenly, you ask the barista what time the coffee shop closes — which they then answer. So, what happens now? You switched the context of the conversation from ordering a coffee to asking about store hours. A human implicitly infers that even though you switched the context of the conversation to talking about store hours, you still want to order a coffee as your intention of asking about store hours has now been fulfilled. In this scenario, the barista would "switch" back to the original context of ordering a coffee. In Voiceflow, you can context-switch using "Commands," which are found in the Home block. Commands allow you to create sub-conversations that perform a task then bring the user back to where they were in their original context once the command is complete. An example is our "store hours" command which is invoked if the user asks about our hours. In practice, this allows the user to break from ordering a coffee to ask about store hours, complete that instance and return to the original question.  Context switching using commands in Voiceflow **3\. Prime contexts** ---------------------- Using the example above, let's say instead of asking about store hours, you say you're experiencing a heart attack — now the barista knows your intention is to get medical assistance, and you no longer care about ordering a coffee. In this case, you do not switch back to your original context, and the primary or prioritized context of the conversation changes. The barista is now focused on getting you medical assistance, that becomes the conversation outcome, and thus becomes the prime context of the conversation. In Voiceflow, prime contexts are designed through continuous multi-turn dialog paths the user is following. Using open-intents, the user can switch their prime context and be on a different conversation path.  Switching prime contexts using open-intents 4\. Contextual intents ---------------------- Sticking with our coffee example, let's say you walk into the coffee shop and say, "I need help." In this case, the barista would ask to clarify what you need 'help with' and establish context for the conversation. Because you haven't entered the conversation with a specific context, 'help' could refer to many things: from finding the washroom to ordering a coffee. Later, you say, "I need help" as you are paying for your coffee. In this case, even though you said the exact same thing, "I need help," which would trigger a "help" intent, the barista knows you need help paying and will say an appropriate response. These are contextualized intents, and they are incredibly crucial to designing a great conversation, yet they are tough to design for as they add another dimension to every response. How Voiceflow handles contextual intents is through a "stack" model where each "flow" can create a layer of context within the stack. Voiceflow's context "stack" model is such a robust system that VF's CTO did a whole video explaining how it works in detail: To create contextualized intents in Voiceflow, you can create two separate flows: a "home" flow and one for "order coffee." Within each flow, you can create a command with the "Help" intent. This will allow for a contextualized response using the same "Help" intent depending on which flow, or "context layer," the user is on.  The "Root" flow has its own "Help" intent contextualized to this flow. In the example above, we can see that we are on the Root flow, which has its own "Help" intent. Because this is the root flow, our Help intent is generalized and allows the user to get medical assistance when they say "Help" as we are imagining they just walked into our store and need help. In the top right, you can see that if the user activates the "Order coffee" intent, they are brought into the "Order coffee" subflow, which you can find a picture of below.  The "order coffee" flow has its own "Help" intent contextualized to this flow. In the picture above, the "Order coffee" flow shows that this flow has its own version of the "Help" intent that is contextualized to ordering a coffee. In this case, because the user is ordering a coffee, we offer them help by showing them our menu. Contextual intents can get as personalized as you, the designer, would like, and you can infinitely stack flows to create additional layers of context. In the example above, you could add a checkout subflow within the order coffee flow with a contextualized help intent to help the user with their payment method. If Voiceflow does not have a contextualized intent for the user's current flow, the system will "look" at the next layer up. Meaning, if there is no help intent on the current flow, Voiceflow will look at the next layer up. This creates a contextual "waterfall" system that allows you to create incredibly engaging and personalized conversations. 5\. Overfilled slot context --------------------------- When ordering your coffee, the barista introduces themselves by name and asks for our order. Feeling the need to reciprocate, we introduce ourselves by name as well and say our order. At checkout, the barista asks our name for the order. To a human, the barista forgetting your name may come across as rude or awkward — after all, you stated your name at the beginning of the conversation. To an assistant, however, this is an example of incorrectly managing an overfilled slot context. Slots are the selected pieces of information we store from a conversation, and often we design conversations to only collect what is immediately relevant to our current intent. In this example, the user gave their name, but we were only looking for their order slots, so later in the conversation, when we needed their name, the assistant didn't have it. An overfilled slot context is hard to design because it requires real-world testing to see where there are frequent opportunities for overfilled slots. Once we find an opportunity, however, capturing information and using it later is a delightful experience sure to wow any user with our assistant's contextual intelligence. To use an overfilled slot context in Voiceflow, you can use dialog management and label potential overfilled slots as "not required." When you do this, the assistant will capture the slot if it's said, however, it's not required to move the conversation forward. This allows the assistant to capture information and then use it later in the conversation without explicitly asking.  In the intent editor, you can see the {name} slot being captured, but not required, and we're using it in the speak step. 6\. Environmental context ------------------------- Continuing with our coffee example, let's consider conversation factors that aren't influenced by what we say, but by the world around us. An example here would be if you were to walk into our fictitious coffee shop at closing time. Even though all other conversation contexts are the same, the dialog will be different simply because of environmental factors, in this case, the store is closed. Now, our dialog will likely be very abrupt "sorry we're closed," or potentially a rushed less fluid outcome-driven dialog if the barista is friendly enough to still try and serve you. These environmental factors take into account a non-conversation driven context that could range from the weather to time of day, or even if it's the user's birthday or returning visitors. Even the best conversation design in the world would feel like a horrible experience if it had the same experience before. Environmental factors need to be taken into account to create more natural conversations. Over time, designs may get larger and larger simply to deal with an increasing number of environmental factors that may change the conversation — and that's just good design. In Voiceflow, you can create an environmental context using the condition step to branch the conversation based on conditional statements (IF statements, like excel). In the below example, we are branching the conversation at the start after checking to see if the store is open based on the store hours variable (which we could get from an API). If the store is open, we proceed as usual, and if it's closed, we give them an appropriate "we're closed" message. This same logic could be applied to bad weather, giving the user variant messages based on the number of sessions they've had, or even a birthday message. Using environmental context, you can personalize your conversations for the user based on the world around them – as a human counterpart would.  We're checking an external variable, {store\_hours}, and giving a contextual response based on the result. Final thoughts -------------- Designing with various contexts is one of the most essential things Cx Designers can do to create great natural-feeling conversations. It's not an easy task and highlights just one layer of depth to conversation design. We're building Voiceflow with context in mind to ensure Conversation Designers have the tools they need to create exceptional, natural conversations. Voiceflow is the only visual tool that allows for the use of context. We'll continue to build on our context model to further empower the creation of amazing, natural conversations. > I'd love your feedback or thoughts, reach out to me: braden@voiceflow.com ---- 1\. Dialog context ------------------ Dialog context is the most common example of context. Dialog context is the memory of a conversation flow as it progresses through its dialog turns. _For example:_ if you are ordering a coffee and the barista asks "do you want sugar," and you say "yes," the barista knows your version of "yes" in the context of their question. This is the most common context used, and it allows for the creation of multi-turn dialogs — without it, every dialog turn would be a new conversation. In Voiceflow, dialog context is handled by "lines," which can be used to connect two dialog turns. You can create the same conversation with or without dialog context either by using an open-intent step, or a choice-intent step.  Cx Design showing the same conversation with, and without dialog context 2\. Context switching --------------------- Context switching is the act of switching the context of a conversation, and for most people, you can think of it like _changing the topic_. _For example:_ let's say you are ordering a coffee through a conversation with a barista, you are now in the "ordering coffee" context. Suddenly, you ask the barista what time the coffee shop closes — which they then answer. So, what happens now? You switched the context of the conversation from ordering a coffee to asking about store hours. A human implicitly infers that even though you switched the context of the conversation to talking about store hours, you still want to order a coffee as your intention of asking about store hours has now been fulfilled. In this scenario, the barista would "switch" back to the original context of ordering a coffee. In Voiceflow, you can context-switch using "Commands," which are found in the Home block. Commands allow you to create sub-conversations that perform a task then bring the user back to where they were in their original context once the command is complete. An example is our "store hours" command which is invoked if the user asks about our hours. In practice, this allows the user to break from ordering a coffee to ask about store hours, complete that instance and return to the original question.  Context switching using commands in Voiceflow **3\. Prime contexts** ---------------------- Using the example above, let's say instead of asking about store hours, you say you're experiencing a heart attack — now the barista knows your intention is to get medical assistance, and you no longer care about ordering a coffee. In this case, you do not switch back to your original context, and the primary or prioritized context of the conversation changes. The barista is now focused on getting you medical assistance, that becomes the conversation outcome, and thus becomes the prime context of the conversation. In Voiceflow, prime contexts are designed through continuous multi-turn dialog paths the user is following. Using open-intents, the user can switch their prime context and be on a different conversation path.  Switching prime contexts using open-intents 4\. Contextual intents ---------------------- Sticking with our coffee example, let's say you walk into the coffee shop and say, "I need help." In this case, the barista would ask to clarify what you need 'help with' and establish context for the conversation. Because you haven't entered the conversation with a specific context, 'help' could refer to many things: from finding the washroom to ordering a coffee. Later, you say, "I need help" as you are paying for your coffee. In this case, even though you said the exact same thing, "I need help," which would trigger a "help" intent, the barista knows you need help paying and will say an appropriate response. These are contextualized intents, and they are incredibly crucial to designing a great conversation, yet they are tough to design for as they add another dimension to every response. How Voiceflow handles contextual intents is through a "stack" model where each "flow" can create a layer of context within the stack. Voiceflow's context "stack" model is such a robust system that VF's CTO did a whole video explaining how it works in detail: To create contextualized intents in Voiceflow, you can create two separate flows: a "home" flow and one for "order coffee." Within each flow, you can create a command with the "Help" intent. This will allow for a contextualized response using the same "Help" intent depending on which flow, or "context layer," the user is on.  The "Root" flow has its own "Help" intent contextualized to this flow. In the example above, we can see that we are on the Root flow, which has its own "Help" intent. Because this is the root flow, our Help intent is generalized and allows the user to get medical assistance when they say "Help" as we are imagining they just walked into our store and need help. In the top right, you can see that if the user activates the "Order coffee" intent, they are brought into the "Order coffee" subflow, which you can find a picture of below.  The "order coffee" flow has its own "Help" intent contextualized to this flow. In the picture above, the "Order coffee" flow shows that this flow has its own version of the "Help" intent that is contextualized to ordering a coffee. In this case, because the user is ordering a coffee, we offer them help by showing them our menu. Contextual intents can get as personalized as you, the designer, would like, and you can infinitely stack flows to create additional layers of context. In the example above, you could add a checkout subflow within the order coffee flow with a contextualized help intent to help the user with their payment method. If Voiceflow does not have a contextualized intent for the user's current flow, the system will "look" at the next layer up. Meaning, if there is no help intent on the current flow, Voiceflow will look at the next layer up. This creates a contextual "waterfall" system that allows you to create incredibly engaging and personalized conversations. 5\. Overfilled slot context --------------------------- When ordering your coffee, the barista introduces themselves by name and asks for our order. Feeling the need to reciprocate, we introduce ourselves by name as well and say our order. At checkout, the barista asks our name for the order. To a human, the barista forgetting your name may come across as rude or awkward — after all, you stated your name at the beginning of the conversation. To an assistant, however, this is an example of incorrectly managing an overfilled slot context. Slots are the selected pieces of information we store from a conversation, and often we design conversations to only collect what is immediately relevant to our current intent. In this example, the user gave their name, but we were only looking for their order slots, so later in the conversation, when we needed their name, the assistant didn't have it. An overfilled slot context is hard to design because it requires real-world testing to see where there are frequent opportunities for overfilled slots. Once we find an opportunity, however, capturing information and using it later is a delightful experience sure to wow any user with our assistant's contextual intelligence. To use an overfilled slot context in Voiceflow, you can use dialog management and label potential overfilled slots as "not required." When you do this, the assistant will capture the slot if it's said, however, it's not required to move the conversation forward. This allows the assistant to capture information and then use it later in the conversation without explicitly asking.  In the intent editor, you can see the {name} slot being captured, but not required, and we're using it in the speak step. 6\. Environmental context ------------------------- Continuing with our coffee example, let's consider conversation factors that aren't influenced by what we say, but by the world around us. An example here would be if you were to walk into our fictitious coffee shop at closing time. Even though all other conversation contexts are the same, the dialog will be different simply because of environmental factors, in this case, the store is closed. Now, our dialog will likely be very abrupt "sorry we're closed," or potentially a rushed less fluid outcome-driven dialog if the barista is friendly enough to still try and serve you. These environmental factors take into account a non-conversation driven context that could range from the weather to time of day, or even if it's the user's birthday or returning visitors. Even the best conversation design in the world would feel like a horrible experience if it had the same experience before. Environmental factors need to be taken into account to create more natural conversations. Over time, designs may get larger and larger simply to deal with an increasing number of environmental factors that may change the conversation — and that's just good design. In Voiceflow, you can create an environmental context using the condition step to branch the conversation based on conditional statements (IF statements, like excel). In the below example, we are branching the conversation at the start after checking to see if the store is open based on the store hours variable (which we could get from an API). If the store is open, we proceed as usual, and if it's closed, we give them an appropriate "we're closed" message. This same logic could be applied to bad weather, giving the user variant messages based on the number of sessions they've had, or even a birthday message. Using environmental context, you can personalize your conversations for the user based on the world around them – as a human counterpart would.  We're checking an external variable, {store\_hours}, and giving a contextual response based on the result. Final thoughts -------------- Designing with various contexts is one of the most essential things Cx Designers can do to create great natural-feeling conversations. It's not an easy task and highlights just one layer of depth to conversation design. We're building Voiceflow with context in mind to ensure Conversation Designers have the tools they need to create exceptional, natural conversations. Voiceflow is the only visual tool that allows for the use of context. We'll continue to build on our context model to further empower the creation of amazing, natural conversations. > I'd love your feedback or thoughts, reach out to me: braden@voiceflow.com  RECOMMENDED [\ \ ### Expanding the definition of conversation design](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) PUBLISHED July 13, 2020 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you should A/B test your conversational experiences](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [](https://www.voiceflow.com/pathways/cxd-center-of-excellence) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Your CxD center of excellence (and why you don’t need one right now)](https://www.voiceflow.com/pathways/cxd-center-of-excellence) [](https://www.voiceflow.com/pathways/celene-osiecka-blog-post) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to design intelligent conversational interfaces](https://www.voiceflow.com/pathways/celene-osiecka-blog-post) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/6-ways-to-build-context-into-your-conversation-designs#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/6-ways-to-build-context-into-your-conversation-designs#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Automation design: The evolution of conversation design | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Automation Design: The Evolution of Conversation Design ======================================================= [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) One cold night in Tokyo, I discussed our industry in a post-ChatGPT world with Greg Bennet—the director of conversation design at Salesforce. We talked about the future of conversation design itself. Is there a future for us? Why are we called conversation designers in the first place? Our questions were answered with shared silence. The title of “conversation designer” that had once rallied our industry behind a common flag now held baggage. Conversation design began to represent a legacy of scripted, deterministic chatbots. This misunderstanding of conversation design’s goal is understandable, but regrettable. To an exec thinking through AI strategy, conversation design sounds like it focuses solely on copy. But this function is changing quickly. The goal of conversation design is more relevant than ever: to ensure customers have a delightful, easy experience with automated systems. After all, it’s one thing to build an automated system—it’s another thing for it to be easy to use and easy to adopt by customers.  Why it’s time for a change: Defining automation design (AxD) ------------------------------------------------------------- I believe conversation design needs to evolve as a title and practice. Automation design (AxD) is my pick, but I’m open to ideas. Automation design would be defined as the application of UX best practices and design-centricity to AI-powered automated systems. As AI models improve, I believe the number of everyday touchpoints we’ll have with automated systems is going to increase exponentially. Today, customer support is the most obvious use case for AI assistants—but I imagine sales, internal operations, healthcare, and so much more will be improved through AI-powered automation. This title change has three benefits. First, this title change allows conversation designers to free themselves from the stigma that CxD is simply copywriting or that their role is from a bygone era of manual chatbot creation. Second, this title change makes it clear that this role helps businesses automate and save costs—all while maintaining a great user experience. And third, automation design frees the function from being tied to any one interface. Automated experiences can be much more than just chatbots or voicebots—this is already the reality for conversation designers. We know this from working with 200,000 global users at Voiceflow whose work consists of automation logic and business process logic behind the scenes, along with their CxD tasks. Conversation designers are already doing this work, and it’s time they receive the recognition they deserve. > In 5 years, I predict automation designers will be working across every industry and use case, ensuring the increasing number of automated experiences we interact with everyday are useful and delightful. The rise of AI is going to lead to a surge in automation across use cases. It’s time for conversation design to evolve and grow into a wider, more critical function. In 5 years, I predict automation designers will be working across every industry and use case, ensuring the increasing number of automated experiences we interact with everyday are useful and delightful. At Voiceflow, we’ll do our part by continuing to build the most collaborative, extensible, and powerful tool to help you automate any use case with AI agents. Why it’s time for a change: Defining automation design (AxD) ------------------------------------------------------------- I believe conversation design needs to evolve as a title and practice. Automation design (AxD) is my pick, but I’m open to ideas. Automation design would be defined as the application of UX best practices and design-centricity to AI-powered automated systems. As AI models improve, I believe the number of everyday touchpoints we’ll have with automated systems is going to increase exponentially. Today, customer support is the most obvious use case for AI assistants—but I imagine sales, internal operations, healthcare, and so much more will be improved through AI-powered automation. This title change has three benefits. First, this title change allows conversation designers to free themselves from the stigma that CxD is simply copywriting or that their role is from a bygone era of manual chatbot creation. Second, this title change makes it clear that this role helps businesses automate and save costs—all while maintaining a great user experience. And third, automation design frees the function from being tied to any one interface. Automated experiences can be much more than just chatbots or voicebots—this is already the reality for conversation designers. We know this from working with 200,000 global users at Voiceflow whose work consists of automation logic and business process logic behind the scenes, along with their CxD tasks. Conversation designers are already doing this work, and it’s time they receive the recognition they deserve. > In 5 years, I predict automation designers will be working across every industry and use case, ensuring the increasing number of automated experiences we interact with everyday are useful and delightful. The rise of AI is going to lead to a surge in automation across use cases. It’s time for conversation design to evolve and grow into a wider, more critical function. In 5 years, I predict automation designers will be working across every industry and use case, ensuring the increasing number of automated experiences we interact with everyday are useful and delightful. At Voiceflow, we’ll do our part by continuing to build the most collaborative, extensible, and powerful tool to help you automate any use case with AI agents.  RECOMMENDED [](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design#) PUBLISHED January 9, 2024 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you should A/B test your conversational experiences](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [](https://www.voiceflow.com/pathways/prompt-chaining-conversational-ai) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Prompt chaining for conversational AI](https://www.voiceflow.com/pathways/prompt-chaining-conversational-ai) [](https://www.voiceflow.com/pathways/best-practices-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Curiosity and conversation beat any best practice](https://www.voiceflow.com/pathways/best-practices-conversation-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Anthropic’s Claude is now in Voiceflow | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Anthropic’s Claude is now in Voiceflow ====================================== [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow](https://www.voiceflow.com/contributors/kim-lu) There’s been [lots of excitement](https://www.voiceflow.com/blog/claude-llm-ai-assistant) around [Claude](https://www.anthropic.com/product) , Anthropic’s impressively quick new model. In fact, we’ve heard many folks in the community tell us they're _still_ on the waiting list to get access. We’ve got some good news: Claude is now available in [Voiceflow’s AI Builder](https://www.voiceflow.com/blog/introducing-voiceflows-gpt-powered-ai-builder) . And there’s no waiting list to try it out in your Voiceflow assistant—Claude and Claude Instant V1 (100K) are both already there, along with several other models like GPT-3.5-Turbo and GPT-4.   How to get started ------------------ When you’re on Voiceflow’s dashboard, select the AI Builder project type. From there, you’ll be able to jump in and start playing around with Claude in our **Knowledge Base**, **AI Steps,** and **Generative Global No Match** features.  What to expect -------------- Claude is fast, concise, and direct. We found that it is a methodical model and performs well when given a focused space of information. Therefore it pairs especially well with our Knowledge Base feature or when it is given organized lists.  Using Claude with our Knowledge Base feature More on the way --------------- This is just the start of multi-LLM vendor support in Voiceflow. What other LLM model vendors would you like us to support? [Tell us on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7071497922571354112?updateEntityUrn=urn%3Ali%3Afs_feedUpdate%3A%28V2%2Curn%3Ali%3Aactivity%3A7071497922571354112%29) . How to get started ------------------ When you’re on Voiceflow’s dashboard, select the AI Builder project type. From there, you’ll be able to jump in and start playing around with Claude in our **Knowledge Base**, **AI Steps,** and **Generative Global No Match** features.  What to expect -------------- Claude is fast, concise, and direct. We found that it is a methodical model and performs well when given a focused space of information. Therefore it pairs especially well with our Knowledge Base feature or when it is given organized lists.  Using Claude with our Knowledge Base feature More on the way --------------- This is just the start of multi-LLM vendor support in Voiceflow. What other LLM model vendors would you like us to support? [Tell us on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7071497922571354112?updateEntityUrn=urn%3Ali%3Afs_feedUpdate%3A%28V2%2Curn%3Ali%3Aactivity%3A7071497922571354112%29) .  RECOMMENDED [\ \ ### Experts share 3 expectations about LLMs to set with business stakeholders](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders) PUBLISHED June 15, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow\ \ A growth and product marketer at Voiceflow who loves to design, build, and grow things. Equipped with an endless curiosity and bad eyesight.](https://www.voiceflow.com/contributors/kim-lu) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Introducing Voiceflow's GPT-powered AI Builder](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) [](https://www.voiceflow.com/pathways/announcing-vfv2) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Introducing #VFV2: The next phase of Voiceflow](https://www.voiceflow.com/pathways/announcing-vfv2) [](https://www.voiceflow.com/pathways/announcing-our-2020-voiceflow-community-award-winners) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Announcing Voiceflow's 2020 Community Award winners!](https://www.voiceflow.com/pathways/announcing-our-2020-voiceflow-community-award-winners) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/anthropic-claude-voiceflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/anthropic-claude-voiceflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # AI will kill apps and I can’t wait | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) AI will kill apps and I can’t wait ================================== [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) I have a confession: I hate apps. Okay. Maybe that’s a little too far. In my years surfing the web—wow, that ages me—I’ve found most apps to be clunky, disjointed, and require far too much mental bandwidth to navigate. In [2022](https://www.businessofapps.com/data/app-statistics/) , smartphone users downloaded 87 billion apps worldwide. That’s dozens of smartphone apps, hundreds of interfaces, and an endless collection of complex passwords. If our best hope for the future is a sea of infinite apps that handle one function before bouncing us to another equally frustrating app, then _count me out_. Instead, I’d like to imagine a future where I could tell my digital assistant what I want to accomplish and it handles the tasks for me behind the scenes through a simple conversational interface. No more specialized apps for this, another for that. Just dynamic conversations that flow naturally, sensing context and adapting on the fly using an intelligent AI.  AI-powered ephemeral apps are science fiction --------------------------------------------- In this vision of the future, traditional apps would be replaced by ephemeral “micro apps”—temporary, single-purpose tools generated by AI in response to a user’s specific needs. Imagine requesting a dinner party planner and your AI assistant generates an app that creates guest lists, shopping lists, recipes, and timers which vanish once the leftovers are cleared. Or asking your assistant for a personalized fitness plan and getting an app tailored to your goals that fades away until the next scheduled workout. These ephemeral apps are a vision of the hyper-personalized future AI could enable—bespoke interactions crafted for the moment, where each conversation becomes an opportunity for us to surprise and delight users. At the moment, it might feel like Star Trek-level science fiction. But, AI-generated personalized experiences prompted by conversation alone are possible right now. There’s been rapid progress in how [large language models (LLMs)](https://www.voiceflow.com/blog/dominant-discourse-llms) are being applied to both consumer and business products with the rise of ChatGPT and [Claude](https://www.voiceflow.com/blog/claude-llm-ai-assistant) . What was promised by tech giants feels like it’s actually here—a personalized assistant that can do more than turn on the lights or tell you the weather. Those who still have their heads buried in the Web 2.0 sand will see the rise of LLMs as an opportunity to integrate it into existing applications (see [Microsoft](https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/) and [Google’s](https://workspace.google.com/blog/product-announcements/generative-ai) AI announcements). But this is just a stopgap. [Conversational AI](https://www.voiceflow.com/blog/future-of-conversational-ai) isn’t another tool in our crowded app drawers—this industry has the potential to replace the drawers altogether. Why start a new spreadsheet or draft an email? Why not ask your assistant to build a budget template from scratch or draft and send that email to your accountant? > "Conversational AI isn’t another tool in our crowded app drawers—this industry has the potential to replace the drawers altogether." One day our most common needs will be easily managed by voice and text requests of our AI assistants, rather than consulting the cacophony of ill-matched apps on our phones. Tech neologisms like "software" and "interface" will disappear from our vocabulary. We won't care _how_ the AI does what it does. We'll just expect that it does the tasks we require of it—quickly and effectively. Tech companies and consulting firms are already bracing for this transition. They're pouring resources into conversational AI in hopes they can outrun obsolescence. The age of AI may arrive one small convenience at a time—today it’s [ChatGPT](https://www.voiceflow.com/blog/when-to-use-chatgpt-gpt-4) writing recipes; tomorrow, it’s the assistant guiding you through each step as you cook. But make no mistake, apps are merely a stop on the journey to a far more innovative destination. The AI revolution has begun, and your apps—all your precious apps—are about to be left in the dust. AI-powered ephemeral apps are science fiction --------------------------------------------- In this vision of the future, traditional apps would be replaced by ephemeral “micro apps”—temporary, single-purpose tools generated by AI in response to a user’s specific needs. Imagine requesting a dinner party planner and your AI assistant generates an app that creates guest lists, shopping lists, recipes, and timers which vanish once the leftovers are cleared. Or asking your assistant for a personalized fitness plan and getting an app tailored to your goals that fades away until the next scheduled workout. These ephemeral apps are a vision of the hyper-personalized future AI could enable—bespoke interactions crafted for the moment, where each conversation becomes an opportunity for us to surprise and delight users. At the moment, it might feel like Star Trek-level science fiction. But, AI-generated personalized experiences prompted by conversation alone are possible right now. There’s been rapid progress in how [large language models (LLMs)](https://www.voiceflow.com/blog/dominant-discourse-llms) are being applied to both consumer and business products with the rise of ChatGPT and [Claude](https://www.voiceflow.com/blog/claude-llm-ai-assistant) . What was promised by tech giants feels like it’s actually here—a personalized assistant that can do more than turn on the lights or tell you the weather. Those who still have their heads buried in the Web 2.0 sand will see the rise of LLMs as an opportunity to integrate it into existing applications (see [Microsoft](https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/) and [Google’s](https://workspace.google.com/blog/product-announcements/generative-ai) AI announcements). But this is just a stopgap. [Conversational AI](https://www.voiceflow.com/blog/future-of-conversational-ai) isn’t another tool in our crowded app drawers—this industry has the potential to replace the drawers altogether. Why start a new spreadsheet or draft an email? Why not ask your assistant to build a budget template from scratch or draft and send that email to your accountant? > "Conversational AI isn’t another tool in our crowded app drawers—this industry has the potential to replace the drawers altogether." One day our most common needs will be easily managed by voice and text requests of our AI assistants, rather than consulting the cacophony of ill-matched apps on our phones. Tech neologisms like "software" and "interface" will disappear from our vocabulary. We won't care _how_ the AI does what it does. We'll just expect that it does the tasks we require of it—quickly and effectively. Tech companies and consulting firms are already bracing for this transition. They're pouring resources into conversational AI in hopes they can outrun obsolescence. The age of AI may arrive one small convenience at a time—today it’s [ChatGPT](https://www.voiceflow.com/blog/when-to-use-chatgpt-gpt-4) writing recipes; tomorrow, it’s the assistant guiding you through each step as you cook. But make no mistake, apps are merely a stop on the journey to a far more innovative destination. The AI revolution has begun, and your apps—all your precious apps—are about to be left in the dust.  RECOMMENDED [\ \ ### Prompt chaining for conversational AI](https://www.voiceflow.com/pathways/prompt-chaining-conversational-ai) PUBLISHED July 31, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### AI automation on a budget: Getting started with high ROI use cases](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [](https://www.voiceflow.com/pathways/future-of-conversational-ai) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### The future of conversational AI](https://www.voiceflow.com/pathways/future-of-conversational-ai) [](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/ai-will-kill-apps#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/ai-will-kill-apps#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How to create an Alexa skill with GPT-4 and Voiceflow | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) How to create an Alexa skill with GPT-4 and Voiceflow ===================================================== [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) **Note: The publish to alexa feature has been deprecated. To connect your Voiceflow project to Alexa you can do so using our Dialog API. You can find a tutorial from one of our team members** [**here.**](https://xavidop.me/alexa/2023-11-21-voiceflow-alexa/) Creating an Alexa skill is an exciting educational project—and with the integration of powerful AI like GPT-4, the possibilities are endless. In this tutorial, you'll learn how to create your own Alexa AI skill using different Voiceflow steps like the API step, JavaScript step, and the Conditions step. You'll walk away with a solid foundation for designing and deploying your own voice-powered GPT chatbot, and the knowledge to customize it further to suit your specific needs. _Please note that this tutorial is for educational purposes only and should not be used for anything other than that._  Step 1: Set up Voiceflow and GPT-4 ---------------------------------- First, sign into your [Voiceflow](https://creator.voiceflow.com/) account and create a new Alexa assistant.  Name your Assistant and choose "Build AI Assistant" Select **Amazon Alexa** for the Channel, then choose the locale you want to use. (_note: this feature has been deprecated)_  Be sure to **add the necessary variables** to your project, such as: _gpt\_reply_ _user\_reply_ _messages_ _OpenAIAPIKey_ _OpenAIOrg_ _request_ _isEnd_ Next, populate the **Open AI Settings** step with your **OpenAI API key** and your **Org ID** (_if you have one_).  Then use a **Javascript Step** below to initiate the variables and push the first content to the {**messages}** array. \[.code-tag\]gpt\_reply = '' user\_reply = '' messages = \[\] messages.push({"role": "system","content": \`You are a AI Assistant who loves helping users! If at anytime the user wants to stop the conversation or means something similar to \\"no\\", \\"stop\\" or don't want any more help, return only \\"1\\" as your answer. If that's not the case, answer the question as truthfully as possible using the provided context, and if you don't know the answer, say \\"I don't know.\\".\`})\[.code-tag\] Here, we're adding some content and rules for the “system” to act as an AI Assistant, answer the user, or return 1 whenever the user wants to end the conversation and doesn’t need more help. _You are an AI Assistant who loves helping users! If at anytime the user wants to stop the conversation or means something similar to ”no”, ”stop” or don’t want any more help, return only ”1” as your answer. If that’s not the case, answer the question as truthfully as possible using the provided context, and if you don’t know the answer, say ”I don’t know”._ Step 2: Handling user input and generating responses ---------------------------------------------------- Capture user input using a **Capture step** and map the entire user’s reply to the {**user\_reply**} variable.  Create a JavaScript code step to process user input, add **user reply** to the **messages array**, and **generate a request object** for the OpenAI API. The request object should include: * Model (_here we are using “gpt-4–0314”_) * Max tokens for the response * Temperature setting (e.g., 0.4) * Messages array \[.code-tag\]messages.push({"role": "user", "content": user\_reply}) request = { "model": "gpt-4-0314", "max\_tokens": 256, "temperature": 0.4, "messages": messages } request = JSON.stringify(request)\[.code-tag\] Last step is to **JSON.stringify** the **request** so we can use it in the API step without any issue.  Next, use the API step to send the request object to the OpenAI API and store the response in the GPT reply variable.  Set up the API step in Voiceflow using the following configuration: * Endpoint: [https://api.openai.com/v1/chat/completions](https://api.openai.com/v1/chat/completions) * Method: **POST** * Headers: \[.code-tag\]Content-Type: application/json Authorization: Bearer {OpenAIAPIKey} OpenAI-Organization: {OpenAIOrg}\[.code-tag\] If you’re not going to use the OpenAI-Organization, simply remove it from the headers. * Body: {request} (the JSON object you created in your Javascript Step) * Capture Response: \[.code-tag\]response.choices\[0\].message.content {gpt\_reply}\[.code-tag\]  The Voiceflow API Step setup to make the OpenAI API request Step 3: Processing GPT-4 responses and managing conversation flow ----------------------------------------------------------------- Create a condition step to check if the **gpt\_reply** value is equal to “**1**.” If it is, this indicates that the user wants to stop the conversation, and the skill should respond with a farewell message like “Chat soon!”  If the **gpt\_reply** value is not equal to “**1**,” continue the conversation by sending the **gpt\_reply** to the user in a **Speak step**.  The **Javascript Step** below the Speak step add the **gtp\_reply** to the **message** array as the assistant content. \[.code-tag\]messages.push({"role": "assistant", "content": gpt\_reply})\[.code-tag\]  Loop back to the **Capture step** to allow the user to ask another question or end the conversation if **gpt\_reply** is “**1**”.  Step 4: Testing and deploying your Alexa skill ---------------------------------------------- **1.** **Test** your assistant in the **Voiceflow test tool**. **2.** Make any necessary adjustments to improve the conversation flow and user experience. **3\. Publish** your skill to the **Amazon Alexa Developer Console**.  Testing your Voiceflow Assistant on the Alexa Developer Console Congrats! You’ve successfully created an Alexa skill using Voiceflow and GPT-4. From here, I encourage you to have some fun exploring the capabilities of OpenAI’s GPT models and engaging in conversations with your new Alexa AI assistant. Step 1: Set up Voiceflow and GPT-4 ---------------------------------- First, sign into your [Voiceflow](https://creator.voiceflow.com/) account and create a new Alexa assistant.  Name your Assistant and choose "Build AI Assistant" Select **Amazon Alexa** for the Channel, then choose the locale you want to use. (_note: this feature has been deprecated)_  Be sure to **add the necessary variables** to your project, such as: _gpt\_reply_ _user\_reply_ _messages_ _OpenAIAPIKey_ _OpenAIOrg_ _request_ _isEnd_ Next, populate the **Open AI Settings** step with your **OpenAI API key** and your **Org ID** (_if you have one_).  Then use a **Javascript Step** below to initiate the variables and push the first content to the {**messages}** array. \[.code-tag\]gpt\_reply = '' user\_reply = '' messages = \[\] messages.push({"role": "system","content": \`You are a AI Assistant who loves helping users! If at anytime the user wants to stop the conversation or means something similar to \\"no\\", \\"stop\\" or don't want any more help, return only \\"1\\" as your answer. If that's not the case, answer the question as truthfully as possible using the provided context, and if you don't know the answer, say \\"I don't know.\\".\`})\[.code-tag\] Here, we're adding some content and rules for the “system” to act as an AI Assistant, answer the user, or return 1 whenever the user wants to end the conversation and doesn’t need more help. _You are an AI Assistant who loves helping users! If at anytime the user wants to stop the conversation or means something similar to ”no”, ”stop” or don’t want any more help, return only ”1” as your answer. If that’s not the case, answer the question as truthfully as possible using the provided context, and if you don’t know the answer, say ”I don’t know”._ Step 2: Handling user input and generating responses ---------------------------------------------------- Capture user input using a **Capture step** and map the entire user’s reply to the {**user\_reply**} variable.  Create a JavaScript code step to process user input, add **user reply** to the **messages array**, and **generate a request object** for the OpenAI API. The request object should include: * Model (_here we are using “gpt-4–0314”_) * Max tokens for the response * Temperature setting (e.g., 0.4) * Messages array \[.code-tag\]messages.push({"role": "user", "content": user\_reply}) request = { "model": "gpt-4-0314", "max\_tokens": 256, "temperature": 0.4, "messages": messages } request = JSON.stringify(request)\[.code-tag\] Last step is to **JSON.stringify** the **request** so we can use it in the API step without any issue.  Next, use the API step to send the request object to the OpenAI API and store the response in the GPT reply variable.  Set up the API step in Voiceflow using the following configuration: * Endpoint: [https://api.openai.com/v1/chat/completions](https://api.openai.com/v1/chat/completions) * Method: **POST** * Headers: \[.code-tag\]Content-Type: application/json Authorization: Bearer {OpenAIAPIKey} OpenAI-Organization: {OpenAIOrg}\[.code-tag\] If you’re not going to use the OpenAI-Organization, simply remove it from the headers. * Body: {request} (the JSON object you created in your Javascript Step) * Capture Response: \[.code-tag\]response.choices\[0\].message.content {gpt\_reply}\[.code-tag\]  The Voiceflow API Step setup to make the OpenAI API request Step 3: Processing GPT-4 responses and managing conversation flow ----------------------------------------------------------------- Create a condition step to check if the **gpt\_reply** value is equal to “**1**.” If it is, this indicates that the user wants to stop the conversation, and the skill should respond with a farewell message like “Chat soon!”  If the **gpt\_reply** value is not equal to “**1**,” continue the conversation by sending the **gpt\_reply** to the user in a **Speak step**.  The **Javascript Step** below the Speak step add the **gtp\_reply** to the **message** array as the assistant content. \[.code-tag\]messages.push({"role": "assistant", "content": gpt\_reply})\[.code-tag\]  Loop back to the **Capture step** to allow the user to ask another question or end the conversation if **gpt\_reply** is “**1**”.  Step 4: Testing and deploying your Alexa skill ---------------------------------------------- **1.** **Test** your assistant in the **Voiceflow test tool**. **2.** Make any necessary adjustments to improve the conversation flow and user experience. **3\. Publish** your skill to the **Amazon Alexa Developer Console**.  Testing your Voiceflow Assistant on the Alexa Developer Console Congrats! You’ve successfully created an Alexa skill using Voiceflow and GPT-4. From here, I encourage you to have some fun exploring the capabilities of OpenAI’s GPT models and engaging in conversations with your new Alexa AI assistant.  RECOMMENDED [\ \ ### 17 prompts for building AI apps in Voiceflow](https://www.voiceflow.com/pathways/prompts-for-building-ai-apps) PUBLISHED April 13, 2023 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Harness the power of OpenAI’s Whisper model for ASR with Voiceflow](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [](https://www.voiceflow.com/pathways/transforming-your-raspberry-pi-into-a-simple-voice-assistant) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Transforming your Raspberry PI into a simple voice assistant](https://www.voiceflow.com/pathways/transforming-your-raspberry-pi-into-a-simple-voice-assistant) [](https://www.voiceflow.com/pathways/a-quick-guide-on-integrating-alexa-with-microsoft-luis) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### A quick guide on integrating Alexa with Microsoft LUIS](https://www.voiceflow.com/pathways/a-quick-guide-on-integrating-alexa-with-microsoft-luis) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/alexa-skill-chat-gpt-4#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/alexa-skill-chat-gpt-4#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 5 powerful templates for AI automation agencies | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) 5 powerful templates for AI automation agencies =============================================== [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow](https://www.voiceflow.com/contributors/kim-lu) AI automation agencies constantly deal with a whole lot of requests. Building website chatbots, running customer support on autopilot, automating lead generation—the list is long. We know managing all these requests is a ton of work. So to help kickstart your next project, we’ve rounded up a collection of templates. **To get started:** pick a template below, clone it to your Voiceflow workspace, watch the tutorial, and refine the flow so it works for you and your client.  1\. Automate lead generation with Zapier and Voiceflow ------------------------------------------------------ Sourced from Voiceflow’s Discord community, [Vinera AI](https://www.youtube.com/@VineraAI) created a chatbot that automates sending lead information into a Google spreadsheet using our API step and Zapier. Vinera AI partners with businesses to maximize efficiency through AI-powered tools. By leveraging chatbots, automations, and more, they help companies reclaim valuable time and resources.  Get the template [**here**](https://www.voiceflow.com/templates/lead-gen-zapier) **.** 2\. Automate your Zendesk ticket creation ----------------------------------------- This template walks through how to create a chatbot that sends a ticket to Zendesk. It shows you how to use our API step to call the Zendesk ticket API and you might use our API step to retrieve user information from a database to execute plan routing.  Get the template [**here**](https://www.voiceflow.com/templates/zendesk-tickets) **.** 3\. Create an AI feature recommender powered by your own sources ---------------------------------------------------------------- Level up your support assistant project with a built-in feature recommender. This assistant will take a customer’s goals, then recommend specific products in your client’s catalog accordingly. Best of all, cut down your creation time by uploading sources in the form of a website or document into our knowledge base to power your assistant’s responses.  Get the template [**here**](https://www.voiceflow.com/templates/ai-recommender) **.** 4\. Create an AI FAQ that recommends sources -------------------------------------------- Learn how to use Voiceflow’s knowledge base API to unlock advanced functionalities such as including the source that a response came from. Add the source as a card that the user can then navigate to.  Get the template [**here**](https://www.voiceflow.com/templates/sources-recommender) **.** 5\. AI analytics summary ------------------------ Create an AI analytics summary bot that takes complex information (e.g. from a report or dashboard) and transforms it into something digestible for the user. Using our template, pull information from an API, then let GPT summarize it to the user.  Get the template [**here**](https://www.voiceflow.com/templates/build-an-ai-analytics-summary-bot) **.** Community templates—built by you, for you ----------------------------------------- Below, you'll find templates built by various AI automation agencies in our community. Browse from this list of use cases, from building a lead generation bot to creating a WhatsApp AI assistant for a local business. ### 1\. Integrating StackAI with Voiceflow for a Youtube AI bot Creator [RedAI](https://www.youtube.com/@RedAIToronto) walks through how to power responses in your chatbot from a Youtube video using StackAI and Voiceflow. ### 2\. Building an AI automation chatbot for e-commerce Join creator [erdinc](https://www.youtube.com/@ciftcierdinc) to create an e-commerce focused AI automation chatbot using Zapier and Voiceflow. ### 3\. Building a martial arts AI chatbot Follow [RedAI’s](https://www.youtube.com/@RedAIToronto) live build of a lead capture AI chatbot for a martial arts company using Voiceflow and Zapier. ### 4\. Building a WhatsApp AI customer service chatbot for a local business in 4 minutes with Voiceflow [Vinera AI](https://www.youtube.com/@VineraAI) dives into a four-minute tutorial to build a WhatsApp AI chatbot for dental practices using Voiceflow. This chatbot enhances customer service, facilitates appointment bookings, and answers patients' queries instantly. 1\. Automate lead generation with Zapier and Voiceflow ------------------------------------------------------ Sourced from Voiceflow’s Discord community, [Vinera AI](https://www.youtube.com/@VineraAI) created a chatbot that automates sending lead information into a Google spreadsheet using our API step and Zapier. Vinera AI partners with businesses to maximize efficiency through AI-powered tools. By leveraging chatbots, automations, and more, they help companies reclaim valuable time and resources.  Get the template [**here**](https://www.voiceflow.com/templates/lead-gen-zapier) **.** 2\. Automate your Zendesk ticket creation ----------------------------------------- This template walks through how to create a chatbot that sends a ticket to Zendesk. It shows you how to use our API step to call the Zendesk ticket API and you might use our API step to retrieve user information from a database to execute plan routing.  Get the template [**here**](https://www.voiceflow.com/templates/zendesk-tickets) **.** 3\. Create an AI feature recommender powered by your own sources ---------------------------------------------------------------- Level up your support assistant project with a built-in feature recommender. This assistant will take a customer’s goals, then recommend specific products in your client’s catalog accordingly. Best of all, cut down your creation time by uploading sources in the form of a website or document into our knowledge base to power your assistant’s responses.  Get the template [**here**](https://www.voiceflow.com/templates/ai-recommender) **.** 4\. Create an AI FAQ that recommends sources -------------------------------------------- Learn how to use Voiceflow’s knowledge base API to unlock advanced functionalities such as including the source that a response came from. Add the source as a card that the user can then navigate to.  Get the template [**here**](https://www.voiceflow.com/templates/sources-recommender) **.** 5\. AI analytics summary ------------------------ Create an AI analytics summary bot that takes complex information (e.g. from a report or dashboard) and transforms it into something digestible for the user. Using our template, pull information from an API, then let GPT summarize it to the user.  Get the template [**here**](https://www.voiceflow.com/templates/build-an-ai-analytics-summary-bot) **.** Community templates—built by you, for you ----------------------------------------- Below, you'll find templates built by various AI automation agencies in our community. Browse from this list of use cases, from building a lead generation bot to creating a WhatsApp AI assistant for a local business. ### 1\. Integrating StackAI with Voiceflow for a Youtube AI bot Creator [RedAI](https://www.youtube.com/@RedAIToronto) walks through how to power responses in your chatbot from a Youtube video using StackAI and Voiceflow. ### 2\. Building an AI automation chatbot for e-commerce Join creator [erdinc](https://www.youtube.com/@ciftcierdinc) to create an e-commerce focused AI automation chatbot using Zapier and Voiceflow. ### 3\. Building a martial arts AI chatbot Follow [RedAI’s](https://www.youtube.com/@RedAIToronto) live build of a lead capture AI chatbot for a martial arts company using Voiceflow and Zapier. ### 4\. Building a WhatsApp AI customer service chatbot for a local business in 4 minutes with Voiceflow [Vinera AI](https://www.youtube.com/@VineraAI) dives into a four-minute tutorial to build a WhatsApp AI chatbot for dental practices using Voiceflow. This chatbot enhances customer service, facilitates appointment bookings, and answers patients' queries instantly.  RECOMMENDED [\ \ ### 17 prompts for building AI apps in Voiceflow](https://www.voiceflow.com/pathways/prompts-for-building-ai-apps) PUBLISHED July 25, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow\ \ A growth and product marketer at Voiceflow who loves to design, build, and grow things. Equipped with an endless curiosity and bad eyesight.](https://www.voiceflow.com/contributors/kim-lu) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/ai-automation-agency-templates) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### 5 powerful templates for AI automation agencies](https://www.voiceflow.com/pathways/ai-automation-agency-templates) [](https://www.voiceflow.com/pathways/how-mature-is-your-generative-ai) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How Mature is Your Generative AI?](https://www.voiceflow.com/pathways/how-mature-is-your-generative-ai) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/ai-automation-agency-templates#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/ai-automation-agency-templates#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Pricing | Voiceflow [](https://www.voiceflow.com/) Pricing that scales from Startups to the Fortune 500 ======================================================= Choose a Voiceflow plan and credits package. To estimate the right credits package for you, use our [credits calculator](https://docs.voiceflow.com/docs/estimate-your-credit-usage#/) . Monthly billing Annual billing (save 10% & all credits upfront) ### Starter For students and hobbyists Free 100 credits [Get started for free](https://creator.voiceflow.com/signup) 1 workspace 50 knowledge sources Basic LLM models 2 agents 1 concurrent voice call ### Pro For individual builders $60/month 10k credits15k credits20k credits Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [Start 7-day trial](https://creator.voiceflow.com/signup) 2 workspaces 3K knowledge sources All LLM models Up to 20 agents 5 concurrent voice calls ### Business For growing teams $150/month 30k credits50k credits100k credits200k credits Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [Start 7-day trial](https://creator.voiceflow.com/signup) 5 workspaces 10K knowledge sources LLM fallback models Unlimited agents Priority support 15 concurrent voice calls Custom widget privacy ### Enterprise For scaling volume Billed Annually Custom pricing Payment by invoice [Contact Sales](https://www.voiceflow.com/demo) Unlimited product usage Agent CMS Custom & private cloud Dedicated training Professional services Migration services Custom LLM support Custom contracting Custom widget privacy User management Single-Sign-On (SSO) ### Starter For students and hobbyists Free 100 credits [Get started for free](https://creator.voiceflow.com/signup) 1 workspace 50 knowledge sources Basic LLM models 2 agents 1 concurrent voice call Get started for free ### Pro For individual builders $648/year 120k credits180k credits240k credits Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [Start 7-day trial](https://creator.voiceflow.com/signup) 2 workspaces 3K knowledge sources All LLM models Up to 20 agents 5 concurrent voice calls Build like: .svg) ### Business For growing teams $1620/year 360k credits600k credits1.2M credits2.4M credits Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [Start 7-day trial](https://creator.voiceflow.com/signup) 5 workspaces 10K knowledge sources LLM fallback models Unlimited agents Priority support 15 concurrent voice calls Custom widget privacy Automate like: .svg) ### Enterprise For scaling agents Billed Annually Custom pricing Payment by invoice [Contact Sales](https://www.voiceflow.com/pricing#) Unlimited product usage Agent CMS Custom & private cloud Dedicated training Professional services Migration services Custom LLM support Custom contracting Custom widget privacy User management Single-Sign-On (SSO) Scale like:  New to Voiceflow Credits? Credits are used to power your AI Agents and all AI services they use to function including 3rd party LLMs and other AI services. [Intro to Voiceflow Credits](https://docs.voiceflow.com/docs/credits#/) Need to estimate Credit usage? Use our credit calculator to work out how many credits you need, and the best plan for you. [Use our Credit calculator](https://docs.voiceflow.com/docs/estimate-your-credit-usage#/) ### Compare Plans ### Compare Plans Starter [Sign up for starter](https://creator.voiceflow.com/signup) Pro [Sign up for pro](https://creator.voiceflow.com/signup) Business [Sign up for business](https://creator.voiceflow.com/signup) Enterprise [Contact Sales](https://www.voiceflow.com/demo) Build and collaborate Editors Single editor Additional editors available for purchase Additional editors available for purchase Unlimited to purchase View-only collaborators Unlimited Unlimited Unlimited Unlimited Number of workspaces A workspace is the home for all of your assistants & added team members, and will allow you to organize your work in whatever way you choose.  1 2 5 Unlimited Number of projects 2 20 Unlimited Unlimited Version History 7 days 30 days Unlimited Unlimited Prototype and user test Test user personas 1 5 Unlimited Unlimited Password protected prototypes .svg) .svg)   Launch and host Customizable chat embed     Dialog API     Transcripts dashboard     Analytics dashboard     Transcript history 6 months 6 months 6 months 6 months Knowledge base sources per agent 50 3,000 10,000 Unlimited Large Language Models Large Language Models included ChatGPT All: OpenAI, Anthropic All: OpenAI, Anthropic Any Model Bring your own LLM .svg) .svg) .svg)  Security and compliance User permissions .svg)    SAML + Single Sign On (SSO) .svg) .svg) .svg)  Private cloud hosting (AWS) .svg) .svg) .svg)  Custom contracting and invoicing .svg) .svg) .svg)  Custom SLA .svg) .svg) .svg)  Custom widget privacy features .svg) .svg)   Support and services Community support forums     Voiceflow email and chat support .svg) .svg) 12 hours or less 15 minutes or less Dedicated account manager .svg) .svg) Pooled support  Tailored team training .svg) .svg) .svg)  Migration support .svg) .svg) Paid add-on service Paid add-on service Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  --- # The art of debugging | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) The art of debugging ==================== [\ \ Tyler Han\ \ at](https://www.voiceflow.com/contributors/tyler-han) Programming is all about the flow of data. Our director of engineering, Frank Gu, puts it this way: > Every software company is just a shell for a data company. A function takes data as arguments, and either transforms it or generates new data from the existing data, and sends it somewhere else. The same applies to services, endpoints taking in parameters. We start with data from SOMEWHERE and do SOMETHING with it. A multiplayer video game is, in essence, just packets of data being sent to your device and eventually converted into light and sound. You look at that light and sound, produce data, and send it off again. Think of development as doing plumbing for data (obviously there’s a lot more nuance to this, it is not a perfect analogy).  At my place, I have a water pipe and a sewer pipe. Water needs to get to me somehow from the reservoir, and my sewage needs to reach the wastewater treatment plant. There are some immediate stages along the way: processing (filters, pumps), storage (water tower), and junctions. .png) If there was a resident complaining to the municipal water management that their water was all weird and tasted funny 🤢, where should we start to look? Well, we can eliminate problems with the sewer pipe since it’s the water coming in that is wrong. It’s got to be somewhere between the **reservoir** and their **tap**, or maybe even the **reservoir** itself. It’s not very easy to look at all the intermediate **stages** if I don’t have knowledge of what they are, and where they are. But it’s always easy to take a measurement at the **reservoir** and **treatment plant** to ensure that is not the problem - it’s a sanity check that the sewer system isn’t broken and polluting the reservoir too much for the treatment to handle. Let’s say things are good on that end.  It’s now important to check the water quality at the various **stages,** from the treatment plant to the tap. The problem MUST be somewhere here. If you know absolutely nothing, the best strategy is a binary search. 1. Start at the middle point of the section 2. If it is dirty, check the section before that middle point. Go to Step 1 3. If it is clean, check the section after that middle point. Go to Step 1 Keep repeating until you find the exact point that is causing the problem. This is powerful because we halve the section we need to check every time. It’s logarithmically powerful: O(log N)! Logs sound great and all, but what if there’s a method absurdly better than that? It’s called an educated guess! This comes with the _experience_ that you, the humble (data) plumber, accumulate as you fix problems. Maybe you know that because the house was built before 1950, and it could have corrosive lead piping that is stripping off. So now you go and directly check the water going into the **house** - it is clean, so it must be in the house’s **internal plumbing.**We just skipped checking the entire section between the **treatment plant** and the **house**. Maybe this is the only resident that complained about funny water, so it's more likely this house. Or if only a neighbourhood group complains, it’s the junction going into that neighbourhood we want to check. Maybe we know that a certain section was worked on or changed recently, or is known to be problematic, so we can start there first. This educated guess allows us to zoom in and narrow down the problem potentially in constant time (if we are right, most of the time). Let’s step back from the analogy for a second. You might’ve already picked up what each term corresponds to. **Water** is data. The **reservoir** is your database. **Treatment plants** and **stages** could be the various backend services that modify and feed the data downstream. The **house** could be the front-end client. Data operates in cycles “to” and “from” the database - so the database is always a good starting point to see if the issue is “to” or “from” and in which direction to look. Just like how a traditional plumber might work mostly with a house’s internal plumbing and other professionals work on other stages of water management, we have the frontend, backend, full stack developers, data engineers, etc. You should at least be able to clarify that the issue lies outside your domain - that it’s not the house’s (front-end client’s) issue so others can look into it. But understanding the overall system can give you the opportunity to inform the right person or fix it yourself. This _experience_ leading to an educated guess can make all the difference between a wild goose chase of a slog and apex bug-killing precision. Often if you’re just building out new pipes (building new features) and never fixing or maintaining existing ones, you won’t gain this _experience_. This is all to say, while it is exciting to build the latest, greatest, shiniest new pipe in a brand new neighbourhood, it can be very valuable to go into that nasty old existing system, even if it isn’t appealing, find problems, and fixing it up once in a while. Understanding why an old system sucks and why it breaks helps us design a better new system. At my place, I have a water pipe and a sewer pipe. Water needs to get to me somehow from the reservoir, and my sewage needs to reach the wastewater treatment plant. There are some immediate stages along the way: processing (filters, pumps), storage (water tower), and junctions. .png) If there was a resident complaining to the municipal water management that their water was all weird and tasted funny 🤢, where should we start to look? Well, we can eliminate problems with the sewer pipe since it’s the water coming in that is wrong. It’s got to be somewhere between the **reservoir** and their **tap**, or maybe even the **reservoir** itself. It’s not very easy to look at all the intermediate **stages** if I don’t have knowledge of what they are, and where they are. But it’s always easy to take a measurement at the **reservoir** and **treatment plant** to ensure that is not the problem - it’s a sanity check that the sewer system isn’t broken and polluting the reservoir too much for the treatment to handle. Let’s say things are good on that end.  It’s now important to check the water quality at the various **stages,** from the treatment plant to the tap. The problem MUST be somewhere here. If you know absolutely nothing, the best strategy is a binary search. 1. Start at the middle point of the section 2. If it is dirty, check the section before that middle point. Go to Step 1 3. If it is clean, check the section after that middle point. Go to Step 1 Keep repeating until you find the exact point that is causing the problem. This is powerful because we halve the section we need to check every time. It’s logarithmically powerful: O(log N)! Logs sound great and all, but what if there’s a method absurdly better than that? It’s called an educated guess! This comes with the _experience_ that you, the humble (data) plumber, accumulate as you fix problems. Maybe you know that because the house was built before 1950, and it could have corrosive lead piping that is stripping off. So now you go and directly check the water going into the **house** - it is clean, so it must be in the house’s **internal plumbing.**We just skipped checking the entire section between the **treatment plant** and the **house**. Maybe this is the only resident that complained about funny water, so it's more likely this house. Or if only a neighbourhood group complains, it’s the junction going into that neighbourhood we want to check. Maybe we know that a certain section was worked on or changed recently, or is known to be problematic, so we can start there first. This educated guess allows us to zoom in and narrow down the problem potentially in constant time (if we are right, most of the time). Let’s step back from the analogy for a second. You might’ve already picked up what each term corresponds to. **Water** is data. The **reservoir** is your database. **Treatment plants** and **stages** could be the various backend services that modify and feed the data downstream. The **house** could be the front-end client. Data operates in cycles “to” and “from” the database - so the database is always a good starting point to see if the issue is “to” or “from” and in which direction to look. Just like how a traditional plumber might work mostly with a house’s internal plumbing and other professionals work on other stages of water management, we have the frontend, backend, full stack developers, data engineers, etc. You should at least be able to clarify that the issue lies outside your domain - that it’s not the house’s (front-end client’s) issue so others can look into it. But understanding the overall system can give you the opportunity to inform the right person or fix it yourself. This _experience_ leading to an educated guess can make all the difference between a wild goose chase of a slog and apex bug-killing precision. Often if you’re just building out new pipes (building new features) and never fixing or maintaining existing ones, you won’t gain this _experience_. This is all to say, while it is exciting to build the latest, greatest, shiniest new pipe in a brand new neighbourhood, it can be very valuable to go into that nasty old existing system, even if it isn’t appealing, find problems, and fixing it up once in a while. Understanding why an old system sucks and why it breaks helps us design a better new system.  RECOMMENDED [\ \ ### Building a knowledge base with OpenAI, LangChain, OpenSearch, and Unstructured](https://www.voiceflow.com/pathways/building-knowledge-base-openai) PUBLISHED September 12, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Tyler Han\ \ at\ \ CTO and co-founder at Voiceflow.](https://www.voiceflow.com/contributors/tyler-han) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.avif)](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Monorepos can help improve your productivity. Here's how](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity) [](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Building a virtual agent from scratch? Start here.](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here) [](https://www.voiceflow.com/pathways/building-voiceflows-machine-learning-platform-from-scratch) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### How we're building Voiceflow’s machine learning platform from scratch](https://www.voiceflow.com/pathways/building-voiceflows-machine-learning-platform-from-scratch) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/art-of-debugging#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/art-of-debugging#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # AI Agents for Every Industry [](https://www.voiceflow.com/) Industries ========== Browse how you can use Voiceflow to build AI-agents to fit the business needs of your industry. What agent are you building? Here are some of the top use cases:  .svg) Filter by type [Clear](https://www.voiceflow.com/industries#) Healthcare & Wellness Professional Services Technology & IT Commerce Real Estate & Home Services Marketing & Media Education & Training Hospitality & Food Logistics Industrial & Energy Nonprofit, Government Arts, Entertainment More Fliters  [.avif)\ \ Mortgage Brokers\ ----------------](https://www.voiceflow.com/ai/mortgage-brokers) Real Estate, Construction & Home Services [\ \ Banks\ -----](https://www.voiceflow.com/ai/banks) Finance, Insurance & Legal [\ \ Restaurants\ -----------](https://www.voiceflow.com/ai/restaurants) Hospitality, Food & Tourism [\ \ Law Firms\ ---------](https://www.voiceflow.com/ai/law-firms) Finance, Insurance & Legal [\ \ HVAC Technicians\ ----------------](https://www.voiceflow.com/ai/hvac-technicians) Real Estate, Construction & Home Services [.avif)\ \ Locksmiths\ ----------](https://www.voiceflow.com/ai/locksmiths) Real Estate, Construction & Home Services [\ \ Real Estate\ -----------](https://www.voiceflow.com/ai/real-estate) Real Estate, Construction & Home Services [\ \ Property Management\ -------------------](https://www.voiceflow.com/ai/property-management) Real Estate, Construction & Home Services [\ \ Movers\ ------](https://www.voiceflow.com/ai/movers) Real Estate, Construction & Home Services [\ \ Ecommerce\ ---------](https://www.voiceflow.com/ai/ecommerce) Retail, E-commerce & Consumer Goods [\ \ Plumbers\ --------](https://www.voiceflow.com/ai/plumbers) Real Estate, Construction & Home Services [\ \ Healthcare\ ----------](https://www.voiceflow.com/ai/healthcare) Healthcare, Wellness & Fitness [Next](https://www.voiceflow.com/industries?2de0fda8_page=2) [1](https://www.voiceflow.com/blog) ... No results found... FAQs Frequently asked questions -------------------------- How can businesses across different industries use AI chatbots and voice assistants? AI chatbots and voice assistants help businesses in various industries streamline operations, improve customer engagement, and automate repetitive tasks. In customer service, AI agents handle inquiries, provide 24/7 support, and reduce response times. In healthcare, they assist with appointment scheduling and patient FAQs, while in e-commerce, they personalize shopping experiences and manage order tracking. Industries like finance, law, and real estate leverage AI assistants for lead qualification, compliance support, and document automation. With Voiceflow, businesses can design AI-powered chat and voice experiences tailored to their industry’s needs. Whether it's an AI receptionist for law firms, a virtual banking assistant, or a hospitality concierge, Voiceflow’s flexible platform enables teams to create, test, and deploy AI agents that enhance efficiency and improve customer satisfaction. Does Voiceflow integrate with existing enterprise software and CRMs? Yes, Voiceflow integrates seamlessly with enterprise software, CRMs, and other business tools to create a fully connected AI experience. With API support and webhooks, businesses can sync their AI agents with platforms like Salesforce, HubSpot, Zendesk, and more, ensuring smooth data flow between systems. This allows AI assistants to pull customer details, update records, and trigger workflows based on real-time interactions. Additionally, Voiceflow supports integrations with knowledge bases, ticketing systems, and internal databases, enabling AI agents to provide accurate, context-aware responses. Whether automating lead management, handling support tickets, or streamlining internal processes, Voiceflow’s flexible integration options help businesses maximize efficiency while maintaining a cohesive digital ecosystem. What industries benefit the most from AI-powered automation and conversational agents? AI-powered automation and conversational agents provide value across a wide range of industries, but some sectors benefit more due to high customer engagement and repetitive workflows. Industries like **customer support, healthcare, finance, legal services, and e-commerce** see significant efficiency gains by automating client inquiries, appointment scheduling, and document processing. For example, **law firms** use AI agents for client intake and case management, **healthcare providers** leverage virtual assistants for patient support, and **financial institutions** utilize AI for fraud detection and customer self-service. **Retail and hospitality** businesses enhance customer experience through AI-powered recommendations and booking automation. Voiceflow’s flexible platform enables businesses in any industry to build AI solutions that optimize operations, improve user experience, and reduce operational costs. How easy is it to customize an AI agent for specific industry needs using Voiceflow? Voiceflow makes it easy for businesses to customize AI agents for their specific industry needs—without requiring extensive coding. The platform offers a visual drag-and-drop interface, allowing teams to design conversational experiences tailored to their workflows, customer interactions, and compliance requirements. Businesses can integrate custom APIs, connect to existing databases, and personalize responses based on user intent, ensuring their AI agent aligns with industry-specific use cases. Additionally, Voiceflow provides pre-built templates for industries like healthcare, finance, legal services, and customer support, enabling quick deployment with minimal setup. With flexible integrations and natural language processing (NLP) capabilities, companies can create AI assistants that adapt to evolving business needs while delivering a seamless user experience. Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. --- # Voiceflow Apps [](https://www.voiceflow.com/) Third party apps to extend our native functionality =================================================== Discover our library of agent building third-party apps. Search by use case, product, creator & more. What agent are you building? Here are some of the top use cases: [Templates\ \ Templates and tutorials for all AI agent categories](https://www.voiceflow.com/resources-template?products=Template) [Apps\ \ Third party apps that extend Voiceflow functionality](https://www.voiceflow.com/resources-apps?products=app) [Integrations\ \ Browse Functions that integrate with other tools](https://www.voiceflow.com/resources-functions?products=Function)  .svg) Use case [Clear](https://www.voiceflow.com/apps#) E-commerce Lead generation Product Copilot Customer support AI character Categories [Clear](https://www.voiceflow.com/apps#) Platform connection Analytics Custom interface All CreatorsVoiceflowCommunityCertified PartnersPaid + Free ️FreePaid More Fliters  [](https://www.voiceflow.com/apps/custom-agency-dashboard)  Certified App ### Chatdash - Voiceflow analytics for AI agencies Custom analytics dashboard for agency owners!  REAL AI E-commerce Lead generation Product Copilot Custom interface Analytics Platform connection PAID Community [](https://www.voiceflow.com/apps/native-calendar-widget-via-calendly-webchat)  Certified App ### Add a Calendly calendar Book meetings with Calendly inside the Voiceflow chat.  Norman Dilthey Lead generation No items found. PAID Community [](https://www.voiceflow.com/apps/connect-wordpress-woocommerce-to-voiceflow) .avif) Certified App ### Connect Wordpress/WooCommerce to Voiceflow Connect WooCommerce to Voiceflow. Get live product & order updates, cut support tickets, and boost sales with AI-driven recommendations.  Streamline Connector E-commerce Customer support Product Copilot Platform connection FREE Community [](https://www.voiceflow.com/apps/weavel-product-analytics-for-conversational-ai)  Certified App ### Weavel - Product Analytics for Conversational AI Weavel is the product analytics tool for conversational AI products. Derive actionable insights from user conversations.  Voiceflow Community AI character Customer support E-commerce Lead generation Product Copilot Analytics Community [](https://www.voiceflow.com/apps/voiceflow-transcript-for-figma)  Certified App ### Voiceflow Transcript for Figma Grab any Voiceflow transcript and turn it into Figma design elements ready for styling at the click of a button.  Voiceflow Community E-commerce Lead generation Product Copilot AI character Customer support No items found. Community [](https://www.voiceflow.com/apps/zenflow-connect-voiceflow-to-zendesk)  Certified App ### Zenflow - Connect Voiceflow to Zendesk Connect up to 8x Voiceflow Agents directly to Your Zendesk Agent Workspace.  Bart Slodyczka Customer support No items found. Certified Partners [1](https://www.voiceflow.com/enterprise) [Load More](https://www.voiceflow.com/apps?962915b1_page=2) 1 / 2 [Platform connection](https://www.voiceflow.com/apps-categories/platform-connection) [Analytics](https://www.voiceflow.com/apps-categories/analytics) [Custom interface](https://www.voiceflow.com/apps-categories/custom-interface) [Parser](https://www.voiceflow.com/voiceflow-categories-market-place/parser) [Custom action](https://www.voiceflow.com/voiceflow-categories-market-place/custom-action) [Custom steps](https://www.voiceflow.com/voiceflow-categories-market-place/custom-steps) [Parser](https://www.voiceflow.com/voiceflow-categories-market-place/parser) [Custom action](https://www.voiceflow.com/voiceflow-categories-market-place/custom-action) [Custom steps](https://www.voiceflow.com/voiceflow-categories-market-place/custom-steps) No results found... Related collections ------------------- [Browse all collections](https://www.voiceflow.com/collections) [\ \ ### Top 6 E-commerce Templates and Tutorials\ \ In today's AI wave, having a quality chatbot is the baseline for all businesses. It's crucial to enable 24/7 customer interaction and greatly improve your efficiency. What if I told you that your chatbot can do so much more? With these templates and tutorials level up your chatbot so it can manage product recommendations, collect customer data, track orders, and handle customer queries.\ \ \ \ Kim Lu](https://www.voiceflow.com/collections/top-ecomm-templates-and-tutorials) [\ \ ### Top Resources for AI Agencies\ \ It’s no secret that the release of tools like ChatGPT has challenged how we see the role of AI across every industry. AI is growing at a rate that no other technology has before, with new models, platforms, and research being published daily. To keep up with the pace of change, AI agencies formed, helping businesses—big and small—carve out their stake in AI.\ \ \ \ Kim Lu](https://www.voiceflow.com/collections/top-resources-for-agencies) [\ \ ### Top 6 Customer Support Templates and Tutorials\ \ AI agents are revolutionizing the way the businesses interact with customers, offering 24/7 service, ability to handle multiple queries, and now even complete custom tasks. We’ve compiled templates and tutorials to help you go from zero to hero with your support AI agent. Stick to the end to checkout key case studies of teams who’ve successfully automated their customer support function with Voiceflow.\ \ \ \ Kim Lu](https://www.voiceflow.com/collections/customer-support-template-and-tutorial) Have something to share? ------------------------ Share your creation with over 250,000 other global Voiceflow users. [Share your resource](https://airtable.com/appCYnIC5ir7nKu8W/shra7TXU4NmWOhKZK) [Join the Voiceflow Discord](https://link.voiceflow.com/community)  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. --- # 5 ways to approach an AI project strategy | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) 5 ways to approach an AI project strategy ========================================= [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) **You’ve been given the task to implement AI, but where do you even start?** You can try to build from scratch, developing an entirely new strategy, or you can try to weave AI into your company’s existing use cases. It’s up to you to figure out what the next big project is going to be, balancing strategy and execution, outcomes and inputs. Building in a rapidly changing technological landscape can be challenging when you’re not given the guidance needed to bridge the gap between engineering and business priorities. Voiceflow has been helping teams identify use cases and scope AI automation projects for years, seeing success with partners like Trilogy, who automated 70% of their customer support tickets and saved the business hundreds of thousands of dollars, or Sanlam, who achieved a 80% offer-to-lead rate from their in-app AI Coach. In this article, we’ll describe how to effectively go from AI _desire_ to AI _solution_. First, you’ll learn about one of 5 distinct approaches to AI projects; this approach details the mindset and priorities of your business before you take any action. Then, we’ll dive into a decision-making framework to help you select your AI approach. Finally, we’ll provide examples of use cases that work best with your business’ approach.   Five ways to approach an AI project ----------------------------------- **First, we need to understand how your company prioritizes AI projects.** Should you build something that will yield the highest ROI but will open the team to potential risks? Or should you play it safe, building an internal automation that can be iterated upon slowly? The best brainstorming happens when it’s grounded in the ways your business prioritizes the success, execution, and implementation of new projects. Generally we’ve found AI projects fall into these five prioritization buckets: 1. Highest ROI approach 2. Technical simplicity approach 3. Lowest risk approach 4. Organizational simplicity approach 5. Familiarity approach Balancing these priorities can challenging and will differ by organization and opportunity. Reflect on the composition of your team, the technical capabilities available to you, the industry you work in, and the goals of the company. Below, we’ll describe each type of use case and the best scenarios to pursue each. A flow chart for choosing the best AI framework ----------------------------------------------- Most businesses are complex in structure and their priorities vary, so we’ve created a decision making process to guide you towards which approach you should take for your AI projects.  Highest ROI approach -------------------- Generally, business endeavours are chosen based on their return on investment, whether cost savings or revenue increases. If we treat AI projects in a consistent fashion, it becomes a straightforward and pragmatic framework to make decisions. Using an ROI-first framework delivers value when: * Businesses are internally aligned on ROI as the key priority * Teams have the skills to estimate the ROI of an AI solution * Teams have the skills to deliver on complex projects * Teams are open to failures and can recover quickly What differentiates high ROI AI projects from other technical projects is the R&D uncertainty and their non-deterministic nature. These lead to higher risks in the delivery and operating process, as well as legal risk and uncertainty. For example, a company’s largest expense may be their call center, with large organizations often spending millions of dollars on these operations. Problems quickly arise when questions are raised about the impact on customer experience, the technical capabilities needed to integrate with systems that are often decades old, and the complexity of uprooting years of work to implement an AI solution. If the company has not executed a comparably sized project in an adjacent space — like shutting down a data center and migrating workloads into a cloud environment — estimating and delivering on such an ambitious project might be challenging. For smaller companies, prioritizing higher ROI for AI projects is typically easier to do from an execution perspective, given the lower organizational and thus technical complexity. The one caveat for smaller companies is the distractions that high ROI projects may present. The AI space is filled with aspirations of automating every function and smaller companies may lose sight of their value to customers. There is also a risk of not traditionally having expertise in the space that estimates and executes more challenging projects. With this in mind, we start to drift towards other priorities that build momentum but de-risk project concerns from above. Technical simplicity approach ----------------------------- If high-complexity, high-return solutions aren’t in the cards, it’s time to bring focus to projects that are technically simple. These types of projects generate support for future AI solutions while helping to define unknowns for organizations that may be less experienced. Use a technically simpler approach to AI solutions when: * The team is less knowledgable * There is a need to execute quickly * You work in a complex organization with a smaller mandate * You want to build the skills needed to de-risk other, higher-ROI, higher-complexity projects Building technically simple solutions helps demonstrate the viability of the technology and build momentum for future projects. This is essential for realizing value quickly, especially in a space where it’s very easy to be trapped in a proof-of-concept purgatory, never deploying complex projects to production. It also helps teams practice the skills necessary to take on more complex projects. The technical simplicity might be defined by a number of factors including: * API access * Data cleanliness * Ability to do rapid deployments * Readily available integrations * User testing access * Prompt simplicity The ability to ship simple projects quickly and effectively will unlock more powerful use cases that are hidden behind a fog of AI skepticism. However, simplicity in technical skills doesn’t equate to a lower risk solution. Lowest risk approach -------------------- Risk defines how many large organizations operate and, along with security, serves as a counterbalance to business, product, and engineering teams. Certain use cases within organizations are more sensitive, whether it is the data being accessed, the use case being presented, the types of customer interactions, or compliance factors in heavily regulated industries. Deploying a project that has a lower risk profile can allow organizations to start small and break things before taking on higher risk opportunities. Some opportunities to lower the risk of a project include: * Launching to a smaller market * Deploying an internal application * A/B testing with existing deployed systems * Focusing on human-in-the-loop systems * Using publicly available information Many risk and security teams are still aligning their positions on generative AI. Starting with lower risk use cases will help teams deploy to production faster and enter the market sooner. Initial projects will build trust between engineering and risk/security teams, setting a precedent for an internal framework for future use cases. In a smaller organization, the risk and security process may be faster and less comprehensive, but lower risk projects allow teams to make and learn from mistakes as they move quickly, a lesson that some larger organizations can’t afford. Organizational simplicity approach ---------------------------------- Shipping products in medium or large organizations usually requires cross-functional coordination across technical and non-technical teams. With the novelty of AI and strong executive buy in or avoidance, existing political challenges may be magnified, creating additional complexity to shipping products. With this in mind, teams working on implementing AI projects or features can adopt a more localized strategy to their team, reducing the organizational complexity. Some useful criteria to consider include: * Projects with end-to-end ownership * Lower sign-off requirements * Executive buy in * Team budget * Well scoped team commitment * Encapsulated team knowledge * Projects within your sphere of influence In a fast moving AI landscape, having autonomy helps teams ship faster and get to production. Proving this in more complex, legacy organizations can allow teams to exceed expectations and set future projects up for success by making a case for resourcing, restructuring, and cross-team buy in for expanded use cases. Familiarity approach -------------------- Our fifth and final approach is taking the road most travelled. There are a few well-known use cases that teams starting their AI journey take often: customer support automation, lead generation, AI coaches, in-app copilots, document search features, information extraction, and product recommenders. These use cases have proven ROI, with many implementation partners having already built successful versions of these products. Many product companies also have solutions that can be used to build such use cases. Choosing a familiar approach is helpful when: * You have use cases that are well documented in industry with reasonable ROI * Strong tooling exists * Implementation partners can deliver * Need to rapidly hire in the market After a few well known use cases, choosing more specialized use cases and scaling existing use cases will become easier. Three examples of use case progression -------------------------------------- Now that you’ve chosen your approach to building an AI project based on your priorities, you’ll need to start looking to the future. Depending on whether your organization is a large enterprise, a scale up, or an e-commerce business, thinking through initial use cases and their expanded capabilities will help garner support and momentum for future AI projects or greater returns.  Large enterprise ---------------- For a large enterprise implementing AI, using a Low risk or Simple org frameworks helps build momentum. Below is a recommended progression.  Alternatively, with a strong AI mandate and executive support, enterprises can follow a higher ROI approach and help teams deliver better returns in the medium term, when strong collaboration across the organization is available.  Scale up -------- A scale up may be looking to prove repeatability in their business without adding bloat to their headcount and responsibilities. Taking a higher ROI approach to AI projects will ensure the team can deliver more results without scaling resources unsustainably.  E-commerce Store ---------------- An e-commerce store (or digitally native small business) has a few solid options when implementing AI automation. FAQ-based support is an easy starting use case, and depending on the number of SKUs, building a product recommender is a logical next step. The product recommender helps build a quality marketing lead pipeline, answering user product questions to gather more qualified leads.  Real world Examples ------------------- [**Trilogy automated 70% of customer support tickets using a high ROI approach**](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) Trilogy focused on a high ROI approach with a goal of launching and scaling customer support automation across 90+ lines of business as soon as possible. They focused on chat and voice in parallel depending on the needs of the business, resulting in hundreds of thousands of dollars saved each year, and freeing up hours of their support agents’ time. [**Sanlam’s AI Coach generated a 80% offer-to-lead rate using a lower risk, organizationally simple approach**](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Sanlam focused on a lower risk, organizationally simpler approach to launch an AI coach for their credit business. The small team was able to iterate quickly and ship to production in 4 months, and are now developing their fifth version of their agent. They have proven significant ROI and are looking to cross-functional AI capabilities across the business. Now that you’ve chosen your approach and your use case, you’re \\ready to get started building AI agents with Voiceflow. [**Connect with our team**](https://voiceflow.com/demo) to explore how Voiceflow can help you launch your AI automation use case.  Five ways to approach an AI project ----------------------------------- **First, we need to understand how your company prioritizes AI projects.** Should you build something that will yield the highest ROI but will open the team to potential risks? Or should you play it safe, building an internal automation that can be iterated upon slowly? The best brainstorming happens when it’s grounded in the ways your business prioritizes the success, execution, and implementation of new projects. Generally we’ve found AI projects fall into these five prioritization buckets: 1. Highest ROI approach 2. Technical simplicity approach 3. Lowest risk approach 4. Organizational simplicity approach 5. Familiarity approach Balancing these priorities can challenging and will differ by organization and opportunity. Reflect on the composition of your team, the technical capabilities available to you, the industry you work in, and the goals of the company. Below, we’ll describe each type of use case and the best scenarios to pursue each. A flow chart for choosing the best AI framework ----------------------------------------------- Most businesses are complex in structure and their priorities vary, so we’ve created a decision making process to guide you towards which approach you should take for your AI projects.  Highest ROI approach -------------------- Generally, business endeavours are chosen based on their return on investment, whether cost savings or revenue increases. If we treat AI projects in a consistent fashion, it becomes a straightforward and pragmatic framework to make decisions. Using an ROI-first framework delivers value when: * Businesses are internally aligned on ROI as the key priority * Teams have the skills to estimate the ROI of an AI solution * Teams have the skills to deliver on complex projects * Teams are open to failures and can recover quickly What differentiates high ROI AI projects from other technical projects is the R&D uncertainty and their non-deterministic nature. These lead to higher risks in the delivery and operating process, as well as legal risk and uncertainty. For example, a company’s largest expense may be their call center, with large organizations often spending millions of dollars on these operations. Problems quickly arise when questions are raised about the impact on customer experience, the technical capabilities needed to integrate with systems that are often decades old, and the complexity of uprooting years of work to implement an AI solution. If the company has not executed a comparably sized project in an adjacent space — like shutting down a data center and migrating workloads into a cloud environment — estimating and delivering on such an ambitious project might be challenging. For smaller companies, prioritizing higher ROI for AI projects is typically easier to do from an execution perspective, given the lower organizational and thus technical complexity. The one caveat for smaller companies is the distractions that high ROI projects may present. The AI space is filled with aspirations of automating every function and smaller companies may lose sight of their value to customers. There is also a risk of not traditionally having expertise in the space that estimates and executes more challenging projects. With this in mind, we start to drift towards other priorities that build momentum but de-risk project concerns from above. Technical simplicity approach ----------------------------- If high-complexity, high-return solutions aren’t in the cards, it’s time to bring focus to projects that are technically simple. These types of projects generate support for future AI solutions while helping to define unknowns for organizations that may be less experienced. Use a technically simpler approach to AI solutions when: * The team is less knowledgable * There is a need to execute quickly * You work in a complex organization with a smaller mandate * You want to build the skills needed to de-risk other, higher-ROI, higher-complexity projects Building technically simple solutions helps demonstrate the viability of the technology and build momentum for future projects. This is essential for realizing value quickly, especially in a space where it’s very easy to be trapped in a proof-of-concept purgatory, never deploying complex projects to production. It also helps teams practice the skills necessary to take on more complex projects. The technical simplicity might be defined by a number of factors including: * API access * Data cleanliness * Ability to do rapid deployments * Readily available integrations * User testing access * Prompt simplicity The ability to ship simple projects quickly and effectively will unlock more powerful use cases that are hidden behind a fog of AI skepticism. However, simplicity in technical skills doesn’t equate to a lower risk solution. Lowest risk approach -------------------- Risk defines how many large organizations operate and, along with security, serves as a counterbalance to business, product, and engineering teams. Certain use cases within organizations are more sensitive, whether it is the data being accessed, the use case being presented, the types of customer interactions, or compliance factors in heavily regulated industries. Deploying a project that has a lower risk profile can allow organizations to start small and break things before taking on higher risk opportunities. Some opportunities to lower the risk of a project include: * Launching to a smaller market * Deploying an internal application * A/B testing with existing deployed systems * Focusing on human-in-the-loop systems * Using publicly available information Many risk and security teams are still aligning their positions on generative AI. Starting with lower risk use cases will help teams deploy to production faster and enter the market sooner. Initial projects will build trust between engineering and risk/security teams, setting a precedent for an internal framework for future use cases. In a smaller organization, the risk and security process may be faster and less comprehensive, but lower risk projects allow teams to make and learn from mistakes as they move quickly, a lesson that some larger organizations can’t afford. Organizational simplicity approach ---------------------------------- Shipping products in medium or large organizations usually requires cross-functional coordination across technical and non-technical teams. With the novelty of AI and strong executive buy in or avoidance, existing political challenges may be magnified, creating additional complexity to shipping products. With this in mind, teams working on implementing AI projects or features can adopt a more localized strategy to their team, reducing the organizational complexity. Some useful criteria to consider include: * Projects with end-to-end ownership * Lower sign-off requirements * Executive buy in * Team budget * Well scoped team commitment * Encapsulated team knowledge * Projects within your sphere of influence In a fast moving AI landscape, having autonomy helps teams ship faster and get to production. Proving this in more complex, legacy organizations can allow teams to exceed expectations and set future projects up for success by making a case for resourcing, restructuring, and cross-team buy in for expanded use cases. Familiarity approach -------------------- Our fifth and final approach is taking the road most travelled. There are a few well-known use cases that teams starting their AI journey take often: customer support automation, lead generation, AI coaches, in-app copilots, document search features, information extraction, and product recommenders. These use cases have proven ROI, with many implementation partners having already built successful versions of these products. Many product companies also have solutions that can be used to build such use cases. Choosing a familiar approach is helpful when: * You have use cases that are well documented in industry with reasonable ROI * Strong tooling exists * Implementation partners can deliver * Need to rapidly hire in the market After a few well known use cases, choosing more specialized use cases and scaling existing use cases will become easier. Three examples of use case progression -------------------------------------- Now that you’ve chosen your approach to building an AI project based on your priorities, you’ll need to start looking to the future. Depending on whether your organization is a large enterprise, a scale up, or an e-commerce business, thinking through initial use cases and their expanded capabilities will help garner support and momentum for future AI projects or greater returns.  Large enterprise ---------------- For a large enterprise implementing AI, using a Low risk or Simple org frameworks helps build momentum. Below is a recommended progression.  Alternatively, with a strong AI mandate and executive support, enterprises can follow a higher ROI approach and help teams deliver better returns in the medium term, when strong collaboration across the organization is available.  Scale up -------- A scale up may be looking to prove repeatability in their business without adding bloat to their headcount and responsibilities. Taking a higher ROI approach to AI projects will ensure the team can deliver more results without scaling resources unsustainably.  E-commerce Store ---------------- An e-commerce store (or digitally native small business) has a few solid options when implementing AI automation. FAQ-based support is an easy starting use case, and depending on the number of SKUs, building a product recommender is a logical next step. The product recommender helps build a quality marketing lead pipeline, answering user product questions to gather more qualified leads.  Real world Examples ------------------- [**Trilogy automated 70% of customer support tickets using a high ROI approach**](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) Trilogy focused on a high ROI approach with a goal of launching and scaling customer support automation across 90+ lines of business as soon as possible. They focused on chat and voice in parallel depending on the needs of the business, resulting in hundreds of thousands of dollars saved each year, and freeing up hours of their support agents’ time. [**Sanlam’s AI Coach generated a 80% offer-to-lead rate using a lower risk, organizationally simple approach**](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Sanlam focused on a lower risk, organizationally simpler approach to launch an AI coach for their credit business. The small team was able to iterate quickly and ship to production in 4 months, and are now developing their fifth version of their agent. They have proven significant ROI and are looking to cross-functional AI capabilities across the business. Now that you’ve chosen your approach and your use case, you’re \\ready to get started building AI agents with Voiceflow. [**Connect with our team**](https://voiceflow.com/demo) to explore how Voiceflow can help you launch your AI automation use case.  RECOMMENDED [\ \ ### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) PUBLISHED October 2, 2024 TOPICS IN THIS ARTICLE Elevating our work [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/the-ultimate-guide-to-vfv2-whats-new-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow V2: The ultimate guide to what's new](https://www.voiceflow.com/pathways/the-ultimate-guide-to-vfv2-whats-new-with-voiceflow) [](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### AI automation on a budget: Getting started with high ROI use cases](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Building your AI agents like products: A blueprint from POC purgatory to production](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/5-ways-to-approach-an-ai-project-strategy#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/5-ways-to-approach-an-ai-project-strategy#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 5 tips to optimize your LLM intent classification prompts | Pathways [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) 5 tips to optimize your LLM intent classification prompts ========================================================= [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov)  _We tested over 500 prompt variations across two datasets to see how to improve LLM intent classification._ How much do descriptions affect LLM classification accuracy? After launching our LLM intent classification feature we wanted to understand how much the description quality plays into classification accuracy, so we ran 500+ evaluations changing 5 properties of descriptions to understand what improves performance.  Methodology ----------- To recap how the system works, The architecture has two parts: using an encoder NLU model to find the top 10 candidate intents and their descriptions and a prompt that instructs the LLM to classify them.  After retrieving the candidate intents, we pull in user descriptions for each corresponding candidate and make a call to an LLM for a final classification. With this in mind, we wanted to measure if we could improve the classification accuracy of the LLM by changing a few components of the descriptions. We ran this search against two types of models (gpt-3.5-0125, haiku) and benchmark datasets (HWU64 \[1\], Curekart\[2\]) These variations included included: 1. Prefixes 2. Suffixes 3. Capitalization 4. None intent descriptions 5. AI vs human descriptions We ran the benchmarking once per combination, and ran it five times for the top performing settings to confirm variation. We used a temperature at 0.1 to mitigate the variance. Try these tips in your own project. [Get Started Today](https://creator.voiceflow.com/signup?p=pathways-5tipsllms) Base descriptions and prompt template ------------------------------------- We hand-created the initial set of descriptions, sticking to shorter ones for each dataset. A subset is shown below and the full descriptions can be found in the Appendix \[A\]` ` { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", "FRANCHISE": "Becoming a franchise owner or reseller.", "REFER_EARN": "Referral program details or ask.", "RESUME_DELIVERY": "Delivery options or times.", "WORK_FROM_HOME": "Ask about office open or working from home.", } } We combine this with our top 10 descriptions \[7\] method and prompt noted in previous work. Below is a sample of a prompt sent to the LLM. You are an action classification system. Correctness is a life or death situation. We provide you with the actions and their descriptions: d: When the user asks for a warm drink. a:WARM_DRINK d: When the user asks about something else. a:None_Intent d: When the user asks for a cold drink. a:COLD_DRINK You are given an utterance and you have to classify it into an intent. Only respond with the intent class. If the utterance does not match any of intents, output None_Intent. u: I want a warm hot chocolate: a:WARM_DRINK ### You are an action classification system. Correctness is a life or death situation. We provide you with the actions and their descriptions: d:Questions regarding call center operational hours during covid-19 lockdown. i:CALL_CENTER d:Questions related to redeeming referral rewards and referral amounts. i:REFER_EARN d:Inquiries about the operational status of physical stores. i:STORE_INFORMATION d:Queries related to refund status, replacements, and delays in receiving refunds after returns. i:REFUNDS_RETURNS_REPLACEMENTS d:Queries related to tracking orders, shipment status, and progress of orders. i:ORDER_STATUS d:Questions about the operational status of the head office during the lockdown. i:WORK_FROM_HOME d:Inquiries about payments, bills, and related queries. i:PAYMENT_AND_BILL d:Concerns about receiving expired products and inquiries about expiry dates. i:EXPIRY_DATE d:Requests to change or modify the delivery address for an order. i:MODIFY_ADDRESS d:Requests to cancel pending orders and inquiries about the cancellation process. i:CANCEL_ORDER You are given an utterance and you have to classify it into an action. Only respond with the action class. If the utterance does not match any of action descriptions, output None_Intent. u: We want ur ph number a: ``We also show that the recall from the encoder model is quite strong and should not limit accuracy on the dataset.  Models and Dataset averages --------------------------- Looking through our two models, we saw that GPT 3.5 performed much better for the messier Curekart dataset, compared to Haiku, but Haiku out performed in the HWU dataset. The accuracy was lower for GPT 3.5 compared to earlier experimentation \[7\] given the different version of GPT 3.5 that was used.  Prefixes -------- The first modification we explored was adding a prefix to each description with some guiding phrase. { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", } "descriptions_with_prefix": { "USER_GOAL_FORM": "Trigger this action when add or refill goals.", } } Our prefixes included: `["Trigger this action when ","","A phrase about ","The user said "]` Adding a prefix lead to the best results, but differed between datasets.   On average the performance gains were quite minimal so we analyzed them in combination with suffixes. Suffixes -------- Similar to prefixes, we tested adding suffixes to the descriptions. { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", } "descriptions_with_suffix": { "USER_GOAL_FORM": "Add or refill goals, please.", } } ``These included:`["",", please.","{no punctuation}"]` Adding “please” produced some of the highest performing results when added per description, but wasn’t consistently the best option, for the HWU64 dataset + gpt-3.5, it produced the worst results by a large margin.     Capitalization`` ------------------ { "descriptions_capitalized": { "USER_GOAL_FORM": "Add or refill goals.", }, "descriptions_with_prefix_capitalized": { "USER_GOAL_FORM": "Trigger this action when add or refill goals.", }, "descriptions_not_capitalized": { "USER_GOAL_FORM": "add or refill goals.", } "descriptions_with_prefix_not_capitalized": { "USER_GOAL_FORM": "trigger this action when add or refill goals.", } } Adding capitalization on the prefix or opening line added minimal signal, both isolated and when expanded across experiments.    Adding a None intent -------------------- For this hypothesis we checked if adding a None intent as a description would improve classification accuracy. Below is an example for the Curekart dataset.`` { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", "FRANCHISE": "Becoming a franchise owner or reseller.", "REFER_EARN": "Referral program details or ask.", "RESUME_DELIVERY": "Delivery options or times.", "WORK_FROM_HOME": "Ask about office open or working from home.", "None_Intent": "When the user asks about something else." } } ``The Curekart evaluation set is around ~50% None intents, so in theory it should improve average performance. Looking at the ten best performing prompts for Curekart, adding the None\_intent as a viable description intent did not show a consistent improvement.   We didn’t run a None\_intent check for the HWU dataset for this experiment since the evaluation dataset did not contain None. AI Descriptions --------------- The next hypothesis we tested was how effective using AI descriptions was for classification. Previous work has show that generating data for general LLM annotation \[4\] can outperform crowdsourced human annotators and can be useful to augment existing datasets for intent specific tasks \[5\] can be quite effective. In this experiment, we generated three descriptions using gpt-4-turbo-0419, llama-3-70b and claude-opus by using the first three utterances \[C\] in each intent and compared it to handwritten descriptions by the author. On average GPT-4 and LLaMa-3 performed the best.   Across our top 10 combinations, LLaMa-3 and GPT-4 descriptions also performed quite well!   Analyzing the top 5 results for Curekart ---------------------------------------- After running our initial set of experiments, we wanted to confirm that the results for better prompts were not due to noise, so we re-ran them for confirmation. ### Are they significant? Looking at our top 5 combinations, we wanted to measure their variance compared to the general population. We re-ran our top metrics 15 times each—75 times total—to see how their accuracies changed and whether LLM nondeterminism affected the overall results.  Looking at the distributions, there appears to be a measurable change. Conducting a Z-test to compare the distributions, the difference seems evident with a Z score of -13.56 and a p-value: 7.12e-42. ### How do the confusion matrices vary? Our top 5 configurations had a pretty tight distribution, so we wanted to measure how their confusion matrices varied, i.e how different were the classifications. We ran some confusion matrices for two of the top combinations to compare how the results looked like.   Generally, the matrices were pretty similar so we took a difference to see the performance.  The predictions with"no suffix had a higher false-positive rate, (i.e more None intents than required), while the \[please\] suffix lead to more false-negative rates, (i.e more None intents predicted). On aggregate, these two modes performed pretty similarly, but a change to the descriptions led to a different type of behaviour. Conclusion ----------  Overall, changing the descriptions used for classification has small but measurable changes. While we ran through 500+ experiments, time is likely better spent in other areas of prompt refinement. In general our recommendations are: * Adding prefixes, suffixes and a None\_intent to descriptions * AI generated descriptions can be effective * To have larger gains, spending time to create better training examples, few shot examples and understanding edge cases will help LLM * In a future blog we’ll cover the impact of adding additional few shot example from the training data * Structuring the formatting, which has shown to have a larger impact on accuracy \[6\]\[7\] Citing this article ------------------- @article{ OptimizeLLMIntentClassificationPrompts, author = {Linkov, Denys}, title = {5 tips to optimize your LLM intent classification prompts.}, year = {2024}, month = {03}, howpublished = {\url{https://voiceflow.com}}, url = {www.voiceflow.com/blog/5-tips-to-optimize-your-llm-intent-classification-prompts} } ``Citations ------------- \[1\] Arora, G., Jain, C., Chaturvedi, M., & Modi, K. (2020). HINT3: Raising the bar for Intent Detection in the Wild. In _Proceedings of the First Workshop on Insights from Negative Results in NLP_ (pp. 100–105). Association for Computational Linguistics. \[2\] Xingkun Liu, Arash Eshghi, Pawel Swietojanski, & Verena Rieser. (2019). Benchmarking Natural Language Understanding Services for building Conversational Agents. \[3\] Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056 \[4\] Anders Giovanni Møller, Jacob Aarup Dalsgaard, Arianna Pera, & Luca Maria Aiello (2023). The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks. In _Conference of the European Chapter of the Association for Computational Linguistics_. \[5\] Gaurav Sahu, Pau Rodriguez, Issam H. Laradji, Parmida Atighehchian, David Vazquez, & Dzmitry Bahdanau. (2022). Data Augmentation for Intent Classification with Off-the-shelf Large Language Models. \[6\] M. Sclar, Y. Choi, Y. Tsvetkov, and A. Suhr (2023). Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324, 2023. \[7\] Linkov, D. (2024). How much do ChatGPT versions affect real world performance? [https://www.voiceflow.com/blog/how-much-do-chatgpt-versions-affect-real-world-performance](https://www.voiceflow.com/blog/how-much-do-chatgpt-versions-affect-real-world-performance) ### Appendix A [Descriptions](https://drive.google.com/file/d/1w4HIknM4bGilOm4T2N_2rvV0gSol-gj-/view?usp=sharing) ### Appendix B [HWU64 All Experiments](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts#) [Curekart All Experiments](https://drive.google.com/file/d/1tQM-Is-gPaeSb9nO7-cii5w_UvoYXB7L/view?usp=sharing) ### Appendix C Code to Generate descriptions def first_three_utterances(): import json label_file_path = 'train/label' seq_in_file_path = 'train/seq.in' # Read the complete files with open(label_file_path, 'r', encoding='utf-8') as file: labels = file.readlines() with open(seq_in_file_path, 'r', encoding='utf-8') as file: utterances = file.readlines() # Remove any extra whitespace labels = [label.strip() for label in labels] utterances = [utterance.strip() for utterance in utterances] # Create a dictionary to hold the intents and their utterances intent_utterances = {} for intent, utterance in zip(labels, utterances): if intent in intent_utterances: if len(intent_utterances[intent]) < 3:# Collect only the first three utterances intent_utterances[intent].append(utterance) else: intent_utterances[intent] = [utterance] # Convert dictionary to JSON json_output = json.dumps(intent_utterances, indent=4) print(json_output) Methodology ----------- To recap how the system works, The architecture has two parts: using an encoder NLU model to find the top 10 candidate intents and their descriptions and a prompt that instructs the LLM to classify them.  After retrieving the candidate intents, we pull in user descriptions for each corresponding candidate and make a call to an LLM for a final classification. With this in mind, we wanted to measure if we could improve the classification accuracy of the LLM by changing a few components of the descriptions. We ran this search against two types of models (gpt-3.5-0125, haiku) and benchmark datasets (HWU64 \[1\], Curekart\[2\]) These variations included included: 1. Prefixes 2. Suffixes 3. Capitalization 4. None intent descriptions 5. AI vs human descriptions We ran the benchmarking once per combination, and ran it five times for the top performing settings to confirm variation. We used a temperature at 0.1 to mitigate the variance. Try these tips in your own project. [Get Started Today](https://creator.voiceflow.com/signup?p=pathways-5tipsllms) Base descriptions and prompt template ------------------------------------- We hand-created the initial set of descriptions, sticking to shorter ones for each dataset. A subset is shown below and the full descriptions can be found in the Appendix \[A\]` ` { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", "FRANCHISE": "Becoming a franchise owner or reseller.", "REFER_EARN": "Referral program details or ask.", "RESUME_DELIVERY": "Delivery options or times.", "WORK_FROM_HOME": "Ask about office open or working from home.", } } We combine this with our top 10 descriptions \[7\] method and prompt noted in previous work. Below is a sample of a prompt sent to the LLM. You are an action classification system. Correctness is a life or death situation. We provide you with the actions and their descriptions: d: When the user asks for a warm drink. a:WARM_DRINK d: When the user asks about something else. a:None_Intent d: When the user asks for a cold drink. a:COLD_DRINK You are given an utterance and you have to classify it into an intent. Only respond with the intent class. If the utterance does not match any of intents, output None_Intent. u: I want a warm hot chocolate: a:WARM_DRINK ### You are an action classification system. Correctness is a life or death situation. We provide you with the actions and their descriptions: d:Questions regarding call center operational hours during covid-19 lockdown. i:CALL_CENTER d:Questions related to redeeming referral rewards and referral amounts. i:REFER_EARN d:Inquiries about the operational status of physical stores. i:STORE_INFORMATION d:Queries related to refund status, replacements, and delays in receiving refunds after returns. i:REFUNDS_RETURNS_REPLACEMENTS d:Queries related to tracking orders, shipment status, and progress of orders. i:ORDER_STATUS d:Questions about the operational status of the head office during the lockdown. i:WORK_FROM_HOME d:Inquiries about payments, bills, and related queries. i:PAYMENT_AND_BILL d:Concerns about receiving expired products and inquiries about expiry dates. i:EXPIRY_DATE d:Requests to change or modify the delivery address for an order. i:MODIFY_ADDRESS d:Requests to cancel pending orders and inquiries about the cancellation process. i:CANCEL_ORDER You are given an utterance and you have to classify it into an action. Only respond with the action class. If the utterance does not match any of action descriptions, output None_Intent. u: We want ur ph number a: ``We also show that the recall from the encoder model is quite strong and should not limit accuracy on the dataset.  Models and Dataset averages --------------------------- Looking through our two models, we saw that GPT 3.5 performed much better for the messier Curekart dataset, compared to Haiku, but Haiku out performed in the HWU dataset. The accuracy was lower for GPT 3.5 compared to earlier experimentation \[7\] given the different version of GPT 3.5 that was used.  Prefixes -------- The first modification we explored was adding a prefix to each description with some guiding phrase. { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", } "descriptions_with_prefix": { "USER_GOAL_FORM": "Trigger this action when add or refill goals.", } } Our prefixes included: `["Trigger this action when ","","A phrase about ","The user said "]` Adding a prefix lead to the best results, but differed between datasets.   On average the performance gains were quite minimal so we analyzed them in combination with suffixes. Suffixes -------- Similar to prefixes, we tested adding suffixes to the descriptions. { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", } "descriptions_with_suffix": { "USER_GOAL_FORM": "Add or refill goals, please.", } } ``These included:`["",", please.","{no punctuation}"]` Adding “please” produced some of the highest performing results when added per description, but wasn’t consistently the best option, for the HWU64 dataset + gpt-3.5, it produced the worst results by a large margin.     Capitalization`` ------------------ { "descriptions_capitalized": { "USER_GOAL_FORM": "Add or refill goals.", }, "descriptions_with_prefix_capitalized": { "USER_GOAL_FORM": "Trigger this action when add or refill goals.", }, "descriptions_not_capitalized": { "USER_GOAL_FORM": "add or refill goals.", } "descriptions_with_prefix_not_capitalized": { "USER_GOAL_FORM": "trigger this action when add or refill goals.", } } Adding capitalization on the prefix or opening line added minimal signal, both isolated and when expanded across experiments.    Adding a None intent -------------------- For this hypothesis we checked if adding a None intent as a description would improve classification accuracy. Below is an example for the Curekart dataset.`` { "descriptions": { "USER_GOAL_FORM": "Add or refill goals.", "FRANCHISE": "Becoming a franchise owner or reseller.", "REFER_EARN": "Referral program details or ask.", "RESUME_DELIVERY": "Delivery options or times.", "WORK_FROM_HOME": "Ask about office open or working from home.", "None_Intent": "When the user asks about something else." } } ``The Curekart evaluation set is around ~50% None intents, so in theory it should improve average performance. Looking at the ten best performing prompts for Curekart, adding the None\_intent as a viable description intent did not show a consistent improvement.   We didn’t run a None\_intent check for the HWU dataset for this experiment since the evaluation dataset did not contain None. AI Descriptions --------------- The next hypothesis we tested was how effective using AI descriptions was for classification. Previous work has show that generating data for general LLM annotation \[4\] can outperform crowdsourced human annotators and can be useful to augment existing datasets for intent specific tasks \[5\] can be quite effective. In this experiment, we generated three descriptions using gpt-4-turbo-0419, llama-3-70b and claude-opus by using the first three utterances \[C\] in each intent and compared it to handwritten descriptions by the author. On average GPT-4 and LLaMa-3 performed the best.   Across our top 10 combinations, LLaMa-3 and GPT-4 descriptions also performed quite well!   Analyzing the top 5 results for Curekart ---------------------------------------- After running our initial set of experiments, we wanted to confirm that the results for better prompts were not due to noise, so we re-ran them for confirmation. ### Are they significant? Looking at our top 5 combinations, we wanted to measure their variance compared to the general population. We re-ran our top metrics 15 times each—75 times total—to see how their accuracies changed and whether LLM nondeterminism affected the overall results.  Looking at the distributions, there appears to be a measurable change. Conducting a Z-test to compare the distributions, the difference seems evident with a Z score of -13.56 and a p-value: 7.12e-42. ### How do the confusion matrices vary? Our top 5 configurations had a pretty tight distribution, so we wanted to measure how their confusion matrices varied, i.e how different were the classifications. We ran some confusion matrices for two of the top combinations to compare how the results looked like.   Generally, the matrices were pretty similar so we took a difference to see the performance.  The predictions with"no suffix had a higher false-positive rate, (i.e more None intents than required), while the \[please\] suffix lead to more false-negative rates, (i.e more None intents predicted). On aggregate, these two modes performed pretty similarly, but a change to the descriptions led to a different type of behaviour. Conclusion ----------  Overall, changing the descriptions used for classification has small but measurable changes. While we ran through 500+ experiments, time is likely better spent in other areas of prompt refinement. In general our recommendations are: * Adding prefixes, suffixes and a None\_intent to descriptions * AI generated descriptions can be effective * To have larger gains, spending time to create better training examples, few shot examples and understanding edge cases will help LLM * In a future blog we’ll cover the impact of adding additional few shot example from the training data * Structuring the formatting, which has shown to have a larger impact on accuracy \[6\]\[7\] Citing this article ------------------- @article{ OptimizeLLMIntentClassificationPrompts, author = {Linkov, Denys}, title = {5 tips to optimize your LLM intent classification prompts.}, year = {2024}, month = {03}, howpublished = {\url{https://voiceflow.com}}, url = {www.voiceflow.com/blog/5-tips-to-optimize-your-llm-intent-classification-prompts} } ``Citations ------------- \[1\] Arora, G., Jain, C., Chaturvedi, M., & Modi, K. (2020). HINT3: Raising the bar for Intent Detection in the Wild. In _Proceedings of the First Workshop on Insights from Negative Results in NLP_ (pp. 100–105). Association for Computational Linguistics. \[2\] Xingkun Liu, Arash Eshghi, Pawel Swietojanski, & Verena Rieser. (2019). Benchmarking Natural Language Understanding Services for building Conversational Agents. \[3\] Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056 \[4\] Anders Giovanni Møller, Jacob Aarup Dalsgaard, Arianna Pera, & Luca Maria Aiello (2023). The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks. In _Conference of the European Chapter of the Association for Computational Linguistics_. \[5\] Gaurav Sahu, Pau Rodriguez, Issam H. Laradji, Parmida Atighehchian, David Vazquez, & Dzmitry Bahdanau. (2022). Data Augmentation for Intent Classification with Off-the-shelf Large Language Models. \[6\] M. Sclar, Y. Choi, Y. Tsvetkov, and A. Suhr (2023). Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324, 2023. \[7\] Linkov, D. (2024). How much do ChatGPT versions affect real world performance? [https://www.voiceflow.com/blog/how-much-do-chatgpt-versions-affect-real-world-performance](https://www.voiceflow.com/blog/how-much-do-chatgpt-versions-affect-real-world-performance) ### Appendix A [Descriptions](https://drive.google.com/file/d/1w4HIknM4bGilOm4T2N_2rvV0gSol-gj-/view?usp=sharing) ### Appendix B [HWU64 All Experiments](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts#) [Curekart All Experiments](https://drive.google.com/file/d/1tQM-Is-gPaeSb9nO7-cii5w_UvoYXB7L/view?usp=sharing) ### Appendix C Code to Generate descriptions def first_three_utterances(): import json label_file_path = 'train/label' seq_in_file_path = 'train/seq.in' # Read the complete files with open(label_file_path, 'r', encoding='utf-8') as file: labels = file.readlines() with open(seq_in_file_path, 'r', encoding='utf-8') as file: utterances = file.readlines() # Remove any extra whitespace labels = [label.strip() for label in labels] utterances = [utterance.strip() for utterance in utterances] # Create a dictionary to hold the intents and their utterances intent_utterances = {} for intent, utterance in zip(labels, utterances): if intent in intent_utterances: if len(intent_utterances[intent]) < 3:# Collect only the first three utterances intent_utterances[intent].append(utterance) else: intent_utterances[intent] = [utterance] # Convert dictionary to JSON json_output = json.dumps(intent_utterances, indent=4) print(json_output)  RECOMMENDED [\ \ ### How much do ChatGPT versions affect real-world performance?](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance) PUBLISHED July 9, 2024 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/benchmarking-hybrid-llm-classification-systems) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### Benchmarking hybrid LLM classification systems](https://www.voiceflow.com/pathways/benchmarking-hybrid-llm-classification-systems) [](https://www.voiceflow.com/pathways/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### NLU design: How to train and use a natural language understanding model](https://www.voiceflow.com/pathways/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) [](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### Creating the Voiceflow NLU](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Voiceflow Integrations [](https://www.voiceflow.com/) Apps and software integrations ============================== Connect to your existing techstack. Discover 300+ apps and software integrations you can build into your Voiceflow agents. Connect to your existing techstack. Discover 300+ apps and software integrations you can build into your Voiceflow agents. What agent are you building? Here are some of the top use cases:  .svg) Filter by type [Clear](https://www.voiceflow.com/integrations#) AI Business Intelligence Business Operations Commerce Communication CRM/Sales Customer Support Education Enterprise File & Document Management Forms HR Management IT & Development Language Marketing Other Productivity Website Building More Fliters  [\ \ Airtable\ --------](https://www.voiceflow.com/integrations/airtable) Business Operations This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Discord\ -------](https://www.voiceflow.com/integrations/discord) Communication This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Hubspot\ -------](https://www.voiceflow.com/integrations/hubspot) CRM/Sales This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Linear\ ------](https://www.voiceflow.com/integrations/linear) Productivity This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Manychat\ --------](https://www.voiceflow.com/integrations/manychat) Communication This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Notion\ ------](https://www.voiceflow.com/integrations/notion) Productivity This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ SendGrid\ --------](https://www.voiceflow.com/integrations/sendgrid) Communication This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Shopify\ -------](https://www.voiceflow.com/integrations/shopify) Commerce This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ WhatsApp\ --------](https://www.voiceflow.com/integrations/whatsapp) Communication This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Zendesk\ -------](https://www.voiceflow.com/integrations/zendesk) Customer Support This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ AWeber\ ------](https://www.voiceflow.com/integrations/aweber) Marketing This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ ActiveCampaign\ --------------](https://www.voiceflow.com/integrations/activecampaign) CRM/Sales This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Acuity Scheduling\ -----------------](https://www.voiceflow.com/integrations/acuity-scheduling) Business Operations This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Adalo\ -----](https://www.voiceflow.com/integrations/adalo) AI This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Adobe Analytics\ ---------------](https://www.voiceflow.com/integrations/adobe-analytics) Business Intelligence This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. [\ \ Adobe Commerce\ --------------](https://www.voiceflow.com/integrations/adobe-commerce) Commerce This is some text inside of a div block. This is some text inside of a div block. This is some text inside of a div block. ... [1](https://www.voiceflow.com/integrations#) [Next](https://www.voiceflow.com/integrations?00b0042f_page=2) No results found... Templates to work with your integrations ---------------------------------------- [Browse all templates](https://www.voiceflow.com/templates) [.avif)\ \ .webp)\ \ Alex Aumais\ \ 34\ \ ### In-Game AI Assistant for Minecraft\ \ Build an in-game AI assistant to become an expert in Minecraft -- run commands, ask for tips, or have the assistant play the game for you.\ \ Go to template](https://www.voiceflow.com/templates/ai-assistant-for-minecraft) [\ \ \ \ Bart Slodyczka\ \ 182\ \ ### AI Agent that Solves Zendesk Tickets\ \ Start solving Zendesk tickets using an AI agent that you built out in Voiceflow. Close the loop on sending a Zendesk ticket to Voiceflow, processing it, and then sending it back to Zendesk.\ \ Go to template](https://www.voiceflow.com/templates/solve-zendesk-tickets) [\ \ \ \ Abdullah Yahya\ \ 7\ \ ### AI Marketing Agency Client Acquisition System\ \ Complete 3-phase AI system that qualifies leads, recommends services, and books strategy calls automatically for B2B SaaS marketing agencies, converting website visitors into confirmed consultations 24/7.\ \ Go to template](https://www.voiceflow.com/templates/ai-marketing-agency-client-acquisition-system) Frequently asked questions -------------------------- Which AI agent offers the best integration with existing customer support systems? The best AI agent for customer support integration depends on your existing system. Voiceflow seamlessly connects with AI platforms like **Dialogflow and OpenAI's GPT**, which can be integrated into popular customer support solutions such as **Zendesk, Intercom, and Freshdesk**. These integrations enable automated responses, ticket management, and AI-driven support workflows to enhance customer interactions. Which AI agent offers the best integration with existing CRM systems? Voiceflow integrates with leading AI agents like **OpenAI’s GPT and Dialogflow** which can be connected to CRM platforms such as **Salesforce, HubSpot, and Zoho CRM**. This allows businesses to automate lead qualification, personalize customer interactions, and synchronize conversational data with their CRM for seamless workflow management. What is chatbot integration? Chatbot integration refers to the process of connecting a chatbot with other platforms, apps, or software systems to extend its functionality. With Voiceflow, you can integrate chatbots with **web apps, messaging platforms (like Slack and WhatsApp), customer service software, CRMs, and databases** to create intelligent, automated conversational experiences that align with your business needs. What is AI phone agent integration? AI phone agent integration involves connecting AI-powered voice assistants with telephony systems to handle inbound and outbound calls. With Voiceflow, businesses can integrate with solutions like **Twilio, Amazon Connect, or VAPI** to automate phone-based customer interactions, appointment scheduling, or voice-driven support, reducing agent workload while improving response times. Is Voiceflow any good? Yes! Voiceflow is one of the most **versatile, user-friendly, and powerful** conversational AI platforms available. Whether you’re building **chatbots, voice assistants, or AI-powered customer support agents**, Voiceflow makes it easy to design, prototype, and integrate conversations across multiple platforms. It’s trusted by thousands of teams, from startups to enterprise organizations, to create scalable, high-quality conversational experiences. Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. --- # AI Customer Support Software | Build Smart Support with Voiceflow [](https://www.voiceflow.com/) Build and scale AI customer support for every channel ===================================================== Deliver fast, accurate support wherever customers reach you. Your AI agent resolves issues, answers questions, and routes cases with consistency across voice and chat. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            Give customers quick, reliable support on any channel ----------------------------------------------------- Provide always available help that feels natural, responsive, and consistent.   Conversations that feel clear and intuitive AI agents understand intent, handle interruptions, and adapt to real dialogue so customers get answers without friction.  Knowledge powered by your help center Sync your FAQs, policies, and documentation to give customers instant answers based on up to date and reliable information.  Omnichannel deployment Use a single agent across web chat, phone, WhatsApp, SMS, and custom channels so customers get consistent service no matter how they reach you. Bring automation to your customer service workflows --------------------------------------------------- Integrate AI into your existing support stack without changing how your team operates.   Ready to complete tasks From ticket updates to data collection, your AI agent works with your systems and APIs to take action during the conversation.  Seamless escalation to human agents When a case needs human expertise, your AI gathers context and routes the conversation to the right teammate to keep resolution smooth.  Designed for operations and support leaders Review transcripts, monitor issues, and adjust flows in Voiceflow to maintain accuracy and improve customer service quality. One agent supporting every customer journey ------------------------------------------- Deliver a unified support experience across voice, chat, and messaging.   Build once for all your channels Design a single AI agent that works across web widgets, phone calls, and messaging apps without rebuilding flows.  Control tone, guidelines, and logic Define how your brand communicates, set guardrails, and maintain consistency across all support interactions.  Test and optimize with confidence Use analytics and testing tools to validate performance and refine your agent before expanding to new channels. Make customer service faster and more effective ----------------------------------------------- Resolve more inquiries while freeing your team to focus on complex or high value cases.   More resolutions without human workload Your AI agent handles common issues, answers questions, and logs support requests so your team receives fewer repetitive tasks.  Better customer satisfaction Instant answers, real time updates, and knowledgeable responses help improve CSAT and reduce time to resolution.  Scales as support demand grows Handle large support volumes automatically. Your AI agent remains available across all channels without additional staffing. Built to integrate with your helpdesk and CRM ---------------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI customer service built to fit your workflow -----------------------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to see how your AI customer service could work? ----------------------------------------------------- [Book a demo](https://www.voiceflow.com/demo)  --- # AI Call Center Solutions | Voiceflow AI Call Center [](https://www.voiceflow.com/) Build and scale AI agents for your call center ================================================= Deliver fast, reliable voice support at scale. AI agents that handle inbound and outbound calls with accuracy, empathy, and efficiency. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            **Turn every inbound call into a great customer experience** ------------------------------------------------------------ Give every caller a fast, helpful experience without the wait.   Built for natural, human-like conversations Voiceflow agents speak clearly, handle interruptions, and adapt to real conversations so your customers feel understood, not managed.  Trained on your brand and policies Agents use your documentation, FAQs, and internal processes to answer questions with consistency and accuracy.  Action-oriented from the first ring From scheduling to account lookups, agents can take action on the caller’s behalf by connecting to your systems and APIs. Plug AI into the heart of your call center ------------------------------------------ Integrate with your phone stack, CRM, and tools without disrupting your workflow. .avif)  Telephony built in Handle inbound and outbound calls through Twilio, Vonage, or SIP, with support for triggered calls like reminders and notifications.  Seamless human handoff Automatically transfer to a live agent when needed, with full context and routing logic that ensures customers reach the right person.  Built for operations teams Track call performance, review transcripts, and update call flows directly from the Voiceflow platform. One agent, every channel, consistent experience ----------------------------------------------- Deliver a consistent customer experience across voice and chat.   One design, multiple channels Build once and use across phone, IVR, and messaging to ensure quality and consistency everywhere your customers interact.  Full control of logic and tone Create structured flows, enforce guardrails, and define how your brand sounds in every conversation.  Built-in feedback and testing Use analytics and testing tools to monitor performance and make updates with confidence. Make every conversation count for your customer and your team ------------------------------------------------------------- Let your customers get what they need faster while driving results for your team.   More resolutions, fewer transfers Handle common requests with precision and let your team focus on high-value interactions.  Higher satisfaction scores Reduce wait times, improve accuracy, and deliver better support experiences that reflect in your CSAT and NPS.  Effortless scale Your AI agent is always available and adapts to changing call volumes without requiring additional staff. From CRM to custom APIs—built to fit your stack -----------------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI agents built for your customers -----------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to hear how your AI call center could sound? -------------------------------------------------- [Book a demo](https://www.voiceflow.com/demo)  --- # AI Agents for Business | Voiceflow Use Cases & Builds [](https://www.voiceflow.com/) AI Agents ========= See what others are building with AI agents on Voiceflow. Browse use cases, explore real projects, and learn how to bring intelligent automation into your own workflows. What agent are you building? Here are some of the top use cases:  More Fliters  [\ \ Phone AI Agent\ --------------](https://www.voiceflow.com/ai/phone) [\ \ Collections & Payment Reminders\ -------------------------------](https://www.voiceflow.com/ai/collections-payment-reminders) [\ \ Customer Support\ ----------------](https://www.voiceflow.com/ai/customer-support) [\ \ Lead Qualification & Routing\ ----------------------------](https://www.voiceflow.com/ai/lead-qualification-routing) [\ \ Virtual Receptionist\ --------------------](https://www.voiceflow.com/ai/virtual-receptionist) [\ \ Scheduling & Appointment Booking\ --------------------------------](https://www.voiceflow.com/ai/scheduling-appointment-booking) [\ \ Cold Calling & Sales Calls\ --------------------------](https://www.voiceflow.com/ai/cold-calling-sales-calls) [\ \ Answering Service\ -----------------](https://www.voiceflow.com/ai/answering-service) [\ \ Call Centers\ ------------](https://www.voiceflow.com/ai/call-centers) Technology & IT [1](https://www.voiceflow.com/blog) ... No results found... FAQs Frequently asked questions -------------------------- What is an AI agent? An AI agent is a system that can understand natural language, make decisions, and take actions to achieve specific goals. AI agents are often powered by large language models (LLMs) and can be used for tasks like answering questions, automating workflows, or engaging in conversations with users. How do AI agents work? AI agents work by processing input (like text or voice), reasoning about the user’s intent, and generating a response or performing an action. Many modern AI agents combine LLMs with knowledge bases, memory, and API integrations to provide contextually aware, goal-driven experiences. What’s the difference between an AI agent and a chatbot? While both AI agents and chatbots interact through conversation, AI agents are more advanced and they use reasoning, context, and memory to make decisions, not just follow pre-written scripts. Chatbots typically respond to set rules, whereas AI agents dynamically generate responses using LLMs. Do I need to code to create an AI agent? Not necessarily. Many modern platforms provide visual builders and drag-and-drop tools to create AI agents without coding. Developers, however, can extend these agents by connecting APIs or integrating with custom systems for advanced functionality. Build AI Agents for customer support and beyond ----------------------------------------------- Ready to explore how Voiceflow can help your team? Let’s talk. [Sign up free](https://creator.voiceflow.com/signup)  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. --- # AI Agent Builder | Create Intelligent AI Agents | Voiceflow [](https://www.voiceflow.com/) Build and scale AI agents for every workflow ============================================ Automate conversations, tasks, and processes with intelligent agents that understand language, take action, and integrate deeply with your systems. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            Deliver instant, intelligent support across your operations ----------------------------------------------------------- Give customers and internal teams fast, accurate help powered by agents that understand context and take decisive action.   Natural language understanding AI agents interpret intent, manage back and forth conversations, and maintain context to provide clear and meaningful responses.  Knowledge from your content Connect documents, URLs, and help centers so your agent can answer questions with reliable, up to date information across domains.  Action ready from the start Whether updating records, routing requests, or triggering workflows, your agent can complete tasks by connecting to your APIs and tools. Build AI agents tailored to your processes ------------------------------------------ Use a visual environment to design, test, and refine agent behavior without writing code.   Visual builder for complex logic Design structured flows, decision paths, and conditional logic on a drag and drop canvas built for teams designing sophisticated agents.  Modular workflows and global triggers Break your agent into purpose specific workflows, allow users to switch tasks naturally, and keep navigation fluid based on real intent.  Built for collaboration Assign workflows, track work in progress, comment on steps, and manage versioning to support teams building agents at scale. One agent supporting all your channels and systems -------------------------------------------------- Deploy a single agent across voice, chat, and messaging to deliver consistent performance everywhere.   Built once, deployed broadly Launch your agent on phone, web chat, SMS, and custom channels to unify experience across your digital and voice touchpoints.  Powered by the model you choose Use GPT, Claude, or bring your own LLM. Select models per workflow for performance, cost, or domain specific needs.  Test and optimize with confidence Validate flows, try variations, evaluate logic branches, and debug within the canvas before launching to customers. Turn conversations into productive outcomes ------------------------------------------- Automate routine tasks while empowering teams to focus on higher value work.   More actions, fewer bottlenecks Your agent handles repetitive questions, gathers context, routes tasks, and executes commands so teams stay focused on strategic work.  Consistent quality at every interaction Accurate responses, reliable task execution, and structured logic help deliver better experiences across teams and customers.  Scales across teams and regions Handle large volumes of queries and actions without additional staffing or operational overhead. Built to integrate with your systems and workflows --------------------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI agents built to connect deeply across your stack ----------------------------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to see how your AI agent could work? ------------------------------------------ [Book a demo](https://www.voiceflow.com/demo)  --- # AI Lead Generation Tools | Voiceflow AI Lead Generation [](https://www.voiceflow.com/) Build and scale AI agents that capture and qualify leads ======================================================== Convert more visitors into sales opportunities with AI that engages prospects, gathers information, and routes high quality leads into your pipeline. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            Turn conversations into qualified leads from the first interaction ------------------------------------------------------------------ Give every prospect a fast, natural experience that collects the information your team needs.   Conversational lead capture AI powered agent steps ask dynamic questions, understand context, and gather lead details without rigid forms, making the process more natural for users.  Structured data collection Use Capture steps to validate emails, extract phone numbers, or confirm company names with automatic reprompts to ensure clean, accurate data.  Personalized using variables Store captured details as variables to tailor responses, guide qualification logic, and prepare clean data for your CRM or database. Build intelligent lead generation flows without code ---------------------------------------------------- Use a visual workspace to design, test, and optimize your entire lead capture and qualification process.   Conversion focused workflows Connect steps, conditions, and branches on a drag and drop canvas to create structured flows that match your sales funnel.  Intelligent qualification paths Use condition steps to qualify based on budget, timeline, interests, or company attributes. Route high intent leads to sales immediately while nurturing others automatically.  Collaboration for growth teams Assign workflows, leave comments, manage versions, and track progress so sales, marketing, and operations can build together. Capture, qualify, and route leads across every channel ------------------------------------------------------ Deploy a single agent across web, SMS, WhatsApp, and voice to meet prospects wherever they convert.   Multi channel deployment Launch on web chat, messaging apps, SMS, or phone to give prospects multiple ways to start a conversation and share their information.  Schedule meetings in the conversation Integrate your scheduling tool to let qualified leads book instantly. Reduce friction and hand off warm prospects directly to your team.  Testing tools built in Validate every path, qualification rule, and integration connection before launch to ensure smooth lead capture from day one. Accelerate your sales pipeline with automation and smart routing ---------------------------------------------------------------- Let your AI agent handle repetitive lead capture while your sales team focuses on closing deals.   Smarter lead qualification Use conditions, product knowledge, and intent signals to prioritize hot leads and personalize the next step for each user.  Higher conversions from educated leads Your agent answers product questions, addresses objections, and uses your content to guide prospects toward informed decisions.  Scales as demand grows Capture and qualify leads at any volume across channels without increasing operational load. Built to integrate with your sales stack ----------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI lead generation agents that plug into your CRM and workflows ----------------------------------------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to see how your AI lead generation agent could work --------------------------------------------------------- [Book a demo](https://www.voiceflow.com/demo)  --- # AI Chatbot Builder | Voiceflow AI Chatbot Platform [](https://www.voiceflow.com/) Build and scale AI chatbots for every customer interaction ========================================================== Deliver fast, intelligent support and automation across your digital channels. Your AI chatbot understands conversations, completes tasks, and adapts to your workflows. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            Create smoother customer interactions from the first message ------------------------------------------------------------ Give users instant, helpful responses with a chatbot built to feel natural and intuitive.   Conversational intelligence built in AI powered steps understand user messages, manage back and forth dialogue, and respond naturally while keeping context throughout the conversation.  Knowledge from your content Connect documents, URLs, help centers, or internal knowledge so your chatbot answers questions with accurate, up to date information.  Designed to take action From account lookups to form submissions, your chatbot uses APIs and dynamic data to complete tasks in real time. Build smarter chatbot experiences without code ---------------------------------------------- Use a visual workspace to design, test, and optimize your chatbot with precision.   Visual canvas for complex workflows Build conversation flows on an infinite drag and drop canvas. Connect steps, conditions, and branches to design structured logic with full visibility.  Modular workflows and triggers Create dedicated workflows for tasks like ordering, tracking, or account updates. Add global triggers so users can switch workflows naturally based on intent.  Collaboration for teams Assign workflows, comment on steps, manage versions, and track progress so teams can build sophisticated chatbots together. One chatbot across all your customer channels --------------------------------------------- Provide consistent, intelligent support everywhere your users interact.   Build once, deploy everywhere Launch your chatbot across web widgets, WhatsApp, SMS, and custom channels with consistent behavior across platforms.  Custom web chat experiences Add a brand aligned chat widget to your site with support for embedded layouts, popovers, and voice input for rich user interactions.  Test and refine before launch Use the built in test tool to validate flows, try variations, run guided tests, and debug complex conditions before deploying. Make conversations more effective for your business and your customers ---------------------------------------------------------------------- Automate routine work while enabling your team to focus on deeper, more meaningful tasks.   More answers with less effort Your chatbot resolves common questions, guides users through workflows, and reduces the need for manual assistance.  Higher user satisfaction Fast responses, accurate information, and smooth navigation create better customer experiences and reduce friction.  Scalable support without added staff Your chatbot handles increasing volumes across channels automatically, providing consistent performance at any scale. Built to connect with your systems and data -------------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI chatbots designed to integrate seamlessly into your stack -------------------------------------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to see how your AI chatbot could work? -------------------------------------------- [Book a demo](https://www.voiceflow.com/demo)  --- # AI Voice Agent Platform | Build Conversational Voice AI [](https://www.voiceflow.com/) Build and scale AI voice agents for real conversations ====================================================== Deliver fast, natural voice support across your website and channels. Your AI agent listens, understands, and responds with clarity while completing tasks and guiding users. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            Bring natural voice interactions to your customer experience ------------------------------------------------------------ Give users a seamless, human-like way to get help through spoken conversation.   Voice input built directly into your site Use the Voiceflow web chat widget to let visitors speak through their browser microphone. Choose from widget, popover, or embedded layouts and deploy in minutes with a simple script.  Accurate speech recognition Configure STT providers like Cartesia, AssemblyAI, or Deepgram to transcribe customer speech with precision. Fine-tune thresholds, timeouts, and custom keywords to reduce errors and maintain fluid conversations.  Intelligent understanding The AI Agent step interprets open-ended requests, searches your knowledge, executes actions, and adapts responses—without mapping every possible path manually. Build intelligent voice experiences without writing code -------------------------------------------------------- Use a visual workspace to craft natural, reliable voice flows with full control over the logic.   Visual canvas for voice-driven flows Design decision paths, function calls, and fallback rules using a drag-and-drop builder built for conversational design.  Knowledge-backed answers Upload documentation or sync your help center so your voice agent retrieves accurate, context-aware information when answering questions.  Collaboration for support and product teams Assign workflows, comment on logic, manage versions, and build together across teams working on voice-based interfaces. Engage visitors proactively and guide them with context ------------------------------------------------------- Deliver timely, relevant help before they even open your widget.   Behavior-based proactive messages Trigger helpful messages based on user actions—lingering on checkout, browsing products, or showing exit intent—to increase engagement and reduce abandonment.  Consistent voice personality Define tone, rules, and domain expertise through prompting to ensure your voice agent sounds on-brand and handles escalation consistently.  Custom UI and effects Use response extensions for interactive inputs and effect extensions to update your website dynamically during the voice interaction. Turn voice conversations into real task completion -------------------------------------------------- Help customers troubleshoot, navigate, and resolve issues through action-oriented voice experiences.   Zendesk connected support Create tickets, update statuses, log notes, and retrieve user details directly during the call. Your agent automatically escalates complex cases with all context included.  Voice-driven actions Run APIs, trigger workflows, create records, or manage user requests directly through spoken interaction.  Adaptive logic paths Use conditions to personalize flows based on intent, customer tier, preferences, or detected sentiment. Built to integrate with your voice and support ecosystem --------------------------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI voice agents connected to your systems ------------------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to hear how your AI voice agent could sound? -------------------------------------------------- [Book a demo](https://www.voiceflow.com/demo)  --- # Cathy Pearl on all things conversation design | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Cathy Pearl on all things conversation design ============================================= [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at](https://www.voiceflow.com/contributors/mark-ammendolia) [Cathy Pearl](https://twitter.com/cpearl42?lang=en) is Head of Conversation Design Outreach at Google. She has been a driving force in voice tech and VUI design for over two decades, creating conversational interfaces for voice, text, and multi-modal since 1999. A world-renown leader in conversation design, Cathy has helped educate thousands in the voice space by spreading her knowledge through a variety of different mediums. She has gained notoriety as an international speaker, having presented at a variety of events, including Cannes Lions International Festival of Creativity, SXSW, O'Reilly Design, Voice Summit, and Project Voice. She has also penned numerous articles, blogs, and her very own book, [_Designing Voice User Interfaces_](https://www.cathypearl.com/book) , the go-to resource for those interested in exploring VUI design. We were lucky enough to chat with Cathy about why conversation design is important, the growing role of the conversation designer, and helpful resources for those passionate about joining this field.  ### Transcript: > **0:01 - 0:13** > > My name is Cathy Pearl. I work at Google. I'm the Head of Conversation Design Outreach, which means I go out in the world and I give talks and write articles about what conversation design is and how to do it. > > **0:14 - 1:26 → When did you fall in love with voice tech? > > **When I was a kid...really uh... I just loved movies like war games and TV shows, like Knight Rider - talking car, talking computer. > > And our family got a computer when I was eight and I learned to program and I had a strong affinity with a computer and writing programs and I wrote a chat bot when I was 12 and I was just so into this idea of getting to have a conversation with a computer, not having any idea that someday there would actually be a career in such a thing. > > Um, but, uh, yeah, so my fascination started pretty, pretty early. Conversation design is essentially at its heart teaching computers to communicate like humans and not the other way around. So not forcing people to adopt a new language in order to get things done. > > It's through this, um, that we've applied a lot of best practices and principles that can enable anyone who wants to create or build these types of conversational experiences to know that we do have patterns. We do have methods to make the conversations effective because the truth is computers are not yet capable of sort of full on regular human to human conversations that we're used to. And so we have to not only write great conversations, but we have to do so in a very technically constrained world and marry those two things together to get an effective user experience. > > **1:27 - 1:57 → Talk about the growing role of conversation designers > > **The job role of conversation designer is still quite new, and I think we're starting to see companies realize that if they're going to get into this space, they can't just hire a couple engineers and have the marketing team tell them what to build, but they actually have to invest a little bit in getting a conversation designer. > > I mean conversation designers are thinking about, first of all, should we even build this thing? Is this a good use case for conversation? Who are our users going to be? What are the questions we need to ask? What order of the questions is going to be in? What are the types of things people might say? > > It's not just about crafting the words, sort of at the end. It's really the conversation is the structure. So getting a conversation designer involved from the get-go is going to mean the whole thing's going to go a lot more smoothly. But I'm not biased or anything. > > **1:58 - 2:41 → Where can I learn more about conversation design? > > **So if you're interested in learning about conversation design, a few things I'd recommend. > > One is, um, Google has published a lot of great guidelines that [**_actions.google.com/design_**](http://actions.google.com/design) > . What is conversation design? How to do it, how to get started. Um, and then at a more personal level, I've got a website, cathypearl.com. There's links to my book, _Designing Voice User Interfaces_ _\-_ as well as blog posts I've written. > > So that's a great place to check things out. I'm also on Twitter at [**@cpearl42**](https://twitter.com/cpearl42?lang=en) > tweeting about conversation design and things like that. ### Transcript: > **0:01 - 0:13** > > My name is Cathy Pearl. I work at Google. I'm the Head of Conversation Design Outreach, which means I go out in the world and I give talks and write articles about what conversation design is and how to do it. > > **0:14 - 1:26 → When did you fall in love with voice tech? > > **When I was a kid...really uh... I just loved movies like war games and TV shows, like Knight Rider - talking car, talking computer. > > And our family got a computer when I was eight and I learned to program and I had a strong affinity with a computer and writing programs and I wrote a chat bot when I was 12 and I was just so into this idea of getting to have a conversation with a computer, not having any idea that someday there would actually be a career in such a thing. > > Um, but, uh, yeah, so my fascination started pretty, pretty early. Conversation design is essentially at its heart teaching computers to communicate like humans and not the other way around. So not forcing people to adopt a new language in order to get things done. > > It's through this, um, that we've applied a lot of best practices and principles that can enable anyone who wants to create or build these types of conversational experiences to know that we do have patterns. We do have methods to make the conversations effective because the truth is computers are not yet capable of sort of full on regular human to human conversations that we're used to. And so we have to not only write great conversations, but we have to do so in a very technically constrained world and marry those two things together to get an effective user experience. > > **1:27 - 1:57 → Talk about the growing role of conversation designers > > **The job role of conversation designer is still quite new, and I think we're starting to see companies realize that if they're going to get into this space, they can't just hire a couple engineers and have the marketing team tell them what to build, but they actually have to invest a little bit in getting a conversation designer. > > I mean conversation designers are thinking about, first of all, should we even build this thing? Is this a good use case for conversation? Who are our users going to be? What are the questions we need to ask? What order of the questions is going to be in? What are the types of things people might say? > > It's not just about crafting the words, sort of at the end. It's really the conversation is the structure. So getting a conversation designer involved from the get-go is going to mean the whole thing's going to go a lot more smoothly. But I'm not biased or anything. > > **1:58 - 2:41 → Where can I learn more about conversation design? > > **So if you're interested in learning about conversation design, a few things I'd recommend. > > One is, um, Google has published a lot of great guidelines that [**_actions.google.com/design_**](http://actions.google.com/design) > . What is conversation design? How to do it, how to get started. Um, and then at a more personal level, I've got a website, cathypearl.com. There's links to my book, _Designing Voice User Interfaces_ _\-_ as well as blog posts I've written. > > So that's a great place to check things out. I'm also on Twitter at [**@cpearl42**](https://twitter.com/cpearl42?lang=en) > tweeting about conversation design and things like that.  RECOMMENDED [](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design#) PUBLISHED February 27, 2020 TOPICS IN THIS ARTICLE Exceptional people [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at\ \ Mark is a Pathways contributor.](https://www.voiceflow.com/contributors/mark-ammendolia) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Designing for voice assistants vs. chatbots](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots) [](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to prioritize requests for conversation design teams](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [](https://www.voiceflow.com/pathways/future-mental-health-support) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### From microbiology to mental health: In conversation with Ebere Wilson](https://www.voiceflow.com/pathways/future-mental-health-support) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Voiceflow Templates [](https://www.voiceflow.com/) Pre-built templates for every use case ====================================== Discover our library of agent-building resources. Search by use case, product, creator & more. What agent are you building? Here are some of the top use cases: [Templates\ \ Templates and tutorials for all AI agent categories](https://www.voiceflow.com/resources-template?products=Template) [Apps\ \ Third party apps that extend Voiceflow functionality](https://www.voiceflow.com/resources-apps?products=app) [Integrations\ \ Browse Functions that integrate with other tools](https://www.voiceflow.com/resources-functions?products=Function)  .svg) Use case [Clear](https://www.voiceflow.com/templates#) This is some text inside of a div block. AI character This is some text inside of a div block. Booking classes This is some text inside of a div block. Customer support This is some text inside of a div block. FAQs This is some text inside of a div block. Internal co-pilot This is some text inside of a div block. Lead capture This is some text inside of a div block. Lead generation This is some text inside of a div block. Lead qualification This is some text inside of a div block. Package explanations This is some text inside of a div block. Product education This is some text inside of a div block. Product recommendation This is some text inside of a div block. Trainer Q&A This is some text inside of a div block. Venue Q&A Interface [Clear](https://www.voiceflow.com/templates#) This is some text inside of a div block. Voice This is some text inside of a div block. Chat Industries [Clear](https://www.voiceflow.com/templates#) This is some text inside of a div block. Gaming This is some text inside of a div block. Finance This is some text inside of a div block. Law firms This is some text inside of a div block. Ecommerce This is some text inside of a div block. Restaurants This is some text inside of a div block. Property management This is some text inside of a div block. Real estate This is some text inside of a div block. Healthcare This is some text inside of a div block. Travel This is some text inside of a div block. Insurance Categories [Clear](https://www.voiceflow.com/templates#) This is some text inside of a div block. Analytics This is some text inside of a div block. Automation platform This is some text inside of a div block. CRM This is some text inside of a div block. Data pipeline platforms This is some text inside of a div block. Data warehouse This is some text inside of a div block. Database This is some text inside of a div block. E-commerce This is some text inside of a div block. Integration platform This is some text inside of a div block. LLM models This is some text inside of a div block. Marketing automation This is some text inside of a div block. Messaging This is some text inside of a div block. Payment service This is some text inside of a div block. Project management This is some text inside of a div block. Support platform This is some text inside of a div block. Video games This is some text inside of a div block. Voice All ProductsTemplateFunctionAll CreatorsVoiceflowCommunityCertified PartnersPaid + Free ️FreePaid More Fliters  [](https://www.voiceflow.com/templates/complete-support-chatbot) .webp) Certified Template 4927 ### Complete support chatbot template Download this template to build a complete support chatbot that answers FAQs, recommends products, captures user information, and more.  Daniel D'Souza No items found. No items found. FREE Voiceflow Template [](https://www.voiceflow.com/templates/ai-agent-with-knowledge-base-memory)  Certified Template 2176 ### AI Agent with Knowledge Base Memory Use this template to create an AI agent that handles follow ups with contextual memory  W. Williams No items found. No items found. FREE Community Template [](https://www.voiceflow.com/templates/building-an-ai-powered-hotel-booking-assistant)  Certified Template 1473 ### Building an AI-Powered Hotel Booking Assistant Pete walks through how to build a personalized hotel booking assistant for Hotels.com by leveraging CRM data, large language models (LLMs), and Airtable integration.  Peter Isaacs No items found. No items found. FREE Voiceflow Template [](https://www.voiceflow.com/templates/complete-retail-chatbot)  Certified Template 1391 ### Complete retail chatbot template Download this template to build a complete retail chatbot that handles customer service, lets users shop products, and more.  Daniel D'Souza No items found. No items found. FREE Voiceflow Template [](https://www.voiceflow.com/templates/shopify-order-checker-ai-agent)  Certified Template 905 ### Shopify Order Checker AI Agent Build an order checker agent for your Shopify store that then displays the data in a carousel format.  Brendan Jowett Customer support E-commerce Certified Partners Template [](https://www.voiceflow.com/templates/ai-order-return-agent-for-new-balance)  Certified Template 453 ### AI Order Return Agent for New Balance how to build an embedded web agent that can track order status, process refunds, and provide product recommendations  Peter Isaacs No items found. No items found. Voiceflow Template [1](https://www.voiceflow.com/templates#) ... [Next](https://www.voiceflow.com/templates?926cb0f5_page=2) 1 / 22 [Platform connection](https://www.voiceflow.com/apps-categories/platform-connection) [Analytics](https://www.voiceflow.com/apps-categories/analytics) [Custom interface](https://www.voiceflow.com/apps-categories/custom-interface) [Parser](https://www.voiceflow.com/voiceflow-categories-market-place/parser) [Custom action](https://www.voiceflow.com/voiceflow-categories-market-place/custom-action) [Custom steps](https://www.voiceflow.com/voiceflow-categories-market-place/custom-steps) [Parser](https://www.voiceflow.com/voiceflow-categories-market-place/parser) [Custom action](https://www.voiceflow.com/voiceflow-categories-market-place/custom-action) [Custom steps](https://www.voiceflow.com/voiceflow-categories-market-place/custom-steps) No results found... Related collections ------------------- [Browse all collections](https://www.voiceflow.com/collections) [\ \ ### Top 6 E-commerce Templates and Tutorials\ \ In today's AI wave, having a quality chatbot is the baseline for all businesses. It's crucial to enable 24/7 customer interaction and greatly improve your efficiency. What if I told you that your chatbot can do so much more? With these templates and tutorials level up your chatbot so it can manage product recommendations, collect customer data, track orders, and handle customer queries.\ \ \ \ Kim Lu](https://www.voiceflow.com/collections/top-ecomm-templates-and-tutorials) [\ \ ### Top Resources for AI Agencies\ \ It’s no secret that the release of tools like ChatGPT has challenged how we see the role of AI across every industry. AI is growing at a rate that no other technology has before, with new models, platforms, and research being published daily. To keep up with the pace of change, AI agencies formed, helping businesses—big and small—carve out their stake in AI.\ \ \ \ Kim Lu](https://www.voiceflow.com/collections/top-resources-for-agencies) [\ \ ### Top 6 Customer Support Templates and Tutorials\ \ AI agents are revolutionizing the way the businesses interact with customers, offering 24/7 service, ability to handle multiple queries, and now even complete custom tasks. We’ve compiled templates and tutorials to help you go from zero to hero with your support AI agent. Stick to the end to checkout key case studies of teams who’ve successfully automated their customer support function with Voiceflow.\ \ \ \ Kim Lu](https://www.voiceflow.com/collections/customer-support-template-and-tutorial) Have something to share? ------------------------ Share your creation with over 250,000 other global Voiceflow users. [Share your resource](https://airtable.com/appCYnIC5ir7nKu8W/shra7TXU4NmWOhKZK) [Join the Voiceflow Discord](https://link.voiceflow.com/community)  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. --- # The evolution and outlook of voice user interfaces | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) The evolution and outlook of voice user interfaces ================================================== [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at](https://www.voiceflow.com/contributors/mark-ammendolia) If you've ever searched out podcasts surrounding conversational AI and voice-enabled technology, there's a very good chance you've come across [VUX World](https://vux.world/) . Founded and hosted by [Kane Simms](https://www.linkedin.com/in/kanesimms/) and [Dustin Coates](https://www.linkedin.com/in/dustincoates/?locale=de_DE) , VUX World has quickly grown into [one of the most popular AI podcasts in its class](https://vux.world/podcast/) . From helping brands innovate on Alexa and Google Assistant to explaining how enterprises can use conversational AI to solve core business challenges across existing channels (contact centers, IVRs, websites, etc.), Kane and Dustin are hard set on helping businesses and individuals further educate themselves on this burgeoning tech. Kane Simms recently joined Voiceflow's [_Ask an Expert_](https://www.youtube.com/watch?v=SXpVNEai17c&t) webinar series, where he responded to a host of questions submitted by our community. We've summarized many of his answers, which focus on the evolving voice landscape, enterprise adoption of VUIs, and the pandemic's impact on current industry trends. More on Kane Simms: Kane Simms is one of the world's premier voice strategists working with top brands on conversational AI initiatives. He's also a VUX designer who has taught hundreds of students how to build conversational user interfaces. Before starting VUX World, Kane spent six years leading corporate digital transformation initiatives and implementation teams. A speaker, host, thought-leader, and well-known personality, Kane is passionate about helping organizations take advantage of conversational AIs, leading to the most significant shift in user behavior since the smartphone. (Source: VUX World, Linkedin)  #### Full Recording Below: ### **What you'll learn by reading this:** 1. Voice is bigger than Alexa and Google Assistant 2. Navigating the changing voice landscape 3. Enterprise adoption of VUIs 4. The biggest mistakes businesses make when adopting voice technology 5. The pandemic's impact on industry trends **Q1: What excites you the most about voice tech right now?** ------------------------------------------------------------- **Kane:** I think what excites me more than anything, is I've got a lot of experience working in service design and digital transformation initiatives. With the team I used to manage at my old job, we'd kind of work across the organization, dig up everything, rebuild it, automate as much as we can. \[It was\] really end-to-end digital transformation. And so we've always been looking at voice through that lens. \[Asking questions like\] how can we use this technology to make companies more efficient? How can we make customer experiences more streamlined? How can we make business units work and function better? How can we increase revenue by providing seamless experiences? A lot of people get into voice from the Alexa Smart Speaker ecosystem and that's where it kind of started for me initially. What happens is you'll discover Alexa or discover the Google Assistant and then you'll start building and playing around with it. Often, this is \[focused on\] education, interactive audio, or entertainment-based use-cases - and so that's where a lot of people begin their journey. But as soon as you start lifting under the hood and investigate about where voice can be used effectively, you start realizing the benefit of it in doing those things. I've mentioned improving customer experience, improving access to streamlined services, enabling self-service, freeing up contact centers. That's what is exciting to me, \[...\] that voice is bigger than Alexa and Google Assistant and that this concept of voice is being broadened to conversational AI. And so we tend to talk about conversational AI now rather than voice specifically. What excites me most, however, is that having gone through this journey — through entertainment, through leisure, through interactive audio — we're now starting to find areas where the technology is good enough and the businesses are ambitious enough to start really searching for those use-cases where this \[technology\] can be used to increase productivity or streamline service access. If you look at Moscow Airport — the ultimate IVR system with a conversational agent — \[they are\] saving 30% of calls from hitting their contact center. That's what's exciting to me about voice and about conversational AI — real practical application of voice technologies that can be used to measurably enhance customer experiences. Q2: **How do you see voice experiences evolving right now? Do you think we're still a long ways from the adoption phase?** -------------------------------------------------------------------------------------------------------------------------- **Kane:** Well, it depends. One of the difficulties about having people adopt stuff right now is that if you're building for Alexa or Google Assistant, you need to draw them into that ecosystem for the purposes that you're creating stuff for. And that's where a lot of the challenges are. People are always talking about discoverability because the Alexa ecosystem is over here, the Google Assistant ecosystem is over there, but everyone is already on social media, and phoning your contact center, and on your website. And so while people are already behaving one way, we want to bring people over here to \[use\] this. People are gradually using these services - Alexa and Google Assistant - for routine things like timers and music. I think that's why it's difficult to get things discovered — you're not only fighting to be that one app that's at the tip of the iceberg, but you're also fighting to bring people over into an ecosystem to discover they can do more than just music. And so if you think about applying these same technologies in other areas - like in an app where you have a voice interface on top of your experience, or on a website where you can streamline user journeys through voice search - there is no struggle or barrier because the technology is already baked in. I don't know if it's early in terms of adoption in those channels — maybe it's early in the adoption of third-party skills for Alexa and Google Assistant — but we're starting to see businesses and enterprises wake up a little. All the big players like Microsoft, Amazon, Google, Oracle, Salesforce have got services. Everyone's building out the capability, and so we're starting to see this beginning to happen at other larger companies. Q3: **Is there an industry that VUIs will impact the most in the future?** -------------------------------------------------------------------------- **Kane**: To be honest — it's anything and everything. I think we're going to continue to see new devices. I think smart speakers are a brand new category. There was no such thing as smart speakers before. There was no such thing as smart displays before. And so I think we're going to continue to see new devices pop up. What will they be? Who knows. But I think that we will continue to see a proliferation of different devices with voice interfaces in them. I also think we're going to see voice interfaces on all our current devices — and they kind of already are. If you've got any Apple gear for example, then you've got Siri on everything. If you've got anything Android, you've got Google Assistant on everything. And so we already see that. We're going to continue to see those platforms — Alexa and Google Assistant — continue to grow. I think we're going to see more devices, more \[voice\] interfaces on current devices, and we're going to see all sorts of stuff spring up on the environments that we already go to to get stuff done. Whether it's in apps or in phone lines — \[it'll be\] everywhere. ### Enterprise adoption of VUIs I think we're going to see enterprise really adopt it and start to open up. But this is the thing — as soon as you start getting into enterprise and begin wanting to integrate with line of business systems, you very quickly realize that many of those systems are legacy-based, and it's very difficult to integrate with them. A lot of the work that we used to do when we were designing services \[involved working with\] systems \[that\] were so old and clunky that you would take data dumps overnight into a data warehouse and integrate with that. And so what voice technology does is the same as mobile, social media, and the internet, which is make businesses question how they operate and function. Voice technology is another tool in the digital transformation arsenal that enterprises can use to be more efficient, operate more cost-effectively, and scale what they do. I think we're going to see a lot more enterprise use-cases as well. Q4: **What are the biggest mistakes you've seen businesses and creators make when getting into voice?** ------------------------------------------------------------------------------------------------------- **Kane:** Just because you think you've got a good idea doesn't mean it is. One of the most crucial things that you can do - as painful as it is - is to put something in front of someone who intends to use it as early as you can. You tend to see a lot of hammer's searching for a nail — or solutions that people think are a good idea and build them, but they might not solve a specific problem. They might not have a specific use-case that's being addressed. And so we tend to advise that you start with the user. Start with the problem. Even though you're thinking about building a voice solution, forget it at the start and try and do your user research. Speak to your end-users. Speak to the person who is ultimately going to use this thing. Find out how they currently use and access your company \[or\] how they currently buy your products and access your services. What's wrong with those existing channels, and what are the expectations? Why do they choose a competitor over you? Then decide and see whether or not there is a solution that a voice interface will solve. With this way, you end up starting with the nail and then building the hammer or finding the nail and then building the hammer, rather than trying to build something that you think is a cool idea. \[Build something\] to learn by all means. If it helps you learn, if it helps you understand the platform, if it helps you get your head around how to build \[and\] design stuff — by all means, build stuff and practice, practice, practice. But to release something into the world — for a business to produce something — it needs to be rooted in what is going to be valuable for the user. If you look at the top 10 skills in the Alexa Skill store in America, none of them are branded. NPR and Jeopardy are the top two \[and\] the rest of \[them\] are things that people have found a use-case for and gone ahead and built themselves. That's the hardest part for businesses — trying to find out where you can align and what you are trying to achieve \[in terms of\] your business objectives \[in relation to\] a genuine user need or problem \[while\] having voice be the solution in the middle. **Q5: Within the voice-tech world, what trends have you seen since the beginning of the pandemic?** --------------------------------------------------------------------------------------------------- **Kane**: What is happening or what we're noticing is that places that have stores or physical locations that are no longer open \[but\] can sustain themselves — \[like\] the big companies — are now looking at ways to increase or improve in-store engagement for after COVID. A lot of conversations we've been having are around how to create in-store experiences using voice technology so that when people go back into stores they don't have to touch quite as much stuff and can self-serve. So that's a future situation that is being worked on now, so when stores do reopen, you might find voice experiences in-stores a little bit more. In terms of the stuff that's happening right now, I mentioned IVRs and voice in the phone lines and things like that. One of our partner organizations that specializes in IVR software and the creation of interactive IVR systems are run off their feet at the moment — building things for government organizations and for the travel industry that has been hit really hard. \[It's really\] conversational technologies in general. It doesn't need to be voice specific — it could be a chatbot on a website or a Facebook messenger bot. A really good example is the Whatsapp bot that the world health organization released a few weeks back. And so \[this technology\] is being used to help organizations manage an increase in demand and scale of their customer support and customer service as well as being thought about now \[as a solution\] to help plan for when the stores eventually reopen. **Q6: What resources did you use to level-up your understanding of VUIs** ------------------------------------------------------------------------- **Kane**: I used to watch the Amazon Teams' Twitch streams quite a lot. They came in handy. I thought they were really, really good because they take you through a step-by-step \[process\] of how to build a skill while talking you through absolutely every single part of it. Even though it was very technical — and i'm not a developer — it helped me understand it enough to hold myself in a conversation with either a client or a guest on a podcast. The Amazon blog was pretty good \[as well\]. Google's also published quite a lot of stuff. And then much of \[my early knowledge\] came from actually talking to people, you know? I also think that teaching is the best form of learning, and so I've been a VUI design instructor for a while now with the VUX Academy. Going through that process helps you uncover things that you might not have thought of before running workshops. We've been doing workshops now for almost two years, and with every single one, you learn something different. So it's a mixture of trying to learn from the guests that we have on the podcast, learning from the people who are building this stuff at Amazon and Google but then also through the experience of building skills, through trying things, and exploring tools like Voiceflow and others. ******** > Interested in joining our next live webinar? Sign up for upcoming events [here](https://www.voiceflow.com/events) > . #### Full Recording Below: ### **What you'll learn by reading this:** 1. Voice is bigger than Alexa and Google Assistant 2. Navigating the changing voice landscape 3. Enterprise adoption of VUIs 4. The biggest mistakes businesses make when adopting voice technology 5. The pandemic's impact on industry trends **Q1: What excites you the most about voice tech right now?** ------------------------------------------------------------- **Kane:** I think what excites me more than anything, is I've got a lot of experience working in service design and digital transformation initiatives. With the team I used to manage at my old job, we'd kind of work across the organization, dig up everything, rebuild it, automate as much as we can. \[It was\] really end-to-end digital transformation. And so we've always been looking at voice through that lens. \[Asking questions like\] how can we use this technology to make companies more efficient? How can we make customer experiences more streamlined? How can we make business units work and function better? How can we increase revenue by providing seamless experiences? A lot of people get into voice from the Alexa Smart Speaker ecosystem and that's where it kind of started for me initially. What happens is you'll discover Alexa or discover the Google Assistant and then you'll start building and playing around with it. Often, this is \[focused on\] education, interactive audio, or entertainment-based use-cases - and so that's where a lot of people begin their journey. But as soon as you start lifting under the hood and investigate about where voice can be used effectively, you start realizing the benefit of it in doing those things. I've mentioned improving customer experience, improving access to streamlined services, enabling self-service, freeing up contact centers. That's what is exciting to me, \[...\] that voice is bigger than Alexa and Google Assistant and that this concept of voice is being broadened to conversational AI. And so we tend to talk about conversational AI now rather than voice specifically. What excites me most, however, is that having gone through this journey — through entertainment, through leisure, through interactive audio — we're now starting to find areas where the technology is good enough and the businesses are ambitious enough to start really searching for those use-cases where this \[technology\] can be used to increase productivity or streamline service access. If you look at Moscow Airport — the ultimate IVR system with a conversational agent — \[they are\] saving 30% of calls from hitting their contact center. That's what's exciting to me about voice and about conversational AI — real practical application of voice technologies that can be used to measurably enhance customer experiences. Q2: **How do you see voice experiences evolving right now? Do you think we're still a long ways from the adoption phase?** -------------------------------------------------------------------------------------------------------------------------- **Kane:** Well, it depends. One of the difficulties about having people adopt stuff right now is that if you're building for Alexa or Google Assistant, you need to draw them into that ecosystem for the purposes that you're creating stuff for. And that's where a lot of the challenges are. People are always talking about discoverability because the Alexa ecosystem is over here, the Google Assistant ecosystem is over there, but everyone is already on social media, and phoning your contact center, and on your website. And so while people are already behaving one way, we want to bring people over here to \[use\] this. People are gradually using these services - Alexa and Google Assistant - for routine things like timers and music. I think that's why it's difficult to get things discovered — you're not only fighting to be that one app that's at the tip of the iceberg, but you're also fighting to bring people over into an ecosystem to discover they can do more than just music. And so if you think about applying these same technologies in other areas - like in an app where you have a voice interface on top of your experience, or on a website where you can streamline user journeys through voice search - there is no struggle or barrier because the technology is already baked in. I don't know if it's early in terms of adoption in those channels — maybe it's early in the adoption of third-party skills for Alexa and Google Assistant — but we're starting to see businesses and enterprises wake up a little. All the big players like Microsoft, Amazon, Google, Oracle, Salesforce have got services. Everyone's building out the capability, and so we're starting to see this beginning to happen at other larger companies. Q3: **Is there an industry that VUIs will impact the most in the future?** -------------------------------------------------------------------------- **Kane**: To be honest — it's anything and everything. I think we're going to continue to see new devices. I think smart speakers are a brand new category. There was no such thing as smart speakers before. There was no such thing as smart displays before. And so I think we're going to continue to see new devices pop up. What will they be? Who knows. But I think that we will continue to see a proliferation of different devices with voice interfaces in them. I also think we're going to see voice interfaces on all our current devices — and they kind of already are. If you've got any Apple gear for example, then you've got Siri on everything. If you've got anything Android, you've got Google Assistant on everything. And so we already see that. We're going to continue to see those platforms — Alexa and Google Assistant — continue to grow. I think we're going to see more devices, more \[voice\] interfaces on current devices, and we're going to see all sorts of stuff spring up on the environments that we already go to to get stuff done. Whether it's in apps or in phone lines — \[it'll be\] everywhere. ### Enterprise adoption of VUIs I think we're going to see enterprise really adopt it and start to open up. But this is the thing — as soon as you start getting into enterprise and begin wanting to integrate with line of business systems, you very quickly realize that many of those systems are legacy-based, and it's very difficult to integrate with them. A lot of the work that we used to do when we were designing services \[involved working with\] systems \[that\] were so old and clunky that you would take data dumps overnight into a data warehouse and integrate with that. And so what voice technology does is the same as mobile, social media, and the internet, which is make businesses question how they operate and function. Voice technology is another tool in the digital transformation arsenal that enterprises can use to be more efficient, operate more cost-effectively, and scale what they do. I think we're going to see a lot more enterprise use-cases as well. Q4: **What are the biggest mistakes you've seen businesses and creators make when getting into voice?** ------------------------------------------------------------------------------------------------------- **Kane:** Just because you think you've got a good idea doesn't mean it is. One of the most crucial things that you can do - as painful as it is - is to put something in front of someone who intends to use it as early as you can. You tend to see a lot of hammer's searching for a nail — or solutions that people think are a good idea and build them, but they might not solve a specific problem. They might not have a specific use-case that's being addressed. And so we tend to advise that you start with the user. Start with the problem. Even though you're thinking about building a voice solution, forget it at the start and try and do your user research. Speak to your end-users. Speak to the person who is ultimately going to use this thing. Find out how they currently use and access your company \[or\] how they currently buy your products and access your services. What's wrong with those existing channels, and what are the expectations? Why do they choose a competitor over you? Then decide and see whether or not there is a solution that a voice interface will solve. With this way, you end up starting with the nail and then building the hammer or finding the nail and then building the hammer, rather than trying to build something that you think is a cool idea. \[Build something\] to learn by all means. If it helps you learn, if it helps you understand the platform, if it helps you get your head around how to build \[and\] design stuff — by all means, build stuff and practice, practice, practice. But to release something into the world — for a business to produce something — it needs to be rooted in what is going to be valuable for the user. If you look at the top 10 skills in the Alexa Skill store in America, none of them are branded. NPR and Jeopardy are the top two \[and\] the rest of \[them\] are things that people have found a use-case for and gone ahead and built themselves. That's the hardest part for businesses — trying to find out where you can align and what you are trying to achieve \[in terms of\] your business objectives \[in relation to\] a genuine user need or problem \[while\] having voice be the solution in the middle. **Q5: Within the voice-tech world, what trends have you seen since the beginning of the pandemic?** --------------------------------------------------------------------------------------------------- **Kane**: What is happening or what we're noticing is that places that have stores or physical locations that are no longer open \[but\] can sustain themselves — \[like\] the big companies — are now looking at ways to increase or improve in-store engagement for after COVID. A lot of conversations we've been having are around how to create in-store experiences using voice technology so that when people go back into stores they don't have to touch quite as much stuff and can self-serve. So that's a future situation that is being worked on now, so when stores do reopen, you might find voice experiences in-stores a little bit more. In terms of the stuff that's happening right now, I mentioned IVRs and voice in the phone lines and things like that. One of our partner organizations that specializes in IVR software and the creation of interactive IVR systems are run off their feet at the moment — building things for government organizations and for the travel industry that has been hit really hard. \[It's really\] conversational technologies in general. It doesn't need to be voice specific — it could be a chatbot on a website or a Facebook messenger bot. A really good example is the Whatsapp bot that the world health organization released a few weeks back. And so \[this technology\] is being used to help organizations manage an increase in demand and scale of their customer support and customer service as well as being thought about now \[as a solution\] to help plan for when the stores eventually reopen. **Q6: What resources did you use to level-up your understanding of VUIs** ------------------------------------------------------------------------- **Kane**: I used to watch the Amazon Teams' Twitch streams quite a lot. They came in handy. I thought they were really, really good because they take you through a step-by-step \[process\] of how to build a skill while talking you through absolutely every single part of it. Even though it was very technical — and i'm not a developer — it helped me understand it enough to hold myself in a conversation with either a client or a guest on a podcast. The Amazon blog was pretty good \[as well\]. Google's also published quite a lot of stuff. And then much of \[my early knowledge\] came from actually talking to people, you know? I also think that teaching is the best form of learning, and so I've been a VUI design instructor for a while now with the VUX Academy. Going through that process helps you uncover things that you might not have thought of before running workshops. We've been doing workshops now for almost two years, and with every single one, you learn something different. So it's a mixture of trying to learn from the guests that we have on the podcast, learning from the people who are building this stuff at Amazon and Google but then also through the experience of building skills, through trying things, and exploring tools like Voiceflow and others. ******** > Interested in joining our next live webinar? Sign up for upcoming events [here](https://www.voiceflow.com/events) > .  RECOMMENDED [](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces#) PUBLISHED December 28, 2020 TOPICS IN THIS ARTICLE No items found. [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at\ \ Mark is a Pathways contributor.](https://www.voiceflow.com/contributors/mark-ammendolia) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 4 things you can do with Airtable and conversation design](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design) [](https://www.voiceflow.com/pathways/how-successful-teams-are-tackling-the-hardest-conversation-design-challenges) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How successful teams are tackling the hardest conversation design challenges](https://www.voiceflow.com/pathways/how-successful-teams-are-tackling-the-hardest-conversation-design-challenges) [](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to write an effective conversation designer job description \[+template\]](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Contributors Content [](https://www.voiceflow.com/) Voiceflow community content =========================== Connect to your existing techstack. Discover 300+ apps and software integrations you can build into your Voiceflow agents. Discover top Voiceflow demos, tutorials, comparisons, by the community  [](https://www.voiceflow.com/contributor-content/build-an-ai-agent-in-5-minutes) Customer support · September 15, 2025 I Built an AI Agent in 5 Minutes with Voiceflow (ChatGPT Killer?) ----------------------------------------------------------------- This video reveals how to build powerful AI Agents in just 5 minutes using Voiceflow, no coding required. [View Content →](https://www.voiceflow.com/contributor-content/build-an-ai-agent-in-5-minutes) [](https://www.voiceflow.com/contributor-content/get-started-with-voiceflow-beginner-tutorial-2025) Customer support · September 8, 2025 Get Started with Voiceflow (Beginner Tutorial 2025) --------------------------------------------------- This beginner tutorial (for 2025) provides a comprehensive guide to Voiceflow, the premier platform for quickly building, managing, and deploying AI agents. [View Content →](https://www.voiceflow.com/contributor-content/get-started-with-voiceflow-beginner-tutorial-2025) [](https://www.voiceflow.com/contributor-content/ai-agent-sends-emails) Outbound · September 10, 2025 An AI agent that can send emails with Tools (Voiceflow Beginner Tutorial 2025) ------------------------------------------------------------------------------ Learn how to build an AI agent using Voiceflow that can send emails by leveraging integrated tools. [View Content →](https://www.voiceflow.com/contributor-content/ai-agent-sends-emails)    All featuresAgent stepIntegrationsKnowledge basePublishingMeasuringVoice Clickup Instagram Facebook Webflow Twilio DeepSeek Telegram Perplexity Mixpanel Facebook Messenger WordPress Dropbox Trello Gmail Google Drive Google Sheets Google Docs Google Calendar PayPal PhantomBuster Microsoft Teams Judge.me Omnisend ActiveCampaign MailerLite Shopware Joomla Adobe Commerce Jimdo Drupal Weebly OpenCart Help Scout Zoho Desk Magento Site123 HappyFox Google Analytics LiveAgent PinnacleCart Freshdesk ChatGPT SendGrid Discord Notion Hubspot WhatsApp Airtable Slack Shift4Shop Gorgias Grok Webydo PrestaShop BigCommerce Contentful Wix Squarespace Brevo Pipedrive ConvertKit Klaviyo Stripe Manychat WooCommerce Github LinkedIn MailChimp TidyCal Zapier VAPI OpenAI ElevenLabs X Intercom Ecwid Linear Groq Google Maps Firebase Basecamp IFTTT Box Make Monday.com Asana Zoom Plivo Ghost Bitbucket Bitrix24 Marketo Heap Tableau Kayako Calendly Looker Wrike Qlik Zoho CRM All integrations All topicsCustomer supportLead generationSchedulingProduct recommendationOutbound  Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [\ \ Voiceflow Community\ \ September 15, 2025\ \ #### I Built an AI Agent in 5 Minutes with Voiceflow (ChatGPT Killer?)\ \ This video reveals how to build powerful AI Agents in just 5 minutes using Voiceflow, no coding required.\ \ Knowledge base\ \ Customer support\ \ View content](https://www.voiceflow.com/contributor-content/build-an-ai-agent-in-5-minutes) [\ \ Voiceflow Team\ \ September 8, 2025\ \ #### Get Started with Voiceflow (Beginner Tutorial 2025)\ \ This beginner tutorial (for 2025) provides a comprehensive guide to Voiceflow, the premier platform for quickly building, managing, and deploying AI agents.\ \ Gmail\ \ Integrations\ \ Customer support\ \ View content](https://www.voiceflow.com/contributor-content/get-started-with-voiceflow-beginner-tutorial-2025) [\ \ Voiceflow Team\ \ September 10, 2025\ \ #### An AI agent that can send emails with Tools (Voiceflow Beginner Tutorial 2025)\ \ Learn how to build an AI agent using Voiceflow that can send emails by leveraging integrated tools.\ \ Gmail\ \ Integrations\ \ Outbound\ \ View content](https://www.voiceflow.com/contributor-content/ai-agent-sends-emails) [\ \ Voiceflow Community\ \ September 7, 2025\ \ #### DAY 15: Home Services Clients Book Themselves: My AI Handles Everything\ \ Learn to build a no-code AI voice agent using Voiceflow for home services, enabling clients to book themselves automatically.\ \ Airtable\ \ Integrations\ \ Scheduling\ \ View content](https://www.voiceflow.com/contributor-content/home-services-booking) [\ \ Voiceflow Community\ \ September 10, 2025\ \ #### Auto-Detect Browser Language For Chatbots | Voiceflow Pro Tutorial\ \ This Voiceflow tutorial shows you how to ditch manual language selection buttons by automatically detecting a user's browser language.\ \ Agent step\ \ Customer support\ \ View content](https://www.voiceflow.com/contributor-content/auto-detect-browser-language) [\ \ Voiceflow Community\ \ September 14, 2025\ \ #### How to Create an AI Chatbot for a Coffee Shop (Works for Any Business!)\ \ Learn how to build a no-code AI chatbot for any business.\ \ Knowledge base\ \ Customer support\ \ View content](https://www.voiceflow.com/contributor-content/ai-chatbot-coffee-shop) [\ \ Voiceflow Community\ \ September 18, 2025\ \ #### Day 17: Why Your AI Chatbot Should Give Free Advice (And How to Build It)\ \ Build a no-code AI chatbot that gives free advice to website visitors, generates goodwill, qualifies leads, and automatically books consultations.\ \ Airtable\ \ Integrations\ \ Scheduling\ \ View content](https://www.voiceflow.com/contributor-content/ai-chatbot-free-advice) [\ \ Voiceflow Community\ \ August 1, 2025\ \ #### The Ultimate AI Chatbot for Home Service Businesses\ \ This video is a guide for home service businesses to build powerful AI chatbots that drive real ROI.\ \ Scheduling\ \ View content](https://www.voiceflow.com/contributor-content/the-ultimate-ai-chatbot-for-home-service-businesses) [\ \ Edwin Chen\ \ August 1, 2025\ \ #### The #1 AI Chatbot Threat You're Missing (Costs $1000s)\ \ Poorly engineered conversational AI can cost businesses thousands. This video delves into real-world case studies of these failures, explaining why they happen and providing crucial strategies to avoid similar costly mistakes.\ \ Knowledge base\ \ View content](https://www.voiceflow.com/contributor-content/the-1-ai-chatbot-threat-youre-missing-costs-1000s) [1](https://www.voiceflow.com/contributors-content#) ... [Next](https://www.voiceflow.com/contributors-content?2f126634_page=2) 1 / 3 No results found... Become a Voiceflow affiliate ---------------------------- Connect to your existing techstack. Discover 300+ apps and software integrations you can build into your Voiceflow agents. Earn revenue, access creator resources, and co-market with us as a Voiceflow affiliate. [Join the program →](https://link.voiceflow.com/affiliate)  --- # AI agent builder for agencies [](https://www.voiceflow.com/) Accelerate your AI agency’s growth ================================== Connect to your existing techstack. Discover 300+ apps and software integrations you can build into your Voiceflow agents. Whether you’re looking to get new leads or wow an existing client, Voiceflow empowers agencies to deliver client facing AI agents faster and at scale [Become a partner →](https://link.voiceflow.com/partner_program_apply) [Become a partner →](https://link.voiceflow.com/partner_program_apply)  300,000+ Total users on Voiceflow "Leveraging Voiceflow allowed us to scale our agency to a team of five in just six months by streamlining our entire conversational design and development process. It's the critical framework that lets us build and test high-end, complex agents, enabling us to prototype and successfully deliver for medium to enterprise level clients." .webp) Edwin Chen Co-Founder at Legacy AI "Joining the Voiceflow Certified Expert Program was a turning point for my business. It gave me instant authority in the AI space, helped attract high-value clients, and enabled me to confidently 4x my rates."  Juan Carlos Quintero AI Engineer and Founder at Moonside AI "The agency program gave me access to ready-to-use templates, powerful AI functions, and a content bounty system that actually pays for your contributions. With Voiceflow, I was able to build and deploy AI agents for e-commerce brands in record time—and even earn while sharing what I’ve built."  Jan Rieck-Blankenburg CEO at Ambotiq Become a partner ---------------- Join the growing network of Voiceflow partners — get rewarded for bringing on new clients [Apply now](https://link.voiceflow.com/partner_program_apply) Connect to every client’s tech stack ------------------------------------ Use our API and tools step to seamlessly bring your client workflows to life [Start building for clients](https://link.voiceflow.com/partner_program_apply) [\ \ Zendesk](https://www.voiceflow.com/integrations/zendesk) [\ \ Make](https://www.voiceflow.com/integrations/make) [\ \ Shopify](https://www.voiceflow.com/integrations/shopify) [\ \ Google Sheets](https://www.voiceflow.com/integrations/google-sheets) [\ \ Salesforce](https://www.voiceflow.com/integrations/salesforce) Get noticed as a Certified Expert --------------------------------- Get on a path to become a Certified Expert where you'll get noticed by leads on our experts portal [Become a partner](https://link.voiceflow.com/partner_program_apply) [\ \ Juan Carlos Quintero\ \ Certified Expert\ \ We build bespoke Voiceflow agents for SMBs to boost conversion and save business owners time\ \ ](https://www.voiceflow.com/contributors/juan-carlos-quintero) [\ \ Conner Burton\ \ Certified Expert\ \ I build AI Agent chatbots for business websites. Sometimes rants on Youtube 📺\ \ ](https://www.voiceflow.com/contributors/conner-burton) [.webp)\ \ Edwin Chen\ \ Certified Expert\ \ We help B2C Service Brands Delivers A Premium Customer Experience on Autopilot In 30 Days. Our expertise comes from building both Voiceflow and Custom coded Conversational Agents in sales, customer experience and customer retention use cases. Some of our current clients include one of the world's largest retreat booking sites, a $250M Restaurant Chain in North America and a premium dog grooming company etc.\ \ .png)](https://www.voiceflow.com/contributors/edwin-chen-legacy-ai) We'll help you get started -------------------------- As a partner, you’ll be given access to helpful resources, workshops, and programs to help you grow your agency [Join the program](https://link.voiceflow.com/partner_program_apply) [\ \ ### How to Build a Successful AI Agency: Strategies for Growth\ \ Looking to grow and scale your AI agency into a successful business? Learn the top strategies from real AI agency owners and experts for 2025 and beyond.\ \ Read more →](https://www.voiceflow.com/blog/build-grow-successful-ai-agency) [\ \ ### What is an AI Agency? New Business Models with the Rise of AI\ \ What is an AI agency? Discover how AI agencies help businesses integrate AI, automate workflows, and enhance marketing strategies for growth and efficiency.\ \ Read more →](https://www.voiceflow.com/blog/what-is-an-ai-agency) [\ \ ### How To Start An AI Automation Agency In 7 Days \[Step by Step\]\ \ The McKinsey Global Institute states that by 2055, around half of all the work on planet Earth can be automated with AI, simplifying jobs and enhancing customer experiences.\ \ Read more →](https://www.voiceflow.com/blog/ai-agency) Ready to grow your agency? -------------------------- Connect to your existing techstack. Discover 300+ apps and software integrations you can build into your Voiceflow agents. Join hundreds of agencies building with Voiceflow [Become a partner →](https://link.voiceflow.com/partner_program_apply)  --- # AI Phone Answering System | Voiceflow AI Phone Answering [](https://www.voiceflow.com/) Build and scale AI phone answering for your business ==================================================== Provide instant, reliable call handling around the clock. Your AI receptionist answers, routes, and supports every caller with accuracy and consistency. [Book a demo](https://www.voiceflow.com/demo) Join 500,000 global teams of every size building AI Agents with Voiceflow            Give every caller a smooth start to their conversation ------------------------------------------------------ Deliver immediate, helpful phone support with no missed calls or long waits.   Natural conversations from the first word AI receptionists speak clearly, understand intent, and adapt to interruptions so callers experience a natural, human-like flow.  Knowledge built around your business Use your FAQs, policies, and service information to provide consistent answers to common questions without human involvement.  Ready to take action on every call From collecting details to routing or taking messages, your AI receptionist works with your systems and APIs to complete tasks in real time. Bring intelligence to your phone system --------------------------------------- Connect Voiceflow to your existing telephony, tools, and workflows without rethinking your operations. .avif)  Connected to your phone provider Use Twilio or Vonage to answer unlimited inbound calls any time of day with immediate responses and zero hold times.  Effortless transfer to your team When escalation is needed, your receptionist passes the call and full context so your team picks up exactly where the AI left off.  Clear oversight for operations Track calls, read transcripts, and adjust flows inside Voiceflow to keep quality and accuracy high. Consistent service across every channel --------------------------------------- Deliver a unified receptionist experience on phone, IVR, and messaging.   Build once, deploy everywhere Design a single receptionist experience and publish it across multiple channels to maintain consistency at every touchpoint.  Full control of tone and logic Manage routing paths, escalation rules, and brand voice so every call reflects your standards.  Test, measure, and optimize Use built-in analytics and testing tools to refine performance and ensure each update maintains reliability. Make every call more productive for customers and your team ----------------------------------------------------------- Automate routine call handling while enabling your team to focus on tasks that require human expertise.   More answers, fewer interruptions Your AI receptionist handles frequent questions, gathers caller info, and routes intelligently so your team receives fewer basic inquiries.  Improved caller satisfaction Faster responses and accurate information drive higher CSAT and smoother experiences for every caller.  Scaling without staffing challenges Your receptionist answers every call instantly and adapts to peak volume without needing additional team members. From CRMs to custom systems built to fit your workflow ------------------------------------------------------  Zendesk  Salesforce  HubSpot  Shopify  Genesys  Ujet  NiCE  Pipedrive  Freshdesk  Intercom  RingCentral  Twilio  Airtable  Snowflake  Tenelyx  Vonage See what’s possible with AI agents built for your customers -----------------------------------------------------------  ### 99% of customer questions answered by an AI chatbot built in just two months Turo launched a global, multilingual AI support chatbot in eight weeks using Voiceflow, successfully handling over 23,000 customer conversations with a 99 percent question resolution rate, only a 1 percent fallback to human agents, and an 82 percent positive user satisfaction score. [Read more](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) ### Built in 90 days and replaced a 12 month development roadmap StubHub International launched a full scale AI support agent in just three months using Voiceflow, enabling a non technical support team to build and manage automation without engineers, accelerating deployment across global markets, and surpassing internal targets for automated ticket resolution. [Read more](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) ### 45% of AI coach conversations convert into qualified leads for human advisors Sanlam Studios launched an AI-powered financial coach with Voiceflow to drive financial literacy at scale while generating demand for advisors. The experience answers 100 percent of user questions, engages more than 350 users per day with no paid marketing, and converts 45 percent of conversations directly into qualified leads. [Read more](https://www.voiceflow.com/pathways/how-sanlam-studios-ai-coach-drives-leads-and-financial-literacy) Ready to hear how your AI phone answering could sound? ------------------------------------------------------ [Book a demo](https://www.voiceflow.com/demo)  --- # Automating 70% of customer support for 90+ products: How AI automation transformed Trilogy | Pathways [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) How Trilogy automated 70% of their customer support =================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) When I first heard about Trilogy’s “AI-first” strategy, I thought it might just be trendy—everyone is looking to AI to help streamline their processes. But for [Colin Guilfoyle](https://www.linkedin.com/in/colinguilfoyle/) , VP for Customer Support (CS), and [Ciprian Nastase](https://www.linkedin.com/search/results/all/?fetchDeterministicClustersOnly=true&heroEntityKey=urn%3Ali%3Afsd_profile%3AACoAACSU9ekBlnWnebj0Ml6ihkFe79BAGqVP6D4&keywords=ciprian%20nastase&origin=RICH_QUERY_SUGGESTION&position=0&searchId=1cc951ed-c596-44b7-92d1-2452792a372a&sid=xo.&spellCorrectionEnabled=false) , L3 Technical Support, at Trilogy, finding ways to get the most out of AI is not just encouraged, it’s expected. At Trilogy, they support hundreds of global B2B and B2C companies to streamline and scale their business and customer operations. The technical and CS teams work together to automate customer experiences for 90 individual products and brands, including handling customer support tickets and answering complex customer questions. When they discovered Voiceflow, they were feeling the pressure. “Our challenge was monumental, honestly,” Colin explains, “We had 90 support lines for the products we managed and each required 24/7 support coverage.” The CS team also had several outages using traditional telecom products, especially in the evenings when it was difficult to find representatives to address support tickets. Alongside the human agent-led IVR channel, the team was also using Forethought to launch support chatbots. This out-of-the-box chat solution was not only siloed from other support channels, but also left Ciprian and Colin with limited customization for handling complex actions and no path to scaling agents across the business lines. The Trilogy team was initially looking for an AI solution that would integrate with their Amazon Connect system, analyze queries, provide level 1 FAQs, and route tickets. When they discovered Voiceflow, they got all that and a whole lot more.  Trilogy’s Atlas Core, the mother of AI agents ---------------------------------------------- Customer support shouldn't feel like rocket science. But when your users are rocket scientists, the stakes change. “The users submitting tickets or querying support are smart—they use our client’s software at a high level of proficiency,” Ciprian, L3 Technical Support, explains, “The challenge in offering advanced users AI support is that your agent must be knowledgeable enough to offer high-quality, contextual solutions to their issues.” The Trilogy team also needed an AI platform that would enable them to collaborate effectively. They worked asynchronously 99% of the time and needed Voiceflow to help them deploy an AI-driven support solution they could iterate quickly. Ready to see how this could work for your team? [Request a Demo](https://www.voiceflow.com/demo?p=pathways-trilogy) ### Here’s how they did it * **Created a core AI agent, then used it to scale:** Twenty percent of the products Trilogy manages produce 80% of the support tickets. So Ciprian and the CS team opted to create what they call Atlas Core: an AI agent in Voiceflow that contains the foundational building blocks for several complex agents. Atlas Core is integrated with Trilogy’s help center, a knowledge base, and core set of user support flows and functions. “We use that core to build the other 80% of standalone projects. This helps us scale AI agents quickly and add features to specific products while maintaining a complex core that we’re continually improving,” Ciprian says. This has worked wonders for standardizing and scaling projects quickly across interfaces, such as AI voice agents. * **Triaged tickets with a knowledge base:** The Trilogy technical team also built a standalone ticket triaging and answering agent, called Atlas Ticket. Using the Voiceflow vector database system powered by [Qdrant](https://qdrant.tech/) and Voiceflow's Knowledge Base API, the agent automatically analyzes user queries, recognizes which are L1 queries, and then answers them, drawing from a database of company-specific information, like FAQs, and product details. Integrating the AI agent with a knowledge base deflects simple queries from requiring human intervention and protects the LLM from hallucinating, all while helping users resolve their tasks efficiently. * **Refined their knowledge base to continually improve responses: “**The prompt can only get you so far, from there it's about the data that's being fed into the LLM,” says Ciprian, “You can have the best prompt in the world, but if you have poorly written knowledge base articles, the AI agent will falter.” To make the most of their AI agent, Ciprian, Colin, and the CS team reviewed hundreds of AI customer interactions that were eventually routed to a human and refined their knowledge base to ensure that the LLM could better address user issues and retrieve information in future interactions. * **Built trust with hesitant customers:** Many users were initially hesitant to engage with AI-driven support. To address this concern, the team chose to include the context of the agent’s answers—links to the website, help articles, or company information—to increase user trust in the agent’s accuracy. Eventually most users realized their queries could be addressed by the agent. > _“We no longer worry about prompts when extracting information. We improve our knowledge base—it’s doing the heavy lifting.” - Colin, VP of Customer Success at Trilogy_ The impact of scaling Voiceflow across 90 products --------------------------------------------------- The effect on Trilogy’s support systems was immediate. Within the first week of deploying Voiceflow, they had a 35% AI resolution rate on support tickets. “The CS and technical teams looked at each other and realized Voiceflow was about to make a huge impact on our support capabilities,” Colin recalls. Since they launched AI agents, Trilogy has seen the biggest strides in four major areas. > _“Of the 7,000 tickets in central support, 59% were solved completely by AI.” - Ciprian, L3 Technical Support at Trilogy_  Trilogy's AI resolution rate week-over-week. 1. **AI resolves over half of all support tickets:** After 12 weeks, 59% of tickets are resolved by AI without any human interaction. Today, 70% of tickets are resolved by AI. 2. **Time savings in reduced human intervention:** “We used to spend 40 hours a week resolving tickets, on average 15-30 minutes per ticket. That research and effort was not scalable,” Colin says. Since launching AI support, Trilogy has reduced its support hours by 57%. Now, support representatives have more time to work on queries that require a human touch or complex research. **** 3. **Reduction in tedious research by using a knowledge base:** Gone are the days of support staff needing to do endless Google searches and tedious research. Now that the CS team has built a robust knowledge base of product information, FAQs, and help articles, they can trust that the LLM will pull from trusted sources to resolve queries. “The AI agent has gotten so good that it fills in the blanks—providing more context and a better response than you’d expect,” says Ciprian. **** 4. **Improving data collection to solve complex problems:** Trilogy’s clients count on them to make operations work. So the pressure is high for the CS team to manage the influx of support tickets that come in when a client's product malfunctions. For example, by expanding the support chat flow to lead users through contextual questions—whether that’s information about user behavior, product data, or past issues—the AI agent retrieves the necessary data to resolve complex queries. With this new flow, the AI agent collects more contextual information than ever before. So if a query does require human intervention, support representatives have access to the chat agent’s conversation log, which enables them to resolve issues quickly.  A transcript example of the agent gathering additional details from a complex question. Trilogy recommends collaboration and failing quickly for better bots --------------------------------------------------------------------- The clear visualizations of flows in the Voiceflow platform allowed the technical and CS teams to speak the same language as they designed customer experiences together. Both teams worked together on the Atlas Core to ensure that scaling AI agents to their 90 products would be seamless. “Without Voiceflow, collaborating on this many agents and projects would be near impossible. Building in Voiceflow allowed us to create a visual journey that anyone could understand, which made approvals and deployment much faster,” Ciprian explained. Ciprian was also adamant that the Trilogy technical team learned valuable lessons when they moved quickly and learned directly from the user. “I learned more in 24 hours of a bot being live than in days of tweaking it to get things ‘perfect’. You can imagine 20% of what customers may throw at your agent, but only after you release into the wild will you discover what the user actually needs,” Ciprian says. Colin adds, “Failing is inevitable. That’s why monitoring is so important—don’t leave your AI agents unattended. But those failures teach our teams where to build next, what to A/B test, and where our easy wins are. Fortunately, due to the guardrails we have in place, none of our failures have resulted in our agents going viral.” What’s next for Trilogy? ------------------------- Trilogy plans to use Voiceflow to continue to standardize and expand its AI capabilities. They plan to roll out Voiceflow for internal projects—for finance, marketing, and sales. For example, if Trilogy employees need to fill out an invoice, instead of searching for a form, they’ll share the information with the internal AI agent and it'll generate the required documents. Also, the technical team is expecting another large jump in what their L2 bot will be able to do. Both L1 and L2 agents will be connected, share data, and source information from several integrations and databases to make them readily available to serve users better. Lastly**,** as Trilogy expands to serve more business’ operational needs, they’ll continue to standardize and scale AI agents across their new customers. Ensuring all new businesses onboard into their AI systems right away. As the technical and CS team’s collaboration continues to go strong, I have no doubt they’ll continue to excel at their AI-first vision. #### Ready to transform your support channels with AI? [Book a demo](https://www.voiceflow.com/demo) with our team. Trilogy’s Atlas Core, the mother of AI agents ---------------------------------------------- Customer support shouldn't feel like rocket science. But when your users are rocket scientists, the stakes change. “The users submitting tickets or querying support are smart—they use our client’s software at a high level of proficiency,” Ciprian, L3 Technical Support, explains, “The challenge in offering advanced users AI support is that your agent must be knowledgeable enough to offer high-quality, contextual solutions to their issues.” The Trilogy team also needed an AI platform that would enable them to collaborate effectively. They worked asynchronously 99% of the time and needed Voiceflow to help them deploy an AI-driven support solution they could iterate quickly. Ready to see how this could work for your team? [Request a Demo](https://www.voiceflow.com/demo?p=pathways-trilogy) ### Here’s how they did it * **Created a core AI agent, then used it to scale:** Twenty percent of the products Trilogy manages produce 80% of the support tickets. So Ciprian and the CS team opted to create what they call Atlas Core: an AI agent in Voiceflow that contains the foundational building blocks for several complex agents. Atlas Core is integrated with Trilogy’s help center, a knowledge base, and core set of user support flows and functions. “We use that core to build the other 80% of standalone projects. This helps us scale AI agents quickly and add features to specific products while maintaining a complex core that we’re continually improving,” Ciprian says. This has worked wonders for standardizing and scaling projects quickly across interfaces, such as AI voice agents. * **Triaged tickets with a knowledge base:** The Trilogy technical team also built a standalone ticket triaging and answering agent, called Atlas Ticket. Using the Voiceflow vector database system powered by [Qdrant](https://qdrant.tech/) and Voiceflow's Knowledge Base API, the agent automatically analyzes user queries, recognizes which are L1 queries, and then answers them, drawing from a database of company-specific information, like FAQs, and product details. Integrating the AI agent with a knowledge base deflects simple queries from requiring human intervention and protects the LLM from hallucinating, all while helping users resolve their tasks efficiently. * **Refined their knowledge base to continually improve responses: “**The prompt can only get you so far, from there it's about the data that's being fed into the LLM,” says Ciprian, “You can have the best prompt in the world, but if you have poorly written knowledge base articles, the AI agent will falter.” To make the most of their AI agent, Ciprian, Colin, and the CS team reviewed hundreds of AI customer interactions that were eventually routed to a human and refined their knowledge base to ensure that the LLM could better address user issues and retrieve information in future interactions. * **Built trust with hesitant customers:** Many users were initially hesitant to engage with AI-driven support. To address this concern, the team chose to include the context of the agent’s answers—links to the website, help articles, or company information—to increase user trust in the agent’s accuracy. Eventually most users realized their queries could be addressed by the agent. > _“We no longer worry about prompts when extracting information. We improve our knowledge base—it’s doing the heavy lifting.” - Colin, VP of Customer Success at Trilogy_ The impact of scaling Voiceflow across 90 products --------------------------------------------------- The effect on Trilogy’s support systems was immediate. Within the first week of deploying Voiceflow, they had a 35% AI resolution rate on support tickets. “The CS and technical teams looked at each other and realized Voiceflow was about to make a huge impact on our support capabilities,” Colin recalls. Since they launched AI agents, Trilogy has seen the biggest strides in four major areas. > _“Of the 7,000 tickets in central support, 59% were solved completely by AI.” - Ciprian, L3 Technical Support at Trilogy_  Trilogy's AI resolution rate week-over-week. 1. **AI resolves over half of all support tickets:** After 12 weeks, 59% of tickets are resolved by AI without any human interaction. Today, 70% of tickets are resolved by AI. 2. **Time savings in reduced human intervention:** “We used to spend 40 hours a week resolving tickets, on average 15-30 minutes per ticket. That research and effort was not scalable,” Colin says. Since launching AI support, Trilogy has reduced its support hours by 57%. Now, support representatives have more time to work on queries that require a human touch or complex research. **** 3. **Reduction in tedious research by using a knowledge base:** Gone are the days of support staff needing to do endless Google searches and tedious research. Now that the CS team has built a robust knowledge base of product information, FAQs, and help articles, they can trust that the LLM will pull from trusted sources to resolve queries. “The AI agent has gotten so good that it fills in the blanks—providing more context and a better response than you’d expect,” says Ciprian. **** 4. **Improving data collection to solve complex problems:** Trilogy’s clients count on them to make operations work. So the pressure is high for the CS team to manage the influx of support tickets that come in when a client's product malfunctions. For example, by expanding the support chat flow to lead users through contextual questions—whether that’s information about user behavior, product data, or past issues—the AI agent retrieves the necessary data to resolve complex queries. With this new flow, the AI agent collects more contextual information than ever before. So if a query does require human intervention, support representatives have access to the chat agent’s conversation log, which enables them to resolve issues quickly.  A transcript example of the agent gathering additional details from a complex question. Trilogy recommends collaboration and failing quickly for better bots --------------------------------------------------------------------- The clear visualizations of flows in the Voiceflow platform allowed the technical and CS teams to speak the same language as they designed customer experiences together. Both teams worked together on the Atlas Core to ensure that scaling AI agents to their 90 products would be seamless. “Without Voiceflow, collaborating on this many agents and projects would be near impossible. Building in Voiceflow allowed us to create a visual journey that anyone could understand, which made approvals and deployment much faster,” Ciprian explained. Ciprian was also adamant that the Trilogy technical team learned valuable lessons when they moved quickly and learned directly from the user. “I learned more in 24 hours of a bot being live than in days of tweaking it to get things ‘perfect’. You can imagine 20% of what customers may throw at your agent, but only after you release into the wild will you discover what the user actually needs,” Ciprian says. Colin adds, “Failing is inevitable. That’s why monitoring is so important—don’t leave your AI agents unattended. But those failures teach our teams where to build next, what to A/B test, and where our easy wins are. Fortunately, due to the guardrails we have in place, none of our failures have resulted in our agents going viral.” What’s next for Trilogy? ------------------------- Trilogy plans to use Voiceflow to continue to standardize and expand its AI capabilities. They plan to roll out Voiceflow for internal projects—for finance, marketing, and sales. For example, if Trilogy employees need to fill out an invoice, instead of searching for a form, they’ll share the information with the internal AI agent and it'll generate the required documents. Also, the technical team is expecting another large jump in what their L2 bot will be able to do. Both L1 and L2 agents will be connected, share data, and source information from several integrations and databases to make them readily available to serve users better. Lastly**,** as Trilogy expands to serve more business’ operational needs, they’ll continue to standardize and scale AI agents across their new customers. Ensuring all new businesses onboard into their AI systems right away. As the technical and CS team’s collaboration continues to go strong, I have no doubt they’ll continue to excel at their AI-first vision. #### Ready to transform your support channels with AI? [Book a demo](https://www.voiceflow.com/demo) with our team.  RECOMMENDED [\ \ ### Introducing Tico: The AI customer support agent resolving 97% of tickets](https://www.voiceflow.com/pathways/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) PUBLISHED February 29, 2024 TOPICS IN THIS ARTICLE Exceptional work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### How Roam saved 30 hours a week in customer support hours](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours) [](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) [](https://www.voiceflow.com/pathways/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### Introducing Tico: The AI customer support agent resolving 97% of tickets](https://www.voiceflow.com/pathways/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Building a knowledge base with OpenAI, LangChain, OpenSearch, and Unstructured | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Building a knowledge base with OpenAI, LangChain, OpenSearch, and Unstructured ============================================================================== [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) We’ve designed this project to help [Voiceflow](https://www.voiceflow.com/) users build a knowledge base using custom APIs. The project utilizes [Open AI](https://www.openai.com/) , [Langchain](https://langchain.com/) , [Redis](https://redis.io/) , [OpenSearch](https://opensearch.org/) , and [Unstructured](https://www.unstructured.io/) to fetch content from various sources such as **URLs**, **sitemaps**, **text**, **PDFs**, **PowerPoints**, **Notion docs (markdown)** and even **images** (OCR). These sources of information are then turned into embeddings/vectors and saved in a local OpenSearch database. This knowledge base can then be used to generate context and answer questions. And the best part? Because it’s an API, you can use it within your Voiceflow Assistant with the help of the **API Step**. Before you dive in, watch the quick overview below.  Installation ------------ You’ll need Node.js 18+ to run this code. Download it [here.](https://nodejs.org/en/download) You'll also need to have [Docker Compose](https://docker.com/) installed. To get started, copy the \`.env\` file and set up required environment variables: To create the containers, install the required dependencies, and launch the server, run: This will create the following containers: * Redis (cache) * Unstructured (handles images, PPT, text, markdown) * OpenSearch (search engine) * OpenSearch-dashboards (search engine dashboard) _OpenSearch dashboard can be accessed at http://localhost:5601_ API documentation ----------------- There are several API endpoints available for various tasks: Add content to OpenSearch: \`POST **/api/add**\` Get a response using a live webpage as context: \`POST **/api/live**\` Get a response using the vector store: \`POST **/api/question**\` Clear Redis cache: \`GET **/api/clearcache**\` Delete a collection: \`DELETE **/api/collection**\` Find more detailed API documentation in the [README.md](https://github.com/voiceflow/demos-and-examples/tree/main/langchain-local-knowledge-base) file on our [repo](https://github.com/voiceflow/demos-and-examples/tree/main/langchain-local-knowledge-base) . Using live data --------------- You can also use the \`**/api/live**\` endpoint to get a response using a live webpage as context without vectorizing the content. Using the knowledge base ------------------------ Once you've added content to OpenSearch, you can use the \`**/api/question**\` endpoint to get answers based on your knowledge base. **Now what?** ------------- Now you can easily set up and use your knowledge base to answer questions and provide valuable information for your users using the API Step in your [Voiceflow](https://www.voiceflow.com/) Assistants. More projects ------------- * [Harness the power of OpenAI's Whisper model for ASR with Voiceflow](https://www.voiceflow.com/blog/openai-whisper-model-asr) * [How to create an Alexa skill with GPT-4 and Voiceflow](https://www.voiceflow.com/blog/alexa-skill-chat-gpt-4) * [Prompt chaining for conversational AI](https://www.voiceflow.com/blog/prompt-chaining-conversational-ai) Installation ------------ You’ll need Node.js 18+ to run this code. Download it [here.](https://nodejs.org/en/download) You'll also need to have [Docker Compose](https://docker.com/) installed. To get started, copy the \`.env\` file and set up required environment variables: To create the containers, install the required dependencies, and launch the server, run: This will create the following containers: * Redis (cache) * Unstructured (handles images, PPT, text, markdown) * OpenSearch (search engine) * OpenSearch-dashboards (search engine dashboard) _OpenSearch dashboard can be accessed at http://localhost:5601_ API documentation ----------------- There are several API endpoints available for various tasks: Add content to OpenSearch: \`POST **/api/add**\` Get a response using a live webpage as context: \`POST **/api/live**\` Get a response using the vector store: \`POST **/api/question**\` Clear Redis cache: \`GET **/api/clearcache**\` Delete a collection: \`DELETE **/api/collection**\` Find more detailed API documentation in the [README.md](https://github.com/voiceflow/demos-and-examples/tree/main/langchain-local-knowledge-base) file on our [repo](https://github.com/voiceflow/demos-and-examples/tree/main/langchain-local-knowledge-base) . Using live data --------------- You can also use the \`**/api/live**\` endpoint to get a response using a live webpage as context without vectorizing the content. Using the knowledge base ------------------------ Once you've added content to OpenSearch, you can use the \`**/api/question**\` endpoint to get answers based on your knowledge base. **Now what?** ------------- Now you can easily set up and use your knowledge base to answer questions and provide valuable information for your users using the API Step in your [Voiceflow](https://www.voiceflow.com/) Assistants. More projects ------------- * [Harness the power of OpenAI's Whisper model for ASR with Voiceflow](https://www.voiceflow.com/blog/openai-whisper-model-asr) * [How to create an Alexa skill with GPT-4 and Voiceflow](https://www.voiceflow.com/blog/alexa-skill-chat-gpt-4) * [Prompt chaining for conversational AI](https://www.voiceflow.com/blog/prompt-chaining-conversational-ai)  RECOMMENDED [\ \ ### Harness the power of OpenAI’s Whisper model for ASR with Voiceflow](https://www.voiceflow.com/pathways/openai-whisper-model-asr) PUBLISHED May 1, 2023 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Harness the power of OpenAI’s Whisper model for ASR with Voiceflow](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Want to try Claude in your Voiceflow assistant? Here's how](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [](https://www.voiceflow.com/pathways/how-to-create-a-simple-telegram-bot-with-voiceflow) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### How to create a simple Telegram bot with Voiceflow](https://www.voiceflow.com/pathways/how-to-create-a-simple-telegram-bot-with-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/building-knowledge-base-openai#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/building-knowledge-base-openai#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Want to try Claude in your Voiceflow assistant? Here's how | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Want to try Claude in your Voiceflow assistant? Here's how ========================================================== [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) Anthropic just launched early access to [Claude](https://www.anthropic.com/index/introducing-claude) , its next-generation AI assistant. Claude is designed to be helpful, honest, and harmless—and it can perform a multitude of tasks, from summarization to coding. Available in two versions, Claude and Claude Instant, this AI assistant has vast potential applications across various industries. If you're looking to harness the power of AI and improve your offerings, drop everything and [get yourself Claude LLM early access](https://www.anthropic.com/earlyaccess) . And if you're wondering how to try out Claude LLM in your Voiceflow assistant, this tutorial will walk you through all the steps.  1\. Get early access to Claude LLM ---------------------------------- First things first: [sign up for Claude LLM early access](https://www.anthropic.com/earlyaccess) . You’ll need to fill out your details and tell the team about your project. Once your request is approved, you’ll gain access to the Anthropic Developer Console, from which you can generate an API key. 2\. Generate an API Key ----------------------- From the Anthropic Developer Console, go to the API section and create a new key. Save that key somewhere—you'll need it for integration into our Voiceflow Assistant.  3\. Integrate Claude into your Voiceflow assistant -------------------------------------------------- To create an integration for Claude into your Voiceflow assistant, follow these steps: 1\. [Log in to your Voiceflow account](https://creator.voiceflow.com/login) and choose the project you want to integrate Claude with. In this demo, I’m using a web chat assistant. 2\. Reach for the API step in your Voiceflow assistant.  3\. Enter the API endpoint URL for Claude. You can find Claude API documentation from the [console](https://console.anthropic.com/docs) . 4\. Set the API method to "POST" and provide the necessary headers, including the one for the API key.  5\. In the "Body" section, include the required JSON payload. You can make it static or use variables to set the model’s settings and the prompt. Our assistant demo ------------------ For detailed steps on how to configure the API step and/or run and test your assistant after Claude integration, watch the video below. Want to test the assistant from the video? [Clone it from this project](https://creator.voiceflow.com/dashboard?import=645cbd221fe1d900079e2b63) .  Feel free to experiment and test Claude while prototyping your Voiceflow assistant. However, keep in mind that commercial deployment of Claude requires a services agreement with Anthropic. Evaluation API keys are provided under non-commercial terms and must not be used in production systems. For commercial deployment, partnership terms, and pricing discussions, reach out to Anthropic at [sales@anthropic.com](mailto:sales@anthropic.com) . Enjoy exploring the powerful features of Claude LLM in your Voiceflow Assistant. Happy prototyping! More on large language models (LLMs) ------------------------------------ * [Why the dominant discourse surrounding LLMs needs to change](https://www.voiceflow.com/blog/dominant-discourse-llms) 1\. Get early access to Claude LLM ---------------------------------- First things first: [sign up for Claude LLM early access](https://www.anthropic.com/earlyaccess) . You’ll need to fill out your details and tell the team about your project. Once your request is approved, you’ll gain access to the Anthropic Developer Console, from which you can generate an API key. 2\. Generate an API Key ----------------------- From the Anthropic Developer Console, go to the API section and create a new key. Save that key somewhere—you'll need it for integration into our Voiceflow Assistant.  3\. Integrate Claude into your Voiceflow assistant -------------------------------------------------- To create an integration for Claude into your Voiceflow assistant, follow these steps: 1\. [Log in to your Voiceflow account](https://creator.voiceflow.com/login) and choose the project you want to integrate Claude with. In this demo, I’m using a web chat assistant. 2\. Reach for the API step in your Voiceflow assistant.  3\. Enter the API endpoint URL for Claude. You can find Claude API documentation from the [console](https://console.anthropic.com/docs) . 4\. Set the API method to "POST" and provide the necessary headers, including the one for the API key.  5\. In the "Body" section, include the required JSON payload. You can make it static or use variables to set the model’s settings and the prompt. Our assistant demo ------------------ For detailed steps on how to configure the API step and/or run and test your assistant after Claude integration, watch the video below. Want to test the assistant from the video? [Clone it from this project](https://creator.voiceflow.com/dashboard?import=645cbd221fe1d900079e2b63) .  Feel free to experiment and test Claude while prototyping your Voiceflow assistant. However, keep in mind that commercial deployment of Claude requires a services agreement with Anthropic. Evaluation API keys are provided under non-commercial terms and must not be used in production systems. For commercial deployment, partnership terms, and pricing discussions, reach out to Anthropic at [sales@anthropic.com](mailto:sales@anthropic.com) . Enjoy exploring the powerful features of Claude LLM in your Voiceflow Assistant. Happy prototyping! More on large language models (LLMs) ------------------------------------ * [Why the dominant discourse surrounding LLMs needs to change](https://www.voiceflow.com/blog/dominant-discourse-llms)  RECOMMENDED [\ \ ### Introducing Voiceflow's GPT-powered AI Builder](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) PUBLISHED May 12, 2023 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Nicholas Laing\ \ at](https://www.voiceflow.com/contributors/nicholas-laing) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/voiceflow-api-export-endpoint) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### How to use Voiceflow’s API export endpoint to fetch your Voiceflow assistant files effortlessly](https://www.voiceflow.com/pathways/voiceflow-api-export-endpoint) [](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Want to try Claude in your Voiceflow assistant? Here's how](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Harness the power of OpenAI’s Whisper model for ASR with Voiceflow](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/claude-llm-ai-assistant#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/claude-llm-ai-assistant#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Architecting the Future of AI Agents: 5 Flexible Conversation Frameworks You Need | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Architecting the future of AI agents: 5 flexible conversation frameworks you need ================================================================================= [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) Most folks familiar with architecture can look at a building designed by Frank Lloyd Wright and recognize it immediately. His design philosophy became what we now call organic architecture—weaving the man-made with the natural world through the design and construction of harmonious buildings. In many ways, conversational AI is going through a similar architectural revolution. Traditional turn-by-turn conversation design is quickly becoming outdated. As language models become more advanced, we need a new approach—one that empowers designers and developers to build agents that handle complex, dynamic interactions with flexibility and context awareness. Organic conversation. I invite you to think of your agent as the house you’re designing with an imaginative architect at the center of the process—you. To build that house, you need five key frameworks that govern areas like context management, integration capabilities, interaction models, and data handling. This new evolution of agents is a future we can build towards now. But in order to reach it, conversation designers and developers must work together closely to build the parameters of how we work with LLMs, agents, and data to build flexible and delightful customer experiences. My goal in this article is to explain the five frameworks you’ll need to continue to see your AI agents evolve—the overarching rules every agent needs to be effective. By approaching the construction of agents as an architect might, with these frameworks to guide structural integrity, we can create agents that do much more, and as a result, save valuable money, effort, and time. Let’s explore how powerful agents are designed based on these frameworks.  Global framework—the architect’s blueprints ------------------------------------------- Like a blueprint, an agent’s global framework contains the overarching rules that an agent will work within. A global framework sets the architect’s vision—how the agent will maintain context across different states and sessions, handle errors, adhere to privacy, and route users correctly. Further, the states we expect an agent to navigate often change, and the purpose of the agent at any given time isn’t fixed. A carpenter cuts the wooden beams for the roof and walls differently—even in one home, there are numerous needs to address. In the same way, you must construct your AI agent to navigate through multiple states in a single session.While it won’t be limited to the following, your global framework will contain areas like: * **User routing logic:** How are you navigating the user through different states based on their queries and context? * **State management:** How do you manage the logic and flow within a specific state? What about transitions to other states? * **Error handling and recovery:** When the user provides unexpected input or the system encounters an error, what are your strategies for the agent to course-correct? * **Global context management:** How are you maintaining user context across different states and sessions? How are you personalizing those interactions? * **Security and compliance:** How are you ensuring all interactions adhere to privacy laws and data protection standards? An agent that has a thoughtfully designed global framework might manage an interaction this way: **_User_**_: I’m planning a trip to Tokyo_ **_Agent_**_: Exciting! When would you like to go?_ **_User_**_: That's a good question, what's the weather like in August?_ **_Agent_**_: In August, Tokyo experiences warm and humid weather. The average temperature ranges from around 25°C to 31°C (77°F to 88°F). Would you like to book then, or would you prefer something cooler?_ **_User_**_: August sounds great!_ **_Agent_**_: Perfect. I've found a window seat on VF Air departing on the first of August. How does that sound?_ __Thanks to a smart designer, the routing logic guides the agent to recognize that the user is asking about booking a trip and places them in that conversation state. The user is thinking about going in August, but instead of saying that explicitly, they ask, ”What’s the weather like in August.” The routing logic enables the agent to pull this information, while the error logic works on getting the user back on track. Then context management kicks into gear, pulling information from prior trips to offer their preferred seat type (window) along with their preferred airline (VF Air). Your global framework is your agent's orchestration layer. You set the parameters for your agent to understand when to engage in a specific conversation state, when to call for a specific back-end integration, and so on. Because of the complexity of planning and building agents like this, your global framework requires robust back-end integrations, data encryption for security, and the ability to understand and maintain context over time and across sessions. The result is setting a foundation that has the potential to be an architectural marvel. Local framework—the plan of each room ----------------------------------------- The local framework of an agent provides relevant, context-aware responses and interactions within defined conversation states or skills. Without localized strategies, agents would struggle to adapt to the requirements and flow of different tasks like booking travel, providing tech support instructions, or processing transactions. This framework must manage how the agent interacts in different states and what information the agent needs within each state. Only then can they work through complex tasks like troubleshooting or action requests like checking someone's balance. For instance, a bedroom might only require windows, power points, and cupboards, while a bathroom will need plumbing, waterproofing, tiling, etc. to function. The local framework is the nitty-gritty of each space and what they need to come together.When designing an agent’s local framework, here are a few questions to explore: * **Execution:** What information do we need to execute the action? How should we execute an action? What should we tell a user once the action has been executed? * **Content retrieval and processing:** What are your specific methods for fetching and processing information relevant to the current state? * **User interaction models:** How do you define how the agent interacts with the user within each state? How do their question-asking strategies or response formulations change in different states? * **Contextual adaptation:** How can you tailor the interaction based on the specific context or user data? To build an agent that handles question and answer pairs, let’s explore an example of an agent supporting a user with the APN setting on their iPhone. **_Agent_**_: Hi, welcome to VF Telcom. How can I help?_ **_User_**_: How do I update my APN settings?_ **_\[LLM added to address query: ‘How do I update the APN settings on my device?’\]_** __Starting with the utterances above, we have the agent write a question that’s optimized for retrieval. Then, the LLM is added to the conversation to make the question more specific to address the query. **_\[LLM: Clarification Needed On: 'Ask the user to specify their device make and model to betteraddress their query.'\]_** __Next, we instruct the LLM to look at both the information it has retrieved, along with the question that was presented to a [knowledge base](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) , and ensure they get clarifying information from the user. **_Agent_**_: Certainly, to assist with that, could you please let me know the device you’re using? That way, I can give you the most accurate information._ **_User_**_: an iPhone 15._ **_\[LLM question generated (using entire conversation history): ‘How do I update the APN settings on an iPhone 15?’_**\ \ **_LLM clarification check: Decision: '1', Clarification Needed On: 'Ask the user to specify the carrier they are using or if they need general instructions for updating APN settings on an iPhone 15 to better address their query.'\]_** **_Agent_**_: And could you confirm if you are already on the VF network?_ **_User_**_: Yes, I’m already on the network._ **_\[LLM question generated: ‘How can I update the APN settings on my iPhone 15 for VF network?’\]_** __Here you can see that the LLM has determined that the user needs to specify their device and confirm their carrier in order to give them the most helpful answer to their query. The user responds with, “iPhone 15,” and is asked for further information so that it can generate the final question for the knowledge base. **_\[LLM check: Decision ‘1’, the output has no inconsistency or inaccuracies.\]_** __Now that we have all the information, the response is created. But before it’s presented, the LLM checks that there are no inconsistencies or hallucinations, by doing a cross-check of the response and the information that was retrieved. **_Agent_**_: To update the APN settings on your iPhone 15 for VF Network, you can follow these steps:_ _1\. Go to Settings on your iPhone._ _2\. Tap on Cellular or Mobile Data._ _3\. Select Cellular Data Network._ _4\. Enter the following APN settings:_ * _APN: fast.VF-mobile.com_ * _Username: JohnSmith_ * _Password: 1234_ _5\. Scroll down and enter the same APN settings under the MMS section._ _6\. Press the Home key to save the APN settings and exit to the main screen._ _7\. Restart your device.For more detailed instructions, you can visit the BYOD Mobile data & APN settings page._ **_Agent_**_: Would you like more information on how to reset your network settings if updating the APN does not resolve your connectivity issues?_ __Finally, the answer is displayed, and another prompt is used to display a follow-up question to the user. These local frameworks give the LLM the guidelines to create questions that have been optimized for retrieval, self-check its own work, and ask follow-up questions. By building an intuitive local framework that handles question-answer pairs, you can go from managing hundreds of FAQs to managing the knowledge source that the overall conversation architecture draws from. Local components need to be flexible to adapt to user needs while being responsive to input—just remember that this approach requires detailed design and testing. Integration framework—the utilities behind the walls ----------------------------------------------------- Like the wiring and plumbing in a house, the stuff behind the drywall can be some of the most important. Similarly, the integrations we build between our agents and our systems can make or break user experience. Your integration framework is about designing what external services your agent has access to, what they're used for, and under which circumstances they should access them. By connecting your agent with integrations, it can automatically and flexibly complete tasks. These components can drastically improve the overall user experience that your agent delivers if they’re implemented non-deterministically. When defining how integrations should work in your overall conversation architecture, here are a few things to ponder: * **API management:** How does the agent interact with APIs? How do you manage those integrations? * **External data access:** How should your agent access and store data from external sources? How should your agent format and share that data with users? * **Service orchestration:** What is your approach for coordinating complex interactions across multiple back-end services? Integrations aren't new in conversational AI. Where this approach differs is that you’re designing integration rules without a deterministic flow to execute them. It's about giving the global (or local) framework all the information it needs to determine which integration would help it action/answer the user's question. For example, when I ask a banking agent, “I want to check my balance," I usually get pushed down a flow that collects information until it calls an API that gives me my total balance (and it’s never what I want it to be). In a future, where we design and construct agents with thoughtful frameworks to guide them, we let the agent decide when they need to use specific integrations. Although this approach to integrations requires secure, efficient, and scalable mechanisms—often involving middleware or service buses—it means there is no singular happy path that the agent is forcing a user down. As a result, the opportunities an agent has to serve multiple needs and reliably help more users go up exponentially. Analytics and data framework—the maintenance and improvements ------------------------------------------------------------- After the home is completely constructed, it’s time for the final inspection. Did we comply with all construction codes? Where should we make more improvements or correct our mistakes? In the same way, a robust analytics and data framework allows you to understand your agent’s performance and manage data effectively. It will define how we pass information to LLMs and derive insights from our interactions. When collecting and managing data and analyzing the success of your agent, here are some questions you should answer: * **Data collection and storage:** What are your methods for securely collecting and storing user data? * **Feedback loops:** How are you incorporating user feedback into your iterative improvements of the agents? How do you analyze interaction outcomes for refinement? * **Implementation considerations:** How are you adhering to data protection regulations? How are you improving your agent in such a way as to enable future scalability and flexibility? How does this framework inform a conversation? Well, during an ecommerce conversation, the AI agent collects data like the user's purchasing history, stated preferences ("I like eco-friendly products"), sentiment signals (frustrated tone), cart abandonment point, and final sale/no sale. Analytics frameworks would process this data, combining it with thousands of other interaction logs, which may reveal that eco-conscious buyers frequently abandon their cart due to a lack of green certifications on product pages. We can use this to update interaction components dynamically—changing the agent's persona, proactively suggesting sustainable options, linking green certifications, and using a more consultative interaction model for this user segment. There are endlessly creative ways to use real-time analytics to update how an agent is responding to users. If you’re not securely collecting data gathered during interactions and analyzing it effectively, you’re not likely to be improving your agents based on what your users actually need. And the gorgeous home you designed, constructed, and inspected will eventually fall to ruin from lack of upkeep. Interaction framework—the interior design ----------------------------------------- When you enter a room, you can tell a lot about the person living there—are they a minimalist or a trinket-lover? Do they exclusively use warm-hued lamps in lieu of the dreaded big light? A single glance into a well-decorated room can tell a story without words. In the case of your digital agent, their interaction framework tells users a story about the vibe of your company and the experience they’re about to receive. Ideally, a great agent is able to capture the essence of your brand in communication style, tone, and techniques. And all that is informed by how you instruct the model to interact with users. So, when designing how your agent interacts with users, here are a few things to consider: * **Style and grammar:** What are your rules on style, tone, and formatting? How do you format hyperlinks or lists? How does it keep the conversation flowing naturally? * **Language and tone:** What are your guidelines for politeness, formality, and brand voice? * **Conversational strategies:** How is your agent programmed to ask for clarification? What do conversational turns look and sound like? How does it give proactive suggestions? I suggest creating and maintaining a style guide and tone-of-voice document to keep your agent’s interaction on brand. This framework requires deep linguistic modeling and an understanding of conversational dynamics, but it also incorporates user feedback and sentiment analysis as you learn more about your agent and your company’s unique needs. (Stay tuned to Pathways for our take on style guides for your agents.) All the makings of an architectural marvel ------------------------------------------ Architect and planner David Pearson once proposed a list of rules for the design of organic architecture—a framework, you might say—and one of his cardinal rules reads, “Let the design follow the flows and be flexible and adaptable.” Like architecture, conversational AI is both a science and an art. A collection of rules, guidelines, and frameworks _and_ the creative mission of many designers, developers, and thinkers. By designing each component of our conversation frameworks thoughtfully—global, local, integrations, interactions, and analytics—you’ll be able to build agents that complete tasks, problem-solve, and delight users. You’ll be able to use these five frameworks as the building blocks to serve a larger conversational AI architecture. Your strategic design choices can make your agents strong, functional, and flexible. Or fragile, ineffective, and rigid. But that’s all up to the architect. Global framework—the architect’s blueprints ------------------------------------------- Like a blueprint, an agent’s global framework contains the overarching rules that an agent will work within. A global framework sets the architect’s vision—how the agent will maintain context across different states and sessions, handle errors, adhere to privacy, and route users correctly. Further, the states we expect an agent to navigate often change, and the purpose of the agent at any given time isn’t fixed. A carpenter cuts the wooden beams for the roof and walls differently—even in one home, there are numerous needs to address. In the same way, you must construct your AI agent to navigate through multiple states in a single session.While it won’t be limited to the following, your global framework will contain areas like: * **User routing logic:** How are you navigating the user through different states based on their queries and context? * **State management:** How do you manage the logic and flow within a specific state? What about transitions to other states? * **Error handling and recovery:** When the user provides unexpected input or the system encounters an error, what are your strategies for the agent to course-correct? * **Global context management:** How are you maintaining user context across different states and sessions? How are you personalizing those interactions? * **Security and compliance:** How are you ensuring all interactions adhere to privacy laws and data protection standards? An agent that has a thoughtfully designed global framework might manage an interaction this way: **_User_**_: I’m planning a trip to Tokyo_ **_Agent_**_: Exciting! When would you like to go?_ **_User_**_: That's a good question, what's the weather like in August?_ **_Agent_**_: In August, Tokyo experiences warm and humid weather. The average temperature ranges from around 25°C to 31°C (77°F to 88°F). Would you like to book then, or would you prefer something cooler?_ **_User_**_: August sounds great!_ **_Agent_**_: Perfect. I've found a window seat on VF Air departing on the first of August. How does that sound?_ __Thanks to a smart designer, the routing logic guides the agent to recognize that the user is asking about booking a trip and places them in that conversation state. The user is thinking about going in August, but instead of saying that explicitly, they ask, ”What’s the weather like in August.” The routing logic enables the agent to pull this information, while the error logic works on getting the user back on track. Then context management kicks into gear, pulling information from prior trips to offer their preferred seat type (window) along with their preferred airline (VF Air). Your global framework is your agent's orchestration layer. You set the parameters for your agent to understand when to engage in a specific conversation state, when to call for a specific back-end integration, and so on. Because of the complexity of planning and building agents like this, your global framework requires robust back-end integrations, data encryption for security, and the ability to understand and maintain context over time and across sessions. The result is setting a foundation that has the potential to be an architectural marvel. Local framework—the plan of each room ----------------------------------------- The local framework of an agent provides relevant, context-aware responses and interactions within defined conversation states or skills. Without localized strategies, agents would struggle to adapt to the requirements and flow of different tasks like booking travel, providing tech support instructions, or processing transactions. This framework must manage how the agent interacts in different states and what information the agent needs within each state. Only then can they work through complex tasks like troubleshooting or action requests like checking someone's balance. For instance, a bedroom might only require windows, power points, and cupboards, while a bathroom will need plumbing, waterproofing, tiling, etc. to function. The local framework is the nitty-gritty of each space and what they need to come together.When designing an agent’s local framework, here are a few questions to explore: * **Execution:** What information do we need to execute the action? How should we execute an action? What should we tell a user once the action has been executed? * **Content retrieval and processing:** What are your specific methods for fetching and processing information relevant to the current state? * **User interaction models:** How do you define how the agent interacts with the user within each state? How do their question-asking strategies or response formulations change in different states? * **Contextual adaptation:** How can you tailor the interaction based on the specific context or user data? To build an agent that handles question and answer pairs, let’s explore an example of an agent supporting a user with the APN setting on their iPhone. **_Agent_**_: Hi, welcome to VF Telcom. How can I help?_ **_User_**_: How do I update my APN settings?_ **_\[LLM added to address query: ‘How do I update the APN settings on my device?’\]_** __Starting with the utterances above, we have the agent write a question that’s optimized for retrieval. Then, the LLM is added to the conversation to make the question more specific to address the query. **_\[LLM: Clarification Needed On: 'Ask the user to specify their device make and model to betteraddress their query.'\]_** __Next, we instruct the LLM to look at both the information it has retrieved, along with the question that was presented to a [knowledge base](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) , and ensure they get clarifying information from the user. **_Agent_**_: Certainly, to assist with that, could you please let me know the device you’re using? That way, I can give you the most accurate information._ **_User_**_: an iPhone 15._ **_\[LLM question generated (using entire conversation history): ‘How do I update the APN settings on an iPhone 15?’_**\ \ **_LLM clarification check: Decision: '1', Clarification Needed On: 'Ask the user to specify the carrier they are using or if they need general instructions for updating APN settings on an iPhone 15 to better address their query.'\]_** **_Agent_**_: And could you confirm if you are already on the VF network?_ **_User_**_: Yes, I’m already on the network._ **_\[LLM question generated: ‘How can I update the APN settings on my iPhone 15 for VF network?’\]_** __Here you can see that the LLM has determined that the user needs to specify their device and confirm their carrier in order to give them the most helpful answer to their query. The user responds with, “iPhone 15,” and is asked for further information so that it can generate the final question for the knowledge base. **_\[LLM check: Decision ‘1’, the output has no inconsistency or inaccuracies.\]_** __Now that we have all the information, the response is created. But before it’s presented, the LLM checks that there are no inconsistencies or hallucinations, by doing a cross-check of the response and the information that was retrieved. **_Agent_**_: To update the APN settings on your iPhone 15 for VF Network, you can follow these steps:_ _1\. Go to Settings on your iPhone._ _2\. Tap on Cellular or Mobile Data._ _3\. Select Cellular Data Network._ _4\. Enter the following APN settings:_ * _APN: fast.VF-mobile.com_ * _Username: JohnSmith_ * _Password: 1234_ _5\. Scroll down and enter the same APN settings under the MMS section._ _6\. Press the Home key to save the APN settings and exit to the main screen._ _7\. Restart your device.For more detailed instructions, you can visit the BYOD Mobile data & APN settings page._ **_Agent_**_: Would you like more information on how to reset your network settings if updating the APN does not resolve your connectivity issues?_ __Finally, the answer is displayed, and another prompt is used to display a follow-up question to the user. These local frameworks give the LLM the guidelines to create questions that have been optimized for retrieval, self-check its own work, and ask follow-up questions. By building an intuitive local framework that handles question-answer pairs, you can go from managing hundreds of FAQs to managing the knowledge source that the overall conversation architecture draws from. Local components need to be flexible to adapt to user needs while being responsive to input—just remember that this approach requires detailed design and testing. Integration framework—the utilities behind the walls ----------------------------------------------------- Like the wiring and plumbing in a house, the stuff behind the drywall can be some of the most important. Similarly, the integrations we build between our agents and our systems can make or break user experience. Your integration framework is about designing what external services your agent has access to, what they're used for, and under which circumstances they should access them. By connecting your agent with integrations, it can automatically and flexibly complete tasks. These components can drastically improve the overall user experience that your agent delivers if they’re implemented non-deterministically. When defining how integrations should work in your overall conversation architecture, here are a few things to ponder: * **API management:** How does the agent interact with APIs? How do you manage those integrations? * **External data access:** How should your agent access and store data from external sources? How should your agent format and share that data with users? * **Service orchestration:** What is your approach for coordinating complex interactions across multiple back-end services? Integrations aren't new in conversational AI. Where this approach differs is that you’re designing integration rules without a deterministic flow to execute them. It's about giving the global (or local) framework all the information it needs to determine which integration would help it action/answer the user's question. For example, when I ask a banking agent, “I want to check my balance," I usually get pushed down a flow that collects information until it calls an API that gives me my total balance (and it’s never what I want it to be). In a future, where we design and construct agents with thoughtful frameworks to guide them, we let the agent decide when they need to use specific integrations. Although this approach to integrations requires secure, efficient, and scalable mechanisms—often involving middleware or service buses—it means there is no singular happy path that the agent is forcing a user down. As a result, the opportunities an agent has to serve multiple needs and reliably help more users go up exponentially. Analytics and data framework—the maintenance and improvements ------------------------------------------------------------- After the home is completely constructed, it’s time for the final inspection. Did we comply with all construction codes? Where should we make more improvements or correct our mistakes? In the same way, a robust analytics and data framework allows you to understand your agent’s performance and manage data effectively. It will define how we pass information to LLMs and derive insights from our interactions. When collecting and managing data and analyzing the success of your agent, here are some questions you should answer: * **Data collection and storage:** What are your methods for securely collecting and storing user data? * **Feedback loops:** How are you incorporating user feedback into your iterative improvements of the agents? How do you analyze interaction outcomes for refinement? * **Implementation considerations:** How are you adhering to data protection regulations? How are you improving your agent in such a way as to enable future scalability and flexibility? How does this framework inform a conversation? Well, during an ecommerce conversation, the AI agent collects data like the user's purchasing history, stated preferences ("I like eco-friendly products"), sentiment signals (frustrated tone), cart abandonment point, and final sale/no sale. Analytics frameworks would process this data, combining it with thousands of other interaction logs, which may reveal that eco-conscious buyers frequently abandon their cart due to a lack of green certifications on product pages. We can use this to update interaction components dynamically—changing the agent's persona, proactively suggesting sustainable options, linking green certifications, and using a more consultative interaction model for this user segment. There are endlessly creative ways to use real-time analytics to update how an agent is responding to users. If you’re not securely collecting data gathered during interactions and analyzing it effectively, you’re not likely to be improving your agents based on what your users actually need. And the gorgeous home you designed, constructed, and inspected will eventually fall to ruin from lack of upkeep. Interaction framework—the interior design ----------------------------------------- When you enter a room, you can tell a lot about the person living there—are they a minimalist or a trinket-lover? Do they exclusively use warm-hued lamps in lieu of the dreaded big light? A single glance into a well-decorated room can tell a story without words. In the case of your digital agent, their interaction framework tells users a story about the vibe of your company and the experience they’re about to receive. Ideally, a great agent is able to capture the essence of your brand in communication style, tone, and techniques. And all that is informed by how you instruct the model to interact with users. So, when designing how your agent interacts with users, here are a few things to consider: * **Style and grammar:** What are your rules on style, tone, and formatting? How do you format hyperlinks or lists? How does it keep the conversation flowing naturally? * **Language and tone:** What are your guidelines for politeness, formality, and brand voice? * **Conversational strategies:** How is your agent programmed to ask for clarification? What do conversational turns look and sound like? How does it give proactive suggestions? I suggest creating and maintaining a style guide and tone-of-voice document to keep your agent’s interaction on brand. This framework requires deep linguistic modeling and an understanding of conversational dynamics, but it also incorporates user feedback and sentiment analysis as you learn more about your agent and your company’s unique needs. (Stay tuned to Pathways for our take on style guides for your agents.) All the makings of an architectural marvel ------------------------------------------ Architect and planner David Pearson once proposed a list of rules for the design of organic architecture—a framework, you might say—and one of his cardinal rules reads, “Let the design follow the flows and be flexible and adaptable.” Like architecture, conversational AI is both a science and an art. A collection of rules, guidelines, and frameworks _and_ the creative mission of many designers, developers, and thinkers. By designing each component of our conversation frameworks thoughtfully—global, local, integrations, interactions, and analytics—you’ll be able to build agents that complete tasks, problem-solve, and delight users. You’ll be able to use these five frameworks as the building blocks to serve a larger conversational AI architecture. Your strategic design choices can make your agents strong, functional, and flexible. Or fragile, ineffective, and rigid. But that’s all up to the architect.  RECOMMENDED [\ \ ### Crawl, walk, run: 28+ tactics for evolving your AI agent](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) PUBLISHED April 1, 2024 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Cathy Pearl on all things conversation design](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design) [](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to prioritize requests for conversation design teams](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### A bit of entity advice: Delight your overfillers](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/architecting-the-future-of-ai-agents-5-flexible-conversation-frameworks-you-need#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/architecting-the-future-of-ai-agents-5-flexible-conversation-frameworks-you-need#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Instacart's Ayesha Saleem shares her 5 best conversational AI practices | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Instacart's Ayesha Saleem shares her 5 best conversational AI practices ======================================================================= [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Conversational AI is constantly changing—but in the creative hands of people like Ayesha Saleem, we can’t help but be excited for the future. In her impressive career, she’s become known as a builder and creative, taking non-existent [conversation design (CxD)](https://www.voiceflow.com/conversation-design) programs and building highly effective operations from scratch—first at Rocket Mortgage (formerly Quicken Loans) and now at Instacart. But her tireless creative energy doesn’t stop with her own teams. Ayesha is sought out in CxD to speak at conferences and on podcasts, build comprehensive CxD guides, and mentor emerging leaders. And she’s shown no signs of slowing down. With these discussions, our goal is to discover exceptional people in conversational AI and find out how they landed in CxD, how they make sense of the chaos, and how their journeys could inspire your own. Today, Ayesha explains how she proves the impact of her team’s work to leadership (and what happens when you don’t), the five best practices she applies to every design project, and how she manages failure while redefining success.  SB: Tell me about your conversational AI career journey. --------------------------------------------------------- AS: I majored in computer science at the University of Michigan and did a bunch of software engineering internships—one of which was at GE, building for Alexa back in 2016. I got the chance to help launch their Alexa assistant when everyone was newly excited about the opportunities with conversation design. That really sparked my love of this space. But I quickly realized engineering was wrong for me. I didn’t enjoy just implementing someone else's vision—it felt like the wrong stage of the creative process. I wanted to find a way to merge all my creativity and excitement around conversation design. That was when I was hired at Rocket Mortgage. My work there was unique because they had no clue what they needed. My title started out as “Cognitive AI Engineering Designer” and I remember thinking, “What? That's 10 different skills. What is that?” But it was the perfect role for me—a little bit of everything. I started building their conversation design operation from the ground up. It only took two years to define goals and success criteria, build a team of five designers, and move us out of spreadsheets and into tools like Voiceflow. And that opportunity led me to Instacart, where I’ve had the pleasure to do that all over again as a conversation design manager. How do you explain the value of your CxD work to leadership? ------------------------------------------------------------- Imagine you're talking to a design manager and you explain it’s taken you 30 hours to iterate on a design. That manager will understand that you’ve been playing around with different designs and thinking through how conversations should flow—you're in the creative process. But when I started my career, I was reporting to an engineering manager who had absolutely no idea what I was doing. He thought I was a _graphic_ designer for a long time. For that engineering manager, he couldn't wrap his mind around that winding, iterative process. He understood progress as lines of code or projects launched. Now when I report to managers or leadership who may not understand conversation design workflows or even how my team spends our time, I report on two things to show how we work and why it’s valuable:impact and data. **1\. Impact.** For example, while testing our designs we’d find people in the office to play the user. It was as simple as reciting the script verbally to one another. We collected their feedback so when it came time for reporting, we could explain the insights we gleaned from those tests and what we were working on to improve the project. Impact can be easy to prove once a product is launched, but if you’re showing that your creative process is already yielding quantifiable results—whether that’s 10 test interviews or a 10% improvement in performance—your leaders will be more likely to trust your process. **2\. Data.** You cannot go into a room and say, "Hey, we're doing this because I said so." For people who are new to conversation design, they need some convincing. And you can't really argue with hard numbers. My foolproof way to prove our process is with A/B testing. When I started at both Rocket and Instacart with A/B testing, I managed to prove that our conversational systems were offering us huge cost savings as well as speed and time efficiencies. > "For people who are new to conversation design, they need some convincing. And you can't really argue with hard numbers." To this day, whenever I update our leaders on our latest progress or pitch new ideas, I always lead with the impact of what we’re doing right now and include real data. What happens if you’re not showing the impact of CxD on the business? ---------------------------------------------------------------------- I’ve worked with people who were more focused on the work itself and unsure how to quantify or explain their output. Sometimes the output of conversational AI doesn’t look like a typical 40-hour work week and that can get you into sticky situations with people who don’t understand our process. In this industry, if you’re not your own advocate, people aren't going to support what you're doing. I've seen people's careers not move as fast as those who have mastered educating others on CxD or advocating for what they do. Or I've seen people burn out because they’re working hard but their efforts aren’t being acknowledged or supported by their management. It’s twofold—your leadership needs to understand what they've hired you to do and give you the agency to do it. And you should learn how to report on your impact, show your value, and educate your leaders and colleagues on conversational AI. What are your CxD best practices? ---------------------------------- I have five that come to mind, which can apply to many conversational AI use cases, both big and small. **1\. Design for your user.** I am in the delivery space and I know my target demographic. We have to design for both older people and busy millennials who are working all the time. You need to understand who your user is and what they're trying to accomplish. When I was working in the mortgage industry, we were designing for folks in the U.S. who were in their 50s—those were the people most likely to get a mortgage. So we were designing for people who were not comfortable with technology and who would opt to talk to a human as fast as they possibly could. That posed a challenge, but knowing that fact made us design better tools for them. Iterate on your personas, perform UX testing, and do user research to understand how comfortable people are with your designs. Much of that knowledge will affect your internal best practices as well—like never writing anything more complex than a sixth-grade reading level. **2\. Listen to your assistant speak.** For voice interfaces, always, always, always listen to your content. Because if you're saying, "Sure, I will order that for you," as opposed to, "Sure, I'll order that for you," you may not realize that including a contraction makes it sound much less robotic. Lots of designers will write a conversation, design it in a Voiceflow or Figma, and then they'll publish it. They won’t even realize how weird it sounds. It's really important to hear the tone and how the system is saying each sentence. **3\. Be conscious of space limitations.** In conversation design, you never have enough space. If you're designing a chatbot that lives on the corner of a page, you may get three lines. Tops. If you're designing an Alexa or Google Home store, you probably have people's attention for about 10 seconds before they've lost interest and gotten annoyed. Become very good at concision. **4\. Focus on accessibility.** If you're using a screen, make sure your colors are accessible and your font is big enough. Make sure your icons are legible. Accessibility is important. And unfortunately, a lot of people forget about it. **5\. Ensure your script is inclusive and translatable.** Write for the audience that you're speaking to. I've seen people release assistants in multiple languages and include puns. The second you translate this joke into a different language, it flops. Inclusive language is about more than avoiding offensive words, it’s about ensuring your designs don’t leave entire cultures and languages out of the conversation. What’s your approach to success and failure? -------------------------------------------- You should determine success criteria before you even start your project so everyone has a shared understanding of what success means and how we’re going to achieve it. Only from that place can we begin the process. Personally, I define insane success goals for myself and then I rarely meet them (laughs). I'm working on it! When you're designing a project, it's all theoretical. So if you push it out and it doesn't do as well as you want it to—which might be considered a failure—you lean into improving it from there. My favorite part is going deep and figuring out why it failed, what we can do right away to fix it, and what we can do better in the future. Honestly, I think it's exciting when things don't go your way 100%. There’s got to be space for iteration and improvement. I think the improvement process is gratifying because you can see how the levers you pull move you closer or further from success. By seeing success and failure as fluid, you have the freedom to try new things. You stop being so afraid. > "Honestly, I think it's exciting when things don't go your way 100%. There’s got to be space for iteration and improvement." Ayesha Saleem—the builder, the creative --------------------------------------- “Failure is just an opportunity,” Ayesha says. At Instacart, she’s exploring how to integrate generative AI into their projects. This new evolution is another opportunity to build, fail, and try—that’s what excites her. “We don’t know all the value AI can bring yet. But we’re building the foundations right now.” In addition to the laundry list of projects she has on the go, Ayesha is currently working on her own farm-to-table vegetable garden. “It’s the creative in me, I have to have my hands in a little bit of everything.” Connect with Ayesha on [LinkedIn](https://www.linkedin.com/in/ayeshasa/) . SB: Tell me about your conversational AI career journey. --------------------------------------------------------- AS: I majored in computer science at the University of Michigan and did a bunch of software engineering internships—one of which was at GE, building for Alexa back in 2016. I got the chance to help launch their Alexa assistant when everyone was newly excited about the opportunities with conversation design. That really sparked my love of this space. But I quickly realized engineering was wrong for me. I didn’t enjoy just implementing someone else's vision—it felt like the wrong stage of the creative process. I wanted to find a way to merge all my creativity and excitement around conversation design. That was when I was hired at Rocket Mortgage. My work there was unique because they had no clue what they needed. My title started out as “Cognitive AI Engineering Designer” and I remember thinking, “What? That's 10 different skills. What is that?” But it was the perfect role for me—a little bit of everything. I started building their conversation design operation from the ground up. It only took two years to define goals and success criteria, build a team of five designers, and move us out of spreadsheets and into tools like Voiceflow. And that opportunity led me to Instacart, where I’ve had the pleasure to do that all over again as a conversation design manager. How do you explain the value of your CxD work to leadership? ------------------------------------------------------------- Imagine you're talking to a design manager and you explain it’s taken you 30 hours to iterate on a design. That manager will understand that you’ve been playing around with different designs and thinking through how conversations should flow—you're in the creative process. But when I started my career, I was reporting to an engineering manager who had absolutely no idea what I was doing. He thought I was a _graphic_ designer for a long time. For that engineering manager, he couldn't wrap his mind around that winding, iterative process. He understood progress as lines of code or projects launched. Now when I report to managers or leadership who may not understand conversation design workflows or even how my team spends our time, I report on two things to show how we work and why it’s valuable:impact and data. **1\. Impact.** For example, while testing our designs we’d find people in the office to play the user. It was as simple as reciting the script verbally to one another. We collected their feedback so when it came time for reporting, we could explain the insights we gleaned from those tests and what we were working on to improve the project. Impact can be easy to prove once a product is launched, but if you’re showing that your creative process is already yielding quantifiable results—whether that’s 10 test interviews or a 10% improvement in performance—your leaders will be more likely to trust your process. **2\. Data.** You cannot go into a room and say, "Hey, we're doing this because I said so." For people who are new to conversation design, they need some convincing. And you can't really argue with hard numbers. My foolproof way to prove our process is with A/B testing. When I started at both Rocket and Instacart with A/B testing, I managed to prove that our conversational systems were offering us huge cost savings as well as speed and time efficiencies. > "For people who are new to conversation design, they need some convincing. And you can't really argue with hard numbers." To this day, whenever I update our leaders on our latest progress or pitch new ideas, I always lead with the impact of what we’re doing right now and include real data. What happens if you’re not showing the impact of CxD on the business? ---------------------------------------------------------------------- I’ve worked with people who were more focused on the work itself and unsure how to quantify or explain their output. Sometimes the output of conversational AI doesn’t look like a typical 40-hour work week and that can get you into sticky situations with people who don’t understand our process. In this industry, if you’re not your own advocate, people aren't going to support what you're doing. I've seen people's careers not move as fast as those who have mastered educating others on CxD or advocating for what they do. Or I've seen people burn out because they’re working hard but their efforts aren’t being acknowledged or supported by their management. It’s twofold—your leadership needs to understand what they've hired you to do and give you the agency to do it. And you should learn how to report on your impact, show your value, and educate your leaders and colleagues on conversational AI. What are your CxD best practices? ---------------------------------- I have five that come to mind, which can apply to many conversational AI use cases, both big and small. **1\. Design for your user.** I am in the delivery space and I know my target demographic. We have to design for both older people and busy millennials who are working all the time. You need to understand who your user is and what they're trying to accomplish. When I was working in the mortgage industry, we were designing for folks in the U.S. who were in their 50s—those were the people most likely to get a mortgage. So we were designing for people who were not comfortable with technology and who would opt to talk to a human as fast as they possibly could. That posed a challenge, but knowing that fact made us design better tools for them. Iterate on your personas, perform UX testing, and do user research to understand how comfortable people are with your designs. Much of that knowledge will affect your internal best practices as well—like never writing anything more complex than a sixth-grade reading level. **2\. Listen to your assistant speak.** For voice interfaces, always, always, always listen to your content. Because if you're saying, "Sure, I will order that for you," as opposed to, "Sure, I'll order that for you," you may not realize that including a contraction makes it sound much less robotic. Lots of designers will write a conversation, design it in a Voiceflow or Figma, and then they'll publish it. They won’t even realize how weird it sounds. It's really important to hear the tone and how the system is saying each sentence. **3\. Be conscious of space limitations.** In conversation design, you never have enough space. If you're designing a chatbot that lives on the corner of a page, you may get three lines. Tops. If you're designing an Alexa or Google Home store, you probably have people's attention for about 10 seconds before they've lost interest and gotten annoyed. Become very good at concision. **4\. Focus on accessibility.** If you're using a screen, make sure your colors are accessible and your font is big enough. Make sure your icons are legible. Accessibility is important. And unfortunately, a lot of people forget about it. **5\. Ensure your script is inclusive and translatable.** Write for the audience that you're speaking to. I've seen people release assistants in multiple languages and include puns. The second you translate this joke into a different language, it flops. Inclusive language is about more than avoiding offensive words, it’s about ensuring your designs don’t leave entire cultures and languages out of the conversation. What’s your approach to success and failure? -------------------------------------------- You should determine success criteria before you even start your project so everyone has a shared understanding of what success means and how we’re going to achieve it. Only from that place can we begin the process. Personally, I define insane success goals for myself and then I rarely meet them (laughs). I'm working on it! When you're designing a project, it's all theoretical. So if you push it out and it doesn't do as well as you want it to—which might be considered a failure—you lean into improving it from there. My favorite part is going deep and figuring out why it failed, what we can do right away to fix it, and what we can do better in the future. Honestly, I think it's exciting when things don't go your way 100%. There’s got to be space for iteration and improvement. I think the improvement process is gratifying because you can see how the levers you pull move you closer or further from success. By seeing success and failure as fluid, you have the freedom to try new things. You stop being so afraid. > "Honestly, I think it's exciting when things don't go your way 100%. There’s got to be space for iteration and improvement." Ayesha Saleem—the builder, the creative --------------------------------------- “Failure is just an opportunity,” Ayesha says. At Instacart, she’s exploring how to integrate generative AI into their projects. This new evolution is another opportunity to build, fail, and try—that’s what excites her. “We don’t know all the value AI can bring yet. But we’re building the foundations right now.” In addition to the laundry list of projects she has on the go, Ayesha is currently working on her own farm-to-table vegetable garden. “It’s the creative in me, I have to have my hands in a little bit of everything.” Connect with Ayesha on [LinkedIn](https://www.linkedin.com/in/ayeshasa/) .  RECOMMENDED [\ \ ### Conversational AI has a massive, UX-shaped hole](https://www.voiceflow.com/pathways/conversational-ai-ux) PUBLISHED August 9, 2023 TOPICS IN THIS ARTICLE Exceptional people Elevating our work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/six-workflows-to-create-amazing-conversational-ai-experiences) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 6 workflows to create amazing conversational AI experiences](https://www.voiceflow.com/pathways/six-workflows-to-create-amazing-conversational-ai-experiences) [](https://www.voiceflow.com/pathways/best-practices-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Curiosity and conversation beat any best practice](https://www.voiceflow.com/pathways/best-practices-conversation-design) [](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Automation Design: The Evolution of Conversation Design](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/best-conversational-ai-practices#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/best-conversational-ai-practices#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Building a virtual agent from scratch? Start here. | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Building a virtual agent from scratch? Start here. ================================================== [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) Nobody is born an excellent conversationalist. It takes years; decades, some would argue. First, we have to learn what language is. Then we have to learn how to use it. And then, over time, we refine, perfect it, (or subvert it—you don’t read the word “pwn” without blinking an eye unless you have many layers of both language and gaming lore to draw on) and make it ours. Building a virtual agent is no different. You can’t just release it fully formed into the world, ready to answer user questions adequately. You have to teach it language first, show it the boundaries it can operate within, and then continue to help it evolve over time. Today, we’ll look at that process in detail. Whether you’re building a voice or chat agent, we’ll walk you through each step, so you can take your virtual agent from n00b to pwning the nuances of customer interactions (sorry, it was right there).  Step 1: Gather utterances and transcripts ----------------------------------------- The process for gathering utterances varies slightly between voice and chat agents. When building an interactive voice response (IVR) agent, we want to start with setting up an utterance capture on the company’s customer service line. In practice, customers would be greeted with something like this when calling in for support: > _“Hi, welcome to Voiceflow Bank. I’m learning how to understand humans. In a short sentence, please tell me why you’re calling today.”_ __ How long you’ll want to run your utterance capture depends on call volume. A good rule of thumb is to keep it open for up to a month, or until you’ve captured at least 5,000 utterances. If you’re building a chat agent, you’ll use your existing chat interface (if you have one) to gather data on first utterances mentioned to your agents. You can also set up a capture box where the user can write why they’re reaching out. This prompt might look something like this: > _“In a short sentence, please tell me why you’re getting in contact today.”_ The overall goal of this step is to gather as many use cases as possible. This is the data you’ll use to build out your intent structure for the rest of the agent. Step 2: Build the intent model ------------------------------ Once you’ve captured enough data, you can start creating your intent model. This involves clustering semantically similar utterances into intents. From there, you’ll refine it further by annotating entities, creating points of disambiguation, splitting intents, and merging others together. Ideally, by the end of the process, your model should understand 80% of all your utterances, at an 80% confidence. Depending on size, your initial model could take anywhere from one week to a month to build. For reference, it took me two weeks to complete the initial build of my last big model, which featured 100,000 utterances and 130 intents. Step 3: Define the scope ------------------------ There’s nothing worse than a two-week project that stretches out into a month. To avoid this, you’ll need to carefully map out the project scope in advance. This starts with agreeing on an ideal outcome for each intent. Working with stakeholders, you can work backwards from each ideal intent to scope out the workload. For example, is the final destination for a certain query in an FAQ redirect? Is it a handover to an agent, or are there multiple integrations to consider? Once you have an idea of the scope of each intent, you can then scope out the entire build. You can assign story points to each intent, like so: * 5 points = 2 days * 3 points = 1 day * 2 points = half day * 1 point = 2 hours Story points create a level of abstraction and a quick translation of effort for each step, making the estimation process quicker and more focused on the complexity of the task than the minutiae of how long something will take. As a rule of thumb, no single task should take longer than two days. And if it does, break it down into sub-tasks. Step 4: Plan your happy paths ----------------------------- Once you’ve built out your intent model and defined the scope of each intent, it’s time to build out your happy flows. These are basically a visual representation of your best-case scenarios, where your customer says what they want you to deliver to them in the most efficient way possible. Of course, happiness is highly subjective, so you’ll want to get sign-off from your stakeholders before building out the long tail of your journeys. Here you’ll need to gather feedback on the design, and maybe even test out your flows with customers before jumping into the long tail of your design. Step 5: Build out long tail design ---------------------------------- After your happy paths are paved, you’ll need to consider the not-so-happy ones. Building out the long tail design of your agent means thinking through each edge case that could arise. That is, every error, unknown, or less-than-ideal outcome that could come up during a customer’s interaction with your virtual agent. For instance, maybe you're asking a customer for their account number. The customer might respond with, “Where do I find my account number?” rather than providing the information you need. This is an edge case you’ll want to handle so you can get the user back on track after giving them that information. Other things that you’ll want to focus on—which already will have been scoped out in Step 3—are things like: * How you’ll use integrations * What variables and entities could be used to skip customers through * What thresholds to put in place before handing a customer over to an agent Much like in the previous step, you’ll want your stakeholders to weigh in with their feedback before you seal off this step and move on. Step 6: QA and launch --------------------- Before your agent is ready to interact with customers, you’ll want to send it over to QA. Ideally, a QA engineer will complete this step. If not, you’ll want to create test cases that ensure the agent and its flows work as expected. If anything goes awry, then you’ll need to fix any bugs (and retest) before launch. Of course, there’s only so much you can test internally before your agent is released into the wild. Building conversational AI systems is an iterative process, so your goal before launching should be to QA each possible path you’ve planned out so far, and then build on them with real-world data. Step 7: Business as usual (BAU) monitoring ------------------------------------------ The long tail design and QA steps will get your virtual agent to a “good enough to launch” state, but that doesn’t mean your work is done. BAU monitoring is an ongoing requirement for virtual agents. By monitoring the way customers interact with it, you can report on performance and track any bugs that weren’t caught during QA. This stage can also involve creating new intents and flows as they emerge in real life. Each new intent can be scoped separately, then sent to stakeholders to decide what should get built next. ### Building an agent IRL takes time. Now that we’ve reviewed each step to building a virtual agent, let’s look at how this process might play out in real life. Here’s an example of an IVR or simple chatbot build, with a few complex flows and agent transfer. This example depicts about 90 days of effort between design development and project management. It’s worth noting that these things don’t always follow a typical waterfall model. There’s room to play here—for example, you could choose to build out integrations during the happy path design. And while I’m positing that there’s a certain timeline you can assign to building out a virtual agent, we all know the work will continue for as long as the agent exists. Just like each person has their own preferred idioms, favorite responses, and new slang to add to their vocabulary over time, your agent will get better—and more distinctive—with each interaction and iteration. Just don’t let it call your customers n00bs. It’s not nice. Step 1: Gather utterances and transcripts ----------------------------------------- The process for gathering utterances varies slightly between voice and chat agents. When building an interactive voice response (IVR) agent, we want to start with setting up an utterance capture on the company’s customer service line. In practice, customers would be greeted with something like this when calling in for support: > _“Hi, welcome to Voiceflow Bank. I’m learning how to understand humans. In a short sentence, please tell me why you’re calling today.”_ __ How long you’ll want to run your utterance capture depends on call volume. A good rule of thumb is to keep it open for up to a month, or until you’ve captured at least 5,000 utterances. If you’re building a chat agent, you’ll use your existing chat interface (if you have one) to gather data on first utterances mentioned to your agents. You can also set up a capture box where the user can write why they’re reaching out. This prompt might look something like this: > _“In a short sentence, please tell me why you’re getting in contact today.”_ The overall goal of this step is to gather as many use cases as possible. This is the data you’ll use to build out your intent structure for the rest of the agent. Step 2: Build the intent model ------------------------------ Once you’ve captured enough data, you can start creating your intent model. This involves clustering semantically similar utterances into intents. From there, you’ll refine it further by annotating entities, creating points of disambiguation, splitting intents, and merging others together. Ideally, by the end of the process, your model should understand 80% of all your utterances, at an 80% confidence. Depending on size, your initial model could take anywhere from one week to a month to build. For reference, it took me two weeks to complete the initial build of my last big model, which featured 100,000 utterances and 130 intents. Step 3: Define the scope ------------------------ There’s nothing worse than a two-week project that stretches out into a month. To avoid this, you’ll need to carefully map out the project scope in advance. This starts with agreeing on an ideal outcome for each intent. Working with stakeholders, you can work backwards from each ideal intent to scope out the workload. For example, is the final destination for a certain query in an FAQ redirect? Is it a handover to an agent, or are there multiple integrations to consider? Once you have an idea of the scope of each intent, you can then scope out the entire build. You can assign story points to each intent, like so: * 5 points = 2 days * 3 points = 1 day * 2 points = half day * 1 point = 2 hours Story points create a level of abstraction and a quick translation of effort for each step, making the estimation process quicker and more focused on the complexity of the task than the minutiae of how long something will take. As a rule of thumb, no single task should take longer than two days. And if it does, break it down into sub-tasks. Step 4: Plan your happy paths ----------------------------- Once you’ve built out your intent model and defined the scope of each intent, it’s time to build out your happy flows. These are basically a visual representation of your best-case scenarios, where your customer says what they want you to deliver to them in the most efficient way possible. Of course, happiness is highly subjective, so you’ll want to get sign-off from your stakeholders before building out the long tail of your journeys. Here you’ll need to gather feedback on the design, and maybe even test out your flows with customers before jumping into the long tail of your design. Step 5: Build out long tail design ---------------------------------- After your happy paths are paved, you’ll need to consider the not-so-happy ones. Building out the long tail design of your agent means thinking through each edge case that could arise. That is, every error, unknown, or less-than-ideal outcome that could come up during a customer’s interaction with your virtual agent. For instance, maybe you're asking a customer for their account number. The customer might respond with, “Where do I find my account number?” rather than providing the information you need. This is an edge case you’ll want to handle so you can get the user back on track after giving them that information. Other things that you’ll want to focus on—which already will have been scoped out in Step 3—are things like: * How you’ll use integrations * What variables and entities could be used to skip customers through * What thresholds to put in place before handing a customer over to an agent Much like in the previous step, you’ll want your stakeholders to weigh in with their feedback before you seal off this step and move on. Step 6: QA and launch --------------------- Before your agent is ready to interact with customers, you’ll want to send it over to QA. Ideally, a QA engineer will complete this step. If not, you’ll want to create test cases that ensure the agent and its flows work as expected. If anything goes awry, then you’ll need to fix any bugs (and retest) before launch. Of course, there’s only so much you can test internally before your agent is released into the wild. Building conversational AI systems is an iterative process, so your goal before launching should be to QA each possible path you’ve planned out so far, and then build on them with real-world data. Step 7: Business as usual (BAU) monitoring ------------------------------------------ The long tail design and QA steps will get your virtual agent to a “good enough to launch” state, but that doesn’t mean your work is done. BAU monitoring is an ongoing requirement for virtual agents. By monitoring the way customers interact with it, you can report on performance and track any bugs that weren’t caught during QA. This stage can also involve creating new intents and flows as they emerge in real life. Each new intent can be scoped separately, then sent to stakeholders to decide what should get built next. ### Building an agent IRL takes time. Now that we’ve reviewed each step to building a virtual agent, let’s look at how this process might play out in real life. Here’s an example of an IVR or simple chatbot build, with a few complex flows and agent transfer. This example depicts about 90 days of effort between design development and project management. It’s worth noting that these things don’t always follow a typical waterfall model. There’s room to play here—for example, you could choose to build out integrations during the happy path design. And while I’m positing that there’s a certain timeline you can assign to building out a virtual agent, we all know the work will continue for as long as the agent exists. Just like each person has their own preferred idioms, favorite responses, and new slang to add to their vocabulary over time, your agent will get better—and more distinctive—with each interaction and iteration. Just don’t let it call your customers n00bs. It’s not nice.  RECOMMENDED [\ \ ### 5 powerful templates for AI automation agencies](https://www.voiceflow.com/pathways/ai-automation-agency-templates) PUBLISHED November 6, 2023 TOPICS IN THIS ARTICLE Learning and APIs Elevating our work [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [%25201.avif)](https://www.voiceflow.com/pathways/jumping-off-the-ai-hype-train-nlus-in-an-llm-dominated-world) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Jumping off the AI hype train: NLUs in an LLM-dominated world](https://www.voiceflow.com/pathways/jumping-off-the-ai-hype-train-nlus-in-an-llm-dominated-world) [](https://www.voiceflow.com/pathways/webpage-assistant-voiceflow-dialog-api) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Create a webpage to talk to your assistant using Voiceflow Dialog API](https://www.voiceflow.com/pathways/webpage-assistant-voiceflow-dialog-api) [](https://www.voiceflow.com/pathways/art-of-debugging) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### The art of debugging](https://www.voiceflow.com/pathways/art-of-debugging) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Breaking and rebuilding Nike's chatbot | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Breaking and rebuilding Nike's chatbot ====================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) In early March, I hosted the first episode of Voiceflow's **Breaking Bots**, an hour-long webinar in which conversation designers break a well-known chatbot or assistant, then rebuild it—only better—live. Inspired by Nicolle Merrill, who broke into the field by breaking bots and writing about her experiences, the series aims to educate through the processes of deconstruction and reconstruction. Our inaugural episode featured two remarkably talented conversation designers: Ali Pinch, Conversation Designer at one of Australia’s largest retailers, and Nicolle Merrill, Senior Conversational AI Designer at Fortune 500 FinTech company Intuit. Ali and Nicolle are both veterans in the field. In this post, we tell the story of how Ali and Nicolle broke Nike’s bot and rebuilt it. To see the breakdown and buildup unfold in real-time, and to discover industry best practices along the way, [watch the episode](https://www.youtube.com/watch?v=R0hTL8Nv4co) .  Breaking Nike’s bot ------------------- [Nike’s bot](https://www.nike.com/help/#contact) is located on its Contact Us page—one of Ali’s biggest pet peeves. She prefers when chatbots are easily accessible from the homepage, as this helps reduce friction for people who need help. People access the bot by clicking on a chat-box icon. At the time of the webinar, the only copy accompanying the icon was “24 hours a day, 7 days a week”—leaving much to be desired, according to Ali and Nicolle. From the copy, people can’t tell whether they’ll be speaking with a human or bot. The lack of copy is also considered a faux pas from an accessibility standpoint. In reality, people can’t interact with Nike’s chatbot between 5am and 11pm. During those hours, they’re instead connected with a member of the company’s support team.  Screenshot of Nike's Contact Us page The authentication flow ----------------------- Before interacting with Nike’s bot, people must enter their name and email address. They’re also asked to select different options depending on what kind of help they need. For example, people can choose to track their orders, get an update on a refund, or ask about specific products. This authentication flow perplexed Ali and Nicolle. Virtual assistants are perfectly capable of processing customer data, so why isn’t Nike’s bot (instead of a form) collecting this information? Companies should also only need a single piece of identification to serve their customers, so why does the form ask for multiple customer data points?  Screenshot of authentication flow User journeys ------------- Ali and Nicolle broke the bot during two journeys: one to track an order and one to ask about a product. Keep reading to discover what went wrong. ### Track my order When Nike’s bot arrives in the chat, it jumps straight to the matter at hand—helping people find their order status—without offering a welcome message. Nicolle argues that welcome messages build trust and set the tone for the rest of the conversation. At minimum, they should inform people that they’ve reached a chatbot and share what it’s designed to help them accomplish. Ideally, they should show off the brand’s personality and guide what happens next. Nike’s bot next asks for the customer’s name and email address, and quickly follows up by asking them to confirm the information they’ve provided. That doesn’t make sense given that the same information was provided during the authentication flow, and then again in the chat. Also, companies should only require a single piece of identifying information. In this case, an order number would make the most sense since it’s publicly available information, according to Ali. If the customer doesn’t remember their order number, Nike’s bot should tell them where to find it or give them another way to access their order. Instead, when the customer enters incorrect information (a no-match response), they are shown not only a fallback response but also the original prompt again. The experience is jarring and leaves people unsure how to respond. The fallback response also instructs customers how to get help from a human. But Ali and Nicolle argue that that’s redundant because most people know how to do that already. One attendee named Daniel also pointed out that it could increase escalations—driving up the company’s support costs. Given that most companies leverage chatbots to lower costs, it’s an unusual choice. All in all, the process of tracking an order was dragged out. A well-designed bot would have helped the customer track their order in just two steps. ### Ask about a product When people indicate they want to ask about a product, Nike’s bot first asks whether they’re searching for a new or existing product. That’s unclear at best and confusing at worst, according to Ali. Nicolle shared that, from a brand perspective, it’s strange that the bot suddenly uses title case. Not only is it unnatural to speak to people that way, but it’s also inconsistently applied throughout the chat. When Ali input Nike’s most in-demand product at the moment, she was transferred to a human—a big waste of money. At minimum, the bot could have directed her to a product page. If it didn’t recognize the name, it could have shown other products. It’s clear from this experience that the bot hasn’t been trained to recognize products. Overall, Ali and Nicolle were disappointed with their experiences.  ### Throughout the experience During the user journey, Ali and Nicolle experienced a slew of other challenges: 1. When they first joined the chat, they were asked not to refresh and told they were in queue. That implied they were waiting for a human, not a bot. This could confuse people and make them believe they’re being connected with a human. 2. On a related note, the UI showed a message suggesting Nike’s virtual agent was typing—something only a human can do. Again, this has the potential to confuse current and prospective customers. 3. The language choice throughout the experience was very formulaic and inconsistent. For example, the bot sent a mix of formal and informal messages. Given Nike’s incredible brand, this missed opportunity was disappointing for Ali and Nicolle. 4. When Ali and Nicolle paused for a moment to discuss their experience, they were given 90 seconds to respond before the chat timed-out. The thing is, everyone gets distracted. By being pushy, you cause feelings of panic. Ali and Nicolle also argued that there’s no reason a bot should time-out. It should infinitely wait for you. 5. Ali and Nicolle also criticized the old-school UI, which they argued looked like it was from 2005. Again, Nike is a prominent brand. Why didn’t the company invest in a simple and clean UI that delights users? 6. Any time Ali and Nicolle wanted to do something different from what they originally chose (e.g., ask about a product instead of tracking an order), they had to exit and reenter the chat, which creates an extra step for the user and slows them down. 7. People think they can toggle back to the main menu from a drop-down menu. Unfortunately, it’s not easy to find and it doesn’t do what it suggests. Instead of bringing you back to the main menu, the bot prompts you to reconfirm what you want to do—presenting only the original prompt and an “other” option. 8. Ali and Nicolle were pleased to see that people could save their chat history and refer back to it later. However, the ways people could save the chat—through a .txt file or by printing the chat—was highly unusual. Watch the rebuild in real-time ------------------------------ The full episode shows Ali and Nicolle rebuilding Nike’s bot in Voiceflow. They not only talk through everything from [generating utterances](https://www.voiceflow.com/blog/ai-assist-llms-voiceflow) and leveraging entities to using logic to create different paths, but they also highlight [conversation design best practices](https://www.voiceflow.com/blog/nike-conversation-design-best-practices) along the way. _Header photo by_ [_Terrance Barksdale_](https://www.pexels.com/photo/a-close-up-shot-of-a-pair-of-nike-shoes-6692038/) _._ Breaking Nike’s bot ------------------- [Nike’s bot](https://www.nike.com/help/#contact) is located on its Contact Us page—one of Ali’s biggest pet peeves. She prefers when chatbots are easily accessible from the homepage, as this helps reduce friction for people who need help. People access the bot by clicking on a chat-box icon. At the time of the webinar, the only copy accompanying the icon was “24 hours a day, 7 days a week”—leaving much to be desired, according to Ali and Nicolle. From the copy, people can’t tell whether they’ll be speaking with a human or bot. The lack of copy is also considered a faux pas from an accessibility standpoint. In reality, people can’t interact with Nike’s chatbot between 5am and 11pm. During those hours, they’re instead connected with a member of the company’s support team.  Screenshot of Nike's Contact Us page The authentication flow ----------------------- Before interacting with Nike’s bot, people must enter their name and email address. They’re also asked to select different options depending on what kind of help they need. For example, people can choose to track their orders, get an update on a refund, or ask about specific products. This authentication flow perplexed Ali and Nicolle. Virtual assistants are perfectly capable of processing customer data, so why isn’t Nike’s bot (instead of a form) collecting this information? Companies should also only need a single piece of identification to serve their customers, so why does the form ask for multiple customer data points?  Screenshot of authentication flow User journeys ------------- Ali and Nicolle broke the bot during two journeys: one to track an order and one to ask about a product. Keep reading to discover what went wrong. ### Track my order When Nike’s bot arrives in the chat, it jumps straight to the matter at hand—helping people find their order status—without offering a welcome message. Nicolle argues that welcome messages build trust and set the tone for the rest of the conversation. At minimum, they should inform people that they’ve reached a chatbot and share what it’s designed to help them accomplish. Ideally, they should show off the brand’s personality and guide what happens next. Nike’s bot next asks for the customer’s name and email address, and quickly follows up by asking them to confirm the information they’ve provided. That doesn’t make sense given that the same information was provided during the authentication flow, and then again in the chat. Also, companies should only require a single piece of identifying information. In this case, an order number would make the most sense since it’s publicly available information, according to Ali. If the customer doesn’t remember their order number, Nike’s bot should tell them where to find it or give them another way to access their order. Instead, when the customer enters incorrect information (a no-match response), they are shown not only a fallback response but also the original prompt again. The experience is jarring and leaves people unsure how to respond. The fallback response also instructs customers how to get help from a human. But Ali and Nicolle argue that that’s redundant because most people know how to do that already. One attendee named Daniel also pointed out that it could increase escalations—driving up the company’s support costs. Given that most companies leverage chatbots to lower costs, it’s an unusual choice. All in all, the process of tracking an order was dragged out. A well-designed bot would have helped the customer track their order in just two steps. ### Ask about a product When people indicate they want to ask about a product, Nike’s bot first asks whether they’re searching for a new or existing product. That’s unclear at best and confusing at worst, according to Ali. Nicolle shared that, from a brand perspective, it’s strange that the bot suddenly uses title case. Not only is it unnatural to speak to people that way, but it’s also inconsistently applied throughout the chat. When Ali input Nike’s most in-demand product at the moment, she was transferred to a human—a big waste of money. At minimum, the bot could have directed her to a product page. If it didn’t recognize the name, it could have shown other products. It’s clear from this experience that the bot hasn’t been trained to recognize products. Overall, Ali and Nicolle were disappointed with their experiences.  ### Throughout the experience During the user journey, Ali and Nicolle experienced a slew of other challenges: 1. When they first joined the chat, they were asked not to refresh and told they were in queue. That implied they were waiting for a human, not a bot. This could confuse people and make them believe they’re being connected with a human. 2. On a related note, the UI showed a message suggesting Nike’s virtual agent was typing—something only a human can do. Again, this has the potential to confuse current and prospective customers. 3. The language choice throughout the experience was very formulaic and inconsistent. For example, the bot sent a mix of formal and informal messages. Given Nike’s incredible brand, this missed opportunity was disappointing for Ali and Nicolle. 4. When Ali and Nicolle paused for a moment to discuss their experience, they were given 90 seconds to respond before the chat timed-out. The thing is, everyone gets distracted. By being pushy, you cause feelings of panic. Ali and Nicolle also argued that there’s no reason a bot should time-out. It should infinitely wait for you. 5. Ali and Nicolle also criticized the old-school UI, which they argued looked like it was from 2005. Again, Nike is a prominent brand. Why didn’t the company invest in a simple and clean UI that delights users? 6. Any time Ali and Nicolle wanted to do something different from what they originally chose (e.g., ask about a product instead of tracking an order), they had to exit and reenter the chat, which creates an extra step for the user and slows them down. 7. People think they can toggle back to the main menu from a drop-down menu. Unfortunately, it’s not easy to find and it doesn’t do what it suggests. Instead of bringing you back to the main menu, the bot prompts you to reconfirm what you want to do—presenting only the original prompt and an “other” option. 8. Ali and Nicolle were pleased to see that people could save their chat history and refer back to it later. However, the ways people could save the chat—through a .txt file or by printing the chat—was highly unusual. Watch the rebuild in real-time ------------------------------ The full episode shows Ali and Nicolle rebuilding Nike’s bot in Voiceflow. They not only talk through everything from [generating utterances](https://www.voiceflow.com/blog/ai-assist-llms-voiceflow) and leveraging entities to using logic to create different paths, but they also highlight [conversation design best practices](https://www.voiceflow.com/blog/nike-conversation-design-best-practices) along the way. _Header photo by_ [_Terrance Barksdale_](https://www.pexels.com/photo/a-close-up-shot-of-a-pair-of-nike-shoes-6692038/) _._  RECOMMENDED [\ \ ### Expanding the definition of conversation design](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) PUBLISHED March 29, 2023 TOPICS IN THIS ARTICLE Elevating our work Learning and APIs [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/the-case-for-a-full-stack-conversation-designer) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The case for a full-stack conversation designer](https://www.voiceflow.com/pathways/the-case-for-a-full-stack-conversation-designer) [](https://www.voiceflow.com/pathways/the-complete-guide-to-better-cxd-workflows) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The Complete Guide to Better CxD Workflows](https://www.voiceflow.com/pathways/the-complete-guide-to-better-cxd-workflows) [](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 5 tips for better multimodal design](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/breaking-nike-chatbot#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/breaking-nike-chatbot#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How we're building Voiceflow’s machine learning platform from scratch | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) How we're building Voiceflow’s machine learning platform from scratch ===================================================================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) We recently embarked on building a machine learning (ML) team at Voiceflow as a strategic bet for the future. As we build our ML Platform at Voiceflow we wanted to share some key ideas that will guide us for Version 1. The platform will inevitably evolve and we hope the values will keep us aligned. Below are seven principles that guide our platform design, they are focused on how we build the platform rather than the team. We’ll be sharing our principles for hiring soon but for now, let’s stick to the platform. Our seven principles -------------------- * Customer Focus * Rapid prototyping * Infrastructure as Code, Templates and Test * Build for Real time * Leverage and contribute to open source * Separate model development and deployment * Hiring for three core technical skills  ### Customer focus When we hear that a company is launching a ML feature, building a ML product or hiring a ML team, we get a little cynical. Sprinkle some ML and AI into your presentation to VCs or managers and suddenly you’re hotly in demand. At Voiceflow we don’t see ML as a magic elixir, rather it’s closer to the search for one: something that can quickly turn into a wild goose chase if you aren’t focused and deliberate. So to keep us from running in circles we focus on our customer, how can adding ML make their lives easier and more productive.  We also recognize the opportunity cost of working on ML, every ML engineer means less funding to hire people to work on our current product and fix current problems. So we’re working closely with our customers to see how we bring some simple, but inspiring features to the platform that leverage our understanding of voice conversations. We’re focused on a highly active segment who’s raised their hand to become beta users and are passionate about the platform. After initial customer tests we can review the qualitative and quantitative data and give the project more time or shelve it. ### Rapid prototyping The beauty of software is the ability to automate and reuse. The darker side of beauty is perfectionism, and in the software world it’s building scalability and impeccable code quality when all you need is a prototype. This perfectionism is often in direct conflict with customer focus in the early days of a project since your goal is to validate, iterate and figure out what to build. As developers we know the struggle of balancing shipping features, minimizing technical debt and evolving requirements. When we want to develop a new capability at Voiceflow, we try to get there as quickly as possible both on the UX and technical side. This has led to us building some tools outside our core application to decouple from the application team and to get results faster. Once we validate that our ML model works and has a positive UX, we want to start integrating into our stack and building higher fidelity experiences. ### Infrastructure as code, templates and testing After an iteration of this process we built some tools and templates along the way. These include customer facing tools, helper functions and a standardized way to create new models. When a ML Engineer at Voiceflow wants to create a new model all they to do is have our MLCLI install and then type **ml create model -n my-mode**l and a sample model will be generated for them. Not only is it generated, it can immediately be deployed with a **ml deploy model -n my-model**. We invested time into the ML CLI because we want to focus on deploying models into prod as quickly as possible, not chasing environment variables and parameters that may linger through bad copy pasting. We’re aiming to automate the boring and error prone part of ML, allowing us to focus on customers and outcomes. Likewise we want developers and data scientists to be able to create isolated end to end Machine Learning environments quickly, so that’s why we created MLEnvs. Using the ML CLI you can create a new environment in 15 minutes, based on our terraform templates. This allows us to try new platform features while stopping stable environments from being contaminated with unstable additions. Each Model and MLEnv is generated using python and is backed by a set of terraform and terragrunt modules. Staging and Prod are always based off of the main branch, while Dev environments allow models to be deployed that are branch specific. This allows our ML team to work in parallel within the same MLEnv when testing changes or doing A/B tests. ### **A Real Time Platform**  One of Voiceflow’s key features is the ability for a team to collaborate and build voice apps in real time. To keep up with our users, we need to provide real time insights from our ML Models, so we incorporated this as a key platform requirement. We’re using sockets and an event driven architecture to handle user requests, process them and respond. This is a relatively niche approach to many ML systems that rely on REST endpoints for inference, which is another reason we decided to build the ML Platform in house. ### **Separate Model Development and Deployment**  This one may seem controversial, given the current movement on MLOps and unifying ML processes. There are a couple reasons for this idea. The first is the idea of starting somewhere. When you draw out a detailed, end to end MLOps architecture it gets overwhelming very quickly. The last time I did so, I had 50+ boxes, 100+ arrows and had to divide it into 14 subsections to keep it organized. So when building for Voiceflow, logically separating model development and deployment makes it easier to focus on building a V1 of a platform. The second reason is that model development is a much more mature space than model deployment and serving. Many platforms like Databricks, Sagemaker, Azure ML and Vertex AI are adding features to create a better model hosting experience, but many are still premature or focused on the model development portion. Due to this situation, decoupling these two components will allow us to focus on building in house where the industry is still maturing while leveraging key SAAS and cloud offerings where it makes sense.  The third reason is that model development and deployment are logically separated, but with the intention of integrating seamlessly. We are still working towards a fully integrated MLOps model, but it’s too early to tightly couple to a given model development framework and methodology. Once we have enough production data, we can make more definite decisions of where we need key integrations between model development and deployment. For now we assume that models will be developed in some pre-existing (cloud/local) environment, which must supply the platform with this information before deployment. ### Leverage and contribute to open source  If we were building a ML platform in 2016 our approach would look very different. Tooling and model architectures were much less common in the open source community, with tensorflow recently released, and pytorch and transformers still early in their development. Now we have many large language models available to us, countless ML frameworks and python packages galore. This has brought down the barrier to entry immensely within the ML space and we’re grateful to all the developers and companies which have contributed to a flourishing community. As we continue to grow as a team we will increase our open source contributions through articles, code snippets and packages. Our first open source post came from our MLCLI development where we shared [some key ideas](https://denyslinkov.medium.com/scaling-your-python-cli-87f74a0fb6cb) and a [repo](https://github.com/Diophontine/ClickCLI) of building a CLI. We also posted about data ingestion using Terraform and [Google PubSub](https://denyslinkov.medium.com/how-to-stream-data-from-pub-sub-to-big-query-with-terraform-37573c729d9c) . ### Hiring for three core technical skills Many times when companies are starting their ML or Data Science teams they don’t know what they want. They want a mixture of business outcomes and technological flare with poorly defined requirements. For a startup it’s even harder because as a company _you_ don’t know what you want yet from a product. So we stick to three things that we know are true. * We want ML to improve our platform and customers’ voice experience * We need to develop capabilities in the NLU, NLP and Recommender system spaces * We need flexibility from our platform and people to accommodate rapidly changing use cases As the company and market matures, our core ML offerings will change, but we’ll continue to stick to these ideas that will guide our team’s growth. So what technical skills are we looking to hire for? * Data science in our core competency areas (NLU, NLP, Rec Systems) * Data engineering * Cloud/Platform Engineering The important thing we recognize is that there are very few people with expertise in all three areas, they are unicorns of this space. So instead we are looking for people with strong experience in one of the three areas, and exposure to the other two. Combined with the key value of flexibility, we are looking for people who are interested in solving interesting problems and will grow with the team. Where we're at -------------- We are currently three months into our ML Platform journey and wanted to share what we have built so far: * V1 of a ML Platform that can deploy models end to end in 5 min * A fairly comprehensive ML CLI that can handle versioning, model promotions and the creation of isolated ML Environments * Five models exiting the R&D stage and two models in a customer validation stage * A comprehensive ML Strategy * A highly capable engineering team We’re excited about what we’ve built, but we are even more excited about what’s to come. Where we're going ----------------- As our team continues evolve we want to focus on three things: * Work closely with our core creators and enterprise customers to improve their user experience * Build a world class ML platform that can scale to millions of users and a comprehensive set of use cases. * Contributing to the ML Community through articles, talks and open source projects As we grow our team, we’d like people who are excited about these three things and bring a unique perspective to tackling these challenges. If you’re excited about these things send us a message at [engineering@voiceflow.com](mailto:engineering@voiceflow.com) Wrapping up ----------- After six months of building our ML platform, we wanted to share the thinking that has guided us so far and will guide our future endeavours. ### Customer focus When we hear that a company is launching a ML feature, building a ML product or hiring a ML team, we get a little cynical. Sprinkle some ML and AI into your presentation to VCs or managers and suddenly you’re hotly in demand. At Voiceflow we don’t see ML as a magic elixir, rather it’s closer to the search for one: something that can quickly turn into a wild goose chase if you aren’t focused and deliberate. So to keep us from running in circles we focus on our customer, how can adding ML make their lives easier and more productive.  We also recognize the opportunity cost of working on ML, every ML engineer means less funding to hire people to work on our current product and fix current problems. So we’re working closely with our customers to see how we bring some simple, but inspiring features to the platform that leverage our understanding of voice conversations. We’re focused on a highly active segment who’s raised their hand to become beta users and are passionate about the platform. After initial customer tests we can review the qualitative and quantitative data and give the project more time or shelve it. ### Rapid prototyping The beauty of software is the ability to automate and reuse. The darker side of beauty is perfectionism, and in the software world it’s building scalability and impeccable code quality when all you need is a prototype. This perfectionism is often in direct conflict with customer focus in the early days of a project since your goal is to validate, iterate and figure out what to build. As developers we know the struggle of balancing shipping features, minimizing technical debt and evolving requirements. When we want to develop a new capability at Voiceflow, we try to get there as quickly as possible both on the UX and technical side. This has led to us building some tools outside our core application to decouple from the application team and to get results faster. Once we validate that our ML model works and has a positive UX, we want to start integrating into our stack and building higher fidelity experiences. ### Infrastructure as code, templates and testing After an iteration of this process we built some tools and templates along the way. These include customer facing tools, helper functions and a standardized way to create new models. When a ML Engineer at Voiceflow wants to create a new model all they to do is have our MLCLI install and then type **ml create model -n my-mode**l and a sample model will be generated for them. Not only is it generated, it can immediately be deployed with a **ml deploy model -n my-model**. We invested time into the ML CLI because we want to focus on deploying models into prod as quickly as possible, not chasing environment variables and parameters that may linger through bad copy pasting. We’re aiming to automate the boring and error prone part of ML, allowing us to focus on customers and outcomes. Likewise we want developers and data scientists to be able to create isolated end to end Machine Learning environments quickly, so that’s why we created MLEnvs. Using the ML CLI you can create a new environment in 15 minutes, based on our terraform templates. This allows us to try new platform features while stopping stable environments from being contaminated with unstable additions. Each Model and MLEnv is generated using python and is backed by a set of terraform and terragrunt modules. Staging and Prod are always based off of the main branch, while Dev environments allow models to be deployed that are branch specific. This allows our ML team to work in parallel within the same MLEnv when testing changes or doing A/B tests. ### **A Real Time Platform**  One of Voiceflow’s key features is the ability for a team to collaborate and build voice apps in real time. To keep up with our users, we need to provide real time insights from our ML Models, so we incorporated this as a key platform requirement. We’re using sockets and an event driven architecture to handle user requests, process them and respond. This is a relatively niche approach to many ML systems that rely on REST endpoints for inference, which is another reason we decided to build the ML Platform in house. ### **Separate Model Development and Deployment**  This one may seem controversial, given the current movement on MLOps and unifying ML processes. There are a couple reasons for this idea. The first is the idea of starting somewhere. When you draw out a detailed, end to end MLOps architecture it gets overwhelming very quickly. The last time I did so, I had 50+ boxes, 100+ arrows and had to divide it into 14 subsections to keep it organized. So when building for Voiceflow, logically separating model development and deployment makes it easier to focus on building a V1 of a platform. The second reason is that model development is a much more mature space than model deployment and serving. Many platforms like Databricks, Sagemaker, Azure ML and Vertex AI are adding features to create a better model hosting experience, but many are still premature or focused on the model development portion. Due to this situation, decoupling these two components will allow us to focus on building in house where the industry is still maturing while leveraging key SAAS and cloud offerings where it makes sense.  The third reason is that model development and deployment are logically separated, but with the intention of integrating seamlessly. We are still working towards a fully integrated MLOps model, but it’s too early to tightly couple to a given model development framework and methodology. Once we have enough production data, we can make more definite decisions of where we need key integrations between model development and deployment. For now we assume that models will be developed in some pre-existing (cloud/local) environment, which must supply the platform with this information before deployment. ### Leverage and contribute to open source  If we were building a ML platform in 2016 our approach would look very different. Tooling and model architectures were much less common in the open source community, with tensorflow recently released, and pytorch and transformers still early in their development. Now we have many large language models available to us, countless ML frameworks and python packages galore. This has brought down the barrier to entry immensely within the ML space and we’re grateful to all the developers and companies which have contributed to a flourishing community. As we continue to grow as a team we will increase our open source contributions through articles, code snippets and packages. Our first open source post came from our MLCLI development where we shared [some key ideas](https://denyslinkov.medium.com/scaling-your-python-cli-87f74a0fb6cb) and a [repo](https://github.com/Diophontine/ClickCLI) of building a CLI. We also posted about data ingestion using Terraform and [Google PubSub](https://denyslinkov.medium.com/how-to-stream-data-from-pub-sub-to-big-query-with-terraform-37573c729d9c) . ### Hiring for three core technical skills Many times when companies are starting their ML or Data Science teams they don’t know what they want. They want a mixture of business outcomes and technological flare with poorly defined requirements. For a startup it’s even harder because as a company _you_ don’t know what you want yet from a product. So we stick to three things that we know are true. * We want ML to improve our platform and customers’ voice experience * We need to develop capabilities in the NLU, NLP and Recommender system spaces * We need flexibility from our platform and people to accommodate rapidly changing use cases As the company and market matures, our core ML offerings will change, but we’ll continue to stick to these ideas that will guide our team’s growth. So what technical skills are we looking to hire for? * Data science in our core competency areas (NLU, NLP, Rec Systems) * Data engineering * Cloud/Platform Engineering The important thing we recognize is that there are very few people with expertise in all three areas, they are unicorns of this space. So instead we are looking for people with strong experience in one of the three areas, and exposure to the other two. Combined with the key value of flexibility, we are looking for people who are interested in solving interesting problems and will grow with the team. Where we're at -------------- We are currently three months into our ML Platform journey and wanted to share what we have built so far: * V1 of a ML Platform that can deploy models end to end in 5 min * A fairly comprehensive ML CLI that can handle versioning, model promotions and the creation of isolated ML Environments * Five models exiting the R&D stage and two models in a customer validation stage * A comprehensive ML Strategy * A highly capable engineering team We’re excited about what we’ve built, but we are even more excited about what’s to come. Where we're going ----------------- As our team continues evolve we want to focus on three things: * Work closely with our core creators and enterprise customers to improve their user experience * Build a world class ML platform that can scale to millions of users and a comprehensive set of use cases. * Contributing to the ML Community through articles, talks and open source projects As we grow our team, we’d like people who are excited about these three things and bring a unique perspective to tackling these challenges. If you’re excited about these things send us a message at [engineering@voiceflow.com](mailto:engineering@voiceflow.com) Wrapping up ----------- After six months of building our ML platform, we wanted to share the thinking that has guided us so far and will guide our future endeavours.  RECOMMENDED [\ \ ### 17 prompts for building AI apps in Voiceflow](https://www.voiceflow.com/pathways/prompts-for-building-ai-apps) PUBLISHED February 23, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/using-machine-learning-to-analyze-customer-support-conversations) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Using machine learning to analyze customer support conversations](https://www.voiceflow.com/pathways/using-machine-learning-to-analyze-customer-support-conversations) [](https://www.voiceflow.com/pathways/art-of-debugging) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### The art of debugging](https://www.voiceflow.com/pathways/art-of-debugging) [](https://www.voiceflow.com/pathways/5-principles-for-good-natural-language-understanding-nlu-design) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### 5 principles for good natural language understanding (NLU) design](https://www.voiceflow.com/pathways/5-principles-for-good-natural-language-understanding-nlu-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/building-voiceflows-machine-learning-platform-from-scratch#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/building-voiceflows-machine-learning-platform-from-scratch#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # AI automation on a budget: Getting started with high ROI use cases | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) AI automation on a budget: Getting started with high ROI use cases ================================================================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) If your AI project feels like building an Iron Man suit out of scraps—you’re not alone. Right now, everyone wants teams to spin AI miracles out of dust and dreams. But we can’t all be Tony Stark, the genius AI wizard. Thankfully, most AI teams aren’t trapped in the desert attempting to create customer support chatbots with rusty metal. But with budget constraints, the risks of deploying large language models (LLMs), and the technical burden on your stack, producing AI agents can feel like you’re fighting for everything you need to succeed. So let’s take a step back and analyze the costs of building, deploying, and expanding effective AI agents. This will not be a listicle on the costs of each line item—although, if you’d like to see something like that, [let me know](https://www.linkedin.com/in/denyslinkov/) . Instead, we’ll be talking about how to approach strategizing and budgeting the time and resources necessary to build a sustainable AI operation. We’ll explore the strategy and costs associated with scaling agents across new use cases, how to weigh the cost of tools and technology like GPUs, choosing the right models (and why you might be choosing wrong), and how to budget for innovation and experimentation.  If you’re more of an audio-visual learner, we suggest tuning into our [webinar](https://www.youtube.com/watch?v=k6rnd_SGNQ4) on this topic (featuring Colin Guilfoyle, VP for Customer Support at Trilogy, who managed to automate over 60% of their customer support in under two months). Stick around for the full breakdown below. Starting small before scaling agents and use cases --------------------------------------------------- Starting with a simple agent is the surefire way to gain traction, before scaling to numerous agents and use cases. Because when you try to walk before you crawl or run before you walk, you’re going to trip over your own feet. \[Starting a new AI agent? [We’ve got you](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here)\ \] But running before you walk isn’t as common an issue for most teams. Most companies struggle to move their first AI proof of concept (POC) out of [production purgatory](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) —it’s often called the cold start problem, named for the difficulty in starting old internal combustion engines when the gas is cold. Once the gas is hot, turning the engine on and off is a breeze. Similarly, once teams have launched one AI agent, they find it much faster and easier to expand to several agents or several use cases. But you don’t have to take my word for it. Colin Guilfoyle, VP for Customer Support at Trilogy, has done it. His team started with one AI build—they call it the Atlas core, very Tony Stark-coded—and have used that build to expand to 90 customer support lines that handle AI support across products. In order to scale production to this level and keep all their agents working smoothly once they got there, they needed to start with how they organized their team. Because there’s so many product lines that require support, their team is made up of code-focused, senior product specialists. These directly responsible individuals (DRIs) are given a subset of products to analyze each week, including how well the customer support automations and tickets are performing. Then, they replicate what goes right and refine what goes wrong—from refining knowledge base searches, training models, ticket raising, building the right retrieval-augmented generation (RAG), and integrating the right tools to solve specific product issues. They apply their best practices to their Atlas core, which they use as a foundation for building and expanding to new agents and products, and the process continues. By effectively dividing and replicating their agents, while continually monitoring and improving them, Trilogy is on track to support 65% of their customer inquiries using AI agents. The next phase of their expansion includes replacing human support on L2 troubleshooting and automating customer changes in the system securely. \[[Read more about how Trilogy automated 60% of their customer support in 12 weeks.\]](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) While you may not have the budget to expand your team or want to create 90 agents, Trilogy’s approach to iterative improvement and replication is a wise one to consider when weighing costs. When it comes to scaling your agent, whether that’s launching more agents or expanding your agent’s current capabilities, [there’s a lot you can do](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) . Start with your minimal viable product (aka your single use case AI agent) and slowly layer in the use cases that expand its problem-solving. You’ll know it’s time to add new use cases when you’ve mastered the one you’re currently on. In fact, you’ll find the cost of ownership remains quite sustainable if you’ve built processes and integrated tools that are growing according to your needs.  As you scale, you’ll experience savings costs when it comes to human support hours. Trilogy reduced their human support hours by 60% after 12 weeks, freeing support staff to focus their efforts more efficiently. In the long run, scaling sustainably allows your support needs to grow as you do while saving your team the one resource they can’t get back—time. “How much should I budget for this?” ------------------------------------ People always ask me this question. And the answer is an unsatisfactory one—it depends. In the case of Trilogy, a large-scale enterprise that streamlines operations and support for hundreds of clients, they use a variety of tools ranging in cost, including: * **Amazon Web Services:** For compute, hosting, backups, etc. They also use Lambas, the first port of call for tickets, which categorizes the issue and offers a response based on a knowledge base * **LLMs:** some text * OpenAI, to generate responses * Enki, to cross-check LLM responses and choose the best one. If there isn’t a viable answer, Enki kicks the response back up a step to generate a better one * Anthropic * Occasionally, Gemini * **Zendesk:** for managing and routing tickets * **Voiceflow:** to design, produce, and launch AI agents Large companies with AI support across channels can spend over $100,000 on LLMs, tokens, and associated AI costs. Similarly, it can cost up to the same per year on Amazon Web Services. And that doesn’t include the cost of engineering a sophisticated support system that automatically generates and cross-checks AI responses across 90 agents. When you’re talking about tooling, people, and time, it’s hard to make estimates about how much you should spend on AI agents unless we talk through the minute details of your circumstances. (Shameless plug for my colleague [Peter Isaacs](https://www.linkedin.com/in/isaacspeter) , who would be stoked to [talk through your AI automation journey in painstaking detail](https://www.voiceflow.com/demo) .) My advice is to talk to your technology and tooling vendors, ask colleagues in your field, and do a lot of research. We’ve also included a RAG cost estimation template for [you](https://docs.google.com/spreadsheets/d/1Vh6NI7Ssy4aILW6O044cYDcEgrUweZHR/edit?gid=334590458#gid=334590458) to forecast costs for your next project. 5 tips for choosing your LLM models (spoiler: versions are underrated) ---------------------------------------------------------------------- The number of LLMs available has exploded in the last year. The influx of [choices](https://en.wikipedia.org/wiki/Large_language_model#List) brings questions about which ones you should be using based on your use cases. There are five things you can do right now to understand models and choose the right ones. 1. **Rely on RAG:** Build a robust [knowledge base](https://www.voiceflow.com/pathways/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) and use RAG to pull that information into your LLM when it’s generating responses. This enhances the effectiveness of your LLM by providing useful context for your responses from your datasets, documentation, and FAQs. Don’t underestimate how powerful RAG can be. The more context your AI agent has, the fewer LLM calls it needs to generate a relevant response, the more cost-effective your agent can be. 2. **Model versions matter:** Many people complain about OpenAI’s GPT, claiming that it’s getting worse with each new version—but it’s unlikely OpenAI is releasing worse versions of their flagship product. What’s happening is that the newest version no longer works for your particular use case. Don’t trust the AI leaderboard. Spend time on prompt engineering. Do multiple tests before you land on the model and version for your use case. For many projects, using an older LLM version will offer results on par with the newest version and be more budget-friendly. 3. **Use models to cross-check responses:** As previously mentioned, Trilogy [layers](https://www.voiceflow.com/pathways/layers-ai-assistant-llms) their LLMs atop one another. This [tiered approach](https://www.voiceflow.com/pathways/layers-ai-assistant-llms) would begin with your agent using an NLU to match your user's response to an intent you’ve already mapped, like collecting account information or surfacing a help link. If your agent can’t find a match, it moves down the order priority and uses RAG to search your knowledge base and find sections of documents you’ve uploaded that have the closest semantic similarity. If it finds a match, it’ll use your LLM to generate an answer to address your user’s intent. Then cross-check that response with a different LLM, generating two responses and choosing the best one. This process has multiple benefits, including better quality control, more accurate AI responses, reduction in hallucinations, more concise responses, and improved data collection. 4. **Test different models for different use cases:** Using a tiered approach can also help you test which models work best for your use cases. If you find that one model consistently “wins” at the quality control cross-check, it might be worth investing in that LLM over the other. For classification tasks, some use cases are better suited to GPT-4, but Haiku, one of the cheapest models, also performs well and should not be discounted. The newest version of Claude may not work as well for your support tasks as the previous one. The key is to test, evaluate, and iterate as you work with different models and versions. 5. **Weigh the cost of prompt engineering vs. upgrading your model:** This is where teams need to make decisions on accuracy, development costs, and runtime costs. You can put a massive context window into GPT-4 or Claude Sonnet 3.5 and you'd be spending a couple of dollars per interaction. You could also use smaller models but you’d need a way to measure the tradeoff—the cost of running the model compared to the business gains of increased latency. This is where having good evaluations is important. Improving the prompts also takes time for both the prompt engineering and surrounding systems. You have to make a large number of LLM calls to actually see a return on investment. You have to weigh how much time that prompt engineering and evaluation is worth. You might increase your LLM costs by upgrading your LLM, but that might be worth it if you’ve already optimized your prompts. Choosing the right LLMs requires thoughtful intention. Use RAG to provide context, making whichever LLM you choose more efficient and cost-effective. Different model versions work better for different tasks, so don’t be afraid to use older versions and cross-check responses to ensure quality and accuracy. You should be balancing the costs of prompt engineering against your models to help you achieve the best performance within your budget. To GPU or not to GPU? That is the question. --------------------------------------------- A graphics processing unit (or GPU) has made the modern world of AI possible. Compared to CPUs, GPUs have many smaller processing cores designed to work in parallel. As a result, LLMs and other Gen AI models use [GPUs to perform massive mathematical and operational tasks](https://theconversation.com/what-is-a-gpu-an-expert-explains-the-chips-powering-the-ai-boom-and-why-theyre-worth-trillions-224637) quickly and simultaneously. Today, enterprise, consumer-grade GPUs serve multiple uses, from model building and low-level testing to deep learning operations, like biometric recognition. We won’t go into all of the GPUs out there, because [there are a bunch](https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference) . But they are typically divided into three categories useful for enterprise: 1. **Consumer-grade GPUs:** Typically sold for gaming, but have been used for local model training and deployment, particularly open source models. 2. **Cloud-based GPUs:** Many cloud providers let you rent GPUs ranging from entry-level (T4s) to state-of-the-art clusters (H100s). A great place to get started when experimenting, training, or running models. 3. **Datacenter GPU clusters:** For larger companies, procuring your own GPU cluster or server becomes an option. These can be just the hardware installed in a data center or platform offering to get started faster. The question is, do you need one? GPUs are expensive resources. For many, using proprietary models and a serverless approach gets them far enough in their AI journey to solve for most use cases. But for the folks interested in AI innovation and playing with bigger, faster, complex AI projects, a GPU has been a critical asset, leading to some supply challenges. Choosing the right hardware for a use case is essential. It’s overkill to build a cluster of H100 GPUs to run a seven billion model inference. It takes a lot of engineering hours to host a model, optimize inference, batch queries, and put up guardrails to make it run efficiently. Instead of investing in a GPU—and spending months installing and deploying models-—my advice is to leave it to platforms until use cases and costs are better defined. When you’re building a large-scale AI operation, hiring a team to run innovation makes sense. But for most use cases, avoid the complexity and use CPUs and smaller models more often. Bigger isn’t always better. Add research, eval-driven development, and experimentation to your budget ------------------------------------------------------------------------- The conversation around AI seems to center around avoiding risk and not getting left behind. It’s a pretty negative approach to an exciting and novel technology, and that affects how we evaluate the value of AI and budget for it. But a budget represents more than just money, it represents time, effort, and strategic thinking. Instead of thinking about all the ways things can go wrong, invite your teams (and even your leaders) to budget for: * **Keeping up with AI:** Budget the time necessary for your team to understand the AI landscape. Colin’s team at Trilogy spends two hours a week on Twitter, LinkedIn, and Reddit, learning, engaging with new information, and expanding their AI knowledge. Because of this, they’re proactive about addressing new use cases and experimenting with tools. When executives come to them with requests, they’re ready to respond, either with a plan to adopt new ideas or an explanation of their previous experiments. Budgeting time for AI makes their team more productive, knowledgeable, and adaptable to change. * **Evaluation-driven development**: AI projects aren’t always clear on returns, but that hasn’t stopped every company under the sun from adopting some form of AI technology. So, if we’ve already accepted that, it would serve us to evaluate the ROI of AI accordingly. Budget your engineering prowess behind [evaluation-driven development](https://levelup.gitconnected.com/evaluation-driven-development-the-swiss-army-knife-for-rag-pipelines-dba24218d47e) (EDD), a methodology for guiding the development of LLM-backed projects using a set of task-specific evaluations, like expected prompts, contexts, and outputs as references. These evaluations guide prompt engineering, model selection, and fine-tuning to help you quickly measure improvements or regressions as your project changes. Don’t just measure how many tickets you automate. Determine what parameters you’d evaluate success in, and work backwards. * **Experimentation and known problems:** Finally, you need a budget to experiment and roll out new tools, tech, and use cases. There needs to be support from leadership for this. AI moves quickly and if your AI team is keeping up with the changes, they’ll also need a budget to experiment and react to those changes. On the other hand, don’t let shiny new tools and ideas have you too focused on [problems under the streetlight](https://en.wikipedia.org/wiki/Streetlight_effect) instead of known issues AI could solve. It’s not too late to invest in the time, evaluation, and experimentation you need to succeed with AI. The most important problems aren’t the easiest ones to solve, but an organization that is forward-thinking about AI will see ROI faster than a reactive one. Let’s build Iron Man-level AI on a start-up budget --------------------------------------------------- Remember, you don't need to be Tony Stark to achieve results with AI. By starting small and scaling up, carefully budgeting for tools and technology, and prioritizing continuous learning and experimentation, you can make the most of your budget, no matter the size. If you’re more of an audio-visual learner, we suggest tuning into our [webinar](https://www.youtube.com/watch?v=k6rnd_SGNQ4) on this topic (featuring Colin Guilfoyle, VP for Customer Support at Trilogy, who managed to automate over 60% of their customer support in under two months). Stick around for the full breakdown below. Starting small before scaling agents and use cases --------------------------------------------------- Starting with a simple agent is the surefire way to gain traction, before scaling to numerous agents and use cases. Because when you try to walk before you crawl or run before you walk, you’re going to trip over your own feet. \[Starting a new AI agent? [We’ve got you](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here)\ \] But running before you walk isn’t as common an issue for most teams. Most companies struggle to move their first AI proof of concept (POC) out of [production purgatory](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) —it’s often called the cold start problem, named for the difficulty in starting old internal combustion engines when the gas is cold. Once the gas is hot, turning the engine on and off is a breeze. Similarly, once teams have launched one AI agent, they find it much faster and easier to expand to several agents or several use cases. But you don’t have to take my word for it. Colin Guilfoyle, VP for Customer Support at Trilogy, has done it. His team started with one AI build—they call it the Atlas core, very Tony Stark-coded—and have used that build to expand to 90 customer support lines that handle AI support across products. In order to scale production to this level and keep all their agents working smoothly once they got there, they needed to start with how they organized their team. Because there’s so many product lines that require support, their team is made up of code-focused, senior product specialists. These directly responsible individuals (DRIs) are given a subset of products to analyze each week, including how well the customer support automations and tickets are performing. Then, they replicate what goes right and refine what goes wrong—from refining knowledge base searches, training models, ticket raising, building the right retrieval-augmented generation (RAG), and integrating the right tools to solve specific product issues. They apply their best practices to their Atlas core, which they use as a foundation for building and expanding to new agents and products, and the process continues. By effectively dividing and replicating their agents, while continually monitoring and improving them, Trilogy is on track to support 65% of their customer inquiries using AI agents. The next phase of their expansion includes replacing human support on L2 troubleshooting and automating customer changes in the system securely. \[[Read more about how Trilogy automated 60% of their customer support in 12 weeks.\]](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) While you may not have the budget to expand your team or want to create 90 agents, Trilogy’s approach to iterative improvement and replication is a wise one to consider when weighing costs. When it comes to scaling your agent, whether that’s launching more agents or expanding your agent’s current capabilities, [there’s a lot you can do](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) . Start with your minimal viable product (aka your single use case AI agent) and slowly layer in the use cases that expand its problem-solving. You’ll know it’s time to add new use cases when you’ve mastered the one you’re currently on. In fact, you’ll find the cost of ownership remains quite sustainable if you’ve built processes and integrated tools that are growing according to your needs.  As you scale, you’ll experience savings costs when it comes to human support hours. Trilogy reduced their human support hours by 60% after 12 weeks, freeing support staff to focus their efforts more efficiently. In the long run, scaling sustainably allows your support needs to grow as you do while saving your team the one resource they can’t get back—time. “How much should I budget for this?” ------------------------------------ People always ask me this question. And the answer is an unsatisfactory one—it depends. In the case of Trilogy, a large-scale enterprise that streamlines operations and support for hundreds of clients, they use a variety of tools ranging in cost, including: * **Amazon Web Services:** For compute, hosting, backups, etc. They also use Lambas, the first port of call for tickets, which categorizes the issue and offers a response based on a knowledge base * **LLMs:** some text * OpenAI, to generate responses * Enki, to cross-check LLM responses and choose the best one. If there isn’t a viable answer, Enki kicks the response back up a step to generate a better one * Anthropic * Occasionally, Gemini * **Zendesk:** for managing and routing tickets * **Voiceflow:** to design, produce, and launch AI agents Large companies with AI support across channels can spend over $100,000 on LLMs, tokens, and associated AI costs. Similarly, it can cost up to the same per year on Amazon Web Services. And that doesn’t include the cost of engineering a sophisticated support system that automatically generates and cross-checks AI responses across 90 agents. When you’re talking about tooling, people, and time, it’s hard to make estimates about how much you should spend on AI agents unless we talk through the minute details of your circumstances. (Shameless plug for my colleague [Peter Isaacs](https://www.linkedin.com/in/isaacspeter) , who would be stoked to [talk through your AI automation journey in painstaking detail](https://www.voiceflow.com/demo) .) My advice is to talk to your technology and tooling vendors, ask colleagues in your field, and do a lot of research. We’ve also included a RAG cost estimation template for [you](https://docs.google.com/spreadsheets/d/1Vh6NI7Ssy4aILW6O044cYDcEgrUweZHR/edit?gid=334590458#gid=334590458) to forecast costs for your next project. 5 tips for choosing your LLM models (spoiler: versions are underrated) ---------------------------------------------------------------------- The number of LLMs available has exploded in the last year. The influx of [choices](https://en.wikipedia.org/wiki/Large_language_model#List) brings questions about which ones you should be using based on your use cases. There are five things you can do right now to understand models and choose the right ones. 1. **Rely on RAG:** Build a robust [knowledge base](https://www.voiceflow.com/pathways/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) and use RAG to pull that information into your LLM when it’s generating responses. This enhances the effectiveness of your LLM by providing useful context for your responses from your datasets, documentation, and FAQs. Don’t underestimate how powerful RAG can be. The more context your AI agent has, the fewer LLM calls it needs to generate a relevant response, the more cost-effective your agent can be. 2. **Model versions matter:** Many people complain about OpenAI’s GPT, claiming that it’s getting worse with each new version—but it’s unlikely OpenAI is releasing worse versions of their flagship product. What’s happening is that the newest version no longer works for your particular use case. Don’t trust the AI leaderboard. Spend time on prompt engineering. Do multiple tests before you land on the model and version for your use case. For many projects, using an older LLM version will offer results on par with the newest version and be more budget-friendly. 3. **Use models to cross-check responses:** As previously mentioned, Trilogy [layers](https://www.voiceflow.com/pathways/layers-ai-assistant-llms) their LLMs atop one another. This [tiered approach](https://www.voiceflow.com/pathways/layers-ai-assistant-llms) would begin with your agent using an NLU to match your user's response to an intent you’ve already mapped, like collecting account information or surfacing a help link. If your agent can’t find a match, it moves down the order priority and uses RAG to search your knowledge base and find sections of documents you’ve uploaded that have the closest semantic similarity. If it finds a match, it’ll use your LLM to generate an answer to address your user’s intent. Then cross-check that response with a different LLM, generating two responses and choosing the best one. This process has multiple benefits, including better quality control, more accurate AI responses, reduction in hallucinations, more concise responses, and improved data collection. 4. **Test different models for different use cases:** Using a tiered approach can also help you test which models work best for your use cases. If you find that one model consistently “wins” at the quality control cross-check, it might be worth investing in that LLM over the other. For classification tasks, some use cases are better suited to GPT-4, but Haiku, one of the cheapest models, also performs well and should not be discounted. The newest version of Claude may not work as well for your support tasks as the previous one. The key is to test, evaluate, and iterate as you work with different models and versions. 5. **Weigh the cost of prompt engineering vs. upgrading your model:** This is where teams need to make decisions on accuracy, development costs, and runtime costs. You can put a massive context window into GPT-4 or Claude Sonnet 3.5 and you'd be spending a couple of dollars per interaction. You could also use smaller models but you’d need a way to measure the tradeoff—the cost of running the model compared to the business gains of increased latency. This is where having good evaluations is important. Improving the prompts also takes time for both the prompt engineering and surrounding systems. You have to make a large number of LLM calls to actually see a return on investment. You have to weigh how much time that prompt engineering and evaluation is worth. You might increase your LLM costs by upgrading your LLM, but that might be worth it if you’ve already optimized your prompts. Choosing the right LLMs requires thoughtful intention. Use RAG to provide context, making whichever LLM you choose more efficient and cost-effective. Different model versions work better for different tasks, so don’t be afraid to use older versions and cross-check responses to ensure quality and accuracy. You should be balancing the costs of prompt engineering against your models to help you achieve the best performance within your budget. To GPU or not to GPU? That is the question. --------------------------------------------- A graphics processing unit (or GPU) has made the modern world of AI possible. Compared to CPUs, GPUs have many smaller processing cores designed to work in parallel. As a result, LLMs and other Gen AI models use [GPUs to perform massive mathematical and operational tasks](https://theconversation.com/what-is-a-gpu-an-expert-explains-the-chips-powering-the-ai-boom-and-why-theyre-worth-trillions-224637) quickly and simultaneously. Today, enterprise, consumer-grade GPUs serve multiple uses, from model building and low-level testing to deep learning operations, like biometric recognition. We won’t go into all of the GPUs out there, because [there are a bunch](https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference) . But they are typically divided into three categories useful for enterprise: 1. **Consumer-grade GPUs:** Typically sold for gaming, but have been used for local model training and deployment, particularly open source models. 2. **Cloud-based GPUs:** Many cloud providers let you rent GPUs ranging from entry-level (T4s) to state-of-the-art clusters (H100s). A great place to get started when experimenting, training, or running models. 3. **Datacenter GPU clusters:** For larger companies, procuring your own GPU cluster or server becomes an option. These can be just the hardware installed in a data center or platform offering to get started faster. The question is, do you need one? GPUs are expensive resources. For many, using proprietary models and a serverless approach gets them far enough in their AI journey to solve for most use cases. But for the folks interested in AI innovation and playing with bigger, faster, complex AI projects, a GPU has been a critical asset, leading to some supply challenges. Choosing the right hardware for a use case is essential. It’s overkill to build a cluster of H100 GPUs to run a seven billion model inference. It takes a lot of engineering hours to host a model, optimize inference, batch queries, and put up guardrails to make it run efficiently. Instead of investing in a GPU—and spending months installing and deploying models-—my advice is to leave it to platforms until use cases and costs are better defined. When you’re building a large-scale AI operation, hiring a team to run innovation makes sense. But for most use cases, avoid the complexity and use CPUs and smaller models more often. Bigger isn’t always better. Add research, eval-driven development, and experimentation to your budget ------------------------------------------------------------------------- The conversation around AI seems to center around avoiding risk and not getting left behind. It’s a pretty negative approach to an exciting and novel technology, and that affects how we evaluate the value of AI and budget for it. But a budget represents more than just money, it represents time, effort, and strategic thinking. Instead of thinking about all the ways things can go wrong, invite your teams (and even your leaders) to budget for: * **Keeping up with AI:** Budget the time necessary for your team to understand the AI landscape. Colin’s team at Trilogy spends two hours a week on Twitter, LinkedIn, and Reddit, learning, engaging with new information, and expanding their AI knowledge. Because of this, they’re proactive about addressing new use cases and experimenting with tools. When executives come to them with requests, they’re ready to respond, either with a plan to adopt new ideas or an explanation of their previous experiments. Budgeting time for AI makes their team more productive, knowledgeable, and adaptable to change. * **Evaluation-driven development**: AI projects aren’t always clear on returns, but that hasn’t stopped every company under the sun from adopting some form of AI technology. So, if we’ve already accepted that, it would serve us to evaluate the ROI of AI accordingly. Budget your engineering prowess behind [evaluation-driven development](https://levelup.gitconnected.com/evaluation-driven-development-the-swiss-army-knife-for-rag-pipelines-dba24218d47e) (EDD), a methodology for guiding the development of LLM-backed projects using a set of task-specific evaluations, like expected prompts, contexts, and outputs as references. These evaluations guide prompt engineering, model selection, and fine-tuning to help you quickly measure improvements or regressions as your project changes. Don’t just measure how many tickets you automate. Determine what parameters you’d evaluate success in, and work backwards. * **Experimentation and known problems:** Finally, you need a budget to experiment and roll out new tools, tech, and use cases. There needs to be support from leadership for this. AI moves quickly and if your AI team is keeping up with the changes, they’ll also need a budget to experiment and react to those changes. On the other hand, don’t let shiny new tools and ideas have you too focused on [problems under the streetlight](https://en.wikipedia.org/wiki/Streetlight_effect) instead of known issues AI could solve. It’s not too late to invest in the time, evaluation, and experimentation you need to succeed with AI. The most important problems aren’t the easiest ones to solve, but an organization that is forward-thinking about AI will see ROI faster than a reactive one. Let’s build Iron Man-level AI on a start-up budget --------------------------------------------------- Remember, you don't need to be Tony Stark to achieve results with AI. By starting small and scaling up, carefully budgeting for tools and technology, and prioritizing continuous learning and experimentation, you can make the most of your budget, no matter the size.  RECOMMENDED [\ \ ### Building your AI agents like products: A blueprint from POC purgatory to production](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production) PUBLISHED July 8, 2024 TOPICS IN THIS ARTICLE Exceptional work [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/voiceflow-raises-3-5-million-to-build-the-future-of-voice-interface-design) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow raises $3.5 million to build the future of voice interface design](https://www.voiceflow.com/pathways/voiceflow-raises-3-5-million-to-build-the-future-of-voice-interface-design) [](https://www.voiceflow.com/pathways/5-ways-to-approach-an-ai-project-strategy) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### 5 ways to approach an AI project strategy](https://www.voiceflow.com/pathways/5-ways-to-approach-an-ai-project-strategy) [](https://www.voiceflow.com/pathways/the-ultimate-guide-to-vfv2-whats-new-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow V2: The ultimate guide to what's new](https://www.voiceflow.com/pathways/the-ultimate-guide-to-vfv2-whats-new-with-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Curiosity and conversation beat any best practice | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Curiosity and conversation beat any best practice ================================================= [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) There are hundreds of thousands of Google Search results for the term “conversation design best practices.” I think. I’m actually unsure. As of about a week ago, I do most of my searching through ChatGPT, which doesn’t reveal that data. And since writing this, I switched to Bing, which does. The number is 2.6 million. But wait—I might switch to Google Bard next week, and... Okay, Bard just confessed its feelings for me and incited a blood feud with Bing over the future of my search traffic. [Smiling devil horns emojis](https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html) are flying, and I guess what I’m really trying to say is I appreciate the simplicity of DuckDuckGo. Don’t you? I’m told they’re working on an offline version, which’ll be great if Bing makes good on some of its threats and my next job is a desert nomad and I want to look up “the before times.” Maybe that was a bit much. But the point I’m trying to make is that the world of [large language models](https://www.voiceflow.com/blog/dominant-discourse-llms) (LLMs) moves hilariously quick. And so does the tangential world of [conversation design (CxD)](https://www.voiceflow.com/blog-categories/conversation-design) . It’s so fast, and so [wide-ranging a discipline](https://www.voiceflow.com/blog/designing-for-voice-assistants-vs-chatbots) , it defies fixed precepts. Most of what I see shared out there as “best practices” are really just someone’s highly situational advice, which quickly grows irrelevant. So here’s what I’ll offer you instead. My only best practice is that there are no best practices save for one: to secret shop others’ experiences and discuss them as a team. Today, I share stories of companies that win by doing just that.  Other people’s “practices” rarely apply to you ---------------------------------------------- How big is your company? [Really big](https://www.voiceflow.com/blog/presenting-conversation-design-work) ? Who’s your customer? All accountants, or just solo practitioners? How online are your users? What goals does your organization have in mind? What properties will your assistant appear on, and alongside what else? I could go on qualifying your situation forever. Without clear answers to these and more, it’s unlikely anybody except a direct competitor or someone already on your team is going to pen a [work-fluencing](https://www.businessinsider.com/linkedin-tiktok-workers-employees-oversharing-about-jobs-work-side-hustle-2022-11?r=US&IR=T) LinkedIn post that perfectly applies to you. Plus, how do you calculate what makes a practice “best”? Did the person who shared it explain how they measured it? Did they bother? Some teams seek to reduce call volume whereas others want good marks for a pleasant and successful dialog. Some teams have high standards and nothing’s a “practice” until it’s provable and replicable. For most others, something’s worth sharing [after it worked just once](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4359000/) . And then, all that knowledge you accumulate changes when you switch jobs or projects. Or when your tools change—like your LLM model. (Cue the present LLM clash of the titans.) All that said, I do think there is still great value in reading and considering all this stuff because it can provide a serendipitous trickle of inspiration and cause you to stumble upon great _questions_. And those questions can lead to new practices—practices that are “best” only for you. Far better to secret shop relentlessly and discuss -------------------------------------------------- There is no way around just doing the work—donning headphones and trying as many experiences as possible—known as “secret shopping the competition.” And I don’t mean direct competition. We all know that if you’re a car manufacturer, consumers also now compare you to Lyft, Geico, and Waze. Your competition is anything that can be tapped or talked to. Go out and “no match” every assistant you can. When you tire of Alexa, Siri, and Google Assistant, ask for help from all the services you pay for. I recently asked [Gina Riley](https://www.voiceflow.com/blog/up-level-user-testing-with-interactive-prototypes) , VP of User Experience and Design Lead at JPMorgan Chase, who is the champion of doing this. Here’s what she’s been inspired by lately: * **KLM**. “The airline KLM’s chat takes your utterance and immediately transfers you to SMS text,” says Gina. “I think that’s a wonderful experience because you don’t have to stay in that channel. You can go anywhere, do anything. I think it’s where the world is headed.” * **Apple’s IVR**. “Apple’s IVR is really intuitive and smart. When I tested it, I expected to ‘no match’ a whole bunch of times, but I only no-matched once,” she says. “Its grasp of natural language is great. Basically anything you say is recognized, and that’s an experience to study.” * **Spectrum**. “What I love about Spectrum’s assistant is if you leave it and come back a day or two later, it’ll say, ‘Welcome back. Want to continue?’ Very few assistants hold that sort of context,” she says. You can also shop for feedback within your own organization, as I wrote about recently here on Pathways: [At PayPal](https://www.voiceflow.com/blog/cxd-most-impossible-divide) , customer success advocates join their assistant design reviews. They weave those support people’s experiences—basically, everything that happens _after_ the assistant—back into the build, in a loop. Take all of those ideas and experiences and feedback and bring those back to your team to discuss. I won’t tell you how many experiences to review, or how exactly to discuss them (no best practice here!) but here’s what’s important about those conversations: ### 1\. It’s everyone’s job to collect and share novel experiences The richness and variety of your inspiration determines the quality of your creation. Ask friends, family, and peers about their assistant interactions. Get everyone on your team looking for new ideas. ### 2\. Everyone should allow their personal preferences to bleed into those choices When people think in terms of what the business needs, the creative part of their brain sometimes shuts off. They filter themselves. Whereas if you treat this like an improv exercise and welcome everyone’s weird, you get exciting ideas that ricochet, accelerate, and ping-pong to life. For example, seeing another business display “wait time” is meh. Seeing that the customer support people at Games Workshop, maker of the set-top game Warhammer, include pictures of their hand-built figurines in their email signatures begs the question, “Could our bot have more personality?” ### 3\. Hold a total teardown where everyone tells you why it would work before you discuss why it wouldn’t The improv principle applies—it’s “Yes, and” first—otherwise people shut down and you never get anywhere. Gather everything, distill, then critique. (Like the Design Council’s [Double Diamond process](https://uxdesign.cc/observing-the-double-diamond-process-in-practice-7c3c7c33a2ba) .) ### 4\. Prioritize a few winning ideas to go back and test some more Novel experiences are just the jumping-off point. Once you’ve found a relevant application for them, go back and test some more. Then prototype in a visual tool that lets you show something realistic enough to get good feedback on. This process may not sound fancy, or like “thought leadership.” But this is the cycle that Gina and countless other experts rely on in the real world. “Good” is subjective, but conversations lead you there ------------------------------------------------------ “Success” is such an ephemeral, conditional thing, and far be it from me to tell you how to get there. That’s why I’m doubtful there is some eternal set of best practices worth sharing, without also providing copious context and testing to confirm they are indeed worthwhile. So, if you also find you’re having trouble applying others’ practices as your world changes rapidly, don’t worry. It’s not just you. Just get really good at discussions. In other words, don’t just be a conversation designer. Be a curious, conversation_al_ one. What’s that? Yes, Bing, getting good at telling stories _would_ also be useful for meeting other desert nomads after an apocalypse. Other people’s “practices” rarely apply to you ---------------------------------------------- How big is your company? [Really big](https://www.voiceflow.com/blog/presenting-conversation-design-work) ? Who’s your customer? All accountants, or just solo practitioners? How online are your users? What goals does your organization have in mind? What properties will your assistant appear on, and alongside what else? I could go on qualifying your situation forever. Without clear answers to these and more, it’s unlikely anybody except a direct competitor or someone already on your team is going to pen a [work-fluencing](https://www.businessinsider.com/linkedin-tiktok-workers-employees-oversharing-about-jobs-work-side-hustle-2022-11?r=US&IR=T) LinkedIn post that perfectly applies to you. Plus, how do you calculate what makes a practice “best”? Did the person who shared it explain how they measured it? Did they bother? Some teams seek to reduce call volume whereas others want good marks for a pleasant and successful dialog. Some teams have high standards and nothing’s a “practice” until it’s provable and replicable. For most others, something’s worth sharing [after it worked just once](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4359000/) . And then, all that knowledge you accumulate changes when you switch jobs or projects. Or when your tools change—like your LLM model. (Cue the present LLM clash of the titans.) All that said, I do think there is still great value in reading and considering all this stuff because it can provide a serendipitous trickle of inspiration and cause you to stumble upon great _questions_. And those questions can lead to new practices—practices that are “best” only for you. Far better to secret shop relentlessly and discuss -------------------------------------------------- There is no way around just doing the work—donning headphones and trying as many experiences as possible—known as “secret shopping the competition.” And I don’t mean direct competition. We all know that if you’re a car manufacturer, consumers also now compare you to Lyft, Geico, and Waze. Your competition is anything that can be tapped or talked to. Go out and “no match” every assistant you can. When you tire of Alexa, Siri, and Google Assistant, ask for help from all the services you pay for. I recently asked [Gina Riley](https://www.voiceflow.com/blog/up-level-user-testing-with-interactive-prototypes) , VP of User Experience and Design Lead at JPMorgan Chase, who is the champion of doing this. Here’s what she’s been inspired by lately: * **KLM**. “The airline KLM’s chat takes your utterance and immediately transfers you to SMS text,” says Gina. “I think that’s a wonderful experience because you don’t have to stay in that channel. You can go anywhere, do anything. I think it’s where the world is headed.” * **Apple’s IVR**. “Apple’s IVR is really intuitive and smart. When I tested it, I expected to ‘no match’ a whole bunch of times, but I only no-matched once,” she says. “Its grasp of natural language is great. Basically anything you say is recognized, and that’s an experience to study.” * **Spectrum**. “What I love about Spectrum’s assistant is if you leave it and come back a day or two later, it’ll say, ‘Welcome back. Want to continue?’ Very few assistants hold that sort of context,” she says. You can also shop for feedback within your own organization, as I wrote about recently here on Pathways: [At PayPal](https://www.voiceflow.com/blog/cxd-most-impossible-divide) , customer success advocates join their assistant design reviews. They weave those support people’s experiences—basically, everything that happens _after_ the assistant—back into the build, in a loop. Take all of those ideas and experiences and feedback and bring those back to your team to discuss. I won’t tell you how many experiences to review, or how exactly to discuss them (no best practice here!) but here’s what’s important about those conversations: ### 1\. It’s everyone’s job to collect and share novel experiences The richness and variety of your inspiration determines the quality of your creation. Ask friends, family, and peers about their assistant interactions. Get everyone on your team looking for new ideas. ### 2\. Everyone should allow their personal preferences to bleed into those choices When people think in terms of what the business needs, the creative part of their brain sometimes shuts off. They filter themselves. Whereas if you treat this like an improv exercise and welcome everyone’s weird, you get exciting ideas that ricochet, accelerate, and ping-pong to life. For example, seeing another business display “wait time” is meh. Seeing that the customer support people at Games Workshop, maker of the set-top game Warhammer, include pictures of their hand-built figurines in their email signatures begs the question, “Could our bot have more personality?” ### 3\. Hold a total teardown where everyone tells you why it would work before you discuss why it wouldn’t The improv principle applies—it’s “Yes, and” first—otherwise people shut down and you never get anywhere. Gather everything, distill, then critique. (Like the Design Council’s [Double Diamond process](https://uxdesign.cc/observing-the-double-diamond-process-in-practice-7c3c7c33a2ba) .) ### 4\. Prioritize a few winning ideas to go back and test some more Novel experiences are just the jumping-off point. Once you’ve found a relevant application for them, go back and test some more. Then prototype in a visual tool that lets you show something realistic enough to get good feedback on. This process may not sound fancy, or like “thought leadership.” But this is the cycle that Gina and countless other experts rely on in the real world. “Good” is subjective, but conversations lead you there ------------------------------------------------------ “Success” is such an ephemeral, conditional thing, and far be it from me to tell you how to get there. That’s why I’m doubtful there is some eternal set of best practices worth sharing, without also providing copious context and testing to confirm they are indeed worthwhile. So, if you also find you’re having trouble applying others’ practices as your world changes rapidly, don’t worry. It’s not just you. Just get really good at discussions. In other words, don’t just be a conversation designer. Be a curious, conversation_al_ one. What’s that? Yes, Bing, getting good at telling stories _would_ also be useful for meeting other desert nomads after an apocalypse.  RECOMMENDED [\ \ ### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) PUBLISHED May 26, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 5 tips for better multimodal design](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design) [](https://www.voiceflow.com/pathways/6-differences-between-mobile-apps-and-voice-apps) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The 6 major differences between mobile apps and voice apps](https://www.voiceflow.com/pathways/6-differences-between-mobile-apps-and-voice-apps) [](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you should A/B test your conversational experiences](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/best-practices-conversation-design#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/best-practices-conversation-design#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Confessions of a CxD killjoy: Why I think your AI assistant shouldn’t be funny | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Confessions of a CxD killjoy: Why I think your AI assistant shouldn’t be funny ============================================================================== [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) “You seem like a person, but you’re just a voice in a computer,” says Theodore to his AI assistant Samatha, in the movie _Her_. Now, a decade after the film’s release, human-like AI is no longer science fiction. AI tools, assistants—even, worryingly enough, [romantic companions](https://www.theguardian.com/technology/2023/jul/22/ai-girlfriend-chatbot-apps-unhealthy-chatgpt) —have become a part of our everyday lives. But, AI tools are not people. And as AI becomes more entwined in our lives, there’s a temptation to imbue it with human characteristics, including humor. I get it, we all want the tools we build to be enjoyable to use. This might be a controversial take, but when it comes to giving your AI assistant a personality, I think CxDs (conversation designers) should be cautious. Before you call me a killjoy, allow me to explain.  What’s the harm in giving your AI assistant a fun personality? --------------------------------------------------------------- Outspoken brand personalities have taken social media by storm—just think of [Wendy’s on Twitter](https://twitter.com/Wendys?s=20) (sorry, I mean X) or [Duolingo on TikTok](https://www.tiktok.com/@duolingo?is_from_webapp=1&sender_device=pc) . Followers love the offbeat humor and [blunt self-awareness](https://www.tiktok.com/@duolingo/video/7229357221837147434?is_from_webapp=1&sender_device=pc&web_id=7232304555244979718) of these accounts. Seeing these successes, it can be tempting to consider building your AI assistant’s voice in a similar way, using humor, charisma, and even shock value to build a relationship with your customers. After all, isn’t that what buyers want? Maybe…but is it what they _need_? A customer’s relationship with an AI assistant is much more complicated than just laughing at a sassy brand mascot’s hot takes and memes. While branded accounts are there primarily to entertain, people rely on AI assistants for support—whether that’s processing returns, answering FAQs, or surfacing customers' information. At the end of the day, your AI assistant is not a social media personality and the risks of slipping up are far higher. I would go so far as to say that anthropomorphizing AI tools can come with the risk of real harm. As humans, we’re susceptible to manipulation, more so than we might think. Research has shown that people tend to trust their friends [more than](https://www.americanpressinstitute.org/publications/reports/survey-research/trust-social-media/) news sources. If you anthropomorphize an AI tool, users may be more likely to believe what it says because they’re building a relationship with it. And we know that AI may mess up and "hallucinate," making it even more important for users to maintain some healthy emotional distance so they can critically evaluate what the AI is telling them. AI is something that does stuff for us—but it’s not our friend. It’s a tool. It’s not infallible. It’s not a person, and it’s important that we keep those distinctions in mind. Chatbot humor falls flat ------------------------ I’m just going to say it—chatbot humor is cringe. While the ethical concerns I’ve highlighted are important, I also have a bone to pick with the way many brands try to add humor to their assistants. For instance, an unconscionable number of businesses have at least one response to the question “What is the meaning of life?” as “42” because of the _Hitchhiker's Guide to the Galaxy._ That joke has been overdone. If you must add humor, at least do so in a way that’s original and makes sense for your brand. Make it interesting and actually worth doing. My colleague, [Ayesha Saleem](https://www.voiceflow.com/blog/best-conversational-ai-practices) , a conversation design leader at Instacart, agrees with me. “I'm pretty anti-humor in most cases,” she says. “There's use cases where it works, but most bots are really gimmicky or based on fads—which means you're going to have to update it every few months, or it will fall flat. It’s something I try to avoid as a best practice.” She brings up another excellent point: Humor doesn’t always translate. “I've seen people release bots in English and they'll put puns in there,” she says. “The second you translate this to a different language, it's not going to work. Your chatbot should be as inclusive as possible, so when and if you do translate it, you’re not leaving anyone out.” If you serve multiple markets or plan to one day, translatability is an important factor to keep in mind. At the end of the day, you know your brand best. If it’s super whimsical, maybe it makes sense to have a chatbot with some personality. But it’s important to be cautious and thoughtful in how you do it, so you’re serving your audience and not irritating or excluding them. What are CxDs’ responsibilities in all this? --------------------------------------------- If there’s anything we’ve learned over the course of the past few years, it’s that people are very bad at deciphering truth from falsehood. And now we’ve created tools that allow us to copy and paste our human experiences over again endlessly in ways that are helpful for us, but can also be detrimental. It’s conversation designers’ responsibility to pepper in cues to remind users that their AI assistant is, in fact, a tool. I like the way ChatGPT says, “As an AI language model, I can’t…” I recommend using language like “As an AI,” or “I’m just a bot,” to nudge users not to put all their faith into the tool. Particularly if you're going to give your assistant a personality, you need to have those moments to bring users back to earth. Let robots be robots -------------------- The movie _Her_ revealed that we have an appetite to make AI assistants that are like us. Now that we have the power to do that, it’s not a question of whether we can, but if we _should_. And if we should, how do we do so responsibly? We’re drawn to brands that are representations of ourselves—there’s a reason why human-like brand personas are so compelling on social media. But the relationship you think you have with a brand mascot and an AI assistant is very different, and the potential for harm with the latter is much greater. Lest you think I hate fun, I do think that pre-written NLU bots can use humor—that’s a use case that’s much closer to web copy and social media. But I have much more hesitation when it comes to LLM-based assistants, because you have less control, and the potential for manipulation is far greater. In _Her_, (spoiler alert), Samantha the AI eventually goes away. But we live in a time where we’ll continue to coexist with AI, and as such, it’s critical to draw clear boundaries so we don’t end up giving it too much power over us. > Disagree? Send me a note on how you see humor in CxD → peter.isaacs \[at\] voiceflow.com What’s the harm in giving your AI assistant a fun personality? --------------------------------------------------------------- Outspoken brand personalities have taken social media by storm—just think of [Wendy’s on Twitter](https://twitter.com/Wendys?s=20) (sorry, I mean X) or [Duolingo on TikTok](https://www.tiktok.com/@duolingo?is_from_webapp=1&sender_device=pc) . Followers love the offbeat humor and [blunt self-awareness](https://www.tiktok.com/@duolingo/video/7229357221837147434?is_from_webapp=1&sender_device=pc&web_id=7232304555244979718) of these accounts. Seeing these successes, it can be tempting to consider building your AI assistant’s voice in a similar way, using humor, charisma, and even shock value to build a relationship with your customers. After all, isn’t that what buyers want? Maybe…but is it what they _need_? A customer’s relationship with an AI assistant is much more complicated than just laughing at a sassy brand mascot’s hot takes and memes. While branded accounts are there primarily to entertain, people rely on AI assistants for support—whether that’s processing returns, answering FAQs, or surfacing customers' information. At the end of the day, your AI assistant is not a social media personality and the risks of slipping up are far higher. I would go so far as to say that anthropomorphizing AI tools can come with the risk of real harm. As humans, we’re susceptible to manipulation, more so than we might think. Research has shown that people tend to trust their friends [more than](https://www.americanpressinstitute.org/publications/reports/survey-research/trust-social-media/) news sources. If you anthropomorphize an AI tool, users may be more likely to believe what it says because they’re building a relationship with it. And we know that AI may mess up and "hallucinate," making it even more important for users to maintain some healthy emotional distance so they can critically evaluate what the AI is telling them. AI is something that does stuff for us—but it’s not our friend. It’s a tool. It’s not infallible. It’s not a person, and it’s important that we keep those distinctions in mind. Chatbot humor falls flat ------------------------ I’m just going to say it—chatbot humor is cringe. While the ethical concerns I’ve highlighted are important, I also have a bone to pick with the way many brands try to add humor to their assistants. For instance, an unconscionable number of businesses have at least one response to the question “What is the meaning of life?” as “42” because of the _Hitchhiker's Guide to the Galaxy._ That joke has been overdone. If you must add humor, at least do so in a way that’s original and makes sense for your brand. Make it interesting and actually worth doing. My colleague, [Ayesha Saleem](https://www.voiceflow.com/blog/best-conversational-ai-practices) , a conversation design leader at Instacart, agrees with me. “I'm pretty anti-humor in most cases,” she says. “There's use cases where it works, but most bots are really gimmicky or based on fads—which means you're going to have to update it every few months, or it will fall flat. It’s something I try to avoid as a best practice.” She brings up another excellent point: Humor doesn’t always translate. “I've seen people release bots in English and they'll put puns in there,” she says. “The second you translate this to a different language, it's not going to work. Your chatbot should be as inclusive as possible, so when and if you do translate it, you’re not leaving anyone out.” If you serve multiple markets or plan to one day, translatability is an important factor to keep in mind. At the end of the day, you know your brand best. If it’s super whimsical, maybe it makes sense to have a chatbot with some personality. But it’s important to be cautious and thoughtful in how you do it, so you’re serving your audience and not irritating or excluding them. What are CxDs’ responsibilities in all this? --------------------------------------------- If there’s anything we’ve learned over the course of the past few years, it’s that people are very bad at deciphering truth from falsehood. And now we’ve created tools that allow us to copy and paste our human experiences over again endlessly in ways that are helpful for us, but can also be detrimental. It’s conversation designers’ responsibility to pepper in cues to remind users that their AI assistant is, in fact, a tool. I like the way ChatGPT says, “As an AI language model, I can’t…” I recommend using language like “As an AI,” or “I’m just a bot,” to nudge users not to put all their faith into the tool. Particularly if you're going to give your assistant a personality, you need to have those moments to bring users back to earth. Let robots be robots -------------------- The movie _Her_ revealed that we have an appetite to make AI assistants that are like us. Now that we have the power to do that, it’s not a question of whether we can, but if we _should_. And if we should, how do we do so responsibly? We’re drawn to brands that are representations of ourselves—there’s a reason why human-like brand personas are so compelling on social media. But the relationship you think you have with a brand mascot and an AI assistant is very different, and the potential for harm with the latter is much greater. Lest you think I hate fun, I do think that pre-written NLU bots can use humor—that’s a use case that’s much closer to web copy and social media. But I have much more hesitation when it comes to LLM-based assistants, because you have less control, and the potential for manipulation is far greater. In _Her_, (spoiler alert), Samantha the AI eventually goes away. But we live in a time where we’ll continue to coexist with AI, and as such, it’s critical to draw clear boundaries so we don’t end up giving it too much power over us. > Disagree? Send me a note on how you see humor in CxD → peter.isaacs \[at\] voiceflow.com  RECOMMENDED [\ \ ### AI will kill apps and I can’t wait](https://www.voiceflow.com/pathways/ai-will-kill-apps) PUBLISHED September 13, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-to-use-open-vs-closed-prompts-in-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to use open vs. closed prompts in conversation design](https://www.voiceflow.com/pathways/how-to-use-open-vs-closed-prompts-in-conversation-design) [](https://www.voiceflow.com/pathways/voiceflow-frog-sxsw) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How a pair of interaction designers used Voiceflow to create an unforgettable experience at SXSW](https://www.voiceflow.com/pathways/voiceflow-frog-sxsw) [](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to prioritize requests for conversation design teams](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/confessions-of-a-cxd-killjoy-why-i-think-your-ai-assistant-shouldnt-be-funny#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/confessions-of-a-cxd-killjoy-why-i-think-your-ai-assistant-shouldnt-be-funny#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Creating the Voiceflow NLU | Pathways [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) Creating the Voiceflow NLU ========================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) Building a NLU is an ambitious task. There are millions of assistants that need to be powered with different domains, languages, and use cases. As Voiceflow continued to grow, it became clear that while building a custom NLU was challenging, this was an important area we needed to invest into to deliver the best customer experience. We started working on the Voiceflow NLU in February 2022, and this is a small slice of our story. ### Chapter 1: Understanding the goals and criteria To start our research efforts, we looked at six important criteria: 1. NLU accuracy, for both intent and entity classifications 2. Latency, how long the model would take to respond 3. Training time, how long it would take to train a new model 4. Generalizability, we have customers across many verticals 5. Operational overhead, maintenance costs of the models 6. Cost $$$, we operate at a scale of 100,000s of models We started with the first two criteria, and began exploring papers and blog posts on the topic. There were existing NLUs and libraries available, but as we dug deeper, the other four criteria started becoming a challenge. We ended up using a pre-trained transformer model, and fine tuning it for intent and entity based tasks. Starting from scratch also allowed us to think through all the features that were important to our customers. It was now March 2022, and we were able to replicate performance of several existing implementations and began thinking of how to deploy the NLU models. We thankfully had our ML deployment platform (ML CLI) already built which made it much easier to do so.  ### Chapter 2: An integrated prototype It was April and we had an internal engineering onsite. One of the demos was the VFNLU, now running in one of our Voiceflow dev environments. Below is the recording of the demo. The VFNLU was no longer an academic exercise, but a feasible product. ### Chapter 3: Feature completeness and Multilingual models After the April excitement, the reality of preparing the models for production were set in. Code was refactored, added and changed to fit our existing systems. We also retrained the NLU several times, including new techniques that boosted performance, along with adding multilingual support. We could A/B test these models, with our ML platform setup, which was handy for doing comparisons. Summer turned into fall, and we began testing load testing. ### Chapter 4: Real time constraints _“This is too slow”_ Our ML platform had been built with realtime in mind, with latencies in the high 100s of ms for most requests. We built it with a queue system, but that technology was not designed for requests that should consistently return in 150ms. Even though our NLU was very fast (sub 20ms) the latency of the system was quite high, especially the long tail of requests. We had to redesign how our NLU deals with requests, which started a 3 month code refactor. It was unexpected but required to achieve the latency that our users expected. ### Chapter 5: Beta testing April 2023, almost a year after the initial VFNLU demo, was when we started rolling out the VFNLU to our free users. It was time to test how well the VFNLU performed in the real world and what bugs we could find. We also began migrating some early customers to help avoid some of the limitations we had with our previous OEM NLU implementation. We implemented a feature flag, directing a % of users to train and run inference on our NLU. Eventually we finalized a full rollout across all our free users, helping us to find some last minute bugs and deficiencies in the systems. ### Chapter 6: The VFNLU And today we release the VFNLU. After 18 months of work VFNLU is ready. Outperforming the most popular NLUs on open source and proprietary benchmarks. We’ve released an open source [testing repo](https://github.com/Diophontine/vfnlu-benchmarks) to test the performance yourself With 30+ languages supported and low training time, you can both power and prototype your next conversational assistant. ### NLUs in a LLM world In December 2022, ChatGPT was in almost every Conversational AI based conversation. A powerful new model that could solve many tasks, handle many languages and respond to almost any user request. We did an internal hackathon before the winter holidays to add a number of Gen AI features to Voiceflow. Outside of the Gen AI features, we kept working on the VFNLU. There were a couple reasons for this: 1. All our Enterprise Customers were continuing to use NLUs to power their assistants 2. NLUs are 100-1000x cheaper than LLMs and perform better on large projects 3. NLUs don’t hallucinate, they might respond with the wrong intent but it will be clear if there’s a false positive.  With this in mind, each piece of technology has their place in the conversational AI world, our goal as a platform is to allow people to experiment and build with both. ### Chapter 2: An integrated prototype It was April and we had an internal engineering onsite. One of the demos was the VFNLU, now running in one of our Voiceflow dev environments. Below is the recording of the demo. The VFNLU was no longer an academic exercise, but a feasible product. ### Chapter 3: Feature completeness and Multilingual models After the April excitement, the reality of preparing the models for production were set in. Code was refactored, added and changed to fit our existing systems. We also retrained the NLU several times, including new techniques that boosted performance, along with adding multilingual support. We could A/B test these models, with our ML platform setup, which was handy for doing comparisons. Summer turned into fall, and we began testing load testing. ### Chapter 4: Real time constraints _“This is too slow”_ Our ML platform had been built with realtime in mind, with latencies in the high 100s of ms for most requests. We built it with a queue system, but that technology was not designed for requests that should consistently return in 150ms. Even though our NLU was very fast (sub 20ms) the latency of the system was quite high, especially the long tail of requests. We had to redesign how our NLU deals with requests, which started a 3 month code refactor. It was unexpected but required to achieve the latency that our users expected. ### Chapter 5: Beta testing April 2023, almost a year after the initial VFNLU demo, was when we started rolling out the VFNLU to our free users. It was time to test how well the VFNLU performed in the real world and what bugs we could find. We also began migrating some early customers to help avoid some of the limitations we had with our previous OEM NLU implementation. We implemented a feature flag, directing a % of users to train and run inference on our NLU. Eventually we finalized a full rollout across all our free users, helping us to find some last minute bugs and deficiencies in the systems. ### Chapter 6: The VFNLU And today we release the VFNLU. After 18 months of work VFNLU is ready. Outperforming the most popular NLUs on open source and proprietary benchmarks. We’ve released an open source [testing repo](https://github.com/Diophontine/vfnlu-benchmarks) to test the performance yourself With 30+ languages supported and low training time, you can both power and prototype your next conversational assistant. ### NLUs in a LLM world In December 2022, ChatGPT was in almost every Conversational AI based conversation. A powerful new model that could solve many tasks, handle many languages and respond to almost any user request. We did an internal hackathon before the winter holidays to add a number of Gen AI features to Voiceflow. Outside of the Gen AI features, we kept working on the VFNLU. There were a couple reasons for this: 1. All our Enterprise Customers were continuing to use NLUs to power their assistants 2. NLUs are 100-1000x cheaper than LLMs and perform better on large projects 3. NLUs don’t hallucinate, they might respond with the wrong intent but it will be clear if there’s a false positive.  With this in mind, each piece of technology has their place in the conversational AI world, our goal as a platform is to allow people to experiment and build with both.  RECOMMENDED [\ \ ### How to create your NLU testing strategy](https://www.voiceflow.com/pathways/how-to-create-your-nlu-testing-strategy) PUBLISHED August 30, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### NLU design: How to train and use a natural language understanding model](https://www.voiceflow.com/pathways/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) [](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### Creating the Voiceflow NLU](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu) [](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### 5 tips to optimize your LLM intent classification prompts](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How a few PayPal’ers tackled CXD’s most impossible divide | Pathways [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) How a few PayPal’ers tackled CXD’s most impossible divide ========================================================= [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) A customer reaches an agent via chat and is breathless with indignation. This occurs over and over across thousands of sessions, and they all share the same complaint—yet the conversation designers never change that bothersome feature. Why? Because in most organizations, there’s a wall between the conversation design (CXD) team and customer support. They have no natural overlap or channel for communicating, and thus the news that many customers express rage at having to select from a confusing set of pill-style buttons that lead them astray never reaches the people who could fix it. Except in some organizations, like at PayPal. In a recent [Expert Conversations](https://www.youtube.com/playlist?list=PLKYemGIohRgBvaBmNVHyY1kCvpnVYqZ2u) workshop on this topic, we heard from PayPal’s [Simon Wolf](https://www.linkedin.com/in/snwolf/) , Conversational Experience Designer, and Nicole Connolly, Customer Support Manager, who worked to break down the wall, unite their teams, and fix all that. You can watch their full recorded conversation below, or scroll down to read their three steps toward a fruitful collaboration.  Step 1: Invite the support team to share their top linguistic gripes -------------------------------------------------------------------- What do people complain about when they finally reach a live agent? Often, they bemoan how long it took to get there—something support teams hear about endlessly. “We’re very used to hearing people’s frustrations and it seems to be the same thing over and over,” says Nicole. “I find the number one issue is the user misunderstanding the bot because it used business jargon. The user wasn’t familiar with the language a company uses internally.” When users misunderstand, they get slotted into the incorrect flow. Time and many taps pass before they get to the right flow. And by then it’s led to a poor perception of the bot’s ability, even though there may be nothing at all wrong with the flow itself—only that users didn’t understand enough to guide themselves. Too much of this across the industry is why people reflexively seek a live agent—they’ve come to distrust bots entirely. Step one for addressing these fixable misdirections is breaking down the conversational design and support silo. Create a forum—perhaps a meeting, maybe even just a Slack or Teams channel—where support teams can report common linguistic issues. These fixes tend to be easy ones—they need only edit the text—and have notable results. They’re easy wins and a foundation for more collaboration. “This is a serious opportunity for knowledge transfer,” says Simon. “Make contact. Reach out. Introduce yourself and say, ‘Here’s what we do, and how it relates to what you do.’ Set a meeting to compare notes, customer pain points and team pain points, and get to know each other.” Here’s Nicole and Simon’s framework: **Initial knowledge transfer** 1. Compile customer pain points from each team 2. Identify common jargon/intents that cause confusion in customers' mental models 3. Review representative transcripts and customer calls 4. Create an initial audit based on extended customer journeys **Ongoing** 1. Regular cadence to review transcripts and customer calls 2. Solution discussion and design review for new projects 3. Heartbeat on what agents are currently seeing/hearing Step 2: Show how support is really a portal into unfiltered user feedback ------------------------------------------------------------------------- Agents sometimes experience the worst of the chatbot overflow. But by that virtue, they are also a lightning rod for extremely candid feedback that wouldn’t have come across in a structured survey delivered after the fact. “Nothing’s as immediate and accurate as an, ‘Oh my god please help,’” says Nicole. It’s very different from what’s reported in surveys, where they’re given a one-to-five option. If the CXD team realizes that these support interactions are a clearer window into what users are feeling during real interactions, that initial sharing can evolve into much more. “Consider getting on a regular cadence where you review transcripts together,” says Nicole. “At PayPal, we have a single teammate champion who collects all that feedback and relays it. But you could just as easily make it a rotating responsibility.” Whoever that person is, CXD should pull them into planning conversations. This effectively draws all that support wisdom into the bot’s design. “There’s a huge benefit to having Nicole, who we can pull into discussions, so it’s not just a retroactive feedback loop. It’s pulling customer support experience into the design process,” says Simon. “This helps us preempt problems and keep the heartbeat of the customer at the center of the experience. And when customer support figures out a really good way to resolve a problem, so does the bot.” Step 3: Pick a targeted challenge to address as a working group --------------------------------------------------------------- Together, conversation designers and customer support agents can tackle a big, common issue like jargon and eliminate it wherever it shows up in customer interactions. That allows them to re-envision how the company communicates and make it friendlier to interact with. “Jargon is a huge problem for every company of a certain size,” says Nicole. “Take a PayPal seller. Say they’re contacting us to access money they received in a sale and are urged to select from a selection of choices including ‘pending funds’ and ‘held funds.’ These sound similar, but are completely different options, with different reasons, time frames, and steps. Mind you, the designer correctly inputted these choices. They followed their instructions and that’s what they’re really called. But it can cause mixups—and if those mixups degrade the user experience, that’s something we can tweak together.” And as both Nicole and Simon point out, agent feedback highlights shortcomings in the design that wouldn't otherwise be apparent until after release. “The reality is that as conversation designers, in user testing we do a lot of controlling for a variable and testing the change, but it doesn’t always capture the emotional charge and the time constraints real users are under,” says Simon. This collaboration gives designers access to all those real, unfiltered, emotional insights. > Curious about what support can teach you? Watch Nicole and Simons’ full talk on how they’ve structured their work together. _Header image by_ [_Hossein Nasr_](https://unsplash.com/@nasrphotos?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) Step 1: Invite the support team to share their top linguistic gripes -------------------------------------------------------------------- What do people complain about when they finally reach a live agent? Often, they bemoan how long it took to get there—something support teams hear about endlessly. “We’re very used to hearing people’s frustrations and it seems to be the same thing over and over,” says Nicole. “I find the number one issue is the user misunderstanding the bot because it used business jargon. The user wasn’t familiar with the language a company uses internally.” When users misunderstand, they get slotted into the incorrect flow. Time and many taps pass before they get to the right flow. And by then it’s led to a poor perception of the bot’s ability, even though there may be nothing at all wrong with the flow itself—only that users didn’t understand enough to guide themselves. Too much of this across the industry is why people reflexively seek a live agent—they’ve come to distrust bots entirely. Step one for addressing these fixable misdirections is breaking down the conversational design and support silo. Create a forum—perhaps a meeting, maybe even just a Slack or Teams channel—where support teams can report common linguistic issues. These fixes tend to be easy ones—they need only edit the text—and have notable results. They’re easy wins and a foundation for more collaboration. “This is a serious opportunity for knowledge transfer,” says Simon. “Make contact. Reach out. Introduce yourself and say, ‘Here’s what we do, and how it relates to what you do.’ Set a meeting to compare notes, customer pain points and team pain points, and get to know each other.” Here’s Nicole and Simon’s framework: **Initial knowledge transfer** 1. Compile customer pain points from each team 2. Identify common jargon/intents that cause confusion in customers' mental models 3. Review representative transcripts and customer calls 4. Create an initial audit based on extended customer journeys **Ongoing** 1. Regular cadence to review transcripts and customer calls 2. Solution discussion and design review for new projects 3. Heartbeat on what agents are currently seeing/hearing Step 2: Show how support is really a portal into unfiltered user feedback ------------------------------------------------------------------------- Agents sometimes experience the worst of the chatbot overflow. But by that virtue, they are also a lightning rod for extremely candid feedback that wouldn’t have come across in a structured survey delivered after the fact. “Nothing’s as immediate and accurate as an, ‘Oh my god please help,’” says Nicole. It’s very different from what’s reported in surveys, where they’re given a one-to-five option. If the CXD team realizes that these support interactions are a clearer window into what users are feeling during real interactions, that initial sharing can evolve into much more. “Consider getting on a regular cadence where you review transcripts together,” says Nicole. “At PayPal, we have a single teammate champion who collects all that feedback and relays it. But you could just as easily make it a rotating responsibility.” Whoever that person is, CXD should pull them into planning conversations. This effectively draws all that support wisdom into the bot’s design. “There’s a huge benefit to having Nicole, who we can pull into discussions, so it’s not just a retroactive feedback loop. It’s pulling customer support experience into the design process,” says Simon. “This helps us preempt problems and keep the heartbeat of the customer at the center of the experience. And when customer support figures out a really good way to resolve a problem, so does the bot.” Step 3: Pick a targeted challenge to address as a working group --------------------------------------------------------------- Together, conversation designers and customer support agents can tackle a big, common issue like jargon and eliminate it wherever it shows up in customer interactions. That allows them to re-envision how the company communicates and make it friendlier to interact with. “Jargon is a huge problem for every company of a certain size,” says Nicole. “Take a PayPal seller. Say they’re contacting us to access money they received in a sale and are urged to select from a selection of choices including ‘pending funds’ and ‘held funds.’ These sound similar, but are completely different options, with different reasons, time frames, and steps. Mind you, the designer correctly inputted these choices. They followed their instructions and that’s what they’re really called. But it can cause mixups—and if those mixups degrade the user experience, that’s something we can tweak together.” And as both Nicole and Simon point out, agent feedback highlights shortcomings in the design that wouldn't otherwise be apparent until after release. “The reality is that as conversation designers, in user testing we do a lot of controlling for a variable and testing the change, but it doesn’t always capture the emotional charge and the time constraints real users are under,” says Simon. This collaboration gives designers access to all those real, unfiltered, emotional insights. > Curious about what support can teach you? Watch Nicole and Simons’ full talk on how they’ve structured their work together. _Header image by_ [_Hossein Nasr_](https://unsplash.com/@nasrphotos?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)  RECOMMENDED [](https://www.voiceflow.com/pathways/cxd-most-impossible-divide#) PUBLISHED January 9, 2023 TOPICS IN THIS ARTICLE Exceptional work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/cxd-most-impossible-divide) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### How a few PayPal’ers tackled CXD’s most impossible divide](https://www.voiceflow.com/pathways/cxd-most-impossible-divide) [](https://www.voiceflow.com/pathways/the-4-customer-experience-trends-you-need-to-know-in-2024) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### The 4 customer experience trends you need to know in 2024](https://www.voiceflow.com/pathways/the-4-customer-experience-trends-you-need-to-know-in-2024) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/cxd-most-impossible-divide#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/cxd-most-impossible-divide#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # From NLU Intents to RAG with Embeddings: Enhancing User Experience | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Enhancing user experience: from NLU intents to RAG with embeddings ================================================================== [\ \ Sanasar Janjughazyan\ \ at](https://www.voiceflow.com/contributors/sanasar-janjughazyan) At Voiceflow, we’re always looking for ways to make every interaction seamless, intuitive, and efficient. For some time, we’ve relied on Natural Language Understanding (NLU) to process your intents. This technology helps us identify your intent by understanding the meaning behind what you say or type to your AI agent, even if the phrasing isn’t perfect. While this system has served us well, we’re excited to introduce a new approach powered by **Retrieval-Augmented Generation (RAG) with embeddings that is set to make the intent recognition experience faster and more efficient**.  What’s Changing? ---------------- With the Natural Language system, when the user asked a question or made a request, our NLU model would identify what they meant by matching the input to pre-defined “intents.” For example, if they said, “What’s the weather today?” the system would recognize it as a weather-related intent and respond accordingly. With this new RAG system, we still rely on predefined intents, but the way we handle and process your utterances has become significantly more advanced. RAG uses embeddings to better understand the context and meaning behind the words, making it more flexible and accurate when matching them to the right intent. This means faster, more efficient intent recognition and an improved ability to handle diverse or complex utterances. Why This Is Better for You -------------------------- **Faster Training And Interaction** RAG training and interactions are significantly faster and more efficient than traditional NLU systems. This means quicker training times for our agents and faster, more accurate classifications when matching your intents, ensuring a smoother conversational experience. For example, an agent with 37 intents and 305 utterances now trains about 20 times faster: in about 1 second. **Automatic agent training** The advanced training speed unlocked by the new RAG system means that explicit training is no longer needed. Test your agent and training will happen automatically. **Greater Understanding of Complex Questions** Embeddings are a way of representing language in a rich, context-aware format. The new RAG system leverages embeddings to understand the deeper meaning behind the words, even if they're phrased differently. Users can also ask detailed, complex questions, and the system will be able to understand the underlying context better. For example, a sentence like, “What’s it like outside, is it going to rain later?” might not have matched perfectly with an intent for “weather today” in the NLU system. Now, embeddings will understand the utterance, "is it going to rain later," as a query into the current weather and match to the correct intent. **A More Natural Experience** Ultimately, this upgrade means a smoother, more conversational experience for you and your customers. Whether you type casually, use slang, or ask detailed questions, our system is designed to understand accurately and efficiently. What Does This Mean for You? ---------------------------- From your perspective, the change should feel effortless—no need to change how you build, test, and launch AI agents. What you will notice is: * fewer misunderstandings; * more accurate responses to your questions, and; * a faster, more personalized experience overall. However, since some user utterances have been specifically tailored for the NLU system, you may notice slight differences when using the new RAG system. In some cases, minimal adjustments to the phrasing of utterances may be needed to optimize performance with the new approach. To make this transition smoother, we will keep both services running for a period of time. This will give you the necessary opportunity to explore the new system, test it thoroughly, and make any adjustments to your agents as needed. You can switch between the NLU and RAG systems in intent classification settings within the Intents CMS.  By adopting cutting-edge technology like RAG with embeddings, we’re making sure our system can grow and adapt alongside your needs, giving you the best experience possible. We’re thrilled to bring you this improved experience, and we’d love to hear your feedback as you interact with the new system. As always, your input helps us continue to innovate and serve you better. What’s Changing? ---------------- With the Natural Language system, when the user asked a question or made a request, our NLU model would identify what they meant by matching the input to pre-defined “intents.” For example, if they said, “What’s the weather today?” the system would recognize it as a weather-related intent and respond accordingly. With this new RAG system, we still rely on predefined intents, but the way we handle and process your utterances has become significantly more advanced. RAG uses embeddings to better understand the context and meaning behind the words, making it more flexible and accurate when matching them to the right intent. This means faster, more efficient intent recognition and an improved ability to handle diverse or complex utterances. Why This Is Better for You -------------------------- **Faster Training And Interaction** RAG training and interactions are significantly faster and more efficient than traditional NLU systems. This means quicker training times for our agents and faster, more accurate classifications when matching your intents, ensuring a smoother conversational experience. For example, an agent with 37 intents and 305 utterances now trains about 20 times faster: in about 1 second. **Automatic agent training** The advanced training speed unlocked by the new RAG system means that explicit training is no longer needed. Test your agent and training will happen automatically. **Greater Understanding of Complex Questions** Embeddings are a way of representing language in a rich, context-aware format. The new RAG system leverages embeddings to understand the deeper meaning behind the words, even if they're phrased differently. Users can also ask detailed, complex questions, and the system will be able to understand the underlying context better. For example, a sentence like, “What’s it like outside, is it going to rain later?” might not have matched perfectly with an intent for “weather today” in the NLU system. Now, embeddings will understand the utterance, "is it going to rain later," as a query into the current weather and match to the correct intent. **A More Natural Experience** Ultimately, this upgrade means a smoother, more conversational experience for you and your customers. Whether you type casually, use slang, or ask detailed questions, our system is designed to understand accurately and efficiently. What Does This Mean for You? ---------------------------- From your perspective, the change should feel effortless—no need to change how you build, test, and launch AI agents. What you will notice is: * fewer misunderstandings; * more accurate responses to your questions, and; * a faster, more personalized experience overall. However, since some user utterances have been specifically tailored for the NLU system, you may notice slight differences when using the new RAG system. In some cases, minimal adjustments to the phrasing of utterances may be needed to optimize performance with the new approach. To make this transition smoother, we will keep both services running for a period of time. This will give you the necessary opportunity to explore the new system, test it thoroughly, and make any adjustments to your agents as needed. You can switch between the NLU and RAG systems in intent classification settings within the Intents CMS.  By adopting cutting-edge technology like RAG with embeddings, we’re making sure our system can grow and adapt alongside your needs, giving you the best experience possible. We’re thrilled to bring you this improved experience, and we’d love to hear your feedback as you interact with the new system. As always, your input helps us continue to innovate and serve you better.  RECOMMENDED [\ \ ### Runtime performance improvements](https://www.voiceflow.com/pathways/runtime-performance-improvements) PUBLISHED TOPICS IN THIS ARTICLE No items found. [\ \ Sanasar Janjughazyan\ \ at\ \ San is a Senior Software Engineer at Voiceflow](https://www.voiceflow.com/contributors/sanasar-janjughazyan) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH No items found. [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/enhancing-user-experience-from-nlu-intents-to-rag-with-embeddings#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/enhancing-user-experience-from-nlu-intents-to-rag-with-embeddings#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Conversation design needs a universal language | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Conversation design needs a universal language ============================================== [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) In the 1960s space race, there was a third entrant—the European space research organization [ESRO](https://www.esa.int/About_Us/ESA_history/History_of_Europe_in_space) . As you’ve probably never seen footage of their moon landing, I’ll bet you can guess that it never happened, because it didn’t. The greatest contributing factor? Mistranslation. Mistranslation in documents and in measurements. Some countries used metrics and some used imperial. Each documented things in their mother tongues. When they brought stages of the rocket together, they could not be made to fit, and cost overruns killed the project. This is all to say, the cost of mistranslation in an industry can be immense. Big enough to hold a continent-wide effort back. Today, the conversation design industry is suffering from this same issue. If all of us can’t agree on what something as basic as a triangle in a flow, or even the title “conversation designer” means—if there are no standards—everyone loses, and everything costs more. So why don’t we have a shared language for conversation design yet?  The inconsistent and shifting language of conversation design ------------------------------------------------------------- The mystery of the indecipherable design triangle is actually a very real issue. Today, within one conversation design team, a triangle dropped into a flow means something completely different than what it means for designers at another company. Moreover, they often can’t even agree on when or how that triangle should be used, let alone what it means. There’s no reason for that. But it has a cost. It means that during a collaboration, designers’ flows aren’t mutually intelligible. Each might as well be hieroglyphics. And that’s true within those companies, across teams and departments. Whenever a conversation design leader is hired from somewhere else and brings their preferences, or two teams work in silos, you get compounding inconsistencies. In the most inconvenient scenarios, you get false positives—instances where one designer thinks they know what another’s triangle means, but is incorrect, and builds a skill upon a faulty assumption. This raises the cost of onboarding conversation designers and bringing on CxD teams. The payroll platform Gusto puts the cost of replacing a technical worker (measured by being a sole contributor with a high salary, which conversation designers mostly fall into) [at $84,000](https://gusto.com/partners/accountants/resources/advising-clients-remote-and-hybrid-workplace) . A large part of that is retraining—teaching them the unique language of that team’s flows. And the triangle is just one example. Every day at Voiceflow we hear: * Should conversation designers design the natural language understanding (NLU) bot, or is that the NLU designer’s job? * Is it “repair path” or “error path”? Can an error path contain repair paths? * Is it a slot, entity, or variable? What’s the difference? At Voiceflow, we find ourselves often having to create our own definitions simply to be able to create our product and work on shared understanding. One example of this was defining the [six variations we see on the term “context”](https://www.voiceflow.com/blog/6-ways-to-build-context-into-your-conversation-designs) . You hear about “context” all the time in conversation design, yet it can have wildly different meanings from team to team. In order to build tooling to cover these use cases, we first had to define it ourselves. How many other teams have written their own definition? > We aren’t going to be able to define, share, and encode best practices if we can’t even agree on what the building blocks are called. Another example of an actual product change to match industry terms, we used to call our entities “slots,” which came from Amazon’s Alexa skill-building lexicon, but have since moved to entities as we saw the industry consolidate on that term. As a toolmaker, it can be difficult to always have the most up-to-date lexicon, as there is no singular institution maintaining a source of truth. It’ll fall to some industry body or collective to agree on and build a true glossary. Until that happens, unfortunately, lots of the software companies and agencies in this space are actively making it worse—often, unintentionally. In an attempt to trademark features and processes, companies add to the pile of terms with new, proprietary synonyms for things that already have well-accepted names. Seeing product launch posts like this on LinkedIn reminds me of the Scooby Doo meme where underneath the so-called “revolutionary” release is … a decades-old thing with a self-serving twist. As an industry, we aren’t going to be able to define, share, and encode best practices if we can’t even agree on what the building blocks are called. We can’t talk about good and bad when we’re speaking different languages. Who’ll provide these standards? ------------------------------- As an industry, we have to commit to building the category, not just our own brands. At Voiceflow, we will provide terms for things we feel aren’t well-named, but, if there is a better, or even just widely used adopted name, we’ll shift our lexicon to match. A rising tide lifts all boats—and ultimately, one of the biggest lifts we can all offer our industry is agreeing on how we describe. And to anyone out there working on a project to define all this, I’d love to chat. When I think about the ESRO’s lunar landing that never happened, I can’t help but also think about the conversation design industry, and what sort of dreams will go unrealized if we can’t end the mistranslation. The inconsistent and shifting language of conversation design ------------------------------------------------------------- The mystery of the indecipherable design triangle is actually a very real issue. Today, within one conversation design team, a triangle dropped into a flow means something completely different than what it means for designers at another company. Moreover, they often can’t even agree on when or how that triangle should be used, let alone what it means. There’s no reason for that. But it has a cost. It means that during a collaboration, designers’ flows aren’t mutually intelligible. Each might as well be hieroglyphics. And that’s true within those companies, across teams and departments. Whenever a conversation design leader is hired from somewhere else and brings their preferences, or two teams work in silos, you get compounding inconsistencies. In the most inconvenient scenarios, you get false positives—instances where one designer thinks they know what another’s triangle means, but is incorrect, and builds a skill upon a faulty assumption. This raises the cost of onboarding conversation designers and bringing on CxD teams. The payroll platform Gusto puts the cost of replacing a technical worker (measured by being a sole contributor with a high salary, which conversation designers mostly fall into) [at $84,000](https://gusto.com/partners/accountants/resources/advising-clients-remote-and-hybrid-workplace) . A large part of that is retraining—teaching them the unique language of that team’s flows. And the triangle is just one example. Every day at Voiceflow we hear: * Should conversation designers design the natural language understanding (NLU) bot, or is that the NLU designer’s job? * Is it “repair path” or “error path”? Can an error path contain repair paths? * Is it a slot, entity, or variable? What’s the difference? At Voiceflow, we find ourselves often having to create our own definitions simply to be able to create our product and work on shared understanding. One example of this was defining the [six variations we see on the term “context”](https://www.voiceflow.com/blog/6-ways-to-build-context-into-your-conversation-designs) . You hear about “context” all the time in conversation design, yet it can have wildly different meanings from team to team. In order to build tooling to cover these use cases, we first had to define it ourselves. How many other teams have written their own definition? > We aren’t going to be able to define, share, and encode best practices if we can’t even agree on what the building blocks are called. Another example of an actual product change to match industry terms, we used to call our entities “slots,” which came from Amazon’s Alexa skill-building lexicon, but have since moved to entities as we saw the industry consolidate on that term. As a toolmaker, it can be difficult to always have the most up-to-date lexicon, as there is no singular institution maintaining a source of truth. It’ll fall to some industry body or collective to agree on and build a true glossary. Until that happens, unfortunately, lots of the software companies and agencies in this space are actively making it worse—often, unintentionally. In an attempt to trademark features and processes, companies add to the pile of terms with new, proprietary synonyms for things that already have well-accepted names. Seeing product launch posts like this on LinkedIn reminds me of the Scooby Doo meme where underneath the so-called “revolutionary” release is … a decades-old thing with a self-serving twist. As an industry, we aren’t going to be able to define, share, and encode best practices if we can’t even agree on what the building blocks are called. We can’t talk about good and bad when we’re speaking different languages. Who’ll provide these standards? ------------------------------- As an industry, we have to commit to building the category, not just our own brands. At Voiceflow, we will provide terms for things we feel aren’t well-named, but, if there is a better, or even just widely used adopted name, we’ll shift our lexicon to match. A rising tide lifts all boats—and ultimately, one of the biggest lifts we can all offer our industry is agreeing on how we describe. And to anyone out there working on a project to define all this, I’d love to chat. When I think about the ESRO’s lunar landing that never happened, I can’t help but also think about the conversation design industry, and what sort of dreams will go unrealized if we can’t end the mistranslation.  RECOMMENDED [\ \ ### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) PUBLISHED October 18, 2022 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/6-best-practices-to-design-more-impactful-conversations) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 6 effective ways to design more impactful conversations](https://www.voiceflow.com/pathways/6-best-practices-to-design-more-impactful-conversations) [](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Automation Design: The Evolution of Conversation Design](https://www.voiceflow.com/pathways/automation-design-the-evolution-of-conversation-design) [](https://www.voiceflow.com/pathways/conversation-design-career-journey) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### From Hobbits to bots—lessons from Simon Mitchell-Wolf’s journey to CxD](https://www.voiceflow.com/pathways/conversation-design-career-journey) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/conversation-design-needs-a-universal-language#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/conversation-design-needs-a-universal-language#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # 3 reasons everyone hates your AI agent (and what you can do about it) | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Chatbot glow-up: 3 reasons everyone hates your AI agent (and what you can do about it) ====================================================================================== [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) It’s not a controversial opinion that most chatbots are bad. Every time I tell someone what I do for a living, I receive a cacophony of veiled insults—”I hate bots” “OMG, you do _that_?” “I never use chatbots, they’re so annoying.” We get it, people hate the bots we make. I only cry about it quarterly now. But customers have been having mediocre conversations with chatbots for years and those poor experiences have left a mark. Traditional bots were technical projects by overworked developers. Those bots weren’t designed so much as they were thrown together with a lack of customer data and a graveyard of FAQs. As a result, they didn’t see a lot of return on investment (ROI), brought back abysmal customer satisfaction (CSAT) scores, and were eventually left to deteriorate as a result. Sound familiar? The thing is, it doesn’t have to be this way anymore. So, let’s talk candidly. Here are the three reasons why your chatbot might suck and what you can do about it.  Your agent might suffer from poor user experience (UX) design (or you’re stuck in a hard-coded, single-interface nightmare) ---------------------------------------------------------------------------------------------------------------------------- Bots have traditionally been seen as a technical challenge rather than a design one. The developers who build them don’t often have an overall design structure or purpose—just a mandate from their higher-ups, some requests from stakeholders, and a list of things that other teams think the bot should do. That method is a recipe for disaster. You're not collaborating across teams to understand your customer needs, collecting user data to personalize agent responses, or designing multiple chat interfaces across channels based on the best of your visual branding. Instead, most companies settle for a cookie-cutter AI chatbot that’s simple to deploy, but can’t handle much more than the simplest of requests (and still sometimes fumbles them) before directing users to a live agent. As you can probably guess, [I have a lot of feelings](https://www.voiceflow.com/blog/conversational-ai-ux) about the massive missteps conversational AI (CAI) teams make when it comes to UX design. For instance, Sephora’s agent sits across their mobile app and website, but its design is so clunky across both that it's consistently a pain to use. ### The fix: Solve your agent’s massive UX problem Start with addressing the underlying problem—the lack of UX support for teams designing AI agents. Now, most CAI teams are small and scrappy, so you’re unlikely to be able to afford a full-fledged UX designer (though, I wish we all could). Instead, make the case to borrow their time from another team. Involve a UX designer at pivotal points in the process—from planning and strategy on what the agent will do, to how the bot should be launched across interfaces, and while you’re collecting user feedback. Folks in UX can help answer questions like: Why do we assume users want a form/FAQ/live agent right here? How are users actually interacting with us today? Is this a problem that needs solving? How can we make this experience smoother? Further, don’t trap your agent on the bottom corner of your homepage. You can embed AI agents across the business and its custom interfaces, including in-app, IVR, webchat, Discord, and your help center. Boxing your AI agent into a webchat wrapper for every interaction might not be the best idea. Customers expect support across their journey, and they don’t want to have to leave their current flow to get it. Instead, bring it to them. Your agent doesn’t use existing internal and user data to make the conversation personalized (or your bot is where your FAQs go to die) ---------------------------------------------------------------------------------------------------------------------------------------- Companies are more concerned about the risk of their AI agent going rogue than they are about the potential of it performing well. Now, there are privacy concerns that are worth considering when integrating your customer data with a LLM. That’s why at Voiceflow we ensure that the data that passes through the LLMs is never used to retrain the AI model. The problem is this fear keeps your AI agents frozen in time and unable to evolve as they should. You end up using bloated dev-only platforms that aren’t accessible to your full team and are difficult to integrate across your data stack. As a result, tailored responses for each unique user are nearly impossible unless they are meticulously planned by the conversation designer. When you build traditional turn-by-turn agents without personalizing the experiences for each user, it’s no wonder bots get a bad rap. Not to mention, who has the time to write every single potential flow from start to finish anymore? Gone are those days, and I’m glad to be rid of them. Even Fortune 500 companies can make these mistakes. For Wells Fargo, providing customers with correct information is paramount—so hallucinations are a big concern. However, by using a [knowledge base](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) along with some clever prompting, we can get an LLM to act as teacher and student, self-checking its own work before it goes out to the customer. ### The fix: Connect your AI agent to your knowledge base and an LLM to ensure flexible, contextual conversations This degree of personalization does more than elevate customer satisfaction—it nurtures loyalty and transforms your customer support. Imagine how impactful your AI agent could be if you could create a deeply personal shopping experience, for example. Your AI agent might suggest products to your user based on past purchases, browsing history, and even current viewing trends. It could even offer care tips or upsell with recommended products. Similarly, you could use your AI assistant to pull up a customer's product usage or to provide personalized troubleshooting instructions based on their past issues logged in your CRM or product database. It can route the query to specialized support paths based on the product type or issue severity, using an internal knowledge base to guide the customer through a resolution. This type of AI-powered conversational magic isn't out of reach for you, no matter what stage of [AI maturity](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) you're in. You can even load your entire website into a knowledge base, creating a central repository of help articles, PDF documents, and .txt files—we even have a helpful [guide](https://learn.voiceflow.com/hc/en-us/articles/19649420517389-How-the-Knowledge-Base-Works#:~:text=The%20Knowledge%20Base%20(KB)%20allows,you%20upload%20a%20KB%20document) to help you do just that. With an AI-powered agent—trained on your brand’s content—you’ll be able to generate responses that feel like an extension of your human customer support team. Your agent can only complete limited actions (and it foists complex tasks over to a live agent) ----------------------------------------------------------------------------------------------- Most users have zero patience for a mediocre agent. A live agent could make a mistake or there could be a typo in a help article, and customers are more forgiving of those snafus than they are of a chatbot that promptly misunderstands a request before responding with something entirely unrelated. It’s borderline unforgivable. Take Nike’s AI agent, for example. It’s not entirely useless—it can complete tasks like processing a refund or tracking an order. But it often fails on really easy things, like understanding what the user is saying. Because it fails at the easy stuff, it makes users lose confidence that it can do anything. ### The fix: build an AI agent to handle complex customer interactions and complete helpful tasks. Traditional agents once required you to deterministically define everything that you wanted your bot to say and do. This made complex tasks hard to design and agents generally inflexible to intents that weren’t accounted for by the CAI team. Today, all it takes is an LLM to search for and generate a semantic match to a user’s natural language intent. With the foundational technology of an LLM at your disposal—and API integrations with your ecommerce site, customer support channel, or customer data source—your AI agent can handle multi-faceted requests such as merging accounts, adjusting service levels, or applying nuanced billing adjustments. You can even equip your agent to make informed decisions on behalf of the business, such as approving discounts or customizing offers based on predefined criteria and customer data. For example, the team at Trilogy managed to automate support tasks across 90 support lines for 24/7 support coverage. It’s an impressive feat that it took a dedicated team of two people—[yes, only two](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) —to design a core AI agent integrated with Trilogy’s help center interface, LLMs, Knowledge Base API, and a set of user support flows and functions. In under 12 weeks, the team achieved the staggering result of having 60% of central support tickets completely resolved by AI. If they can do it, so can you. These complex tasks can help boost your resolution rate and CSAT scores while slashing your resolution time. The key is to deploy LLM and NLU models thoughtfully at various parts of the interaction to facilitate specific tasks—not just implementing a generalized LLM model. My advice? Explore how you can continue to use the power of LLMs to handle complex flows from end to end. Working with LLMs is just more fun. Learning how to tame them and build frameworks for your prompt chains to work within is so much more satisfying than designing turn-by-turn conversations. Sometimes you look at the results and just think wow, that's magic. You can learn more about building with LLMs from the [Making Bots series](https://www.youtube.com/playlist?list=PLKYemGIohRgBdoIl2cOWJQ3xSBUgMGW5j) or how to prompt LLMs from [Learn Prompting](https://learnprompting.org/) . Most importantly, you can learn by playing. Your AI agent doesn't have to suck ---------------------------------- Remember, if your agent is struggling to win over hearts, it's time for a revamp. First, ditch the UX design that's as appealing as a dial-up tone and get some cross-team collaboration going. Spruce up that bot with a bit of personalization by hooking it up to your user data and knowledge base because nobody likes one-size-fits-all conversations. And finally, use LLMs as your partner in automating complex tasks—trust me, the reward is worth the risk. With these improvements, your AI agent can evolve from a source of frustration to a valuable asset that impresses users and streamlines support. Your AI agent has the potential to be something special—or at the very least, not the thing people love to hate. Your agent might suffer from poor user experience (UX) design (or you’re stuck in a hard-coded, single-interface nightmare) ---------------------------------------------------------------------------------------------------------------------------- Bots have traditionally been seen as a technical challenge rather than a design one. The developers who build them don’t often have an overall design structure or purpose—just a mandate from their higher-ups, some requests from stakeholders, and a list of things that other teams think the bot should do. That method is a recipe for disaster. You're not collaborating across teams to understand your customer needs, collecting user data to personalize agent responses, or designing multiple chat interfaces across channels based on the best of your visual branding. Instead, most companies settle for a cookie-cutter AI chatbot that’s simple to deploy, but can’t handle much more than the simplest of requests (and still sometimes fumbles them) before directing users to a live agent. As you can probably guess, [I have a lot of feelings](https://www.voiceflow.com/blog/conversational-ai-ux) about the massive missteps conversational AI (CAI) teams make when it comes to UX design. For instance, Sephora’s agent sits across their mobile app and website, but its design is so clunky across both that it's consistently a pain to use. ### The fix: Solve your agent’s massive UX problem Start with addressing the underlying problem—the lack of UX support for teams designing AI agents. Now, most CAI teams are small and scrappy, so you’re unlikely to be able to afford a full-fledged UX designer (though, I wish we all could). Instead, make the case to borrow their time from another team. Involve a UX designer at pivotal points in the process—from planning and strategy on what the agent will do, to how the bot should be launched across interfaces, and while you’re collecting user feedback. Folks in UX can help answer questions like: Why do we assume users want a form/FAQ/live agent right here? How are users actually interacting with us today? Is this a problem that needs solving? How can we make this experience smoother? Further, don’t trap your agent on the bottom corner of your homepage. You can embed AI agents across the business and its custom interfaces, including in-app, IVR, webchat, Discord, and your help center. Boxing your AI agent into a webchat wrapper for every interaction might not be the best idea. Customers expect support across their journey, and they don’t want to have to leave their current flow to get it. Instead, bring it to them. Your agent doesn’t use existing internal and user data to make the conversation personalized (or your bot is where your FAQs go to die) ---------------------------------------------------------------------------------------------------------------------------------------- Companies are more concerned about the risk of their AI agent going rogue than they are about the potential of it performing well. Now, there are privacy concerns that are worth considering when integrating your customer data with a LLM. That’s why at Voiceflow we ensure that the data that passes through the LLMs is never used to retrain the AI model. The problem is this fear keeps your AI agents frozen in time and unable to evolve as they should. You end up using bloated dev-only platforms that aren’t accessible to your full team and are difficult to integrate across your data stack. As a result, tailored responses for each unique user are nearly impossible unless they are meticulously planned by the conversation designer. When you build traditional turn-by-turn agents without personalizing the experiences for each user, it’s no wonder bots get a bad rap. Not to mention, who has the time to write every single potential flow from start to finish anymore? Gone are those days, and I’m glad to be rid of them. Even Fortune 500 companies can make these mistakes. For Wells Fargo, providing customers with correct information is paramount—so hallucinations are a big concern. However, by using a [knowledge base](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) along with some clever prompting, we can get an LLM to act as teacher and student, self-checking its own work before it goes out to the customer. ### The fix: Connect your AI agent to your knowledge base and an LLM to ensure flexible, contextual conversations This degree of personalization does more than elevate customer satisfaction—it nurtures loyalty and transforms your customer support. Imagine how impactful your AI agent could be if you could create a deeply personal shopping experience, for example. Your AI agent might suggest products to your user based on past purchases, browsing history, and even current viewing trends. It could even offer care tips or upsell with recommended products. Similarly, you could use your AI assistant to pull up a customer's product usage or to provide personalized troubleshooting instructions based on their past issues logged in your CRM or product database. It can route the query to specialized support paths based on the product type or issue severity, using an internal knowledge base to guide the customer through a resolution. This type of AI-powered conversational magic isn't out of reach for you, no matter what stage of [AI maturity](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) you're in. You can even load your entire website into a knowledge base, creating a central repository of help articles, PDF documents, and .txt files—we even have a helpful [guide](https://learn.voiceflow.com/hc/en-us/articles/19649420517389-How-the-Knowledge-Base-Works#:~:text=The%20Knowledge%20Base%20(KB)%20allows,you%20upload%20a%20KB%20document) to help you do just that. With an AI-powered agent—trained on your brand’s content—you’ll be able to generate responses that feel like an extension of your human customer support team. Your agent can only complete limited actions (and it foists complex tasks over to a live agent) ----------------------------------------------------------------------------------------------- Most users have zero patience for a mediocre agent. A live agent could make a mistake or there could be a typo in a help article, and customers are more forgiving of those snafus than they are of a chatbot that promptly misunderstands a request before responding with something entirely unrelated. It’s borderline unforgivable. Take Nike’s AI agent, for example. It’s not entirely useless—it can complete tasks like processing a refund or tracking an order. But it often fails on really easy things, like understanding what the user is saying. Because it fails at the easy stuff, it makes users lose confidence that it can do anything. ### The fix: build an AI agent to handle complex customer interactions and complete helpful tasks. Traditional agents once required you to deterministically define everything that you wanted your bot to say and do. This made complex tasks hard to design and agents generally inflexible to intents that weren’t accounted for by the CAI team. Today, all it takes is an LLM to search for and generate a semantic match to a user’s natural language intent. With the foundational technology of an LLM at your disposal—and API integrations with your ecommerce site, customer support channel, or customer data source—your AI agent can handle multi-faceted requests such as merging accounts, adjusting service levels, or applying nuanced billing adjustments. You can even equip your agent to make informed decisions on behalf of the business, such as approving discounts or customizing offers based on predefined criteria and customer data. For example, the team at Trilogy managed to automate support tasks across 90 support lines for 24/7 support coverage. It’s an impressive feat that it took a dedicated team of two people—[yes, only two](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) —to design a core AI agent integrated with Trilogy’s help center interface, LLMs, Knowledge Base API, and a set of user support flows and functions. In under 12 weeks, the team achieved the staggering result of having 60% of central support tickets completely resolved by AI. If they can do it, so can you. These complex tasks can help boost your resolution rate and CSAT scores while slashing your resolution time. The key is to deploy LLM and NLU models thoughtfully at various parts of the interaction to facilitate specific tasks—not just implementing a generalized LLM model. My advice? Explore how you can continue to use the power of LLMs to handle complex flows from end to end. Working with LLMs is just more fun. Learning how to tame them and build frameworks for your prompt chains to work within is so much more satisfying than designing turn-by-turn conversations. Sometimes you look at the results and just think wow, that's magic. You can learn more about building with LLMs from the [Making Bots series](https://www.youtube.com/playlist?list=PLKYemGIohRgBdoIl2cOWJQ3xSBUgMGW5j) or how to prompt LLMs from [Learn Prompting](https://learnprompting.org/) . Most importantly, you can learn by playing. Your AI agent doesn't have to suck ---------------------------------- Remember, if your agent is struggling to win over hearts, it's time for a revamp. First, ditch the UX design that's as appealing as a dial-up tone and get some cross-team collaboration going. Spruce up that bot with a bit of personalization by hooking it up to your user data and knowledge base because nobody likes one-size-fits-all conversations. And finally, use LLMs as your partner in automating complex tasks—trust me, the reward is worth the risk. With these improvements, your AI agent can evolve from a source of frustration to a valuable asset that impresses users and streamlines support. Your AI agent has the potential to be something special—or at the very least, not the thing people love to hate.  RECOMMENDED [\ \ ### Crawl, walk, run: 28+ tactics for evolving your AI agent](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) PUBLISHED April 8, 2024 TOPICS IN THIS ARTICLE Exceptional work Learning and APIs [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.avif)](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 10 things every voice app should do](https://www.voiceflow.com/pathways/10-things-every-voice-app-should-do) [](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) [](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Chatbot glow-up: 3 reasons everyone hates your AI agent (and what you can do about it)](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Conversational AI has a massive, UX-shaped hole | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Conversational AI has a massive, UX-shaped hole =============================================== [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) If you lived in 1972 and thought the brand-new Cartrivision system that recorded TV shows would revolutionize watching, you would be disappointed. That wouldn’t become widespread until 1999. The reason was, the [Cartrivision](https://www.mentalfloss.com/article/642374/vcr-history) required “two hands to program … with one pressing a button while the other twirled a knob.” It was finicky and expensive. Cartrivision, and many similarly difficult devices that followed, gave rise to a software design meme “[blinking twelve](https://en.wikipedia.org/wiki/Blinking_twelve_problem) ,” a term for a system so difficult to operate, users don’t even try. (Most VCR owners never learned to change the clock, so they just left it “blinking 12:00.”) Which brings us to now. You can build all the fancy conversation technology in the world but if that final interface is no good, your idea is three decades too early. Right now, I think this is a massive problem in conversational AI (CAI) and there’s a need for these teams to bring UX designers into conversations. This sort of user empathy is missing, leading to many interfaces that aren’t all that friendly. If we involve more UX designers, who excel at fostering familiarity with accessible, intuitive experiences, I think we can unlock a new paradigm for CAI: One that will really start to transform not just how we interface with assistants, but think in more dimensions about what they can do for us. Here are three places I believe their talent is urgently needed.  1\. Rethinking the linear web interface --------------------------------------- Conventional website design and information architecture tend to be linear, emphasizing headlines and hierarchy. But why? Because it was based on paper documents? Computers no longer have to think linearly, neither do we, and I think UX designers can help us break out of this. If we joined forces with UX designers, we could create far more dynamic user interactions with websites and dashboards. Rather than dashboards being a big grid of tiles, users could simply ask an assistant questions about the data. Those users would no longer need to use a mouse to point and click. You wouldn’t even have to train them on the dashboard—only on asking questions. And this is just one idea. I’m not even a UX designer. I think they could utterly revolutionize the AI landscape by introducing novel interaction patterns that transcend traditional chatbot constraints. They could integrate features like adaptable conversation flows, context sensitivity, and dynamic interfaces. > "If we joined forces with UX designers, we could create far more dynamic user interactions with websites and dashboards." Because again, UX designers aren’t starting with the technology. They’re starting with people and their intents. And just like the Google Assistant team knows, users don’t always want to repeat the wake word. It’s unnatural. Instead, they want to engage in a more organic back and forth. The result is more fluid and human-like. UX designers could help us answer the question: How could everything about CAI be more fluid and human-like? 2\. Building micro interfaces (UIs) and ephemeral apps ------------------------------------------------------ Generative AI can code. With that realization comes the reality that many apps may soon cost nothing. There’ll always be a place for the large-enterprise, high-design, high-security iOS app, or the voice assistant that creates a genuinely delightful experience, but if you want to build your own tiny app and run it in your own environment? That’s basically free. Smart people are predicting this will create a category of what is known as “ephemeral apps” and micro user interfaces. **Ephemeral apps will need to be easy to use** The idea with ephemeral apps is if your assistant knows what you want to achieve, it can field-code its own tiny, temporary app to help you accomplish this task. For example, it could amalgamate shopping lists, recipe suggestions, and guest list management into one tool for a dinner party. And once the party is over, the app would disappear, leaving no trace of its existence. Those sorts of applications will need clear and intuitive conventions, drawn from everything UX designers already know about such interfaces. **Micro interfaces will need to know how people think** Your assistant will be able to code a new micro interface to display or collect information in a format that’s even more intuitive than text chat. For example, “Okay, please adjust these color selectors,” “Assemble your ideal car,” or “Here’s your personal calendar, why don’t you select a time when you’d like a callback in between your work?” The assistant could design those interactions to reveal these UI elements as users progress through the conversation to make the conversation much easier. The entire micro UI would be custom to that user and situation. And it’d need to follow UX design principles to ensure it is, in fact, easier and more satisfying than the alternative. 3\. Integrating UX research into the CAI process ------------------------------------------------ I’ve often tried to get a UX designer on our team at past companies, but of course, budgetary constraints stopped us. That’s usually the only rational objection to having a truly [multidisciplinary CAI team](https://www.voiceflow.com/blog/multidisciplinary-cai-teams) . But let’s say you’re able to hire someone or borrow them from another team. Then, you’ll need to shift the way you work to make room for their contributions. I’ll bet you’ll want to include them in the whole process, but be very tactical about what projects you most need their input on at the start, middle, and end. * At the start, you want them asking questions like, “Why is our default assumption that users want a form?” and “What data do we have about how users are actually interacting with these elements today?” * In the middle, you’ll want to make space for them being the user’s champion, and asking, “But are people _actually_ asking for this? Is this truly a problem that needs solving?” * And throughout, but especially at the end, you want them doing UX research—talking to customers to figure out what they were hoping to accomplish, how it went, and areas that you might improve. At first, they may have to unlearn the linear interface ------------------------------------------------------- It’s also probably the case that many UX designers schooled in linear web design won’t yet understand what a profound shift CAI is having on the way we interact with websites, apps, and data. It may take a moment to get their head around the fact that everything they know about page hierarchy and presentation is no longer the center of the known universe—that there are other planets. (Especially because they’ve probably only interacted with mediocre chatbots.) But I’m convinced those who see the potential for things like non-linear interfaces, micro UIs, and ephemeral apps will understand. And they’ll help you completely rethink how you’re going about it. Let’s party like it’s 1999 -------------------------- So what happened in 1999? TiVo launched. It was a set-top box that recorded TV while it played so you could pause, fast forward, and rewind just like with any recorded medium. Unlike any of its predecessors, [TiVo was loved](https://www.tivopedia.com/tivo-history.php) . The remote was obvious and simple. Once set up, anyone in the family could work it. It was so successful, it was a commercial failure—it was so heavily copied, it’s now an intrinsic feature of most TVs and presaged the entire streaming era. For a few years, TiVo was a verb. [It won awards](https://spectrum.ieee.org/the-consumer-electronics-hall-of-fame-tivo) . And it normalized the idea that one ought to be able to record and watch shows at will. It was the revolution the Cartrivision had wished for. And it was a revolution because of its UX. **Here’s a bonus: How to make a case for a UX designer** * Identify the biggest performance challenges your CAI team faces * Identify your boss’ targets and the most important thing they’re measured on * Quantify the opportunity cost to their targets in not addressing these challenges * Propose ways a UX designer might address those challenges * Quantify the benefit of having those challenges fixed * Present this to your boss in the above order * Tell them I sent you! 1\. Rethinking the linear web interface --------------------------------------- Conventional website design and information architecture tend to be linear, emphasizing headlines and hierarchy. But why? Because it was based on paper documents? Computers no longer have to think linearly, neither do we, and I think UX designers can help us break out of this. If we joined forces with UX designers, we could create far more dynamic user interactions with websites and dashboards. Rather than dashboards being a big grid of tiles, users could simply ask an assistant questions about the data. Those users would no longer need to use a mouse to point and click. You wouldn’t even have to train them on the dashboard—only on asking questions. And this is just one idea. I’m not even a UX designer. I think they could utterly revolutionize the AI landscape by introducing novel interaction patterns that transcend traditional chatbot constraints. They could integrate features like adaptable conversation flows, context sensitivity, and dynamic interfaces. > "If we joined forces with UX designers, we could create far more dynamic user interactions with websites and dashboards." Because again, UX designers aren’t starting with the technology. They’re starting with people and their intents. And just like the Google Assistant team knows, users don’t always want to repeat the wake word. It’s unnatural. Instead, they want to engage in a more organic back and forth. The result is more fluid and human-like. UX designers could help us answer the question: How could everything about CAI be more fluid and human-like? 2\. Building micro interfaces (UIs) and ephemeral apps ------------------------------------------------------ Generative AI can code. With that realization comes the reality that many apps may soon cost nothing. There’ll always be a place for the large-enterprise, high-design, high-security iOS app, or the voice assistant that creates a genuinely delightful experience, but if you want to build your own tiny app and run it in your own environment? That’s basically free. Smart people are predicting this will create a category of what is known as “ephemeral apps” and micro user interfaces. **Ephemeral apps will need to be easy to use** The idea with ephemeral apps is if your assistant knows what you want to achieve, it can field-code its own tiny, temporary app to help you accomplish this task. For example, it could amalgamate shopping lists, recipe suggestions, and guest list management into one tool for a dinner party. And once the party is over, the app would disappear, leaving no trace of its existence. Those sorts of applications will need clear and intuitive conventions, drawn from everything UX designers already know about such interfaces. **Micro interfaces will need to know how people think** Your assistant will be able to code a new micro interface to display or collect information in a format that’s even more intuitive than text chat. For example, “Okay, please adjust these color selectors,” “Assemble your ideal car,” or “Here’s your personal calendar, why don’t you select a time when you’d like a callback in between your work?” The assistant could design those interactions to reveal these UI elements as users progress through the conversation to make the conversation much easier. The entire micro UI would be custom to that user and situation. And it’d need to follow UX design principles to ensure it is, in fact, easier and more satisfying than the alternative. 3\. Integrating UX research into the CAI process ------------------------------------------------ I’ve often tried to get a UX designer on our team at past companies, but of course, budgetary constraints stopped us. That’s usually the only rational objection to having a truly [multidisciplinary CAI team](https://www.voiceflow.com/blog/multidisciplinary-cai-teams) . But let’s say you’re able to hire someone or borrow them from another team. Then, you’ll need to shift the way you work to make room for their contributions. I’ll bet you’ll want to include them in the whole process, but be very tactical about what projects you most need their input on at the start, middle, and end. * At the start, you want them asking questions like, “Why is our default assumption that users want a form?” and “What data do we have about how users are actually interacting with these elements today?” * In the middle, you’ll want to make space for them being the user’s champion, and asking, “But are people _actually_ asking for this? Is this truly a problem that needs solving?” * And throughout, but especially at the end, you want them doing UX research—talking to customers to figure out what they were hoping to accomplish, how it went, and areas that you might improve. At first, they may have to unlearn the linear interface ------------------------------------------------------- It’s also probably the case that many UX designers schooled in linear web design won’t yet understand what a profound shift CAI is having on the way we interact with websites, apps, and data. It may take a moment to get their head around the fact that everything they know about page hierarchy and presentation is no longer the center of the known universe—that there are other planets. (Especially because they’ve probably only interacted with mediocre chatbots.) But I’m convinced those who see the potential for things like non-linear interfaces, micro UIs, and ephemeral apps will understand. And they’ll help you completely rethink how you’re going about it. Let’s party like it’s 1999 -------------------------- So what happened in 1999? TiVo launched. It was a set-top box that recorded TV while it played so you could pause, fast forward, and rewind just like with any recorded medium. Unlike any of its predecessors, [TiVo was loved](https://www.tivopedia.com/tivo-history.php) . The remote was obvious and simple. Once set up, anyone in the family could work it. It was so successful, it was a commercial failure—it was so heavily copied, it’s now an intrinsic feature of most TVs and presaged the entire streaming era. For a few years, TiVo was a verb. [It won awards](https://spectrum.ieee.org/the-consumer-electronics-hall-of-fame-tivo) . And it normalized the idea that one ought to be able to record and watch shows at will. It was the revolution the Cartrivision had wished for. And it was a revolution because of its UX. **Here’s a bonus: How to make a case for a UX designer** * Identify the biggest performance challenges your CAI team faces * Identify your boss’ targets and the most important thing they’re measured on * Quantify the opportunity cost to their targets in not addressing these challenges * Propose ways a UX designer might address those challenges * Quantify the benefit of having those challenges fixed * Present this to your boss in the above order * Tell them I sent you!  RECOMMENDED [\ \ ### The future of conversational AI](https://www.voiceflow.com/pathways/future-of-conversational-ai) PUBLISHED July 14, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) [](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Cathy Pearl on all things conversation design](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design) [](https://www.voiceflow.com/pathways/getting-started-in-conversation-design-heres-what-you-need-to-know) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Getting started in conversation design? Here's what to know](https://www.voiceflow.com/pathways/getting-started-in-conversation-design-heres-what-you-need-to-know) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/conversational-ai-ux#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/conversational-ai-ux#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Behind the hype: How Klarna's customer support agent falls short | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Behind the hype: How Klarna's customer support agent falls short ================================================================ [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) Earlier this year, Klarna sent out a press release that sent shockwaves through the AI community. They partnered with OpenAI to release a customer support agent that addresses two-thirds of their customer support tickets, reportedly doing the equivalent work of 700 full-time agents. The reaction was swift. Vitriol for the lost jobs. Praise for the productivity of AI. Cheers and jeers all around. Heck, we even launched a campaign to share how you could achieve similar ([or better](https://www.linkedin.com/posts/braden-ream_keepitupkim-activity-7172939389289787392-9ZOE?utm_source=share&utm_medium=member_desktop) ) results. But right away I had questions: How effective is the Klarna agent, really? If it put 700 people out of a job (allegedly), it ought to be exceptional, right? Is an OpenAI-cosign the key to creating more effective AI agents? In this race towards early AI automation, real competitive advantage will come from thoughtful design, testing, and iteration of these experiences. Otherwise, the flashy automation numbers will be surface level only—never making a consistent impact on customer experience or business outcomes. So, I decided it was time to look closer at the Klarna AI agent now that the fanfare has quieted. And my experience was… underwhelming. Despite the flashy press, the Klarna agent is riddled with issues from the mildly annoying to the outrageously incorrect. Let’s break down six of those problems together, while I share practical advice you can use to avoid similar pitfalls. By the end, I think you’ll agree you don’t need a special partnership with Sam Altman to create an effective agent that handles two-thirds (or even more) of your customer support. _Disclaimer: This is all in good fun. We’re not starting beef with Klarna or OpenAI, just breaking down an agent that will help us learn a few things together. Let’s be kind to the cool devs and designers involved in making this bot possible._  I’ve got 6 problems with the Klarna agent, here’s how to fix them: ------------------------------------------------------------------- ### 1\. Generates verbose responses I started my tests by asking a simple question: _What is Klarna?_ I got a response that was lengthy and filled the screen with text.  It is possible to answer this question without going into a brand story—in fact, you should be concise. To answer this question without taking up all this valuable real estate, you should explain your company in 2-3 sentences, followed by top product features, and what briefly sets you apart from competitors. That’s a conversation design best practice. But this verbosity was consistent with the Klarna agent. No matter the question, from simple to complex, the response I received was always longer than necessary and took up most of the chat window. At this point, I was starting to understand Kevin’s humble adage. Now, if you’re using an LLM paired with a knowledge base to address basic questions from users ([which you should](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) ), then dealing with verbosity from your LLM just requires some clever prompting. Here’s an example of what this prompt could look like: `As a chatbot support agent, provide a clear and concise response to the user’s question: “{userQuestion}” ` `With the provided details: “{chunks}” ` `Instructions:` `- Summarize key points briefly, within three to four sentences.` `- Use bullet points for clarity and numbered lists for step-by-step instructions.` `- Start with a direct answer before using bullet points for additional details.` `- Write in simple, clear language.` `- Apply markdown syntax for formatting.` `- Ensure accuracy with the core information given.` `- Respond with "chunks" if essential details are missing.` `- Include hyperlinks only when relevant, formatted as: "Name of the page page."` `- Avoid phrases like 'information provided.'` `IMPORTANT:` `If ‘chunks’ do not contain the needed information to answer a question, respond with “NOANSWER”` These instructions not only work well for an LLM, but also for static responses. Always remember, users will skim responses—so break up lengthy responses with numbered or bulleted lists where you can. You can also do things like: * Provide buttons with common user responses * Offer a maximum of three options for each conversation item * If more options are necessary, group them into a “more options” button And remember, a conversation might not always be the best way to automate something. You can use other UI elements like forms, images, attachments, and carousels. ### 2\. Unable to perform simple actions When prompted, the Klarna agent has a list of actions it says it can complete—updating personal details, changing payment information, or sharing information about my purchases. In my experience, it is unable to complete simple actions on behalf of the user.  When asking the agent to update my name in my profile, it offered me a button to update my personal information by navigating away from the chat. That link led to nowhere. When I asked the agent to tell me what past purchases I had made—an easy task, because I had none—it prompted me to leave the chat to find them myself. I asked the agent to update my payment details, and once again I was prompted to follow a list of instructions to complete the action myself, outside the chat window. When I asked the agent if I could give the details in the chat, it said no and pushed me to a live agent. Over and over, I asked the Klarna agent to complete actions it had told me it could do, and every time it asked me to do the action myself, it pushed me to a live agent or failed altogether.  To improve the performance right away, I would ensure that it was very clear to the user what the agent can and can’t do. Setting expectations is key. Then, for actions the agent can’t complete, I’d ensure I have clear instructions in a knowledge base that an agent can pull from. For actions an agent can complete, it’s all about setting up intents that help capture utterances and then push them down the business process you’re trying to take. If an agent can’t automate the thing a user is trying to do, then clearly tell users how to do it. And if the agent can automate it, do it. ### 3\. Asks users to exit support to complete actions I will admit I’m not a Klarna customer—I started a blank account to run these tests—so this is not a scientific experiment. But if an agent claims it can do a collection of actions for me as a user, then I expect the agent to complete those actions. I don’t expect to be sent exploring my own profile with a laundry list of instructions. Now, generating instructions for a customer to DIY actions is not inherently bad. But what Klarna gets wrong in this case is the list of instructions in a static support window. When you select a button or go to a different page, that window disappears. Imagine the experience from the user’s perspective. They must read those verbose instructions, exit the support window, remember those instructions well enough to navigate the site, complete the action, and navigate back to the support window. No one will remember those instructions after five clicks, not to mention how to navigate back to the support. Which means they’ll be unable to complete the action they sought support for. The whole experience is just frustrating. So, what’s the solution? If you must ask your user to complete the action themselves (although I’d argue, there’s a bunch of [actions your agent can do](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) ), include a link that goes to where you want your user to go. (And make sure it works.) This is more deterministic flow but it ensures that the agent is more accurately supporting users. ### 4\. Hides the customer support chat In my opinion, finding the customer support chat was a hassle every time. Here’s a clip of me struggling, which happened repeatedly as the agent directed me to complete actions elsewhere. [](https://youtu.be/1Mkycplnm5w) Instead, Klarna should host their customer support in the bottom corner of the page as a dynamic window that stays with the user as they navigate the site (and you should too). Also, since they’re offering customer support already, I don’t recommend hiding it away. Adding unnecessary friction to the support process is counterintuitive. And when users are inevitably transferred to a live agent, they’ll take out those frustrations on them. ### 5\. Employs a sickly sweet tone Now, this may be because I am a [self-proclaimed killjoy](https://www.voiceflow.com/blog/confessions-of-a-cxd-killjoy-why-i-think-your-ai-assistant-shouldnt-be-funny) when it comes to agent’s personalities. But, the inhuman, enthusiastic tone of this agent often undermines its intentions. For example, I asked a common question for Klarna: _What happens if I can’t pay on time?_ And its answers were run-of-the-mill boilerplate. That isn’t my problem with the response—my issue is that it lacks empathy. It lacked anything that addressed the potential hardship of a customer who needs to ask that question in the first place.  Conversation designers do [a lot of things](https://www.voiceflow.com/blog/the-case-for-a-full-stack-conversation-designer) and one of the most important is keeping the human qualities of conversation alive in AI agents. In the case of a live customer support agent or an in-person interaction, there’s usually acknowledgement and empathy in response to someone sharing their difficulty in making payments on time. I encourage designers and developers to include moments of empathy, it could be as simple as starting the response with “I’m sorry to hear you’re having trouble making payments.” You can disagree with me, in fact, I welcome it (shoot me a message on [LinkedIn](https://www.linkedin.com/in/isaacspeter/) ). But I think moments like these really add up to a better user experience. I’d also suggest removing static pleasantries that you drop at the end of every paragraph. That’s not how people talk. ### 6\. Suffers from lengthy response times After receiving an input, the Klarna agent generated responses after 15-20 seconds. That doesn’t sound like much, but imagine asking your barista for a chai latte and watching them stare at you for 20 seconds before responding—it’s bearable, but painfully awkward. And from a technical standpoint, it’s unacceptable. I have a theory as to why the Klarna agent’s responses lag—I suspect using the beta of OpenAI’s assistants API is the culprit. For each response, the agent is accessing multiple tools at once, like knowledge retrieval, function calling, voice and tone tuning, and persistent threads. As a result, the response time is impacted. If you’re planning to use the new OpenAI assistants API, these current constraints are ones to consider. Setbacks are to be expected when an API is in beta, even with a company as innovative as OpenAI. In reality, this API is great at helping you launch an agent quickly, but not ideal at helping your agent perform actions quickly. To avoid lengthy response times with your agent, it’s best to use a hybrid approach. Use a knowledge base for information retrieval and deterministic flows (along with LLMs) to guide users through actions. Also, don’t use GPT-4, or Claude Opus for everything—it’s like driving a Ferrari two doors down to pick up milk. Sure, it’s cool but totally unnecessary. Instead, you can use cheaper and faster models to give the same feeling of flexibility, while also using deterministic flows to get things done. ### You don’t need to be BFFs with Sam Altman to automate your customer support If there’s one thing I want you to take away from this breakdown it’s this—you don’t need a special relationship with OpenAI to automate your customer support. With a [knowledge base](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) , LLMs, and an agent orchestration platform (like Voiceflow, obviously), you can consistently automate an impressive batch of your level 1 support tickets. In fact, [you can move beyond level 1 tickets](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) . Developers at [Trilogy](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) are making it happen—their team has automated 60% of their customer support across multiple product lines, in under 12 weeks. Meanwhile, our customer support agent [Tico](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) , offers multi-channel support for 97% of tickets and completes complex actions on behalf of users. And those are real results. I will say, for simple FAQs and level 1 inquiries, the Klarna bot excels. There is a team of incredible developers and designers who are working tirelessly to make the Klarna agent possible. And I don’t doubt that it will improve over time. This AI agent is an incredibly successful proof concept (POC) with issues that can be fixed with testing and iteration—the most important steps to avoid a POC that doesn’t deliver results. But given all the spectacle when that press release dropped, I expected to be wowed. Maybe next time. I’ve got 6 problems with the Klarna agent, here’s how to fix them: ------------------------------------------------------------------- ### 1\. Generates verbose responses I started my tests by asking a simple question: _What is Klarna?_ I got a response that was lengthy and filled the screen with text.  It is possible to answer this question without going into a brand story—in fact, you should be concise. To answer this question without taking up all this valuable real estate, you should explain your company in 2-3 sentences, followed by top product features, and what briefly sets you apart from competitors. That’s a conversation design best practice. But this verbosity was consistent with the Klarna agent. No matter the question, from simple to complex, the response I received was always longer than necessary and took up most of the chat window. At this point, I was starting to understand Kevin’s humble adage. Now, if you’re using an LLM paired with a knowledge base to address basic questions from users ([which you should](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) ), then dealing with verbosity from your LLM just requires some clever prompting. Here’s an example of what this prompt could look like: `As a chatbot support agent, provide a clear and concise response to the user’s question: “{userQuestion}” ` `With the provided details: “{chunks}” ` `Instructions:` `- Summarize key points briefly, within three to four sentences.` `- Use bullet points for clarity and numbered lists for step-by-step instructions.` `- Start with a direct answer before using bullet points for additional details.` `- Write in simple, clear language.` `- Apply markdown syntax for formatting.` `- Ensure accuracy with the core information given.` `- Respond with "chunks" if essential details are missing.` `- Include hyperlinks only when relevant, formatted as: "Name of the page page."` `- Avoid phrases like 'information provided.'` `IMPORTANT:` `If ‘chunks’ do not contain the needed information to answer a question, respond with “NOANSWER”` These instructions not only work well for an LLM, but also for static responses. Always remember, users will skim responses—so break up lengthy responses with numbered or bulleted lists where you can. You can also do things like: * Provide buttons with common user responses * Offer a maximum of three options for each conversation item * If more options are necessary, group them into a “more options” button And remember, a conversation might not always be the best way to automate something. You can use other UI elements like forms, images, attachments, and carousels. ### 2\. Unable to perform simple actions When prompted, the Klarna agent has a list of actions it says it can complete—updating personal details, changing payment information, or sharing information about my purchases. In my experience, it is unable to complete simple actions on behalf of the user.  When asking the agent to update my name in my profile, it offered me a button to update my personal information by navigating away from the chat. That link led to nowhere. When I asked the agent to tell me what past purchases I had made—an easy task, because I had none—it prompted me to leave the chat to find them myself. I asked the agent to update my payment details, and once again I was prompted to follow a list of instructions to complete the action myself, outside the chat window. When I asked the agent if I could give the details in the chat, it said no and pushed me to a live agent. Over and over, I asked the Klarna agent to complete actions it had told me it could do, and every time it asked me to do the action myself, it pushed me to a live agent or failed altogether.  To improve the performance right away, I would ensure that it was very clear to the user what the agent can and can’t do. Setting expectations is key. Then, for actions the agent can’t complete, I’d ensure I have clear instructions in a knowledge base that an agent can pull from. For actions an agent can complete, it’s all about setting up intents that help capture utterances and then push them down the business process you’re trying to take. If an agent can’t automate the thing a user is trying to do, then clearly tell users how to do it. And if the agent can automate it, do it. ### 3\. Asks users to exit support to complete actions I will admit I’m not a Klarna customer—I started a blank account to run these tests—so this is not a scientific experiment. But if an agent claims it can do a collection of actions for me as a user, then I expect the agent to complete those actions. I don’t expect to be sent exploring my own profile with a laundry list of instructions. Now, generating instructions for a customer to DIY actions is not inherently bad. But what Klarna gets wrong in this case is the list of instructions in a static support window. When you select a button or go to a different page, that window disappears. Imagine the experience from the user’s perspective. They must read those verbose instructions, exit the support window, remember those instructions well enough to navigate the site, complete the action, and navigate back to the support window. No one will remember those instructions after five clicks, not to mention how to navigate back to the support. Which means they’ll be unable to complete the action they sought support for. The whole experience is just frustrating. So, what’s the solution? If you must ask your user to complete the action themselves (although I’d argue, there’s a bunch of [actions your agent can do](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) ), include a link that goes to where you want your user to go. (And make sure it works.) This is more deterministic flow but it ensures that the agent is more accurately supporting users. ### 4\. Hides the customer support chat In my opinion, finding the customer support chat was a hassle every time. Here’s a clip of me struggling, which happened repeatedly as the agent directed me to complete actions elsewhere. [](https://youtu.be/1Mkycplnm5w) Instead, Klarna should host their customer support in the bottom corner of the page as a dynamic window that stays with the user as they navigate the site (and you should too). Also, since they’re offering customer support already, I don’t recommend hiding it away. Adding unnecessary friction to the support process is counterintuitive. And when users are inevitably transferred to a live agent, they’ll take out those frustrations on them. ### 5\. Employs a sickly sweet tone Now, this may be because I am a [self-proclaimed killjoy](https://www.voiceflow.com/blog/confessions-of-a-cxd-killjoy-why-i-think-your-ai-assistant-shouldnt-be-funny) when it comes to agent’s personalities. But, the inhuman, enthusiastic tone of this agent often undermines its intentions. For example, I asked a common question for Klarna: _What happens if I can’t pay on time?_ And its answers were run-of-the-mill boilerplate. That isn’t my problem with the response—my issue is that it lacks empathy. It lacked anything that addressed the potential hardship of a customer who needs to ask that question in the first place.  Conversation designers do [a lot of things](https://www.voiceflow.com/blog/the-case-for-a-full-stack-conversation-designer) and one of the most important is keeping the human qualities of conversation alive in AI agents. In the case of a live customer support agent or an in-person interaction, there’s usually acknowledgement and empathy in response to someone sharing their difficulty in making payments on time. I encourage designers and developers to include moments of empathy, it could be as simple as starting the response with “I’m sorry to hear you’re having trouble making payments.” You can disagree with me, in fact, I welcome it (shoot me a message on [LinkedIn](https://www.linkedin.com/in/isaacspeter/) ). But I think moments like these really add up to a better user experience. I’d also suggest removing static pleasantries that you drop at the end of every paragraph. That’s not how people talk. ### 6\. Suffers from lengthy response times After receiving an input, the Klarna agent generated responses after 15-20 seconds. That doesn’t sound like much, but imagine asking your barista for a chai latte and watching them stare at you for 20 seconds before responding—it’s bearable, but painfully awkward. And from a technical standpoint, it’s unacceptable. I have a theory as to why the Klarna agent’s responses lag—I suspect using the beta of OpenAI’s assistants API is the culprit. For each response, the agent is accessing multiple tools at once, like knowledge retrieval, function calling, voice and tone tuning, and persistent threads. As a result, the response time is impacted. If you’re planning to use the new OpenAI assistants API, these current constraints are ones to consider. Setbacks are to be expected when an API is in beta, even with a company as innovative as OpenAI. In reality, this API is great at helping you launch an agent quickly, but not ideal at helping your agent perform actions quickly. To avoid lengthy response times with your agent, it’s best to use a hybrid approach. Use a knowledge base for information retrieval and deterministic flows (along with LLMs) to guide users through actions. Also, don’t use GPT-4, or Claude Opus for everything—it’s like driving a Ferrari two doors down to pick up milk. Sure, it’s cool but totally unnecessary. Instead, you can use cheaper and faster models to give the same feeling of flexibility, while also using deterministic flows to get things done. ### You don’t need to be BFFs with Sam Altman to automate your customer support If there’s one thing I want you to take away from this breakdown it’s this—you don’t need a special relationship with OpenAI to automate your customer support. With a [knowledge base](https://www.voiceflow.com/blog/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) , LLMs, and an agent orchestration platform (like Voiceflow, obviously), you can consistently automate an impressive batch of your level 1 support tickets. In fact, [you can move beyond level 1 tickets](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) . Developers at [Trilogy](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) are making it happen—their team has automated 60% of their customer support across multiple product lines, in under 12 weeks. Meanwhile, our customer support agent [Tico](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) , offers multi-channel support for 97% of tickets and completes complex actions on behalf of users. And those are real results. I will say, for simple FAQs and level 1 inquiries, the Klarna bot excels. There is a team of incredible developers and designers who are working tirelessly to make the Klarna agent possible. And I don’t doubt that it will improve over time. This AI agent is an incredibly successful proof concept (POC) with issues that can be fixed with testing and iteration—the most important steps to avoid a POC that doesn’t deliver results. But given all the spectacle when that press release dropped, I expected to be wowed. Maybe next time.  RECOMMENDED [\ \ ### Chatbot glow-up: 3 reasons everyone hates your AI agent (and what you can do about it)](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) PUBLISHED May 15, 2024 TOPICS IN THIS ARTICLE Exceptional work [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### AI automation on a budget: Getting started with high ROI use cases](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) [](https://www.voiceflow.com/pathways/voiceflow-winter-product-roundup) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### ICYMI: Voiceflow news and updates from our winter product roundup](https://www.voiceflow.com/pathways/voiceflow-winter-product-roundup) [.avif)](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Design and prototype production-ready conversational experiences with user personas](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/behind-the-hype-how-klarnas-customer-support-agent-falls-short#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/behind-the-hype-how-klarnas-customer-support-agent-falls-short#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Conversation design best practices | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) The best practices of conversation design ========================================= [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) [Conversation design](https://www.voiceflow.com/glossary/conversation-design) is a fairly recent design principle that has risen to popularity due to the worldwide emergence of voice technology and smart speakers. But whether you’re interacting with technology or someone in-person, we’ve all experienced awkward engagements that could’ve gone better. Therefore, to be a great conversation designer takes a similar approach to being a great conversationalist. Here are 6 ways in which we can design more natural conversations for chat or voice. > “The key to being a good conversationalist is probably a genuine unselfish interest in others. That, and practice.” –Frank Crane  Conversation design best practice #1: Set your intentions --------------------------------------------------------- Have you ever tried to insert yourself into a conversation and feel completely lost? It’s confusing at best and leaves you discouraged and unwelcomed. Before designing your conversational interface, get to know who you may be interacting with, the context of the conversation, and what the users may want to get out of it. Purpose provides direction which can help lead to more fruitful engagements. Conversation design best practice #2: Let people be heard --------------------------------------------------------- Take interest in what the user has to say and build trust through validation. That’s how healthy relationships are formed. Practice active listening and acknowledge frequently - whether by confirming the user’s response or empathizing with emotion. This ensures you are hearing both correctly, and that they know you are. _Learn more about ‘_[_slot confirmations_](https://www.voiceflow.com/blog/dialogue-management-new-editors-highlight-newest-voiceflow-release) _’ in Voiceflow._ Conversation design best practice #3: Elicit cooperative conversations ---------------------------------------------------------------------- Bot conversations should follow the [Cooperative Principle](https://www.thoughtco.com/cooperative-principle-conversation-1689928) where parties in a conversation cooperate in order to advance the conversation. Interactions should account for implicit confirmations and forward-moving questions to guide users to the conversation’s intentions. Conversation design should also account for overanswers by users so that a question is not asked if an implicit answer has already been given. _This can be done via ‘slot requirements’ in Voiceflow._ _Bot: “What would you like to order?”_ _User: “A_ **_grande_** _dark-roast coffee. I’m so tired”_ _🙅♂️ Bot: “What size would you like?”_ _🙌🏼 Bot: “Sounds good. You ordered a_ **_large_** _dark-roast coffee. Will that be all?”_ Conversation design best practice #4: Anticipate for conversational error ------------------------------------------------------------------------- It happens in everyday conversation: maybe they misheard you (no match), get distracted and don’t respond (no input), or provide an answer that doesn’t necessarily fit your question (system error). They’re hard to expect but you can hide errors by designing conversation repairs or creating rapid reprompts. * **Conversational repair**: The process by which a speaker recognizes a speech error and repeats what has been said with some sort of correction. _Ex. “Did you mean, \_\_\_\_\_\_?”_ * **_Rapid reprompt_**: Asking a short immediate question to give users another chance to respond. _Ex. “Sorry, what was that?” vs. “Sorry, could you please provide me with your telephone number?”_ Conversation design best practice #5: Develop personas for your voice experience --------------------------------------------------------------------------------- Chatbots or voice assistants are not human (clearly). But it can still anthropomorphize human qualities that make a conversation more engaging. Having a persona/personality is one of those qualities, especially with voice design. That’s because words carry emotions. The Google Assistant team has discovered that the conversational bots with the best retention also had the strongest personas. This also ensures that users connect with your brand inline with your values. _Create a persona style guide and learn more about Voiceflow’s_ [_SSML Editor_](https://www.voiceflow.com/blog/dialogue-management-new-editors-highlight-newest-voiceflow-release) _._ Conversation design best practice #6: Rehearse Your Conversations ------------------------------------------------------------------ Before designing your conversations, run through the flow of the conversation preferably with others’ feedback. This will allow you to provide structure (& intention) to the conversation, anticipate technical limitations, and discover alternative interpretations of your script that you did not originally account for. **Putting it all together** ------------------------------ Conversations and to a larger extent, language, is complex and filled with nuances. But that’s not to say we can’t improve our ability to engage in better conversations. The goal isn’t to recreate human interaction. The goal is to elicit an engaging conversation that feels less automated and helps users with what’s intended. The best conversations make people feel comfortable, adjusting to the subtle changes without pause and dialogue flows naturally. Let us know what you think makes for great conversation design! Sign up for our blog and shoot us a message. Conversation design best practice #1: Set your intentions --------------------------------------------------------- Have you ever tried to insert yourself into a conversation and feel completely lost? It’s confusing at best and leaves you discouraged and unwelcomed. Before designing your conversational interface, get to know who you may be interacting with, the context of the conversation, and what the users may want to get out of it. Purpose provides direction which can help lead to more fruitful engagements. Conversation design best practice #2: Let people be heard --------------------------------------------------------- Take interest in what the user has to say and build trust through validation. That’s how healthy relationships are formed. Practice active listening and acknowledge frequently - whether by confirming the user’s response or empathizing with emotion. This ensures you are hearing both correctly, and that they know you are. _Learn more about ‘_[_slot confirmations_](https://www.voiceflow.com/blog/dialogue-management-new-editors-highlight-newest-voiceflow-release) _’ in Voiceflow._ Conversation design best practice #3: Elicit cooperative conversations ---------------------------------------------------------------------- Bot conversations should follow the [Cooperative Principle](https://www.thoughtco.com/cooperative-principle-conversation-1689928) where parties in a conversation cooperate in order to advance the conversation. Interactions should account for implicit confirmations and forward-moving questions to guide users to the conversation’s intentions. Conversation design should also account for overanswers by users so that a question is not asked if an implicit answer has already been given. _This can be done via ‘slot requirements’ in Voiceflow._ _Bot: “What would you like to order?”_ _User: “A_ **_grande_** _dark-roast coffee. I’m so tired”_ _🙅♂️ Bot: “What size would you like?”_ _🙌🏼 Bot: “Sounds good. You ordered a_ **_large_** _dark-roast coffee. Will that be all?”_ Conversation design best practice #4: Anticipate for conversational error ------------------------------------------------------------------------- It happens in everyday conversation: maybe they misheard you (no match), get distracted and don’t respond (no input), or provide an answer that doesn’t necessarily fit your question (system error). They’re hard to expect but you can hide errors by designing conversation repairs or creating rapid reprompts. * **Conversational repair**: The process by which a speaker recognizes a speech error and repeats what has been said with some sort of correction. _Ex. “Did you mean, \_\_\_\_\_\_?”_ * **_Rapid reprompt_**: Asking a short immediate question to give users another chance to respond. _Ex. “Sorry, what was that?” vs. “Sorry, could you please provide me with your telephone number?”_ Conversation design best practice #5: Develop personas for your voice experience --------------------------------------------------------------------------------- Chatbots or voice assistants are not human (clearly). But it can still anthropomorphize human qualities that make a conversation more engaging. Having a persona/personality is one of those qualities, especially with voice design. That’s because words carry emotions. The Google Assistant team has discovered that the conversational bots with the best retention also had the strongest personas. This also ensures that users connect with your brand inline with your values. _Create a persona style guide and learn more about Voiceflow’s_ [_SSML Editor_](https://www.voiceflow.com/blog/dialogue-management-new-editors-highlight-newest-voiceflow-release) _._ Conversation design best practice #6: Rehearse Your Conversations ------------------------------------------------------------------ Before designing your conversations, run through the flow of the conversation preferably with others’ feedback. This will allow you to provide structure (& intention) to the conversation, anticipate technical limitations, and discover alternative interpretations of your script that you did not originally account for. **Putting it all together** ------------------------------ Conversations and to a larger extent, language, is complex and filled with nuances. But that’s not to say we can’t improve our ability to engage in better conversations. The goal isn’t to recreate human interaction. The goal is to elicit an engaging conversation that feels less automated and helps users with what’s intended. The best conversations make people feel comfortable, adjusting to the subtle changes without pause and dialogue flows naturally. Let us know what you think makes for great conversation design! Sign up for our blog and shoot us a message.  RECOMMENDED [\ \ ### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) PUBLISHED March 3, 2020 TOPICS IN THIS ARTICLE Elevating our work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The evolution and outlook of voice user interfaces](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces) [](https://www.voiceflow.com/pathways/6-differences-between-mobile-apps-and-voice-apps) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The 6 major differences between mobile apps and voice apps](https://www.voiceflow.com/pathways/6-differences-between-mobile-apps-and-voice-apps) [](https://www.voiceflow.com/pathways/4-tips-for-great-conversational-ai-product-management) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 4 tips for great conversational AI product management](https://www.voiceflow.com/pathways/4-tips-for-great-conversational-ai-product-management) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/conversation-design-best-practices#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/conversation-design-best-practices#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Design conversations to be helpful, not human | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Design conversations to be helpful, not human ============================================= [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) Throughout my time at Voiceflow, I’ve seen and worked with a lot of conversation design teams. One trend I’ve noticed, especially with designers just getting into conversation design, is focusing too much on making conversations sound _human_. I’ve seen designers focus their time on small talk, jokes, and over-the-top personas, all to create a more human experience. As conversation designers, our goal is not to design human conversations; our goal is to create helpful conversations that help users achieve their goals. Small talk in conversation design --------------------------------- When humans converse with each other, we follow the [cooperative principle](https://en.wikipedia.org/wiki/Cooperative_principle) , which states that both people work together to achieve an objective through conversation. This objective could be as lighthearted as telling a story or transactional as buying a car. The foundation of these human conversations is built on empathy or the mutual understanding of each other’s desires and emotions. When two people have a relationship, i.e., empathy, conversations are more fluid and less rigid as both parties know the other is empathetic toward their needs. This base of empathy is why we, as humans, lean on small talk like “how’s your day going?”—although we don’t always _care_ how the other person’s day is going, we leverage this sentence to showcase our empathy toward other humans.  Unlike humans, computers are not empathetic (yet), and because of this, assistant small talk is often considered invaluable or unnecessary. When I talk to my Starbucks assistant, I don’t want to tell the app how my day is going — I want a coffee. Unlike my barista, I know that the assistant is incapable of being empathetic, so expecting an emotional base, the same question would add friction and unnecessary steps to my order. Furthermore, today’s NLP/NLU technology is far from perfect and can only manage well designed structured conversations. Small talk is the prime example of an unstructured conversational element that can only serve to trip up and break assistants, or worse, be contextually irrelevant or bland. _For example:_ > **Coffee bot:** _“How are you doing today?_**Customer:** _“I’m actually going through a really rough day,”_** > Coffee bot:** _“Great! What can I get for you?”_ Yikes. The perils of blanket responses to unstructured prompts. When designing conversations, ditch the small talk. While it may be an impressive tech demo in the board room with perfectly planned utterances, it’s not always going to match your users’ expectations or context. Tangential functions -------------------- When designing an assistant, it’s essential to design with a goal for the user in mind. If you’re designing a pizza ordering assistant, you shouldn’t have to design flows for Intents like “tell me a joke.” These flows fall into the “pointless small talk board room demo” category as they are not critical to the user’s experience. Instead, these side tangents often bloat the experience and hide what’s really important. If your Intent analytics show that users are consistently asking your Pizza design for a joke, so be it, but otherwise, make sure to focus on core flows/experiences. Human personas -------------- Conversational personas are important, but designers need to remember that in many cases, these personas represent the brand as an assistant across all sorts of situations and environments – not necessarily the brand as a human. Although there are several engaging ways to leverage human-like characteristics and humor in conversation design, they aren’t always necessary. For instance, in games, education, or perhaps even brand creation, using voice actors, custom rhetoric, or personality can drastically improve a conversational experience. In contrast, transactional experiences such as banking, POS system ordering, or logistics may not leave room for such liberties. In these situations, added small talk could slow down the conversation and increase the time it takes to complete the user’s goal. In many transactional use cases, users know they are talking to an assistant, not a human, so any dialog unrelated to their objective could be considered a bad experience. Remember your goal ------------------ When designing conversations, remember that your goal is to create a pleasant conversation with a computer that achieves an objective. In most cases, your goal is not to pass the [Turing test](https://searchenterpriseai.techtarget.com/definition/Turing-test#:~:text=A%20Turing%20Test%20is%20a,cryptanalyst%2C%20mathematician%20and%20theoretical%20biologist.) and foster the most human conversational AI out there. Therefore, when designing transactional or utility-focused conversations, avoid small talk, tangential functions, and overly human personas and focus on building intents for their core needs/goals. Unlike humans, computers are not empathetic (yet), and because of this, assistant small talk is often considered invaluable or unnecessary. When I talk to my Starbucks assistant, I don’t want to tell the app how my day is going — I want a coffee. Unlike my barista, I know that the assistant is incapable of being empathetic, so expecting an emotional base, the same question would add friction and unnecessary steps to my order. Furthermore, today’s NLP/NLU technology is far from perfect and can only manage well designed structured conversations. Small talk is the prime example of an unstructured conversational element that can only serve to trip up and break assistants, or worse, be contextually irrelevant or bland. _For example:_ > **Coffee bot:** _“How are you doing today?_**Customer:** _“I’m actually going through a really rough day,”_** > Coffee bot:** _“Great! What can I get for you?”_ Yikes. The perils of blanket responses to unstructured prompts. When designing conversations, ditch the small talk. While it may be an impressive tech demo in the board room with perfectly planned utterances, it’s not always going to match your users’ expectations or context. Tangential functions -------------------- When designing an assistant, it’s essential to design with a goal for the user in mind. If you’re designing a pizza ordering assistant, you shouldn’t have to design flows for Intents like “tell me a joke.” These flows fall into the “pointless small talk board room demo” category as they are not critical to the user’s experience. Instead, these side tangents often bloat the experience and hide what’s really important. If your Intent analytics show that users are consistently asking your Pizza design for a joke, so be it, but otherwise, make sure to focus on core flows/experiences. Human personas -------------- Conversational personas are important, but designers need to remember that in many cases, these personas represent the brand as an assistant across all sorts of situations and environments – not necessarily the brand as a human. Although there are several engaging ways to leverage human-like characteristics and humor in conversation design, they aren’t always necessary. For instance, in games, education, or perhaps even brand creation, using voice actors, custom rhetoric, or personality can drastically improve a conversational experience. In contrast, transactional experiences such as banking, POS system ordering, or logistics may not leave room for such liberties. In these situations, added small talk could slow down the conversation and increase the time it takes to complete the user’s goal. In many transactional use cases, users know they are talking to an assistant, not a human, so any dialog unrelated to their objective could be considered a bad experience. Remember your goal ------------------ When designing conversations, remember that your goal is to create a pleasant conversation with a computer that achieves an objective. In most cases, your goal is not to pass the [Turing test](https://searchenterpriseai.techtarget.com/definition/Turing-test#:~:text=A%20Turing%20Test%20is%20a,cryptanalyst%2C%20mathematician%20and%20theoretical%20biologist.) and foster the most human conversational AI out there. Therefore, when designing transactional or utility-focused conversations, avoid small talk, tangential functions, and overly human personas and focus on building intents for their core needs/goals.  RECOMMENDED [\ \ ### Expanding the definition of conversation design](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) PUBLISHED January 15, 2021 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to write an effective conversation designer job description \[+template\]](https://www.voiceflow.com/pathways/how-to-write-an-effective-conversation-designer-job-description-template) [](https://www.voiceflow.com/pathways/breaking-nike-chatbot) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Breaking and rebuilding Nike's chatbot](https://www.voiceflow.com/pathways/breaking-nike-chatbot) [](https://www.voiceflow.com/pathways/prompt-chaining-conversational-ai) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Prompt chaining for conversational AI](https://www.voiceflow.com/pathways/prompt-chaining-conversational-ai) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/design-conversations-to-be-helpful-not-human#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/design-conversations-to-be-helpful-not-human#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Designing conversational bots that fix themselves | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Designing conversational bots that fix themselves ================================================= [.avif)\ \ Terry Lee\ \ at](https://www.voiceflow.com/contributors/terry-lee) When I was in college taking discrete mathematics, I had this professor who was terrible at communicating. If I asked him to explain something, he’d pause a few seconds, and then say exactly what he just said, word for word. On busier days, he might just say: “Go find a tutor after class”. These conversations went nowhere. More than half of his students failed that year. Fast-forward to today, I design and build conversational bots, and think about how to keep conversations going when they break down. But rather than endlessly loop or hand things off at the first sight of trouble, the key, I’ve found, is to guide the user back to something they want, in a socially appropriate manner, without any mention of an error. The path I took to settling on this approach is something I want to share with you. My goal is to shorten the learning curve about conversation repair strategies, create more ambitious conversation designers, and show repair pathing, not as an afterthought, but a central feature of conversational bots.  There's A Right Way To Fail --------------------------- Let’s begin with a dialogue I pieced together from multiple Google Action horoscope bots to highlight not so effective ways bots respond to unexpected user responses: **Bot**: What’s your zodiac sign? **User**: #$&@! **Bot**: Sorry I didn’t understand. Can you say that again? **User**: #$&@! **Bot**: Sorry I didn’t quite catch that. What was that? **User**: #$&@! **Bot**: Sorry I can’t help. Maybe you’re wondering at this point as I did: why does the bot say “sorry” so much? Does it even need to say this? Isn’t this defeatist? When I hear a bot say "sorry", too often or too quickly, I tend to think “this bot sucks”. And then there’s the strategy of admitting to a mistake when things go wrong, like saying "I didn’t understand”. This seems like a step in the right direction, but not if you don’t actually know what went wrong. Then it’d be presumptuous. For example, what if I mumbled? Or what if the device didn’t pick up what I said? Or what if the recording of my audio got through, but something else happened on the backend beyond my control? You may also have noticed some nuance in the bot’s tone. Surely, softening the blow does some good. You certainly don’t want to alienate the user. But a gentler confession like “I didn’t quite catch that” by itself, without any attempt to make things right, sounds wishy washy. I’ve even seen some voice apps take a more neutral route and say “something happened”. But that comes across as vague, even ominous. Regardless of whether you give a million sorrys or no apology at all, you still need to help users do what they came to your app for. And that means getting possible fixes out in front of them when things go wrong. For example, a miscommunication scenario with an Alexa Skill I designed, called Interview Practice, would go down like this: **Bot**: Why do you think you’re a good fit for the role? Are you ready to practice this question? **User**: #$&@! **Bot**: Again, the question is: Why do you think you’re a good fit for the role? Ready? You can also ask for a new question. .png) Notice the only thing that matters is moving the conversation forward. First, it repeats the interview question at play in case you didn’t hear it. Then, it re-asks the original prompt in condensed form (“Ready?"). Finally, if that’s not enough, you get another option that addresses a difficulty you might be having. I’ve designed it so you hear straightforward fixes like “you can also repeat the question” the very first time you say something that doesn’t register. Less obvious ones like “you can also ask me to wait” surface later if triggered. These secondary fixes rotate endlessly in a succession from more to less obvious. Ideally, any options suggested would be based on specific user utterances (so not random). In fact, a [2019 chatbot repair study by IBM researchers](https://research.ibm.com/publications/resilient-chatbots-repair-strategy-preferences-for-conversational-breakdowns) came to this exact conclusion: when things go wrong, the most preferred repair strategy is giving the user options based on what the user just said. Even “problems” such as unexpected, out-of-context, barge-in requests, when anticipated and designed for, can open up new vistas of engagement. For example, in early testing of my app I noticed some users needed more time to think, even a few seconds, before starting their answers. So I included this detour: **Bot**: What are your salary expectations? Are you ready to answer this? **User**: Wait! **Bot**: No problem, I’ll come back in a moment” (10 silent seconds go by). I’m back. Again, the question is: what are your salary expectations? Ready? Or do you need help? .png) If taking a pause doesn’t work, you can always ask for help, which opens up other possible fixes. For example, you may want to re-practice the previous question. Or you may wish to do a related question and dive deeper into a specific category. Or maybe you just want to spin up new questions at random until you find a good one. These and other options are suggested when you say help. The goal is to gracefully redirect the user back to one of several possible fixes, getting them back on the “happy path”, so the user can get done what they came to your app for. By the way, in my example I used human intuition and good ol’ fashion guessing to generate possible fixes. In a more perfect system, solutions would be educated guesses (predictive) and leverage not only intuition-based rules from human experts but also machine learning models that mimic memory and contextual knowledge (see [OpenAI’s GPT-3](https://openai.com/blog/gpt-3-apps/) and [Facebook’s BlenderBot 2.0)](https://ai.facebook.com/blog/blender-bot-2-an-open-source-chatbot-that-builds-long-term-memory-and-searches-the-internet/) . Using [_socially appropriate, repair-focused re-prompts_](https://design.google/library/conversation-design-speaking-same-language/) like reminding, paraphrasing, and giving options, when intents are mis-recognized or missing, is an approach I came to through trial and error, and is inspired by James Giangola, a Google conversation design expert. > Steve Jobs also has a saying on design that I love, which easily applies here: “Most people make the mistake of thinking (conversation) design is what it looks (and sounds) like. But it's not just what it looks (sounds) and feels like. (Conversation) design is how it _works_”. What matters is not just what words mean, it’s what they do. Paraphrasing as a tactic reminds me of my account management days years ago, when I had a client with a crazy strong accent. He spoke fast too. I never understood what he was saying. One day it dawned on me: asking him what he meant only made things worse if he couldn’t easily be understood. And you can only apologize so much. So instead I paraphrased what I thought he said, based on bits and pieces I could make out - guesses really - and then asking him for confirmation. When this worked, he’d usually say back to me: “THAT’S what I’ve been SAYING the WHOLE TIME!” Redirecting as a tactic is something skilled parents do. I have a friend who does this masterfully. Her kid would start climbing the table or pick up a bowl to throw, and then my friend, instead of losing her cool, would suddenly point to the window and say “Wow, look at that bird with the huge beak!” This would stop her kid in his tracks. He’d look over to the window, turn to his mom and say “Really? and forget all about whatever mischief he was about to cause. Buddhists and relationship therapists call this “skillful means”. Repair strategies are central to making bots (and humans) better and more focused. Why? Because repair is a universal aspect of language itself. > “That’s repair. Ubiquitous and necessary. The human language fixing device. The more you start thinking about repair, the more you start to notice it. Without it, we would be lost in an ocean of intersubjective failures. Good thing language invented it”, says Emma Hoffstetter, a linguistic researcher. The more happy paths, the better. Conversation design is about building a network of more traveled and less traveled roads that all lead to great destinations. In this way, effective conversational bots and good conversationalists do and want the same things. They keep things moving. They fix things. They take you places. Above all, they don’t focus on themselves. They make it about you. **Further references ** * A [video presentation](https://youtu.be/wuDP_eygsvs?t=916) entitled “Applying Built-in Hacks of Conversation to Your Voice UI” (starts at 15:16 for repair discussion) by Google conversation design expert James Giangola * A [video lecture](https://www.youtube.com/watch?v=H4D3zGk2wds) on “Repair” as a central feature of human conversation by linguistic researcher Emma Hoffstetter * A [video presentation](https://youtu.be/yFDwi83p-UI) and [paper](http://qveraliao.com/chi19-1.pdf) entitled “Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns” by IBM researchers About the author: [linktr.ee/terryjlee](https://linktr.ee/terryjlee) About Prepfit: [linktr.ee/prepfit](https://linktr.ee/prepfit) Thanks to Elaine Anzaldo, Nicolas Arcay Bermejo, Don Irwin, and Sara Pion for reading drafts of this. There's A Right Way To Fail --------------------------- Let’s begin with a dialogue I pieced together from multiple Google Action horoscope bots to highlight not so effective ways bots respond to unexpected user responses: **Bot**: What’s your zodiac sign? **User**: #$&@! **Bot**: Sorry I didn’t understand. Can you say that again? **User**: #$&@! **Bot**: Sorry I didn’t quite catch that. What was that? **User**: #$&@! **Bot**: Sorry I can’t help. Maybe you’re wondering at this point as I did: why does the bot say “sorry” so much? Does it even need to say this? Isn’t this defeatist? When I hear a bot say "sorry", too often or too quickly, I tend to think “this bot sucks”. And then there’s the strategy of admitting to a mistake when things go wrong, like saying "I didn’t understand”. This seems like a step in the right direction, but not if you don’t actually know what went wrong. Then it’d be presumptuous. For example, what if I mumbled? Or what if the device didn’t pick up what I said? Or what if the recording of my audio got through, but something else happened on the backend beyond my control? You may also have noticed some nuance in the bot’s tone. Surely, softening the blow does some good. You certainly don’t want to alienate the user. But a gentler confession like “I didn’t quite catch that” by itself, without any attempt to make things right, sounds wishy washy. I’ve even seen some voice apps take a more neutral route and say “something happened”. But that comes across as vague, even ominous. Regardless of whether you give a million sorrys or no apology at all, you still need to help users do what they came to your app for. And that means getting possible fixes out in front of them when things go wrong. For example, a miscommunication scenario with an Alexa Skill I designed, called Interview Practice, would go down like this: **Bot**: Why do you think you’re a good fit for the role? Are you ready to practice this question? **User**: #$&@! **Bot**: Again, the question is: Why do you think you’re a good fit for the role? Ready? You can also ask for a new question. .png) Notice the only thing that matters is moving the conversation forward. First, it repeats the interview question at play in case you didn’t hear it. Then, it re-asks the original prompt in condensed form (“Ready?"). Finally, if that’s not enough, you get another option that addresses a difficulty you might be having. I’ve designed it so you hear straightforward fixes like “you can also repeat the question” the very first time you say something that doesn’t register. Less obvious ones like “you can also ask me to wait” surface later if triggered. These secondary fixes rotate endlessly in a succession from more to less obvious. Ideally, any options suggested would be based on specific user utterances (so not random). In fact, a [2019 chatbot repair study by IBM researchers](https://research.ibm.com/publications/resilient-chatbots-repair-strategy-preferences-for-conversational-breakdowns) came to this exact conclusion: when things go wrong, the most preferred repair strategy is giving the user options based on what the user just said. Even “problems” such as unexpected, out-of-context, barge-in requests, when anticipated and designed for, can open up new vistas of engagement. For example, in early testing of my app I noticed some users needed more time to think, even a few seconds, before starting their answers. So I included this detour: **Bot**: What are your salary expectations? Are you ready to answer this? **User**: Wait! **Bot**: No problem, I’ll come back in a moment” (10 silent seconds go by). I’m back. Again, the question is: what are your salary expectations? Ready? Or do you need help? .png) If taking a pause doesn’t work, you can always ask for help, which opens up other possible fixes. For example, you may want to re-practice the previous question. Or you may wish to do a related question and dive deeper into a specific category. Or maybe you just want to spin up new questions at random until you find a good one. These and other options are suggested when you say help. The goal is to gracefully redirect the user back to one of several possible fixes, getting them back on the “happy path”, so the user can get done what they came to your app for. By the way, in my example I used human intuition and good ol’ fashion guessing to generate possible fixes. In a more perfect system, solutions would be educated guesses (predictive) and leverage not only intuition-based rules from human experts but also machine learning models that mimic memory and contextual knowledge (see [OpenAI’s GPT-3](https://openai.com/blog/gpt-3-apps/) and [Facebook’s BlenderBot 2.0)](https://ai.facebook.com/blog/blender-bot-2-an-open-source-chatbot-that-builds-long-term-memory-and-searches-the-internet/) . Using [_socially appropriate, repair-focused re-prompts_](https://design.google/library/conversation-design-speaking-same-language/) like reminding, paraphrasing, and giving options, when intents are mis-recognized or missing, is an approach I came to through trial and error, and is inspired by James Giangola, a Google conversation design expert. > Steve Jobs also has a saying on design that I love, which easily applies here: “Most people make the mistake of thinking (conversation) design is what it looks (and sounds) like. But it's not just what it looks (sounds) and feels like. (Conversation) design is how it _works_”. What matters is not just what words mean, it’s what they do. Paraphrasing as a tactic reminds me of my account management days years ago, when I had a client with a crazy strong accent. He spoke fast too. I never understood what he was saying. One day it dawned on me: asking him what he meant only made things worse if he couldn’t easily be understood. And you can only apologize so much. So instead I paraphrased what I thought he said, based on bits and pieces I could make out - guesses really - and then asking him for confirmation. When this worked, he’d usually say back to me: “THAT’S what I’ve been SAYING the WHOLE TIME!” Redirecting as a tactic is something skilled parents do. I have a friend who does this masterfully. Her kid would start climbing the table or pick up a bowl to throw, and then my friend, instead of losing her cool, would suddenly point to the window and say “Wow, look at that bird with the huge beak!” This would stop her kid in his tracks. He’d look over to the window, turn to his mom and say “Really? and forget all about whatever mischief he was about to cause. Buddhists and relationship therapists call this “skillful means”. Repair strategies are central to making bots (and humans) better and more focused. Why? Because repair is a universal aspect of language itself. > “That’s repair. Ubiquitous and necessary. The human language fixing device. The more you start thinking about repair, the more you start to notice it. Without it, we would be lost in an ocean of intersubjective failures. Good thing language invented it”, says Emma Hoffstetter, a linguistic researcher. The more happy paths, the better. Conversation design is about building a network of more traveled and less traveled roads that all lead to great destinations. In this way, effective conversational bots and good conversationalists do and want the same things. They keep things moving. They fix things. They take you places. Above all, they don’t focus on themselves. They make it about you. **Further references ** * A [video presentation](https://youtu.be/wuDP_eygsvs?t=916) entitled “Applying Built-in Hacks of Conversation to Your Voice UI” (starts at 15:16 for repair discussion) by Google conversation design expert James Giangola * A [video lecture](https://www.youtube.com/watch?v=H4D3zGk2wds) on “Repair” as a central feature of human conversation by linguistic researcher Emma Hoffstetter * A [video presentation](https://youtu.be/yFDwi83p-UI) and [paper](http://qveraliao.com/chi19-1.pdf) entitled “Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns” by IBM researchers About the author: [linktr.ee/terryjlee](https://linktr.ee/terryjlee) About Prepfit: [linktr.ee/prepfit](https://linktr.ee/prepfit) Thanks to Elaine Anzaldo, Nicolas Arcay Bermejo, Don Irwin, and Sara Pion for reading drafts of this.  RECOMMENDED [](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves#) PUBLISHED April 4, 2022 TOPICS IN THIS ARTICLE No items found. [.avif)\ \ Terry Lee\ \ at](https://www.voiceflow.com/contributors/terry-lee) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Designing for voice assistants vs. chatbots](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots) [](https://www.voiceflow.com/pathways/best-conversational-ai-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Instacart's Ayesha Saleem shares her 5 best conversational AI practices](https://www.voiceflow.com/pathways/best-conversational-ai-practices) [](https://www.voiceflow.com/pathways/cxd-center-of-excellence) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Your CxD center of excellence (and why you don’t need one right now)](https://www.voiceflow.com/pathways/cxd-center-of-excellence) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # From Hobbits to bots—lessons from Simon Mitchell-Wolf’s journey to CxD | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) From Hobbits to bots—lessons from Simon Mitchell-Wolf’s journey to CxD ====================================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) [Simon Mitchell-Wolf](https://www.linkedin.com/in/snwolf) fell in love with linguistics when he was just a kid reading fantasy novels. Something about the languages authors like J.R.R. Tolkien created for their characters captured his imagination and wouldn’t let go. Much like one of Tolkien’s Hobbit protagonists, Simon has since been on a great journey—one that started in the pages of a book but eventually led him to his current real-world role as a conversational experience designer at PayPal. After dedicating years of his life to studying linguistics, Simon’s initial forays into the world of technology left him feeling a little out of his element, not unlike Bilbo Baggins when he was conscripted to join Gandalf’s brawling band of dwarves for the journey to Smaug’s mountain. Happily, like Bilbo, Simon found a way to shine by bringing his unique set of talents and perspective to bear. Bilbo used his innate stealth and understanding of “human” nature to become the party’s MVP burglar. Simon used his linguistics background to look at digitally mediated communication in a very different way that focuses less on the technology involved, and more on using words to bring people together and build authentic connections and communities. In my recent conversation with Simon, he shared some of his story along with some tips and inspiration for anyone looking to make their own way into the [conversation design](https://www.voiceflow.com/blog-categories/conversation-design) arena. There may not be a single clear path to a career in this space, but people like Simon are forging their own way ahead and building roles that are perfectly suited to their particular skills and passions.  Follow your passion ------------------- After discovering his passion through exposure to constructed languages in fantasy novels, Simon was already on his path. By the time he was in middle school, he knew he wanted to study linguistics seriously. And, unlike most middle schoolers’ aspirations, the dream stuck. He taught some peer-to-peer linguistics in high school, did some related research at boarding school, and continued his studies in college where he ultimately received an accelerated master’s degree. He wrote his thesis on mediated language. His paper, _OMG BREAK A LEGGG LOL: Digital Discourse-Pragmatic Variation In A Theater-Based Community Of Practice,_ analyzes the function of three digital language phenomena (LOL, OMG, and typographic lengthening) to demonstrate the importance of digital language and discourse for building ties in within a community of student artists working together in a theater group. Even though his specific interests and sub-disciplines shifted over the years, Simon was all in from day one and never let anyone or anything derail his passion. **Be like Simon: Believe in what you love, and give it your all.** Even if your passion seems irrational or a little unrealistic. Even if you’re not sure how it will translate in real life, stick with what gets you really excited. You never know where your interests might take you. Think outside the \[career path\] box --------------------------------------  Simon Mitchell-Wolf With a master’s degree in hand, Simon’s most obvious path forward was to go into teaching or research. But he didn’t want to go that route. “I realized I didn’t want to go into academia, and I wasn’t that excited about the research,” Simon explained. “The idea of getting a PhD and a research degree, and then going into the tenure job market—which is like doing a PhD all over again—just didn’t seem like a sustainable plan.” Floundering a little, Simon shifted gears and did some work in arts management and theater production. His was a common story. Many folks with social science degrees struggle when they try to transition into industry roles. “It seems like nothing really fits,” Simon said. “You always feel like you’re having to justify your experience and explain why your skills are relevant to the job.” It was all a little discouraging and exhausting, but there were better things just around the corner. **Be like Simon: Don’t assume that the only path forward is the one right in front of you.** Traditional career strategies don’t always work in emerging fields. Also, it’s okay to feel a little lost—that’s often a normal part of the process. Don’t give up. Build a bot ----------- Before we get to how Simon initially broke into the industry, we want to share a piece of advice that he wishes he’d been given back when he was just starting out: [build a bot](https://www.voiceflow.com/chatbots) . “One thing I wish is that I’d given myself the opportunity to start building my own bots early on,” Simon said. “Just thinking about how to accomplish tasks, answer questions, and communicate information would have given me a really great jumpstart.” Simon stressed that your initial bots don’t have to be anything fancy. In fact, they don’t even need to be built with technology—you can just draft them out on paper. The main thing is to train yourself to think in terms of user-oriented design. Write it down. Read it out loud. And talk to people about it. For example, Simon and his wife take turns standing in for bots and users to work through how a conversation should be designed to deliver the most natural and comfortable experience. **Be like Simon: Start before you think you’re ready.** Don’t assume you have to know everything or have massive technical know-how to start testing your theories and honing your skills. Just go for it. Dip your toes in with freelance ------------------------------- Simon’s first formal foray into conversation design-related career territory came in the form of some freelance [conversational AI](https://www.voiceflow.com/blog/future-of-conversational-ai) consulting gigs. The work was mostly text-to-speech projects, which were an obvious fit because of his background in phonetics, phonology, and acoustic analysis. Simon said, “I got my foot in the door with some short-term, sporadic contracts, which got some tech on my resume.” Eventually, he got on the radar of a recruiter who was working with Google. That connection landed Simon a shot at a “dialogue design” job with the search giant. “I had no idea what ‘dialogue design’ was, but it sounded interesting,” Simon recalled. “And once I started talking with the recruiter, I realized that I was very qualified for the job.” Initially, Simon’s role at Google was primarily analyzing transcripts and doing a lot of maintenance and improvement. He was developing labeling techniques and ways to look at user data and metrics. “I was kind of flying by the seat of my pants,” he said. It wasn’t until near the end of his contract that he was given any opportunities to do actual design work. **Be like Simon: Stay open to opportunities.** Even if a gig isn’t what you think you’re looking for, every experience helps you get to the next experience. Push past your fear to find your place -------------------------------------- When Simon came across the opportunity at PayPal, he had mixed feelings. “It was super exciting, but also super scary,” he recalled. “It was starting a new team pretty much from scratch, fully internal, at a huge company.” What he found, once he overcame his fears and decided to go for it, was that moving to PayPal helped him feel more comfortable engaging with the industry overall. “I started feeling like part of a community at PayPal because we had a real team,” he said. “We were a team of conversation designers who were on the same level with all these other teams.” Today, Simon’s role is focused on actual conversation design—figuring out processes, [defining best practices](https://www.voiceflow.com/blog/best-practices-conversation-design) , assembling great documentation, and really starting a CxD practice from the ground up. It’s the kind of work he set out to do, and now he’s doing it with a great team at a prestigious company. **Be like Simon: Have faith in yourself, and don’t assume that everyone else knows more than you.** Conversation design is a new area of expertise. It’s easy to doubt yourself, but the truth is everyone is trying to figure things out. The people who get in there now—even if they aren’t sure about everything—are the ones who will become leaders in the field as it matures. Bring a new perspective ----------------------- The tech industry revolves around development. Everything is driven and shaped by developer work styles and priorities. But just because that’s the primary focus today doesn’t mean there isn’t room for other ways of doing things. Simon brought not only his technical linguistic skills to the table, but also what he calls a “human focus,” which he’s found is often missing in tech environments. Simon explained, “I felt like what I could bring to this world was the ability to use my technical knowledge and expertise in service to user advocacy, human advocacy.” Whatever angle you take, ensure that conversation design has a seat at the table. “One of the struggles we face as conversation designers is about how we create space for ourselves within the tech environment and empower ourselves to push back on some of the old patterns,” Simon said. He found it helpful to look to the history of user experience (UX) for examples of how to do this well. UX professionals faced a similar uphill battle to carve out their territory and integrate with other teams, and today they're a critical part of any tech team. **Be like Simon: Know what you bring to the table, and help others understand that value.** Conversation design is new to everyone, and to people not already steeped in it, it can be both confusing and daunting. Find creative ways to educate and collaborate with your peers so that they become supporters and allies. Be a part of the community -------------------------- Throughout his journey, Simon has always sought to learn from others, whether they are people on his immediate team, collaborators, or industry peers. “I’m learning a lot from other people in the industry like Tara Nair Shah and [Gina Riley](https://www.linkedin.com/in/g-riley/) ,” he said. “These are people who have already positioned themselves in the industry, have seniority, and are advocating for not only users, but also for their teams.” Simon is learning from these folks not only about the practice itself, but also about how to create space for CxD. “When we have really great leadership in the conversation design space, it helps us create space and hold our ground,” he said. “It helps empower us in the tough conversations about prioritizing, timing, and resource management.” In addition to engaging with specific individuals, Simon also recommends getting out there into the larger CxD community. He’s not a fan of networking, but he has found that connecting with fellow CxD folks never feels like traditional networking. “CxD is still a pretty small world, and the people in it are always excited to talk about conversation design,” he said. “It’s not too late to make friends and get established.” And don’t forget that you bring a lot to the exchange as well. Talking with your peers isn’t only about learning from them. You have experiences and insights to share as well. It’s a two-way street. **Be like Simon: Step outside your comfort zone, and find your people.** They are out there—and they’re just as passionate and excited as you are. This is an exciting time in conversation design. What are you waiting for? -------------------------------------------------------------------------- At the end of the day, Simon feels like his journey from academic interest to practical application has been a really natural progression. “I’ve always been so interested in how language, context, and medium affects how we interact,” he said. “And these topics are even more relevant today because more and more we are interacting with machines mediated by whatever technology is between people. What’s happening now is really starting to push the boundaries of linguistics into overall communication studies.” Simon’s last and most generous piece of advice for anyone interested in pursuing a career in conversation design is to [reach out to him](https://www.linkedin.com/in/snwolf) . He continues to be just as passionate about linguistics as he was back in middle school when he was geeking out about the fantasy languages of elves and dwarves. And now he’s also excited about conversation design—where it is today, and where it’s going tomorrow. He’d love to help more like-minded folks get a jump on shaping the future of the industry. _Header image by_ [_Midjourney_](https://www.midjourney.com/) _._ Follow your passion ------------------- After discovering his passion through exposure to constructed languages in fantasy novels, Simon was already on his path. By the time he was in middle school, he knew he wanted to study linguistics seriously. And, unlike most middle schoolers’ aspirations, the dream stuck. He taught some peer-to-peer linguistics in high school, did some related research at boarding school, and continued his studies in college where he ultimately received an accelerated master’s degree. He wrote his thesis on mediated language. His paper, _OMG BREAK A LEGGG LOL: Digital Discourse-Pragmatic Variation In A Theater-Based Community Of Practice,_ analyzes the function of three digital language phenomena (LOL, OMG, and typographic lengthening) to demonstrate the importance of digital language and discourse for building ties in within a community of student artists working together in a theater group. Even though his specific interests and sub-disciplines shifted over the years, Simon was all in from day one and never let anyone or anything derail his passion. **Be like Simon: Believe in what you love, and give it your all.** Even if your passion seems irrational or a little unrealistic. Even if you’re not sure how it will translate in real life, stick with what gets you really excited. You never know where your interests might take you. Think outside the \[career path\] box --------------------------------------  Simon Mitchell-Wolf With a master’s degree in hand, Simon’s most obvious path forward was to go into teaching or research. But he didn’t want to go that route. “I realized I didn’t want to go into academia, and I wasn’t that excited about the research,” Simon explained. “The idea of getting a PhD and a research degree, and then going into the tenure job market—which is like doing a PhD all over again—just didn’t seem like a sustainable plan.” Floundering a little, Simon shifted gears and did some work in arts management and theater production. His was a common story. Many folks with social science degrees struggle when they try to transition into industry roles. “It seems like nothing really fits,” Simon said. “You always feel like you’re having to justify your experience and explain why your skills are relevant to the job.” It was all a little discouraging and exhausting, but there were better things just around the corner. **Be like Simon: Don’t assume that the only path forward is the one right in front of you.** Traditional career strategies don’t always work in emerging fields. Also, it’s okay to feel a little lost—that’s often a normal part of the process. Don’t give up. Build a bot ----------- Before we get to how Simon initially broke into the industry, we want to share a piece of advice that he wishes he’d been given back when he was just starting out: [build a bot](https://www.voiceflow.com/chatbots) . “One thing I wish is that I’d given myself the opportunity to start building my own bots early on,” Simon said. “Just thinking about how to accomplish tasks, answer questions, and communicate information would have given me a really great jumpstart.” Simon stressed that your initial bots don’t have to be anything fancy. In fact, they don’t even need to be built with technology—you can just draft them out on paper. The main thing is to train yourself to think in terms of user-oriented design. Write it down. Read it out loud. And talk to people about it. For example, Simon and his wife take turns standing in for bots and users to work through how a conversation should be designed to deliver the most natural and comfortable experience. **Be like Simon: Start before you think you’re ready.** Don’t assume you have to know everything or have massive technical know-how to start testing your theories and honing your skills. Just go for it. Dip your toes in with freelance ------------------------------- Simon’s first formal foray into conversation design-related career territory came in the form of some freelance [conversational AI](https://www.voiceflow.com/blog/future-of-conversational-ai) consulting gigs. The work was mostly text-to-speech projects, which were an obvious fit because of his background in phonetics, phonology, and acoustic analysis. Simon said, “I got my foot in the door with some short-term, sporadic contracts, which got some tech on my resume.” Eventually, he got on the radar of a recruiter who was working with Google. That connection landed Simon a shot at a “dialogue design” job with the search giant. “I had no idea what ‘dialogue design’ was, but it sounded interesting,” Simon recalled. “And once I started talking with the recruiter, I realized that I was very qualified for the job.” Initially, Simon’s role at Google was primarily analyzing transcripts and doing a lot of maintenance and improvement. He was developing labeling techniques and ways to look at user data and metrics. “I was kind of flying by the seat of my pants,” he said. It wasn’t until near the end of his contract that he was given any opportunities to do actual design work. **Be like Simon: Stay open to opportunities.** Even if a gig isn’t what you think you’re looking for, every experience helps you get to the next experience. Push past your fear to find your place -------------------------------------- When Simon came across the opportunity at PayPal, he had mixed feelings. “It was super exciting, but also super scary,” he recalled. “It was starting a new team pretty much from scratch, fully internal, at a huge company.” What he found, once he overcame his fears and decided to go for it, was that moving to PayPal helped him feel more comfortable engaging with the industry overall. “I started feeling like part of a community at PayPal because we had a real team,” he said. “We were a team of conversation designers who were on the same level with all these other teams.” Today, Simon’s role is focused on actual conversation design—figuring out processes, [defining best practices](https://www.voiceflow.com/blog/best-practices-conversation-design) , assembling great documentation, and really starting a CxD practice from the ground up. It’s the kind of work he set out to do, and now he’s doing it with a great team at a prestigious company. **Be like Simon: Have faith in yourself, and don’t assume that everyone else knows more than you.** Conversation design is a new area of expertise. It’s easy to doubt yourself, but the truth is everyone is trying to figure things out. The people who get in there now—even if they aren’t sure about everything—are the ones who will become leaders in the field as it matures. Bring a new perspective ----------------------- The tech industry revolves around development. Everything is driven and shaped by developer work styles and priorities. But just because that’s the primary focus today doesn’t mean there isn’t room for other ways of doing things. Simon brought not only his technical linguistic skills to the table, but also what he calls a “human focus,” which he’s found is often missing in tech environments. Simon explained, “I felt like what I could bring to this world was the ability to use my technical knowledge and expertise in service to user advocacy, human advocacy.” Whatever angle you take, ensure that conversation design has a seat at the table. “One of the struggles we face as conversation designers is about how we create space for ourselves within the tech environment and empower ourselves to push back on some of the old patterns,” Simon said. He found it helpful to look to the history of user experience (UX) for examples of how to do this well. UX professionals faced a similar uphill battle to carve out their territory and integrate with other teams, and today they're a critical part of any tech team. **Be like Simon: Know what you bring to the table, and help others understand that value.** Conversation design is new to everyone, and to people not already steeped in it, it can be both confusing and daunting. Find creative ways to educate and collaborate with your peers so that they become supporters and allies. Be a part of the community -------------------------- Throughout his journey, Simon has always sought to learn from others, whether they are people on his immediate team, collaborators, or industry peers. “I’m learning a lot from other people in the industry like Tara Nair Shah and [Gina Riley](https://www.linkedin.com/in/g-riley/) ,” he said. “These are people who have already positioned themselves in the industry, have seniority, and are advocating for not only users, but also for their teams.” Simon is learning from these folks not only about the practice itself, but also about how to create space for CxD. “When we have really great leadership in the conversation design space, it helps us create space and hold our ground,” he said. “It helps empower us in the tough conversations about prioritizing, timing, and resource management.” In addition to engaging with specific individuals, Simon also recommends getting out there into the larger CxD community. He’s not a fan of networking, but he has found that connecting with fellow CxD folks never feels like traditional networking. “CxD is still a pretty small world, and the people in it are always excited to talk about conversation design,” he said. “It’s not too late to make friends and get established.” And don’t forget that you bring a lot to the exchange as well. Talking with your peers isn’t only about learning from them. You have experiences and insights to share as well. It’s a two-way street. **Be like Simon: Step outside your comfort zone, and find your people.** They are out there—and they’re just as passionate and excited as you are. This is an exciting time in conversation design. What are you waiting for? -------------------------------------------------------------------------- At the end of the day, Simon feels like his journey from academic interest to practical application has been a really natural progression. “I’ve always been so interested in how language, context, and medium affects how we interact,” he said. “And these topics are even more relevant today because more and more we are interacting with machines mediated by whatever technology is between people. What’s happening now is really starting to push the boundaries of linguistics into overall communication studies.” Simon’s last and most generous piece of advice for anyone interested in pursuing a career in conversation design is to [reach out to him](https://www.linkedin.com/in/snwolf) . He continues to be just as passionate about linguistics as he was back in middle school when he was geeking out about the fantasy languages of elves and dwarves. And now he’s also excited about conversation design—where it is today, and where it’s going tomorrow. He’d love to help more like-minded folks get a jump on shaping the future of the industry. _Header image by_ [_Midjourney_](https://www.midjourney.com/) _._  RECOMMENDED [\ \ ### How a few PayPal’ers tackled CXD’s most impossible divide](https://www.voiceflow.com/pathways/cxd-most-impossible-divide) PUBLISHED July 7, 2023 TOPICS IN THIS ARTICLE Exceptional people [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-to-prove-the-roi-of-conversational-ai-through-buyer-journey-mapping-framework) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to prove the ROI of conversational AI through buyer journey mapping: Framework](https://www.voiceflow.com/pathways/how-to-prove-the-roi-of-conversational-ai-through-buyer-journey-mapping-framework) [](https://www.voiceflow.com/pathways/architecting-the-future-of-ai-agents-5-flexible-conversation-frameworks-you-need) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Architecting the future of AI agents: 5 flexible conversation frameworks you need](https://www.voiceflow.com/pathways/architecting-the-future-of-ai-agents-5-flexible-conversation-frameworks-you-need) [](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 4 things you can do with Airtable and conversation design](https://www.voiceflow.com/pathways/4-things-you-can-do-with-airtable-and-conversation-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/conversation-design-career-journey#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/conversation-design-career-journey#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Design and prototype production-ready conversational experiences with user personas | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Design and prototype production-ready conversational experiences with user personas =================================================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Production-ready conversational designs should be as reflective of the live experience as possible when you’re performing high-fidelity prototyping for internal and [external user testing](https://www.voiceflow.com/blog/how-to-run-unmoderated-user-testing-for-your-conversation-designs) . The results from those tests gives conversation design teams better insight not only into how users will interact with the experience in the wild, but how much development work will be necessary in order to get the design into production faster. When conversation designers are using tools that force them to include steps that apply business logic in your conversation design, it adds unnecessary steps to the canvas, takes away from the high fidelity prototype experience for users, and can create confusion for development. .avif) The canvas on which you design conversations should be an artifact and running users through your prototypes should feel like a seamless experience, where they are funneled into the appropriate part of your conversation design without being aware of it. That’s why today we are excited to introduce User Personas in Voiceflow: The best way to design conversations for every user. Organize Your Conversation Design Canvas ---------------------------------------- Before User Personas, conversation designers using Voiceflow had to apply additional paths on their designs to account for business logic that would be applied in production but couldn’t be replicated on the canvas. Designers could accidentally create experiences where users entered an endless loop of repeating paths and blocks because conversation designers couldn’t see each individual experience for the different types of users who would be interacting with the live assistant. This resulted in a messy canvas with steps that weren’t reflective of the live, production environment of the conversational experience. ### **The Old View:**  In the example above, users would have to click a button in the beginning of the prototyping experience to test out a specific path. This step wouldn’t be necessary in a live environment but was necessary to ensure users testing the experience could understand each individual path that a customer could go down. ### **The New View** With User Personas, you’ll be able to test your designs and send custom prototype links that have a specific variable set on different user personas.  That way, when you're sending links out for user testing, you've already determined the experience the user will get, creating a seamless and high-fidelity testing experience that mimics your live environment. In the example used below, you can set up user persona states for users who would would be run through a specific scenario based on user account information, eliminating the need for a user to indicate which path they want to go down through buttons in the beginning of the conversation. Not only will your canvas be cleaner, there will be clearer delineations for where persona-specific experiences begin. With User Personas, you can choose at which Block on the canvas, in any [Topic](https://www.voiceflow.com/blog/how-to-leverage-topics-and-components-in-your-conversation-designs) the persona should be set. No more accidental endless loops because you forgot to design for a specific use case, user personas help design teams create clarity on who is accounted for in each experience.  ### **All Project Variables View** When you’re debugging your projects, you’ll notice that the All Project Variables settings have moved to account for user personas. The variables used to be shown in the lefthand sidebar in testing mode.  The variables have no been moved to be listed under the dropdown to manage and create your user personas. You’ll also notice that the testing settings are now next to the Share Prototype button versus the lefthand sidebar.  _For a more in depth view on how to access and create User Personas,_ [_check out this help doc._](https://www.voiceflow.com/docs/prototyping-test-tool#toc-3) When Should You Leverage User Personas -------------------------------------- Conversations that have a semblance of variable data in their designs - different experiences for new and returning users, for users who indicate they have or have not accomplished a task, for users who fit certain account criteria - should have conditional paths that are based on these variable states specific to different kinds of user personas in Voiceflow. > This is exactly what we need. Most of our conversational experiences require account disambiguation, so easily showcasing the separate paths each type of user can go down will save us at least 10-15 minutes of design work for each persona. > > **Gina Riley, VP, Conversation & UX Design Lead** @ JP Morgan, Chase In order to get the most out of User Personas, your conversation design should include these [conditional paths](https://www.voiceflow.com/docs/designing-logic-steps#toc-0) , indicating that there are different scenarios and paths for users who fit different criteria. When you’re collaborating with other stakeholders to get your conversational experience from design to launch, highlighting individual paths for different user personas make it easier to pressure test your assistant for any and all scenarios it could encounter when it’s live. > **If you’re interested in using User Personas,** [**create your free Voiceflow account here**](https://creator.voiceflow.com/signup) > **and start designing more human conversations. If you're a Voiceflow user who wants more information on User Personas, check out our help documentation here and** [**our tutorial on using User Personas here.**](https://www.youtube.com/watch?v=Cf8Vp5N-5pE) The canvas on which you design conversations should be an artifact and running users through your prototypes should feel like a seamless experience, where they are funneled into the appropriate part of your conversation design without being aware of it. That’s why today we are excited to introduce User Personas in Voiceflow: The best way to design conversations for every user. Organize Your Conversation Design Canvas ---------------------------------------- Before User Personas, conversation designers using Voiceflow had to apply additional paths on their designs to account for business logic that would be applied in production but couldn’t be replicated on the canvas. Designers could accidentally create experiences where users entered an endless loop of repeating paths and blocks because conversation designers couldn’t see each individual experience for the different types of users who would be interacting with the live assistant. This resulted in a messy canvas with steps that weren’t reflective of the live, production environment of the conversational experience. ### **The Old View:**  In the example above, users would have to click a button in the beginning of the prototyping experience to test out a specific path. This step wouldn’t be necessary in a live environment but was necessary to ensure users testing the experience could understand each individual path that a customer could go down. ### **The New View** With User Personas, you’ll be able to test your designs and send custom prototype links that have a specific variable set on different user personas.  That way, when you're sending links out for user testing, you've already determined the experience the user will get, creating a seamless and high-fidelity testing experience that mimics your live environment. In the example used below, you can set up user persona states for users who would would be run through a specific scenario based on user account information, eliminating the need for a user to indicate which path they want to go down through buttons in the beginning of the conversation. Not only will your canvas be cleaner, there will be clearer delineations for where persona-specific experiences begin. With User Personas, you can choose at which Block on the canvas, in any [Topic](https://www.voiceflow.com/blog/how-to-leverage-topics-and-components-in-your-conversation-designs) the persona should be set. No more accidental endless loops because you forgot to design for a specific use case, user personas help design teams create clarity on who is accounted for in each experience.  ### **All Project Variables View** When you’re debugging your projects, you’ll notice that the All Project Variables settings have moved to account for user personas. The variables used to be shown in the lefthand sidebar in testing mode.  The variables have no been moved to be listed under the dropdown to manage and create your user personas. You’ll also notice that the testing settings are now next to the Share Prototype button versus the lefthand sidebar.  _For a more in depth view on how to access and create User Personas,_ [_check out this help doc._](https://www.voiceflow.com/docs/prototyping-test-tool#toc-3) When Should You Leverage User Personas -------------------------------------- Conversations that have a semblance of variable data in their designs - different experiences for new and returning users, for users who indicate they have or have not accomplished a task, for users who fit certain account criteria - should have conditional paths that are based on these variable states specific to different kinds of user personas in Voiceflow. > This is exactly what we need. Most of our conversational experiences require account disambiguation, so easily showcasing the separate paths each type of user can go down will save us at least 10-15 minutes of design work for each persona. > > **Gina Riley, VP, Conversation & UX Design Lead** @ JP Morgan, Chase In order to get the most out of User Personas, your conversation design should include these [conditional paths](https://www.voiceflow.com/docs/designing-logic-steps#toc-0) , indicating that there are different scenarios and paths for users who fit different criteria. When you’re collaborating with other stakeholders to get your conversational experience from design to launch, highlighting individual paths for different user personas make it easier to pressure test your assistant for any and all scenarios it could encounter when it’s live. > **If you’re interested in using User Personas,** [**create your free Voiceflow account here**](https://creator.voiceflow.com/signup) > **and start designing more human conversations. If you're a Voiceflow user who wants more information on User Personas, check out our help documentation here and** [**our tutorial on using User Personas here.**](https://www.youtube.com/watch?v=Cf8Vp5N-5pE)  RECOMMENDED [](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas#) PUBLISHED March 29, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/voiceflow-raises-3-5-million-to-build-the-future-of-voice-interface-design) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow raises $3.5 million to build the future of voice interface design](https://www.voiceflow.com/pathways/voiceflow-raises-3-5-million-to-build-the-future-of-voice-interface-design) [.avif)](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Design and prototype production-ready conversational experiences with user personas](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas) [](https://www.voiceflow.com/pathways/behind-the-hype-how-klarnas-customer-support-agent-falls-short) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Behind the hype: How Klarna's customer support agent falls short](https://www.voiceflow.com/pathways/behind-the-hype-how-klarnas-customer-support-agent-falls-short) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/design-and-prototype-production-ready-conversational-experiences-with-user-personas#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Why the dominant discourse surrounding LLMs needs to change | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Why the dominant discourse surrounding LLMs needs to change =========================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Since the advent of ChatGPT, what feels like a tsunami of spectators have questioned the inherent value of large language models (LLMs). People have debated not whether LLMs can be useful for content generation, but whether this technology _should_ be leveraged at all. Skeptics fear that LLMs will replace us and, in doing so, eradicate a time-honored human art form. Underlying this argument is the belief that content generated with the assistance of artificial intelligence is somehow inferior or “less than” content created purely by humans—it’s all science and no art, they say. For this reason, LLMs can’t possibly be used to build authentic relationships with customers. But “LLMs aren’t intended to replace human ingenuity,” explains [Anna Rosen](https://twitter.com/whatsupanna) , Enterprise Solutions Engineer at Voiceflow. “They’re simply tools that empower people to craft better content faster.” > “LLMs aren’t intended to replace human ingenuity. They’re simply tools that empower people to craft better content faster.” > _–Anna Rosen_ In this blog post, we highlight three arguments that support Anna’s claim.  1\. It’s all about the context ------------------------------ When making a judgment call about whether to use LLMs, Anna believes it’s important to take your unique context into account. As conversation designers, we don’t write wedding speeches. We guide technology in discussing business matters with our customers. That anchors our work soundly within the realm of commercial writing. Unlike creative writing, commercial writing is full of constraints. You take in information from select sources and write what helps you achieve business goals. Along the way, brand guidelines dictate not only what you can say but also how you say it. LLMs make sense in this context because, similar to commercial writing, what you feed it has a direct impact on what’s generated. The input determines the output. 2\. It’s all part of the process -------------------------------- Anna believes LLMs should be used for part, but not all, of the writing process. As a life-long writer who majored in English literature, Anna heard one quote over and over again during university: “There’s no such thing as good writing—only good _re_writing.” > “There’s no such thing as good writing—only good _re_writing.”_ > –Countless university professors_ Countless professors echoed this sentiment to help students understand that professional writers never publish a first draft. It was a memorable way for them to communicate that revision is an indispensable part of the writing process. The same rule applies to content generated by LLMs. They can help you generate a V1, but that’s not what you should publish. Between that first step and the finish line lies the revision process, which humans are uniquely qualified to handle. According to the University of North Carolina at Chapel Hill, revision means [looking at something again with a fresh perspective](https://writingcenter.unc.edu/tips-and-tools/revising-drafts/) . It typically involves “reconsidering your arguments, reviewing your evidence, refining your purpose, reorganizing your presentation, \[and\] reviving stale prose”—tasks artificial intelligence tends to struggle with. When working with LLMs, it’s also important to _validate_ what’s generated. That’s how you ensure facts are accurate and what’s said is intentional. 3\. It’s all because of the benefits ------------------------------------ “If it’s not broke, don’t fix it” can be a comforting thought. However, because companies stand to benefit from greater efficiency and better outcomes, Anna believes it’s worthwhile to embrace this change. We’re all familiar with the plague of writer’s block. No one enjoys staring at a blank page. By getting words on paper quickly, LLMs effectively speed up the writing process, leaving more time for the higher-value work of revising (not to mention unrelated, mission-critical projects). LLMs also quickly generate a large variety of utterances. In the traditional content creation process, that task requires little skill but can take a lot of time. LLMs thus eliminate a lot of tedium from your workflow. Take these two utterance workflows, for example. In the time it took to manually write and edit 10 utterances for the intent from scratch (left), the designer using generative AI could create eight different intents with as many utterances. The efficiency is great, but the benefits of LLMs truly show when conversation designers can spend this newly freed time on the art of the assistant experience. The terms and concepts presented by generative AI can also trigger critical thinking and foster innovation. By prompting your brain to form new connections and consider things you hadn’t thought of before, this fuels original content development. In sum, LLMs enable you to reclaim valuable brain space while designing better user experiences. 1\. It’s all about the context ------------------------------ When making a judgment call about whether to use LLMs, Anna believes it’s important to take your unique context into account. As conversation designers, we don’t write wedding speeches. We guide technology in discussing business matters with our customers. That anchors our work soundly within the realm of commercial writing. Unlike creative writing, commercial writing is full of constraints. You take in information from select sources and write what helps you achieve business goals. Along the way, brand guidelines dictate not only what you can say but also how you say it. LLMs make sense in this context because, similar to commercial writing, what you feed it has a direct impact on what’s generated. The input determines the output. 2\. It’s all part of the process -------------------------------- Anna believes LLMs should be used for part, but not all, of the writing process. As a life-long writer who majored in English literature, Anna heard one quote over and over again during university: “There’s no such thing as good writing—only good _re_writing.” > “There’s no such thing as good writing—only good _re_writing.”_ > –Countless university professors_ Countless professors echoed this sentiment to help students understand that professional writers never publish a first draft. It was a memorable way for them to communicate that revision is an indispensable part of the writing process. The same rule applies to content generated by LLMs. They can help you generate a V1, but that’s not what you should publish. Between that first step and the finish line lies the revision process, which humans are uniquely qualified to handle. According to the University of North Carolina at Chapel Hill, revision means [looking at something again with a fresh perspective](https://writingcenter.unc.edu/tips-and-tools/revising-drafts/) . It typically involves “reconsidering your arguments, reviewing your evidence, refining your purpose, reorganizing your presentation, \[and\] reviving stale prose”—tasks artificial intelligence tends to struggle with. When working with LLMs, it’s also important to _validate_ what’s generated. That’s how you ensure facts are accurate and what’s said is intentional. 3\. It’s all because of the benefits ------------------------------------ “If it’s not broke, don’t fix it” can be a comforting thought. However, because companies stand to benefit from greater efficiency and better outcomes, Anna believes it’s worthwhile to embrace this change. We’re all familiar with the plague of writer’s block. No one enjoys staring at a blank page. By getting words on paper quickly, LLMs effectively speed up the writing process, leaving more time for the higher-value work of revising (not to mention unrelated, mission-critical projects). LLMs also quickly generate a large variety of utterances. In the traditional content creation process, that task requires little skill but can take a lot of time. LLMs thus eliminate a lot of tedium from your workflow. Take these two utterance workflows, for example. In the time it took to manually write and edit 10 utterances for the intent from scratch (left), the designer using generative AI could create eight different intents with as many utterances. The efficiency is great, but the benefits of LLMs truly show when conversation designers can spend this newly freed time on the art of the assistant experience. The terms and concepts presented by generative AI can also trigger critical thinking and foster innovation. By prompting your brain to form new connections and consider things you hadn’t thought of before, this fuels original content development. In sum, LLMs enable you to reclaim valuable brain space while designing better user experiences.  RECOMMENDED [\ \ ### LLMs won’t replace NLUs. Here’s why](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) PUBLISHED March 31, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Building a virtual agent from scratch? Start here.](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here) [](https://www.voiceflow.com/pathways/webpage-assistant-voiceflow-dialog-api) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Create a webpage to talk to your assistant using Voiceflow Dialog API](https://www.voiceflow.com/pathways/webpage-assistant-voiceflow-dialog-api) [](https://www.voiceflow.com/pathways/5-principles-for-good-natural-language-understanding-nlu-design) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### 5 principles for good natural language understanding (NLU) design](https://www.voiceflow.com/pathways/5-principles-for-good-natural-language-understanding-nlu-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/dominant-discourse-llms#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/dominant-discourse-llms#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How eSnipe automated 70% of their help center tickets with an AI-powered search agent | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) How eSnipe automated 70% of their help center tickets with an AI-powered search agent ===================================================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) ### **The scaling customer support problem** Scaling effective customer support was a challenge for [eSnipe.com](https://www.esnipe.com/) - a fast-growing eBay auction bidding tool. The company's more than 2,000,000 customers and 100s of product features meant a substantial number of user questions, resulting in a high ticket volume (average: 9,000 tickets/month). Users struggled to self-serve relevant articles in their existing help center, leading to many manual, expensive support tickets and frustrated users. Led by Winston, a team of developers and customer service managers set out to automate this complex customer workflow.  ### **An AI Agent with a trained knowledge base** ****With customer experience in mind, Winston and his team used Voiceflow to automate their help center for better customer support. The team designed and deployed a knowledge base-powered agent that integrated with their existing help center interface. Here's how they did it:**** 1. **Curated a Knowledge Base (KB)**: Winston's team loaded their entire website into a Voiceflow knowledge base, creating a central repository of eSnipe information. The knowledge base consisted of help article URLs, PDF documents, and .txt files. **** 2. **Deployed the Knowledge Base Query API**: They used the [Knowledge Base Query API](https://developer.voiceflow.com/reference/post_knowledge-base-query) to enable external access to the curated KB information. This would allow for quick responses to user queries whenever the API was called. 3. **Connected to a custom interface:** A script was created for integrating their AI Agent and eSnipe.com using Voiceflow's API step and CloudFlare. This ensured accurate, rapid answers at their existing help center frontend.  "Customer support ticket volume has decreased by 70% since launching an AI agent in our help center." ----------------------------------------------------------------------------------------------------- ### **Impact on customers & support** Winston and team have seen remarkable impact since launching their AI agent. * **Total ticket volume has drastically decreased.** The original 9,000 tickets/month is down to 2,700 tickets/month - a 70% decrease. This is reducing the burden on their support team and signaling that customers are more frequently self-servicing their questions.**** * **Response times are faster.** Customer questions are handled more promptly than the manual ticketing support process. * **Live agents are allocating time better.** They can fully focus their time on the 30% of remaining support tickets that need valuable live agent skills and time.**** * **Support is always user-friendly**. The AI-powered responses are trained on the company's content and prompted in their branded voice to make sure the help center agent is an extension of their customer support team. Hear more from our chat with Winston, including a deep dive into his integration build here. > **What's next? Winston and his team are expanding their knowledge base and AI agent build into a chat-based interface, offering features such as account lookup, transaction-specific assistance, and page-specific guidance.** ### **An AI Agent with a trained knowledge base** ****With customer experience in mind, Winston and his team used Voiceflow to automate their help center for better customer support. The team designed and deployed a knowledge base-powered agent that integrated with their existing help center interface. Here's how they did it:**** 1. **Curated a Knowledge Base (KB)**: Winston's team loaded their entire website into a Voiceflow knowledge base, creating a central repository of eSnipe information. The knowledge base consisted of help article URLs, PDF documents, and .txt files. **** 2. **Deployed the Knowledge Base Query API**: They used the [Knowledge Base Query API](https://developer.voiceflow.com/reference/post_knowledge-base-query) to enable external access to the curated KB information. This would allow for quick responses to user queries whenever the API was called. 3. **Connected to a custom interface:** A script was created for integrating their AI Agent and eSnipe.com using Voiceflow's API step and CloudFlare. This ensured accurate, rapid answers at their existing help center frontend.  "Customer support ticket volume has decreased by 70% since launching an AI agent in our help center." ----------------------------------------------------------------------------------------------------- ### **Impact on customers & support** Winston and team have seen remarkable impact since launching their AI agent. * **Total ticket volume has drastically decreased.** The original 9,000 tickets/month is down to 2,700 tickets/month - a 70% decrease. This is reducing the burden on their support team and signaling that customers are more frequently self-servicing their questions.**** * **Response times are faster.** Customer questions are handled more promptly than the manual ticketing support process. * **Live agents are allocating time better.** They can fully focus their time on the 30% of remaining support tickets that need valuable live agent skills and time.**** * **Support is always user-friendly**. The AI-powered responses are trained on the company's content and prompted in their branded voice to make sure the help center agent is an extension of their customer support team. Hear more from our chat with Winston, including a deep dive into his integration build here. > **What's next? Winston and his team are expanding their knowledge base and AI agent build into a chat-based interface, offering features such as account lookup, transaction-specific assistance, and page-specific guidance.**  RECOMMENDED [\ \ ### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) PUBLISHED December 1, 2023 TOPICS IN THIS ARTICLE Exceptional work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### How eSnipe automated 70% of their help center tickets with an AI-powered search agent](https://www.voiceflow.com/pathways/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Your CxD center of excellence (and why you don’t need one right now) | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Your CxD center of excellence (and why you don’t need one right now) ==================================================================== [\ \ Ayesha Saleem\ \ AI Design Lead at Instacart](https://www.voiceflow.com/contributors/ayesha-saleem) The conversational AI industry is notoriously difficult to define. Like a cresting wave, it seems like every time we discover a new “[best practice](https://www.voiceflow.com/blog/best-practices-conversation-design) ” or a simplified way of working, it’s washed away by the influx of the new. So when the expectations from customers are high and industry-wide best practices are still taking shape, it’s hard to imagine creating something like a conversational AI center of excellence (CoE). Because, what _is_ excellent right now? But let’s define this term. “Center of excellence” is a pithy buzzword that’s come to mean different things across industries. For tech companies, a CoE is often defined as a team that provides best practices, governance, and support to the rest of the organization. They help their fellow teammates be great at that thing, usually in the form of performing training, providing support, standardizing processes, and solidifying governance. For example, [a writing center of excellence](https://hbr.org/2017/02/why-your-organization-needs-a-writing-center) . If you’re leading your own CxD team, it’ll be some time until our industry can mature enough to build true centers of CxD excellence. Which is why I’m here to let you off the hook. In the meantime, the goal for CxD leaders should be to develop intentional best practices and governance right now, while creating forward-thinking process documentation. This article is about how to begin thinking in this way—one foot in the present and one foot in the future of your budding CoE.  The four things you need to begin building your CxD center of excellence ------------------------------------------------------------------------- Developing a CoE within your team means you are equally focused on how to achieve success in your current projects, and how you might teach those exact skills to someone else, or replicate successful results later on. You do this by solidifying the good processes you already have in place and refining the others that are in flux. So, if you’re building your CxD team, expanding your team’s output, or you just want to become the kind of team that creates repeatable success, here are four strategies that’ll help. ### 1\. Learn from anyone who will talk to you When I started as a CxD leader at Instacart, I knew I couldn’t apply everything I’d learned at GE or Rocket Mortgage to the new team I was building. Every single CxD team speaks a different language and has different goals and ways of achieving them. For instance, I found that well-documented governance was one of the most important things I could do in my new role. And the way I discovered that was to talk to anyone who could spare 15 minutes. I began by looking up people's titles—UX researcher, design lead, operations, even people managers—and scheduled meetings with each of them to gather information. I wanted to know how they built their governance, processes, and operational structures. I asked how they defined “center of excellence.” I even asked how they decided which practices were best—how they enshrined them and made them repeatable. I read internal research, engineering and process documents, scoured our Confluence libraries, Wikis, and Notions. Then I met with the people who developed those resources and asked them my questions, so I could recreate them for my CxD team. I also asked who I should meet next—who is doing a phenomenal job of building their team and their own center of excellence? I had two conversations a week for six months with folks across the organization. I got hand cramps from all the notetaking. But the results were astonishing. One conversation with the leader of the people operations team led to me borrowing their spreadsheet templates and reworking them for our own use. They were so organized and their processes have helped inform our own—something that wouldn’t have happened had I not sat down to learn from a more established team. This approach allowed me to also get to know the folks I would be collaborating with—especially engineers and product managers. I knew that if I learned from them (and even made a few friends), our projects would go smoother and make our lives a lot easier. I turned out to be right. ### 2\. Don’t skip defining clear roles and expectations I’ve had plenty of experience with conversation design roles that weren’t defined or didn’t have expectations. [Honestly, that kind of role kinda works for me](https://www.voiceflow.com/blog/best-conversational-ai-practices) . But for most people, a vaguely defined role paired with unclear expectations could mean that an entire team of CxDs feels untethered. I've been in scenarios where two or three conversation designers are working on competing priorities and no one can decide whether there’s a singular path forward. A shifting conversational AI industry shouldn’t mean there’s a constant state of internal confusion in your CxD team. Building a one-pager with the roles and responsibilities of each person on the team frees everyone to focus on what they do best. Even if every day feels like a different job—and as CxDs, we know it often does— there should be basic expectations for each role. Only from there can we add tasks to each role that contribute to building a center of excellence—like creating process documentation or a style guide (more on that in part 4). ### 3\. Choose the right tools When I started out in CxD, I used whatever tool I thought would be cool that week. It'd be like buying a new notebook. For a time, half my projects were in Miro and the other half were divided between Voiceflow and Excel. As fun as it was to try a bunch of new tools all the time, it’s hard to scale a cohesive CxD team when you don’t have the right tools. A big part of the center of excellence is finding your tools and creating a shared library of resources to help your designers fully utilize those tools. In the end, I tried 15 to 20 different tools before landing on the ones we’d use regularly. And I still dedicate a couple of hours a month to trying out new tools because our space is changing so much. My advice? There are a few industry leaders. But not every tool will work for every team and use case. [Find yours](https://www.voiceflow.com/blog/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) and build training, support, and process documentation around those tools. Your goal is to create an operation that enables a new designer to join your team and asynchronously and independently get up to speed on your tech stack as soon as possible. ### 4\. Create a living style guide A style guide is essential if you want to maintain brand consistency. If your AI assistant reads as if it’s been written by several different people, you’re doing something very wrong. The most obvious way to avoid this is to standardize certain stylistic rules—dates, numbers, contractions, and word choice. That portion of the style guide should be a ready-made checklist anyone can follow. But the most important part of the style guide is the voice and tone section. We even have a persona for our bot. So for a hypothetical example, this bot is a 34-year-old, gender-neutral person who grew up in the Midwest, whose major characteristics are care and empathy. Their humor sounds like an elder millennial, but they never make pop culture references. Maybe they're helpful, but they're not a pushover. They're confident, but not arrogant. This level of detail and direction on voice and tone will keep your team on track. We created the style guide as a main document that supports our CoE—it's written and designed to help us create one cohesive persona even though we’re all working on disparate parts of the AI assistant. Style guides take a lot of people to create—hence why building internal relationships is paramount. Our UX research team defined our target audience, our marketing team helped us edit the guide for clarity, and customer success helped us refine it with user feedback. We relied on the real-life experiences of users and internal team members to understand how our assistant should speak and react. We’re constantly tweaking the style guide as our best practices evolve. Spread the excellence around ---------------------------- In our continued attempts to build systems, refine processes, and document our successes, we can’t forget that our way isn’t the only right one. Whether it’s gut-checking our ongoing projects using peer reviews or swapping stories with other conversational AI teams at networking events, we all know that in the pursuit of excellence, we all have something to offer and to gain. To give and to take. Conversation design is an ever-evolving beast—the rise of generative AI, the increasingly sophisticated expectations of customers, and access to technology means we’re entering a new era. That’s why starting to create the processes and documents that lead to excellence is something you should be doing right now. The four things you need to begin building your CxD center of excellence ------------------------------------------------------------------------- Developing a CoE within your team means you are equally focused on how to achieve success in your current projects, and how you might teach those exact skills to someone else, or replicate successful results later on. You do this by solidifying the good processes you already have in place and refining the others that are in flux. So, if you’re building your CxD team, expanding your team’s output, or you just want to become the kind of team that creates repeatable success, here are four strategies that’ll help. ### 1\. Learn from anyone who will talk to you When I started as a CxD leader at Instacart, I knew I couldn’t apply everything I’d learned at GE or Rocket Mortgage to the new team I was building. Every single CxD team speaks a different language and has different goals and ways of achieving them. For instance, I found that well-documented governance was one of the most important things I could do in my new role. And the way I discovered that was to talk to anyone who could spare 15 minutes. I began by looking up people's titles—UX researcher, design lead, operations, even people managers—and scheduled meetings with each of them to gather information. I wanted to know how they built their governance, processes, and operational structures. I asked how they defined “center of excellence.” I even asked how they decided which practices were best—how they enshrined them and made them repeatable. I read internal research, engineering and process documents, scoured our Confluence libraries, Wikis, and Notions. Then I met with the people who developed those resources and asked them my questions, so I could recreate them for my CxD team. I also asked who I should meet next—who is doing a phenomenal job of building their team and their own center of excellence? I had two conversations a week for six months with folks across the organization. I got hand cramps from all the notetaking. But the results were astonishing. One conversation with the leader of the people operations team led to me borrowing their spreadsheet templates and reworking them for our own use. They were so organized and their processes have helped inform our own—something that wouldn’t have happened had I not sat down to learn from a more established team. This approach allowed me to also get to know the folks I would be collaborating with—especially engineers and product managers. I knew that if I learned from them (and even made a few friends), our projects would go smoother and make our lives a lot easier. I turned out to be right. ### 2\. Don’t skip defining clear roles and expectations I’ve had plenty of experience with conversation design roles that weren’t defined or didn’t have expectations. [Honestly, that kind of role kinda works for me](https://www.voiceflow.com/blog/best-conversational-ai-practices) . But for most people, a vaguely defined role paired with unclear expectations could mean that an entire team of CxDs feels untethered. I've been in scenarios where two or three conversation designers are working on competing priorities and no one can decide whether there’s a singular path forward. A shifting conversational AI industry shouldn’t mean there’s a constant state of internal confusion in your CxD team. Building a one-pager with the roles and responsibilities of each person on the team frees everyone to focus on what they do best. Even if every day feels like a different job—and as CxDs, we know it often does— there should be basic expectations for each role. Only from there can we add tasks to each role that contribute to building a center of excellence—like creating process documentation or a style guide (more on that in part 4). ### 3\. Choose the right tools When I started out in CxD, I used whatever tool I thought would be cool that week. It'd be like buying a new notebook. For a time, half my projects were in Miro and the other half were divided between Voiceflow and Excel. As fun as it was to try a bunch of new tools all the time, it’s hard to scale a cohesive CxD team when you don’t have the right tools. A big part of the center of excellence is finding your tools and creating a shared library of resources to help your designers fully utilize those tools. In the end, I tried 15 to 20 different tools before landing on the ones we’d use regularly. And I still dedicate a couple of hours a month to trying out new tools because our space is changing so much. My advice? There are a few industry leaders. But not every tool will work for every team and use case. [Find yours](https://www.voiceflow.com/blog/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) and build training, support, and process documentation around those tools. Your goal is to create an operation that enables a new designer to join your team and asynchronously and independently get up to speed on your tech stack as soon as possible. ### 4\. Create a living style guide A style guide is essential if you want to maintain brand consistency. If your AI assistant reads as if it’s been written by several different people, you’re doing something very wrong. The most obvious way to avoid this is to standardize certain stylistic rules—dates, numbers, contractions, and word choice. That portion of the style guide should be a ready-made checklist anyone can follow. But the most important part of the style guide is the voice and tone section. We even have a persona for our bot. So for a hypothetical example, this bot is a 34-year-old, gender-neutral person who grew up in the Midwest, whose major characteristics are care and empathy. Their humor sounds like an elder millennial, but they never make pop culture references. Maybe they're helpful, but they're not a pushover. They're confident, but not arrogant. This level of detail and direction on voice and tone will keep your team on track. We created the style guide as a main document that supports our CoE—it's written and designed to help us create one cohesive persona even though we’re all working on disparate parts of the AI assistant. Style guides take a lot of people to create—hence why building internal relationships is paramount. Our UX research team defined our target audience, our marketing team helped us edit the guide for clarity, and customer success helped us refine it with user feedback. We relied on the real-life experiences of users and internal team members to understand how our assistant should speak and react. We’re constantly tweaking the style guide as our best practices evolve. Spread the excellence around ---------------------------- In our continued attempts to build systems, refine processes, and document our successes, we can’t forget that our way isn’t the only right one. Whether it’s gut-checking our ongoing projects using peer reviews or swapping stories with other conversational AI teams at networking events, we all know that in the pursuit of excellence, we all have something to offer and to gain. To give and to take. Conversation design is an ever-evolving beast—the rise of generative AI, the increasingly sophisticated expectations of customers, and access to technology means we’re entering a new era. That’s why starting to create the processes and documents that lead to excellence is something you should be doing right now.  RECOMMENDED [\ \ ### What Nike's chatbot taught us about conversation design best practices](https://www.voiceflow.com/pathways/nike-conversation-design-best-practices) PUBLISHED August 24, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Ayesha Saleem\ \ AI Design Lead at Instacart\ \ Ayesha is a whip-smart CxD leader who’s sought out to speak at conferences, create comprehensive CxD guides, and mentor emerging leaders. She loves to build conversational AI teams from the ground up, not unlike her farm-to-table veggie garden.](https://www.voiceflow.com/contributors/ayesha-saleem) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you should A/B test your conversational experiences](https://www.voiceflow.com/pathways/why-you-should-a-b-test-your-conversational-experiences-a) [](https://www.voiceflow.com/pathways/the-case-for-a-full-stack-conversation-designer) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The case for a full-stack conversation designer](https://www.voiceflow.com/pathways/the-case-for-a-full-stack-conversation-designer) [](https://www.voiceflow.com/pathways/how-to-use-open-vs-closed-prompts-in-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to use open vs. closed prompts in conversation design](https://www.voiceflow.com/pathways/how-to-use-open-vs-closed-prompts-in-conversation-design) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/cxd-center-of-excellence#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/cxd-center-of-excellence#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Experts share 3 expectations about LLMs to set with business stakeholders | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Experts share 3 expectations about LLMs to set with business stakeholders ========================================================================= [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Since the launch of ChatGPT three months ago, large language models (LLMs) have rapidly transformed how people work across entire industries and disciplines. From content creation to customer service, conversational AI is well on its way to becoming embedded into our daily workflows. After [introducing AI Assist](https://www.voiceflow.com/blog/ai-assist-llms-voiceflow) in January, Voiceflow brought together a panel of experts to discuss the implications of this trend: [Greg Bennett](https://www.linkedin.com/in/gab45/) (Salesforce), [Daryl Thomas](https://www.linkedin.com/in/darylthomas/) (Cisco), [Kenny Tan](https://www.linkedin.com/in/tkl86?trk=public_post_feed-actor-name) (Intuit), [Sibel Ciddi](https://www.linkedin.com/in/sibelciddi/) (U.S. Bank), and [Rob Hayes](https://www.linkedin.com/in/therobhayes/) (Voiceflow). In this blog post, we highlight their top three takeaways around how to set proper expectations around LLMs with business stakeholders.  1\. LLMs won’t steal your job ----------------------------- It’s important to communicate that ChatGPT and other LLMs can’t replace humans, the experts agreed. Otherwise, workers may panic and managers may plan for a future that never arrives. The truth is, people have [unique skills that machines can’t mimic](https://hbr.org/2021/03/ai-should-augment-human-intelligence-not-replace-it) , like the ability to empathize, imagine, and judge. We may be informed by the data in our environment, but we’re not dependent on it to function. That makes us inherently valuable to the companies we’re employed by. 2\. Teams will get more done, faster ------------------------------------ While LLMs won’t replace workers, they will empower professionals to move faster and get more done. This is one benefit of embedding LLMs into your workflows that is worth highlighting, the panelists agreed. By accelerating workflows, LLMs will restore employee bandwidth—empowering people to spend more time elevating the quality of their execution and improving the systems and processes that facilitate their activities. 3\. It’s not as simple as flipping a switch ------------------------------------------- Many executives are excited to invest in LLMs because they recognize their potential to drive positive business outcomes. But few leaders understand what conversational designers do behind the scenes to make LLMs useful. Our panel of experts agreed that guardrails are necessary—a layer between what’s generated by LLMs and what’s delivered to the customer. And building rules, policy, and governance into conversational AI requires both careful consideration and cross-functional collaboration. For these reasons, it’s important to set clear expectations around the need for human supervision when discussing LLMs. ### More on LLMs and conversation design LLMs are now in Voiceflow. [Read about these game-changing new features—and even more we plan to roll out soon](https://www.voiceflow.com/blog/voiceflow-winter-product-roundup) . 1\. LLMs won’t steal your job ----------------------------- It’s important to communicate that ChatGPT and other LLMs can’t replace humans, the experts agreed. Otherwise, workers may panic and managers may plan for a future that never arrives. The truth is, people have [unique skills that machines can’t mimic](https://hbr.org/2021/03/ai-should-augment-human-intelligence-not-replace-it) , like the ability to empathize, imagine, and judge. We may be informed by the data in our environment, but we’re not dependent on it to function. That makes us inherently valuable to the companies we’re employed by. 2\. Teams will get more done, faster ------------------------------------ While LLMs won’t replace workers, they will empower professionals to move faster and get more done. This is one benefit of embedding LLMs into your workflows that is worth highlighting, the panelists agreed. By accelerating workflows, LLMs will restore employee bandwidth—empowering people to spend more time elevating the quality of their execution and improving the systems and processes that facilitate their activities. 3\. It’s not as simple as flipping a switch ------------------------------------------- Many executives are excited to invest in LLMs because they recognize their potential to drive positive business outcomes. But few leaders understand what conversational designers do behind the scenes to make LLMs useful. Our panel of experts agreed that guardrails are necessary—a layer between what’s generated by LLMs and what’s delivered to the customer. And building rules, policy, and governance into conversational AI requires both careful consideration and cross-functional collaboration. For these reasons, it’s important to set clear expectations around the need for human supervision when discussing LLMs. ### More on LLMs and conversation design LLMs are now in Voiceflow. [Read about these game-changing new features—and even more we plan to roll out soon](https://www.voiceflow.com/blog/voiceflow-winter-product-roundup) .  RECOMMENDED [\ \ ### LLMs won’t replace NLUs. Here’s why](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) PUBLISHED March 9, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/voiceflow-frog-sxsw) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How a pair of interaction designers used Voiceflow to create an unforgettable experience at SXSW](https://www.voiceflow.com/pathways/voiceflow-frog-sxsw) [](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The evolution and outlook of voice user interfaces](https://www.voiceflow.com/pathways/ask-an-expert-the-evolution-and-future-of-voice-user-interfaces) [](https://www.voiceflow.com/pathways/best-conversational-ai-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Instacart's Ayesha Saleem shares her 5 best conversational AI practices](https://www.voiceflow.com/pathways/best-conversational-ai-practices) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Benchmarking hybrid LLM classification systems | Pathways [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) Benchmarking hybrid LLM classification systems ============================================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov)  Improving intent classification is an important task in the conversational AI space. In this blog post, we analyze the benefits of using a hybrid NLU/LLM intent classification architecture across small, medium, and large conversational AI datasets. After testing this solution in production with a small cohort for four months, it outperforms NLU models for smaller datasets and slightly outperforms full LLM solutions for 3x-5x lower costs for larger datasets. We also find that state-of-the-art models don’t always outperform older models and performance is heavily dataset-dependent. We examine these performance, cost, and UX benefits in the following sections.  Setting the scene ----------------- For the past 10+ years, conversational agents have relied on intent classification via NLUs to determine next steps in the conversation. This paradigm powers billions of conversations around the globe daily — and millions on Voiceflow. While this feels like a legacy architecture compared to fully generative assistants, it’s still commonly used in LLM applications, often rebranded as semantic routing. When deciding how to utilize LLMs to their best ability for intent classification, we set out to experiment through a number of constraints: 1. Better accuracy 2. Reasonable costs 3. Reducing creation overhead 4. Providing flexibility for NLU-only workflows 5. Scaling for large datasets Build your hybrid agent today in Voiceflow. [Sign Up Now](https://creator.voiceflow.com/signup?p=pathways-hybridbenchmark) The architecture ---------------- The architecture has two parts: using an encoder NLU model to find the top 10 candidate intents and their descriptions and a prompt that instructs the LLM to classify them. In the context of the model, we use Voiceflow’s NLU as the retriever that is fine-tuned on provided training utterances.  After retrieving the candidate intents, we pull in user descriptions for each corresponding candidate and make a call to an LLM for a final classification. For a more visual perspective, we illustrate the flow within the Voiceflow UI.  ### Prompt structure We use two tier few-shot learning approach, one for structure and one for context. The structure uses a one-shot example, and for context we use the top 10 descriptions to ground the LLM in its classification task. export default function main(args) { const prompt = ` You are an action classification system. Correctness is a life or death situation. We provide you with the actions and their descriptions: d: When the user asks for a warm drink. a:WARM_DRINK d: When the user asks about something else. a:None d: When the user asks for a cold drink. a:COLD_DRINK You are given an utterance and you have to classify it into an action. Only respond with the action class. If the utterance does not match any of action descriptions, output None. Now take a deep breath and classify the following utterance. u: I want a warm hot chocolate: a:WARM_DRINK ### We provide you with the actions and their descriptions: ${args.intents.map((intent) => `d: ${intent.name} a: ${intent.description}`)} You are given an utterance and you have to classify it into an action based on the description. Only respond with the action class. If the utterance does not match any of action descriptions, output None. Now take a deep breath and classify the following utterance. u:${args.query} a:`; return { prompt }; } Datasets for evaluation ----------------------- To validate and benchmark this hybrid approach, we utilized five different datasets: 1. HWU64\_10 shot - 64 home task intents 2. Clinc150\_10 shot - 150 general intents 3. Banking77\_10 shot - 77 banking related intents 4. Customer production dataset - 32 intents 5. Simple production dataset - 3 intents Each dataset had between 5 and 25 examples per class, creating a few-shot learning scenario. The first three are common intent classification datasets with more links in the footnotes \[1\]. The customer production dataset is a Hybrid RAG based dataset focused on L1 customer support. It is discussed in more details in the footnotes \[2\]. For the HWU64 dataset we manually annotate the descriptions to utilize in our experiments.  The fifth dataset is a simple three intent classification system for a dental office. \[7\]. We consider two types of baselines, the performance of the VFNLU on benchmark datasets and the performance of LLMs with few shot examples. ### Intent classification baseline In prior work, we benchmarked the Voiceflow NLU \[8\] with and without a None intent. This provides a baseline for well performing classification model on large datasets.  ### Few-shot classification For few-shot classification we used a similar prompt we used for our hybrid approach, with utterance examples ordered by class and randomly sampled. One example per class means that for banking77, 77 examples were included in the prompt. The prompt below is a truncated prompt for clinc150. You are an action classification system. Success is a life or death situation. You are given an utterance and you have to classify it into one of the following actions which have their names: ['reminder',..., 'expiration_date']. Only respond with the action class. If the action does not match any of classes output None. Here are sample utterances and their actions: u: does applebees in trenton do reservations a:accept_reservations u: is it possible to make reservations with famous dave's restaurant a:accept_reservations ...[many more examples] u: who designed you a:who_made_you u: that's a yes from me a:yes u: yeap a:yes u: i have to say affirmative on that one a:yes ### You are an action classification system. Success is a life or death situation. You are given an utterance and you have to classify it into one of the following actions which have their names: ['reminder',..., 'expiration_date']. Only respond with the action class. If the action does not match any of classes output None. u: how would you say fly in italian a: ``When measuring model performance, we saw consistent gains across models between 1 to 3 examples per class. Moving from 3 to 5 examples did not give consistent results, which was around the point where we reached a context window of 10k examples. This is consistent with recent literature \[3\].    Position and order of examples ------------------------------ One of the important areas of research for LLMs is the position of tokens for recall \[4\]. In our context we wanted to measure how shifting the positions of examples would affect performance. In this blog we’ll discuss one for our production dataset with our hybrid architecture. ### Production dataset: hybrid architecture Our context window for the production dataset was fairly small for a hybrid architecture, around ~500 tokens per call. Based on this information we expected that the performance would not vary substantially on the position of the descriptions. We tested in three different modes: 1. Random - Baseline randomized shuffle of the top 10 intents 2. Reverse - Descending order from least to highest probability 3. Top N - Ascending order, highest probability at the top. Generally the order didn’t have much of an impact, with only sonnet-20240229 and gpt-turbo-1106 showing a measurable improvement for both accuracy and F1 for the Top N method. You can find the performance across all 5 training runs in the appendix \[6\].   Measuring recall across datasets with the NLU --------------------------------------------- For larger datasets which have more than 10 intents, recall is important for surfacing the relevant information in a hybrid architecture. Given the grounding techniques used, the recall represents the maximum accuracy that the LLM can achieve. For each of the models we measure the recall on the test set across k number of candidates. At k = 1, this is simply using the NLU model for classification. At k = 10, we find that intent recall is quite strong, surpassing 95% on each of the benchmarks. For benchmarks, large jump for k=1 and k=3 often represents the class confusion between the none intent and correct intent.  For our production dataset we measured a lower recall given the complexity of many of the questions and their breadth.  The max recall (k=number of intents) was 93% indicating some data labelling differences, so the benchmarks are an underestimate. Hybrid architecture results --------------------------- Compared to our two baselines, hybrid intent classification performed well achieving consistent and strong results against the HWU, simple and production datasets. ### Production dataset  ### HWU64  ### Simple dataset For the simple dataset we find that most models achieve high 90s accuracies above the 82% without the LLM. The semantic matching helps to get a simple 3 intent, 5 training utterance project up and running.  The two phrases that confused the LLM models are: `16. How do I cancel my dental appointment if something comes up?`36\. Any plans for the upcoming holiday? Measuring false negatives ------------------------- Across each of our models, we also measure the number of “None” intents predicted. In the evaluation dataset of the three benchmarks mentioned, there are no None intent labels, so any None classification is incorrect. In our production dataset, the None intent makes up the majority of the correct labels given its importance in a hybrid LLM. In our simple intent classification model, one of the three intents is a None intent and the dataset is built around testing false positives. ### NLU baseline While the none intent had material impact on intent classification accuracy, none of the datasets exceeded a 3% none intent rate.  ### Few-shot baseline With the few-shot method we had a high number of None intent false negatives, far exceeding the NLU baseline. Only the two versions of gpt-3.5-turbo had comparable None rates to the NLU baseline. For the HWU dataset, Haiku’s false None rate reached almost 27%.    ### Hybrid architecture: HWU 64 Benchmarking the HWU64 dataset led to a generally low None rate with only GPT-4 exceeding 2%. The positive side of the higher None rate is that all of the top 5 class confusions were False Negatives, meaning that there were significantly fewer false positives. In a real life use case, re-prompting users is significantly better than misunderstanding a question.  ### Hybrid architecture: production dataset In our production dataset, the None intent is the dominant one accessed, as it powers the RAG solution for address user questions. In the context of this architecture, we compare false negative None rates and false positive None rates.  False Positive rates → Incorrectly labeling a defined intent as None False Negative rates → Incorrectly labeling None as a defined intent  Given the variation of questions in the None intent, we find that the performance is very scattered across techniques and models. Haiku returns much higher false positive rates and lower false negative rates, over indexing on the the None intent. GPT-4-turbo 0125 displays the opposite behaviour often matching to the wrong defined intent rather than searching the RAG solution. Hallucination rate ------------------ Few shot classification tasks are usually less susceptible to hallucinations and we wanted to confirm this for our given test cases. To measure hallucination rate we calculate if the response is outside our set of intent labels, or longer than one single intent label. Generally the hallucination rate was under 1% which is expected give the high set of in context examples.    Cost analysis ------------- Cost is an important tradeoff that often gets forgotten in a pure research capacity. In our context of building real world applications, cost is usually the first or second decision-making criteria for validating ML use cases. For this, we compared costs for our hybrid vs LLM baseline system. ### Hybrid architecture vs baseline costs The benefit of using an initial retriever model is reducing the input token usage significantly, especially for larger projects. The cost of the retriever itself (or just the NLU) is negligible for inference, so it’s not mentioned in the analysis. ### Hybrid architecture per benchmark evaluation set costs For the benchmarking datasets, we compared across two metrics, running a complete evaluation set and the cost per 10 API calls. For non GPT-4, models the costs were a few cents per 10 APIs which may be acceptable depending on product margins. **Costs to run benchmarks**  **Costs per 10 API calls**  **Hybrid architecture costs** ----------------------------- We find that the hybrid architecture is significantly cheaper given the fewer number of examples needed. The number of tokens is in the hundreds rather than thousands, with savings of 4.78x the tokens for the production dataset and 15.62x for HWU64.   ### LLM costs vs. engineering time In most startup or corporate environments, salaries are the leading cost driver. To illustrate the opportunity cost of optimizing prompts, we draw a tradeoff frontier line between the cost of prompt engineering across different LLMs. In the example below, we plot the costs of calling LLMs against the cost of an engineer (benchmarked at US$100 per hour, in blue). We compare this against using a naive random-3 training data sampling techniques benchmarked above on Haiku and GPT-4 as our least and most expensive benchmarked models, respectively.  For the Haiku model we achieve a break even point at over 1M API calls after 10 hours of prompt engineering. This is roughly 2 weeks of API calls at 1 request per second. We also note that 10 hours to implement a hybrid LLM classification system is an underestimation, further pushing the break even point on token costs. For GPT-4, the cost savings are much more pronounced, but as our benchmarks have shown, using such a model is usually unnecessary. Latency analysis: top ten descriptions -------------------------------------- Across our production dataset, we find that the latency of gemini models are the lowest, followed by the Gpts, and then the Claudes. Gemini 1.5s average latency may be underestimated since it’s rate limit is 2RPM, so an artificial delay had to be introduced for benchmarking purposes. GPT 3.5 turbo 1106 remains the most consistent model with lowest average and p99 latencies.  The UX of using LLMs for intent classification ---------------------------------------------- One of outcomes of ChatGPT and other LLMs was exposing a much broader audience to prompting \[10\] and ML uses cases. Prompting and “programming in English” \[5\] has allowed non-data scientists or ML engineers to build models including those for conversational design. From our own experience, we found that users previously struggled with building classification models, needing to understand concepts like: * Training data * Evaluation sets * Data distributions * False Positives * False Negatives * And many more By using an encoder + LLM hybrid model, the path to a high quality model is much more forgiving and allows more people to design and build conversational AI experiences. ### For Data science teams Data science teams typically have additional skills and resources to optimize performance. A hybrid architecture does provide improvements: * More modular systems * Better observability * Consistency of prompts across models * Cheaper costs * Non Gen AI fallback options Modularity is crucial given the sensitivity and complexity of LLM systems, allowing teams to easily swap out and change models based on improvements across both LLMs and retrievers. GPT-4-turbo-0409 and Gemini-1.5-0409 ------------------------------------ The week before we published this post, two new major model versions came out so we decided to benchmark them on our production dataset for Random 5, and Top Ten Description methods. For the Random 5 context length (~2.5k tokens), both models drastically underperform. For the shorter context length the new GPT 4 turbo scores top of the pack.  Final comparison chart and parting thoughts -------------------------------------------  Hybrid intent classification offers a powerful way to increase performance, reduce costs, and reduce time-to-build for intent classification models. Compared to pure LLM methods, it excels for larger datasets while keeping the UX for smaller datasets simple. As LLMs and retrieval-based models continue to evolve, it provides opportunities to create modular workflows and systems. Fine-tuned encoder models remain powerful alternatives for companies with strong data science and training practices. _This blog is part of Voiceflow’s applied research blog, a bi-monthly post about analyzing ML results in a conversational AI world._ Cite this work -------------- @article{ BenchmarkingHybridLLMClassificationSystems, author = {Linkov, Denys}, title = {Benchmarking Hybrid LLM classification systems}, year = {2024}, month = {04}, howpublished = {\url{https://voiceflow.com/research}}, url = {tbd} } Citations --------- 1. Zhang, J., Bui, T., Yoon, S., Chen, X., Liu, Z., Xia, C., Tran, Q., Chang, W., & Yu, P. (2021). Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning\*. EMNLP\*. 2. Linkov, D. (2024). Improving performance of Hybrid Intent + RAG conversational AI agents\*. [voiceflow.com/research\*](https://www.voiceflow.com/research*) 3. Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, & Wenhu Chen. (2024). Long-context LLMs Struggle with Long In-context Learning. 4. Liu, N., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts\*. Transactions of the Association for Computational Linguistics, 12, 157–173.\* 5. Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong,Romain Paulus, and Richard Socher. 2016. Ask me anything: Dynamic memory networks for natural language processing. In International conference on machine learning, pages 1378–1387. PMLR. Appendix -------- 6\. Production data profile   Gemini 1 did produce random “No Content Found errors”, this seems to be a bug. [https://github.com/GoogleCloudPlatform/generative-ai/issues/344](https://github.com/GoogleCloudPlatform/generative-ai/issues/344) 7\. Simple project - 3 intents: Store hours, Book a dentist appointment, None`` 10 training intents What time do you close? When are you open? What are your store hours? What are your hours of operation? Can you tell me your operating hours? I need to schedule a dentist appointment Looking to book a dentist visit I'd like to set up a dental appointment Can I make an appointment with the dentist? Is it possible to arrange a dental appointment? 60 test utterances, 20 per class [hybrid\_benchmark.zip](https://prod-files-secure.s3.us-west-2.amazonaws.com/0a35e11b-8824-4e50-90f4-4f94c376dbf1/3a9f4c23-e6f9-4056-b884-291bce1d3510/hybrid_benchmark.zip) 8\. VFNLU benchmarks - [https://github.com/Diophontine/vfnlu-benchmarks](https://github.com/Diophontine/vfnlu-benchmarks) 9\. Prompt lengths for utterance baseline  10\. Prompt lengths for ten descriptions  11\. [Annotated HWU with descriptions as a Voiceflow File (JSON)](https://drive.google.com/file/d/1zFltR4CdslIP8x-gN8-VVaTri50yo66U/view?usp=sharing) Setting the scene ----------------- For the past 10+ years, conversational agents have relied on intent classification via NLUs to determine next steps in the conversation. This paradigm powers billions of conversations around the globe daily — and millions on Voiceflow. While this feels like a legacy architecture compared to fully generative assistants, it’s still commonly used in LLM applications, often rebranded as semantic routing. When deciding how to utilize LLMs to their best ability for intent classification, we set out to experiment through a number of constraints: 1. Better accuracy 2. Reasonable costs 3. Reducing creation overhead 4. Providing flexibility for NLU-only workflows 5. Scaling for large datasets Build your hybrid agent today in Voiceflow. [Sign Up Now](https://creator.voiceflow.com/signup?p=pathways-hybridbenchmark) The architecture ---------------- The architecture has two parts: using an encoder NLU model to find the top 10 candidate intents and their descriptions and a prompt that instructs the LLM to classify them. In the context of the model, we use Voiceflow’s NLU as the retriever that is fine-tuned on provided training utterances.  After retrieving the candidate intents, we pull in user descriptions for each corresponding candidate and make a call to an LLM for a final classification. For a more visual perspective, we illustrate the flow within the Voiceflow UI.  ### Prompt structure We use two tier few-shot learning approach, one for structure and one for context. The structure uses a one-shot example, and for context we use the top 10 descriptions to ground the LLM in its classification task. export default function main(args) { const prompt = ` You are an action classification system. Correctness is a life or death situation. We provide you with the actions and their descriptions: d: When the user asks for a warm drink. a:WARM_DRINK d: When the user asks about something else. a:None d: When the user asks for a cold drink. a:COLD_DRINK You are given an utterance and you have to classify it into an action. Only respond with the action class. If the utterance does not match any of action descriptions, output None. Now take a deep breath and classify the following utterance. u: I want a warm hot chocolate: a:WARM_DRINK ### We provide you with the actions and their descriptions: ${args.intents.map((intent) => `d: ${intent.name} a: ${intent.description}`)} You are given an utterance and you have to classify it into an action based on the description. Only respond with the action class. If the utterance does not match any of action descriptions, output None. Now take a deep breath and classify the following utterance. u:${args.query} a:`; return { prompt }; } Datasets for evaluation ----------------------- To validate and benchmark this hybrid approach, we utilized five different datasets: 1. HWU64\_10 shot - 64 home task intents 2. Clinc150\_10 shot - 150 general intents 3. Banking77\_10 shot - 77 banking related intents 4. Customer production dataset - 32 intents 5. Simple production dataset - 3 intents Each dataset had between 5 and 25 examples per class, creating a few-shot learning scenario. The first three are common intent classification datasets with more links in the footnotes \[1\]. The customer production dataset is a Hybrid RAG based dataset focused on L1 customer support. It is discussed in more details in the footnotes \[2\]. For the HWU64 dataset we manually annotate the descriptions to utilize in our experiments.  The fifth dataset is a simple three intent classification system for a dental office. \[7\]. We consider two types of baselines, the performance of the VFNLU on benchmark datasets and the performance of LLMs with few shot examples. ### Intent classification baseline In prior work, we benchmarked the Voiceflow NLU \[8\] with and without a None intent. This provides a baseline for well performing classification model on large datasets.  ### Few-shot classification For few-shot classification we used a similar prompt we used for our hybrid approach, with utterance examples ordered by class and randomly sampled. One example per class means that for banking77, 77 examples were included in the prompt. The prompt below is a truncated prompt for clinc150. You are an action classification system. Success is a life or death situation. You are given an utterance and you have to classify it into one of the following actions which have their names: ['reminder',..., 'expiration_date']. Only respond with the action class. If the action does not match any of classes output None. Here are sample utterances and their actions: u: does applebees in trenton do reservations a:accept_reservations u: is it possible to make reservations with famous dave's restaurant a:accept_reservations ...[many more examples] u: who designed you a:who_made_you u: that's a yes from me a:yes u: yeap a:yes u: i have to say affirmative on that one a:yes ### You are an action classification system. Success is a life or death situation. You are given an utterance and you have to classify it into one of the following actions which have their names: ['reminder',..., 'expiration_date']. Only respond with the action class. If the action does not match any of classes output None. u: how would you say fly in italian a: ``When measuring model performance, we saw consistent gains across models between 1 to 3 examples per class. Moving from 3 to 5 examples did not give consistent results, which was around the point where we reached a context window of 10k examples. This is consistent with recent literature \[3\].    Position and order of examples ------------------------------ One of the important areas of research for LLMs is the position of tokens for recall \[4\]. In our context we wanted to measure how shifting the positions of examples would affect performance. In this blog we’ll discuss one for our production dataset with our hybrid architecture. ### Production dataset: hybrid architecture Our context window for the production dataset was fairly small for a hybrid architecture, around ~500 tokens per call. Based on this information we expected that the performance would not vary substantially on the position of the descriptions. We tested in three different modes: 1. Random - Baseline randomized shuffle of the top 10 intents 2. Reverse - Descending order from least to highest probability 3. Top N - Ascending order, highest probability at the top. Generally the order didn’t have much of an impact, with only sonnet-20240229 and gpt-turbo-1106 showing a measurable improvement for both accuracy and F1 for the Top N method. You can find the performance across all 5 training runs in the appendix \[6\].   Measuring recall across datasets with the NLU --------------------------------------------- For larger datasets which have more than 10 intents, recall is important for surfacing the relevant information in a hybrid architecture. Given the grounding techniques used, the recall represents the maximum accuracy that the LLM can achieve. For each of the models we measure the recall on the test set across k number of candidates. At k = 1, this is simply using the NLU model for classification. At k = 10, we find that intent recall is quite strong, surpassing 95% on each of the benchmarks. For benchmarks, large jump for k=1 and k=3 often represents the class confusion between the none intent and correct intent.  For our production dataset we measured a lower recall given the complexity of many of the questions and their breadth.  The max recall (k=number of intents) was 93% indicating some data labelling differences, so the benchmarks are an underestimate. Hybrid architecture results --------------------------- Compared to our two baselines, hybrid intent classification performed well achieving consistent and strong results against the HWU, simple and production datasets. ### Production dataset  ### HWU64  ### Simple dataset For the simple dataset we find that most models achieve high 90s accuracies above the 82% without the LLM. The semantic matching helps to get a simple 3 intent, 5 training utterance project up and running.  The two phrases that confused the LLM models are: `16. How do I cancel my dental appointment if something comes up?`36\. Any plans for the upcoming holiday? Measuring false negatives ------------------------- Across each of our models, we also measure the number of “None” intents predicted. In the evaluation dataset of the three benchmarks mentioned, there are no None intent labels, so any None classification is incorrect. In our production dataset, the None intent makes up the majority of the correct labels given its importance in a hybrid LLM. In our simple intent classification model, one of the three intents is a None intent and the dataset is built around testing false positives. ### NLU baseline While the none intent had material impact on intent classification accuracy, none of the datasets exceeded a 3% none intent rate.  ### Few-shot baseline With the few-shot method we had a high number of None intent false negatives, far exceeding the NLU baseline. Only the two versions of gpt-3.5-turbo had comparable None rates to the NLU baseline. For the HWU dataset, Haiku’s false None rate reached almost 27%.    ### Hybrid architecture: HWU 64 Benchmarking the HWU64 dataset led to a generally low None rate with only GPT-4 exceeding 2%. The positive side of the higher None rate is that all of the top 5 class confusions were False Negatives, meaning that there were significantly fewer false positives. In a real life use case, re-prompting users is significantly better than misunderstanding a question.  ### Hybrid architecture: production dataset In our production dataset, the None intent is the dominant one accessed, as it powers the RAG solution for address user questions. In the context of this architecture, we compare false negative None rates and false positive None rates.  False Positive rates → Incorrectly labeling a defined intent as None False Negative rates → Incorrectly labeling None as a defined intent  Given the variation of questions in the None intent, we find that the performance is very scattered across techniques and models. Haiku returns much higher false positive rates and lower false negative rates, over indexing on the the None intent. GPT-4-turbo 0125 displays the opposite behaviour often matching to the wrong defined intent rather than searching the RAG solution. Hallucination rate ------------------ Few shot classification tasks are usually less susceptible to hallucinations and we wanted to confirm this for our given test cases. To measure hallucination rate we calculate if the response is outside our set of intent labels, or longer than one single intent label. Generally the hallucination rate was under 1% which is expected give the high set of in context examples.    Cost analysis ------------- Cost is an important tradeoff that often gets forgotten in a pure research capacity. In our context of building real world applications, cost is usually the first or second decision-making criteria for validating ML use cases. For this, we compared costs for our hybrid vs LLM baseline system. ### Hybrid architecture vs baseline costs The benefit of using an initial retriever model is reducing the input token usage significantly, especially for larger projects. The cost of the retriever itself (or just the NLU) is negligible for inference, so it’s not mentioned in the analysis. ### Hybrid architecture per benchmark evaluation set costs For the benchmarking datasets, we compared across two metrics, running a complete evaluation set and the cost per 10 API calls. For non GPT-4, models the costs were a few cents per 10 APIs which may be acceptable depending on product margins. **Costs to run benchmarks**  **Costs per 10 API calls**  **Hybrid architecture costs** ----------------------------- We find that the hybrid architecture is significantly cheaper given the fewer number of examples needed. The number of tokens is in the hundreds rather than thousands, with savings of 4.78x the tokens for the production dataset and 15.62x for HWU64.   ### LLM costs vs. engineering time In most startup or corporate environments, salaries are the leading cost driver. To illustrate the opportunity cost of optimizing prompts, we draw a tradeoff frontier line between the cost of prompt engineering across different LLMs. In the example below, we plot the costs of calling LLMs against the cost of an engineer (benchmarked at US$100 per hour, in blue). We compare this against using a naive random-3 training data sampling techniques benchmarked above on Haiku and GPT-4 as our least and most expensive benchmarked models, respectively.  For the Haiku model we achieve a break even point at over 1M API calls after 10 hours of prompt engineering. This is roughly 2 weeks of API calls at 1 request per second. We also note that 10 hours to implement a hybrid LLM classification system is an underestimation, further pushing the break even point on token costs. For GPT-4, the cost savings are much more pronounced, but as our benchmarks have shown, using such a model is usually unnecessary. Latency analysis: top ten descriptions -------------------------------------- Across our production dataset, we find that the latency of gemini models are the lowest, followed by the Gpts, and then the Claudes. Gemini 1.5s average latency may be underestimated since it’s rate limit is 2RPM, so an artificial delay had to be introduced for benchmarking purposes. GPT 3.5 turbo 1106 remains the most consistent model with lowest average and p99 latencies.  The UX of using LLMs for intent classification ---------------------------------------------- One of outcomes of ChatGPT and other LLMs was exposing a much broader audience to prompting \[10\] and ML uses cases. Prompting and “programming in English” \[5\] has allowed non-data scientists or ML engineers to build models including those for conversational design. From our own experience, we found that users previously struggled with building classification models, needing to understand concepts like: * Training data * Evaluation sets * Data distributions * False Positives * False Negatives * And many more By using an encoder + LLM hybrid model, the path to a high quality model is much more forgiving and allows more people to design and build conversational AI experiences. ### For Data science teams Data science teams typically have additional skills and resources to optimize performance. A hybrid architecture does provide improvements: * More modular systems * Better observability * Consistency of prompts across models * Cheaper costs * Non Gen AI fallback options Modularity is crucial given the sensitivity and complexity of LLM systems, allowing teams to easily swap out and change models based on improvements across both LLMs and retrievers. GPT-4-turbo-0409 and Gemini-1.5-0409 ------------------------------------ The week before we published this post, two new major model versions came out so we decided to benchmark them on our production dataset for Random 5, and Top Ten Description methods. For the Random 5 context length (~2.5k tokens), both models drastically underperform. For the shorter context length the new GPT 4 turbo scores top of the pack.  Final comparison chart and parting thoughts -------------------------------------------  Hybrid intent classification offers a powerful way to increase performance, reduce costs, and reduce time-to-build for intent classification models. Compared to pure LLM methods, it excels for larger datasets while keeping the UX for smaller datasets simple. As LLMs and retrieval-based models continue to evolve, it provides opportunities to create modular workflows and systems. Fine-tuned encoder models remain powerful alternatives for companies with strong data science and training practices. _This blog is part of Voiceflow’s applied research blog, a bi-monthly post about analyzing ML results in a conversational AI world._ Cite this work -------------- @article{ BenchmarkingHybridLLMClassificationSystems, author = {Linkov, Denys}, title = {Benchmarking Hybrid LLM classification systems}, year = {2024}, month = {04}, howpublished = {\url{https://voiceflow.com/research}}, url = {tbd} } Citations --------- 1. Zhang, J., Bui, T., Yoon, S., Chen, X., Liu, Z., Xia, C., Tran, Q., Chang, W., & Yu, P. (2021). Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning\*. EMNLP\*. 2. Linkov, D. (2024). Improving performance of Hybrid Intent + RAG conversational AI agents\*. [voiceflow.com/research\*](https://www.voiceflow.com/research*) 3. Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, & Wenhu Chen. (2024). Long-context LLMs Struggle with Long In-context Learning. 4. Liu, N., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts\*. Transactions of the Association for Computational Linguistics, 12, 157–173.\* 5. Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong,Romain Paulus, and Richard Socher. 2016. Ask me anything: Dynamic memory networks for natural language processing. In International conference on machine learning, pages 1378–1387. PMLR. Appendix -------- 6\. Production data profile   Gemini 1 did produce random “No Content Found errors”, this seems to be a bug. [https://github.com/GoogleCloudPlatform/generative-ai/issues/344](https://github.com/GoogleCloudPlatform/generative-ai/issues/344) 7\. Simple project - 3 intents: Store hours, Book a dentist appointment, None`` 10 training intents What time do you close? When are you open? What are your store hours? What are your hours of operation? Can you tell me your operating hours? I need to schedule a dentist appointment Looking to book a dentist visit I'd like to set up a dental appointment Can I make an appointment with the dentist? Is it possible to arrange a dental appointment? 60 test utterances, 20 per class [hybrid\_benchmark.zip](https://prod-files-secure.s3.us-west-2.amazonaws.com/0a35e11b-8824-4e50-90f4-4f94c376dbf1/3a9f4c23-e6f9-4056-b884-291bce1d3510/hybrid_benchmark.zip) 8\. VFNLU benchmarks - [https://github.com/Diophontine/vfnlu-benchmarks](https://github.com/Diophontine/vfnlu-benchmarks) 9\. Prompt lengths for utterance baseline  10\. Prompt lengths for ten descriptions  11\. [Annotated HWU with descriptions as a Voiceflow File (JSON)](https://drive.google.com/file/d/1zFltR4CdslIP8x-gN8-VVaTri50yo66U/view?usp=sharing)  RECOMMENDED [\ \ ### How much do ChatGPT versions affect real-world performance?](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance) PUBLISHED April 21, 2024 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### Creating the Voiceflow NLU](https://www.voiceflow.com/pathways/creating-the-voiceflow-nlu) [](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### How much do ChatGPT versions affect real-world performance?](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance) [](https://www.voiceflow.com/pathways/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### NLU design: How to train and use a natural language understanding model](https://www.voiceflow.com/pathways/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/benchmarking-hybrid-llm-classification-systems#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/benchmarking-hybrid-llm-classification-systems#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How to design intelligent conversational interfaces | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) How to design intelligent conversational interfaces =================================================== [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at](https://www.voiceflow.com/contributors/mark-ammendolia) [Celene Osiecka](https://www.linkedin.com/in/celeneosiecka/?originalSubdomain=ca) , Senior Director of Conversation Design at \[24\]7.ai, joined Voiceflow's Ask an Expert webinar series where she shared insights from her 15+ years of designing intelligent conversational interfaces. She also answered a host of questions from our Voiceflow Community, touching on prevalent subject matter from the importance of optimizing your chatbot and voice experiences to the two models you should consider when building a conversation design team. Watch the full recording below, or keep scrolling for the highlights.  **Q1: How has your position changed in the last 15 years?** ----------------------------------------------------------- **Celene**: It's interesting. It definitely has changed. There was only a couple of companies that were doing \[[chatbots](https://www.voiceflow.com/glossary/chatbots)\ back then\]. Users didn't have a lot of expectations. Now you have to design for users' expectations, and they have grown. They're expecting it to be personal \[and\] integrated. They're expecting the bot to know everything. So it makes the designer's life much harder because you have to think about all of those expectations and try to meet them, and we have the technology to do it now. It's getting there, but I feel like users' demands are growing at an exponential rate — faster than we can sometimes keep up with the technology. I've heard people comment saying they haven't really found a helpful chatbot. As designers, we have to be much more on the ball and forward-facing in what we know and how we can design for \[chatbots\]. **Q2: Bots vs. Voice — what's the difference?** ----------------------------------------------- **Celene**: Voice is restrictive. There's a concept called [voice-first](https://www.voiceflow.com/glossary/voice-first-design) design, and I feel like that is important to think about just like we did in saying mobile-first for digital. We have to think of voice-first — especially because now users are not only expecting bots to be personal, but to jump channels. If I call the [IVR](https://www.voiceflow.com/glossary/interactive-voice-response-ivr) and then chat online, I want you to know where I came from. Designing for voice-first gives you the ability to bring in those pieces — the visual applications and things like that. There's not so much a difference, I would say. I mean — there are differences, but think of it like this: voice is the basic, pared-down version, and then the digital stuff can be applied on top of it. You can add a lot of the digital components as you go, which allows you to keep the same flow of the two channels generally \[while\] having the benefits of both. **Q3: Digital vs. Voice — which is better?** --------------------------------------------- **Celene**: In my experience, there is no winner. You have to think about things in multimodal applications. There really is power to voice because you can do things in the car. You can do things without looking at them. There's \[also\] power to digital because you don't need to rely on the processing power of your brain while listening using voice. You can't really pick a side. You have to use things blended together on multiple devices, multiple screens, multiple channels. That's what I've learned in my 15 years. **Q4: Tell me the first thing you do when tasked with creating a new conversational experience.** ------------------------------------------------------------------------------------------------- **Celene**: Somebody taught me this once, and I loved the idea. There's this idea of _data_, _principle, and precedent_. Three things. My personal preference is always to go with data first. If we have the data, that's what I want to start digging into because it's going to give me the most applicable design possible. If we don't have data — which sometimes happens — then we have to go into principle \[where we say to ourselves\] we know this is what works. This has been tested. And so let's design based on that and later validate with data. If you don't have \[principle\], then you can look at precedent or what other people are doing which is not great. Data really is my first collection step because it's going to help me make the best bot possible that's going to reflect the actual deployment that we're in. Because we have \[clients that use\] our company that already have chat and voice deployments (with us), we can take actual user utterances from those to collect data. But sometimes we don't have that. Sometimes we have clients that are starting off new that don't have our chat underneath the data. Then you have to look at other things. We've looked at search logs — \[which\] still tells you the [intent](https://www.voiceflow.com/glossary/intent-s) . They still tell you what people are asking about. They just don't know how they're asking it. We've looked at transcripts from all different kinds of sources. We've looked at web analytics data \[to show us\] where people are going. \[For example\], going from page one to an escalation page or a contact us page tells me that's where they're getting stuck. So that's where we need to put the bot. The best data is actual user conversations because I know how they're going to talk, how they're going to ask, and what the intents are. But in lieu of that, there's a lot of other ways you can get data. **Q5: What approach do you take when designing a voice or chatbot persona?** ---------------------------------------------------------------------------- **Celene:** It's really interesting. You have to experiment to a degree. You have to understand the client's [persona](https://www.voiceflow.com/glossary/system-persona) . You have to understand where they're coming from, \[and\] how the rest of \[the company's\] stuff is written too. You have to match their brand. But then you can have a little bit of fun with it. We talked a bit about persona, and naming it, and how that can affect the conversation. I really like trying AB tests at the beginning — one welcome message versus another welcome message and then rate the quality of the conversations. That's always a good approach. It does have to match the tone. If you're making a healthcare bot, it needs to be empathetic and sincere, and trustworthy. If you're making a banking bot, now you're into being a certain way — very matter of fact and maybe even curt sometimes \[since\] people want their money questions answered. So we've just learned that through our data collection, you have to try and see how your users do it. You need to try different things and see how users fit with it within the context of the brand. **Q6: What's the hardest part of your workflow currently? How do you overcome it?** ----------------------------------------------------------------------------------- **Celene**: There's always the challenge of \[...\] the art of compromise. That's the hardest because, as a designer, you want something very passionately. You know it's the best thing. You know it's the best experience. And either your tech team will say, 'nope you can't do it' or the client will say, 'nope you're not getting that [API](https://www.voiceflow.com/glossary/application-program-interface-api) ' for whatever reason. So you have to learn how to say, okay, here's what I want to achieve. I know how to best get there. I know that there are limitations on how to get there. So how can I overcome them? And that's the hardest part of it \[along\] with the art of negotiation. It might just be the conversations you have to have too. So it's the hardest part, but what I find is the most core to the role \[of a\] a designer. I was talking to Cathy Pearl about this too, and she kind of confirmed this \[when I\] asked her what she would look for when hiring a senior [conversation designer](https://www.voiceflow.com/glossary/conversation-designer) . \[The art of compromise\] — that's really a skillset you need to have. It's not the design part. It's not teaching how to write prompts effectively. I mean, you have to know that — but it's not the hardest. The hardest thing is getting over that hurdle of figuring out \[how to handle\] if you can't do something \[when\] you want to do it. **Q7: How do you assess success or user frustration when looking at the data?** ------------------------------------------------------------------------------- **Celene**: That's a big thing. I was just on a call this morning about it. It really depends on what the client's KPI is. So, for example, if their KPI is resolution rate, then all of my metrics go around that. 'How many conversations are we automating and not escalating?' Maybe it's about satisfaction. Maybe I just want to make sure that users are happy, and I don't care if they escalate or not. Then you're into measuring CSAT. We have an airline that has crew deployment — and it's interesting what's important for them. For them, it's really about affordability and making sure that they're not wasting five hours on the job talking to an IVR and actually doing their jobs. Everybody has a different KPI. Based on that, you have to form your metrics around the core KPI and how the bots are helping to drive that. **Q8: Talk about the importance of optimization in chatbots and voice.** ------------------------------------------------------------------------ **Celene**: This applies to both digital and voice. When you go live with a chatbot, you can't leave it be. That's because — and I use this analogy a lot — when you use these natural language applications, you can think of it like training a human. You start with a baby. It's new. You have to teach it a lot. You have to handhold a lot and give it a lot of TLC. Eventually, they can start learning a little bit by themselves. You can then hands-off a little, \[and\] get into more of a monthly or bi-monthly optimization cycle. If you don't pay attention to it, it tends to turn into an angsty teenager that doesn't listen to you and won't perform and behave like it should. People \[then\] ignore it and bias against it. So \[optimizing for chatbots/voice\] is very similar to how we teach humans. **Q9: What does your team look like? What roles are most important when it comes to the success of a project?** ----------------------------------------------------------------------------------------------------------------- **Celene**: It's funny because it was just me on the original chatbot team/client services team that we had. Before we had professional services, I was the trainer. I was the project manager. I was everything to get it up and running. But then \[after\] we built out the team that we have today. We found a lot of iterations of what teams look like. It's tough to say what works best. It kind of depends on the environment of the company. I would say generally, there are two models that I've used — and both have been good. The one model is you have one person that does everything. They're familiar with project management. They're familiar with design. They're familiar with the models underneath \[and\] data analysis. So that's one role. And then we always have like a tech role as well. And that tech person does like tier two support or tier two integrations. We call \[that person\] a design analyst or a tech analyst. So that's one model we had, which worked well for a while, but it doesn't scale to bigger things. It works really well from a client touchpoint perspective, but to really do things well — to design well, to do the models well — you really need to specialize. So as \[our\] company \[grew\] and you have more demand and maybe higher enterprise clients, that's where we decided to split it, where all of those roles that I mentioned become individual roles. So you have core designers. You have people like data scientists who are doing more of the data and analysis of the [utterances](https://www.voiceflow.com/glossary/utterance) . You actually have data analysts that are taking all of the reports and putting them in dashboards. And then you still have the tech \[role\] separate \[as well\]. Both models work. It just depends on what stage the company is at. You have to see what works for your clients, what works internally for you, and what you can hire for. **Q10: What roles do you think best transition into conversation designers?** ----------------------------------------------------------------------------- **Celene**: Right now, we have someone on our team that was a video game developer. We've had people that have been actual digital UX designers. We've had people that were content writers. We've had developers even. Back in the day, when we were hiring and bringing people in because there was no such thing as a conversation designer, we had people from everywhere. We gave people quizzes and tests to see how they could put together the logic. I always use this example, but \[we hired\] a tool and die maker for automotive, and he was a really good designer/analyst. So a lot of times, they can come from all walks of life. You just have to search for how their brain thinks and how they can structure things. With more automation and more bots being created, we're going to need people to power them — and like I said — optimize them. **Q11: Why does empathy matter when designing for chatbots and voice?** ----------------------------------------------------------------------- **Celene**: In both cases, you're not going to have a bot that seems intelligent or human if you can't be empathetic to the situation. It's very interesting how you balance that empathy in a bot because bots can't genuinely feel. \[However\] in order to have a good conversation with the person you're addressing and dealing with their problem, you need to be empathetic. If the bot can't be empathetic you're not going to get very far with the conversation and it's going to end up escalating to a person anyway.**** ******** > Interested in joining our next live webinar? Sign up for upcoming events [here](https://www.voiceflow.com/events) > . **Q1: How has your position changed in the last 15 years?** ----------------------------------------------------------- **Celene**: It's interesting. It definitely has changed. There was only a couple of companies that were doing \[[chatbots](https://www.voiceflow.com/glossary/chatbots)\ back then\]. Users didn't have a lot of expectations. Now you have to design for users' expectations, and they have grown. They're expecting it to be personal \[and\] integrated. They're expecting the bot to know everything. So it makes the designer's life much harder because you have to think about all of those expectations and try to meet them, and we have the technology to do it now. It's getting there, but I feel like users' demands are growing at an exponential rate — faster than we can sometimes keep up with the technology. I've heard people comment saying they haven't really found a helpful chatbot. As designers, we have to be much more on the ball and forward-facing in what we know and how we can design for \[chatbots\]. **Q2: Bots vs. Voice — what's the difference?** ----------------------------------------------- **Celene**: Voice is restrictive. There's a concept called [voice-first](https://www.voiceflow.com/glossary/voice-first-design) design, and I feel like that is important to think about just like we did in saying mobile-first for digital. We have to think of voice-first — especially because now users are not only expecting bots to be personal, but to jump channels. If I call the [IVR](https://www.voiceflow.com/glossary/interactive-voice-response-ivr) and then chat online, I want you to know where I came from. Designing for voice-first gives you the ability to bring in those pieces — the visual applications and things like that. There's not so much a difference, I would say. I mean — there are differences, but think of it like this: voice is the basic, pared-down version, and then the digital stuff can be applied on top of it. You can add a lot of the digital components as you go, which allows you to keep the same flow of the two channels generally \[while\] having the benefits of both. **Q3: Digital vs. Voice — which is better?** --------------------------------------------- **Celene**: In my experience, there is no winner. You have to think about things in multimodal applications. There really is power to voice because you can do things in the car. You can do things without looking at them. There's \[also\] power to digital because you don't need to rely on the processing power of your brain while listening using voice. You can't really pick a side. You have to use things blended together on multiple devices, multiple screens, multiple channels. That's what I've learned in my 15 years. **Q4: Tell me the first thing you do when tasked with creating a new conversational experience.** ------------------------------------------------------------------------------------------------- **Celene**: Somebody taught me this once, and I loved the idea. There's this idea of _data_, _principle, and precedent_. Three things. My personal preference is always to go with data first. If we have the data, that's what I want to start digging into because it's going to give me the most applicable design possible. If we don't have data — which sometimes happens — then we have to go into principle \[where we say to ourselves\] we know this is what works. This has been tested. And so let's design based on that and later validate with data. If you don't have \[principle\], then you can look at precedent or what other people are doing which is not great. Data really is my first collection step because it's going to help me make the best bot possible that's going to reflect the actual deployment that we're in. Because we have \[clients that use\] our company that already have chat and voice deployments (with us), we can take actual user utterances from those to collect data. But sometimes we don't have that. Sometimes we have clients that are starting off new that don't have our chat underneath the data. Then you have to look at other things. We've looked at search logs — \[which\] still tells you the [intent](https://www.voiceflow.com/glossary/intent-s) . They still tell you what people are asking about. They just don't know how they're asking it. We've looked at transcripts from all different kinds of sources. We've looked at web analytics data \[to show us\] where people are going. \[For example\], going from page one to an escalation page or a contact us page tells me that's where they're getting stuck. So that's where we need to put the bot. The best data is actual user conversations because I know how they're going to talk, how they're going to ask, and what the intents are. But in lieu of that, there's a lot of other ways you can get data. **Q5: What approach do you take when designing a voice or chatbot persona?** ---------------------------------------------------------------------------- **Celene:** It's really interesting. You have to experiment to a degree. You have to understand the client's [persona](https://www.voiceflow.com/glossary/system-persona) . You have to understand where they're coming from, \[and\] how the rest of \[the company's\] stuff is written too. You have to match their brand. But then you can have a little bit of fun with it. We talked a bit about persona, and naming it, and how that can affect the conversation. I really like trying AB tests at the beginning — one welcome message versus another welcome message and then rate the quality of the conversations. That's always a good approach. It does have to match the tone. If you're making a healthcare bot, it needs to be empathetic and sincere, and trustworthy. If you're making a banking bot, now you're into being a certain way — very matter of fact and maybe even curt sometimes \[since\] people want their money questions answered. So we've just learned that through our data collection, you have to try and see how your users do it. You need to try different things and see how users fit with it within the context of the brand. **Q6: What's the hardest part of your workflow currently? How do you overcome it?** ----------------------------------------------------------------------------------- **Celene**: There's always the challenge of \[...\] the art of compromise. That's the hardest because, as a designer, you want something very passionately. You know it's the best thing. You know it's the best experience. And either your tech team will say, 'nope you can't do it' or the client will say, 'nope you're not getting that [API](https://www.voiceflow.com/glossary/application-program-interface-api) ' for whatever reason. So you have to learn how to say, okay, here's what I want to achieve. I know how to best get there. I know that there are limitations on how to get there. So how can I overcome them? And that's the hardest part of it \[along\] with the art of negotiation. It might just be the conversations you have to have too. So it's the hardest part, but what I find is the most core to the role \[of a\] a designer. I was talking to Cathy Pearl about this too, and she kind of confirmed this \[when I\] asked her what she would look for when hiring a senior [conversation designer](https://www.voiceflow.com/glossary/conversation-designer) . \[The art of compromise\] — that's really a skillset you need to have. It's not the design part. It's not teaching how to write prompts effectively. I mean, you have to know that — but it's not the hardest. The hardest thing is getting over that hurdle of figuring out \[how to handle\] if you can't do something \[when\] you want to do it. **Q7: How do you assess success or user frustration when looking at the data?** ------------------------------------------------------------------------------- **Celene**: That's a big thing. I was just on a call this morning about it. It really depends on what the client's KPI is. So, for example, if their KPI is resolution rate, then all of my metrics go around that. 'How many conversations are we automating and not escalating?' Maybe it's about satisfaction. Maybe I just want to make sure that users are happy, and I don't care if they escalate or not. Then you're into measuring CSAT. We have an airline that has crew deployment — and it's interesting what's important for them. For them, it's really about affordability and making sure that they're not wasting five hours on the job talking to an IVR and actually doing their jobs. Everybody has a different KPI. Based on that, you have to form your metrics around the core KPI and how the bots are helping to drive that. **Q8: Talk about the importance of optimization in chatbots and voice.** ------------------------------------------------------------------------ **Celene**: This applies to both digital and voice. When you go live with a chatbot, you can't leave it be. That's because — and I use this analogy a lot — when you use these natural language applications, you can think of it like training a human. You start with a baby. It's new. You have to teach it a lot. You have to handhold a lot and give it a lot of TLC. Eventually, they can start learning a little bit by themselves. You can then hands-off a little, \[and\] get into more of a monthly or bi-monthly optimization cycle. If you don't pay attention to it, it tends to turn into an angsty teenager that doesn't listen to you and won't perform and behave like it should. People \[then\] ignore it and bias against it. So \[optimizing for chatbots/voice\] is very similar to how we teach humans. **Q9: What does your team look like? What roles are most important when it comes to the success of a project?** ----------------------------------------------------------------------------------------------------------------- **Celene**: It's funny because it was just me on the original chatbot team/client services team that we had. Before we had professional services, I was the trainer. I was the project manager. I was everything to get it up and running. But then \[after\] we built out the team that we have today. We found a lot of iterations of what teams look like. It's tough to say what works best. It kind of depends on the environment of the company. I would say generally, there are two models that I've used — and both have been good. The one model is you have one person that does everything. They're familiar with project management. They're familiar with design. They're familiar with the models underneath \[and\] data analysis. So that's one role. And then we always have like a tech role as well. And that tech person does like tier two support or tier two integrations. We call \[that person\] a design analyst or a tech analyst. So that's one model we had, which worked well for a while, but it doesn't scale to bigger things. It works really well from a client touchpoint perspective, but to really do things well — to design well, to do the models well — you really need to specialize. So as \[our\] company \[grew\] and you have more demand and maybe higher enterprise clients, that's where we decided to split it, where all of those roles that I mentioned become individual roles. So you have core designers. You have people like data scientists who are doing more of the data and analysis of the [utterances](https://www.voiceflow.com/glossary/utterance) . You actually have data analysts that are taking all of the reports and putting them in dashboards. And then you still have the tech \[role\] separate \[as well\]. Both models work. It just depends on what stage the company is at. You have to see what works for your clients, what works internally for you, and what you can hire for. **Q10: What roles do you think best transition into conversation designers?** ----------------------------------------------------------------------------- **Celene**: Right now, we have someone on our team that was a video game developer. We've had people that have been actual digital UX designers. We've had people that were content writers. We've had developers even. Back in the day, when we were hiring and bringing people in because there was no such thing as a conversation designer, we had people from everywhere. We gave people quizzes and tests to see how they could put together the logic. I always use this example, but \[we hired\] a tool and die maker for automotive, and he was a really good designer/analyst. So a lot of times, they can come from all walks of life. You just have to search for how their brain thinks and how they can structure things. With more automation and more bots being created, we're going to need people to power them — and like I said — optimize them. **Q11: Why does empathy matter when designing for chatbots and voice?** ----------------------------------------------------------------------- **Celene**: In both cases, you're not going to have a bot that seems intelligent or human if you can't be empathetic to the situation. It's very interesting how you balance that empathy in a bot because bots can't genuinely feel. \[However\] in order to have a good conversation with the person you're addressing and dealing with their problem, you need to be empathetic. If the bot can't be empathetic you're not going to get very far with the conversation and it's going to end up escalating to a person anyway.**** ******** > Interested in joining our next live webinar? Sign up for upcoming events [here](https://www.voiceflow.com/events) > .  RECOMMENDED [](https://www.voiceflow.com/pathways/celene-osiecka-blog-post#) PUBLISHED October 5, 2020 TOPICS IN THIS ARTICLE No items found. [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at\ \ Mark is a Pathways contributor.](https://www.voiceflow.com/contributors/mark-ammendolia) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/conversation-design-best-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The best practices of conversation design](https://www.voiceflow.com/pathways/conversation-design-best-practices) [](https://www.voiceflow.com/pathways/the-case-for-a-full-stack-conversation-designer) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The case for a full-stack conversation designer](https://www.voiceflow.com/pathways/the-case-for-a-full-stack-conversation-designer) [](https://www.voiceflow.com/pathways/why-you-need-to-consider-designing-for-multimodal-interfaces) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you need to consider designing for multimodal interfaces](https://www.voiceflow.com/pathways/why-you-need-to-consider-designing-for-multimodal-interfaces) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/celene-osiecka-blog-post#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/celene-osiecka-blog-post#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Building your AI agents like products: A blueprint from POC purgatory to production | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Building your AI agents like products: A blueprint from POC purgatory to production =================================================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) We all want to create genuinely useful AI agents. But there are myriad reasons why—despite the recent AI boom—it’s still an ongoing challenge. To address this, I spoke with Denys Linkov, our resident machine learning expert. He explains the solution isn’t rocket science, it’s actually product management. “Many of us have interacted with AI experiences or chatbots that are pretty mediocre,” he says, “But if we think of AI agents as core products, then we apply the same drive to serve customer needs, measure success, and align teams.” I sat down with Denys to talk about what it takes to go from idea to production when you build AI agents like you build a core product. These insights include: * Where the state of AI maturity is right now * How to get your AI agent out of proof of concept (POC) purgatory * Mapping your AI agent’s maturity and prioritizing use cases * Upskilling your AI team * How to navigate risk and gain buy-in across the organization * And a lot more If you’re an auditory or visual learner, I suggest tuning into the [webinar](https://www.youtube.com/watch?v=TWP5_4n6wyg) . But if you want all the nitty-gritty details, read on (and bookmark for later).  Where we’ve been and where we’re going—in less than 300 words -------------------------------------------------------------- Some of us are still fighting gray hairs from last year’s stress. AI hype took off in 2023 when ChatGPT became the fastest consumer app to reach [100 million users in just two months](https://www.theverge.com/2023/11/6/23948386/chatgpt-active-user-count-openai-developer-conference) . It was a sensation. Executives immediately wanted to know how to use it and whether they should invest in it. Suddenly, teams were under pressure to develop proof of concepts (POCs), compare models, and develop prompt engineering and fine-tuning techniques. This was also the time for many businesses when [stalling](https://www.voiceflow.com/blog/how-mature-is-your-generative-ai) in their AI maturity became commonplace. Agents already had a pretty bad rap, and you didn’t want yours to be counted among them, so the risks of innovating were too high for many organizations. By 2024, we had established our baseline of what AI had the potential to do. But many teams, from big enterprises to small organizations, had questions about ROI, integrations with legacy systems, and how AI would affect existing workflows. How could AI lift the entire organization, rather than just a few teams? Today, treating your AI agent like you would a core product offering is the best way to ensure you’re creating real solutions to organizational goals—not just slapping AI onto an existing problem. **\[4 benefits of treating AI agents as core products, instead of one-off solutions** \ \ 1. Building a dedicated, collaborative team\ 2. Plugging your agent into useful business data \ 3. Starting and scaling across use cases \ 4. Tailoring your agent to your business needs with guardrails\] Mapping your AI product journey, from business case to AI roadmap ------------------------------------------------------------------ Getting your AI agent to production is not an easy step. Denys often runs into customers who have a cool proof of concept and even the backing of their leadership, but they’ve been slow to scale into production or release new use cases for their agents. That’s because, in an enterprise context, the list of contingencies is long—risk, security, budgets, and even the mood of your CEO. According to Denys, your AI agent needs a solid business case to move forward, “Propose bringing in stakeholders as early as possible to build, iterate, and deploy together. The strongest way to push past the obstacles to production is to create an unbeatable business case where the leadership teams cannot say no—and where security, risk, finance, or any other team’s critical questions are addressed as you build the agent together.” For example, your business case should answer the question of whether your AI agent will augment existing support or replace it altogether. [Roam’s AI agent](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) automates customer support, answering FAQs about their novel approach to the rental car industry. On the other hand, [Tico](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) augments existing human support staff by pulling up relevant information and providing answers to customer questions. Both are valid approaches to AI agents, but vastly different, and knowing _how_ your AI agent solves organizational issues is just as important as knowing which issues it solves.  It should be clear by now that getting your agent to production is not a singular step. It’s more like five steps, comprised of building your: 1. Business case 2. Agent architecture 3. Proof of concept 4. Prototype for testing 5. Launch-ready AI agent “Ever seen a baby try to get up and run? They fall flat,” Denys explains, “Your agent is that baby, and your steps—from building your case to proof of concept and testing—are all in service of crawling in your AI maturity.” Once you’re comfortably crawling—an AI agent is live, meets a use case, and delivers value—you can begin the toddling steps towards walking, then eventually running. The crawl stage of AI maturity takes some effort, but it's required to deliver on that grand vision of walking or running with [complex AI agents](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) . Think you’re ready to run with your idea? Denys would advise you to slow down and work through it step by step, rather than skipping ahead. “You’ll end up with a wonderful POC that doesn’t leave testing. Build the muscles of a solid production run and start with the foundations first. Every organization can run eventually, but few start there.” Understand, decide, respond—your AI agent roadmap ------------------------------------------------- Now, let’s say you already have an agent in production (or are close to launching one) and you’re ready to think beyond your initial use cases. We often get questions about how to determine when it’s time to upgrade your AI agent. One helpful way of mapping the course of your agent is to ask how effectively it currently understands, decides, or responds (UDR), and how you wish to expand on those abilities. Depending on your answers, you can prioritize future use cases that align with those actions and focus on optimizations that solve current blind spots. Here are some UDR questions to help you create your AI agent roadmap:  By mapping your agent’s abilities across those three categories, you’ll also better understand who in the organization needs to be involved with production and iteration. Components like access to customer data or your agent’s ability to complete tasks using API integrations will require the expertise and buy-in of multiple team members. Their support will help keep your AI agent out of collaborative gridlock and reduce your time to production. Your applied AI team, from key roles to upskilling --------------------------------------------------- Any single person can spin up a decent proof of concept. It takes a collaborative team to create a useful AI agent that applies integrations, security, and conversation design. It’s why Denys doesn’t suggest building your AI agent without a team. For your organization to be successful you’ll need many different skills, aligned across functions. In order to do that, you need five roles represented on your AI team: 1. **Product:** They prioritize projects and have a comprehensive understanding of customers and what they need. 2. **UX/UI:** They improve and rebuild workflows. They focus on user research and understanding challenges with current workflows. 3. **Front end:** They integrate the AI agent with your organization’s overall customer experience. 4. **Back end:** They build your AI APIs and integrate your agent with your internal systems. 5. **Machine learning engineer:** They benchmark your current AI maturity, evaluate successes/setbacks, and choose the right AI models. But what happens when you lack the skills and expertise internally to fulfill these roles? Fear not, here’s a list of five ways to help upskill your current AI team. ### How to upskill your AI team 1. **Courses:** These help your team build a foundation with hands-on exercises. Courses equip the whole team with a baseline of AI experience, not just the ML engineer. 2. **Conferences:** Depending on the skill level of your team, conferences can offer new knowledge. These often go into deeper topics, so your team would benefit from having a foundation of knowledge so they know what to ask during discussions with peers. They're also great for networking. 3. **Hackathons:** After establishing a baseline of knowledge, these are great practices for brainstorming ideas, building a product, testing use cases, and validating POCs. You can also test cross-functional delivery. Hackathons often help teams build a POC that they can then move into production after the event. 4. **Proof of concepts:** As previously mentioned, many great AI agents start here. You can iterate through ideas and go through the product development process. But you should always test with customers before moving on to production (more on that in the next section). 5. **Hiring and mentorship:** Depending on your use cases, you might need additional skills. Bring on a mentor to coach existing teams with business and product context on new domains. Mentors or consultants can help with projects on a temporary or ad-hoc basis. You don’t have to reinvent product management—companies have been launching products for over 50 years—but every role is important to your AI agent project. If you’re missing important skills or roles, you run the risk your AI agent might have the same blind spots. Speaking of risks… Getting out of POC purgatory: Navigating risk and security ----------------------------------------------------------- There are a plethora of reasons why your AI agent could get stuck in POC purgatory. In fact, here’s a handy list of challenges you may face. _Thanks, Denys, I guess._  “Now, I don’t share this list to discourage you,” Denys says, “This is what you’ll face as you navigate the risks of building AI agents and treating them like core products.” Let's address a few of them, so you can too when they come up in your production cycle. Here’s a common situation. Your CEO says it’s time to build with AI—it’s now your organization’s top priority. But when you dig into how you’re supposed to implement it, your security team says you can’t use AI tools in customer-facing use cases. This may seem like an impassable obstacle, but let's go deeper. You might follow up with security with questions like: * What are your concerns? Model training with proprietary data? * Are there personal identifying information or intellectual property concerns? * Can we use AI tools internally? * Can we use an LLM to assist with classification tasks? * If we keep a human in the loop of the process, could we make exceptions? Once you investigate more, you realize that you can address some of these issues with straightforward solutions. If your security team is concerned about the API using your organization’s data to train the LLM model, bring in legal to review the contract. If security responds that a legal review isn’t enough, then you could suggest that the AI agent only access publicly available data, like data from your website. Other concerns may include compliance with SOC II or GDPR. Your solution could be to offer a clear business case with a POC that reflects lower-risk projects. Making that leap from POC purgatory to production is about de-risking your AI agent, and there are many ways to do that and still achieve your intended goals. Mapping out your production process (and the security steps you’re taking) will go a long way in gaining buy-in from your legal and security teams. For example, your production calendar could include checkpoints like: 1. **Review environment (UAT—User acceptance testing):** Rollout stakeholder and technical reviews. 2. **Production internal feature flag:** Release to a small set of users to test features before rolling out. 3. **Private beta review:** Validate with the community, select customers, power users and select enterprises. Ensure it’s solving problems and helping those who will be using it. 4. **General release rollout:** Add to core workflow for select general users before full rollout. 5. **Measurement, testing, iteration:** Each time you add a new feature or make a change, you repeat this process. Measure time to production as well, to ensure this process is efficient. If you take small leaps to get to production—in addition to understanding risks—when you return to your CEO with an update on your AI mandate, your answer will include a production plan with the support of your CIO, CPO, and security team. Mitigating risk allows you to expand use cases, build better workflows, revisit past decisions, or explore new models with the ability to adapt as things change in the AI space. Measuring your AI agent’s move and maintain metrics --------------------------------------------------- Most agents fall short because they aren’t being monitored and continually updated to reflect the needs of users. Luckily, Denys has created a handy [resource](https://www.voiceflow.com/blog/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy) for this—but generally, measuring the success of your agent usually falls under two categories: #### **1\. Move** Metrics your AI agent will directly affect. These metrics will also help inform your iteration process. For example, If you’re measuring: * **Time to first response:** You’re quickly acknowledging user issues that come up. If not, you may need to review your AI model and token usage. Focus on not making the workflow too complicated to avoid dropping user messages. * **Resolution rate/speed:** You want a faster resolution for customers. That means future iterations you make will prioritize efficiency. * **Ticket throughput:** You may realize that your tickets are increasing and your company needs to scale support to keep up, either human or AI. * **Repeat questions:** If you have an influx of repeat questions, you may need to focus on debugging your AI agent. #### **2\. Maintain** Metrics that your AI agent should have no impact upon. For example: * You’d like to use AI to augment your support team so you can maintain a flat team structure and size. * You don’t want your AI agent to impact customer satisfaction or net promoter score (NPS). Keeping a close eye on your move and maintain metrics helps guide the iteration conversation and, if your agent is working, proves the [ROI of your AI agent investment](https://docs.google.com/document/d/1VmW75dpHMmocgIF5k80URduvRWkozxHW2Nyg9FmPQOE/edit?usp=sharing) . But what happens when your agent is not working as planned? \[Don’t know how to measure the success of your AI agent? [We can help with that](https://www.voiceflow.com/blog/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy)\ \] ### Let’s talk tokens—LLMs and the cost of AI agents First of all, Denys explains, it’s really easy to blame the shiny new AI agent when something goes awry. “Before everyone gets confused about metrics dropping, start with the cause. CSAT may have decreased because of a product change that coincides with the AI agent launch,” he says. Start with the cause of the problem and get more context, it might reveal the issue isn’t the agent, or if it is, the solution can be iterated and solved quickly. Let’s imagine another common scenario. You’d like to understand the cost per interaction of your LLM—you’re noticing the costs are higher than expected. Firstly, Denys warns against getting comfortable designing an AI agent that calls on your LLM too often. “There's a reason they're called large language models. They’re resource-intensive. If you stack a dollar per interaction and your agent is designed to make 30 LLM calls, you have an AI agent that costs $30 to do a task.” Needless to say, if your finance team hears about it, they’re unlikely to be pleased. Secondly, you should weigh the engineering costs of optimizing for LLM prompts. If the LLM Haiku costs $0.05 per call, but an engineer is working 10 hours or more on optimizing your AI agent, the cost of their time may not be worth the 900,000 Haiku calls you’d be able to make. “It may seem counterintuitive,” Deny says, “But depending on the model, it may be worth a first production launch with satisfactory accuracy and guardrails, without the hours of labor to optimize for the LLM.” The good news is that LLM costs are dropping all the time. Today, cheaper models often replace expensive ones in their abilities and can be easily swapped out. Denys advises you to weigh the costs thoughtfully—AI and human—when building and iterating your AI agent. Just like you would any other core product. \[Want to see what AI agent success looks like? Meet Trilogy, the team that automated [60% of their support tickets within 19 weeks](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)\ .\] AI agents are core products and they should be treated that way --------------------------------------------------------------- To go from ideas to useful AI agents takes effort and determination. You need to prioritize your use cases and align them closely with what customers want. You need to define success and always be thinking about ROI. And you need to align your AI team with the right skills while helping them collaborate across the organization by mitigating and addressing risk. But Denys is right, none of this is rocket science. We’re just applying commonly held product management principles to the AI space. The difference now is that we have the chance to make really cool, useful agents. Where we’ve been and where we’re going—in less than 300 words -------------------------------------------------------------- Some of us are still fighting gray hairs from last year’s stress. AI hype took off in 2023 when ChatGPT became the fastest consumer app to reach [100 million users in just two months](https://www.theverge.com/2023/11/6/23948386/chatgpt-active-user-count-openai-developer-conference) . It was a sensation. Executives immediately wanted to know how to use it and whether they should invest in it. Suddenly, teams were under pressure to develop proof of concepts (POCs), compare models, and develop prompt engineering and fine-tuning techniques. This was also the time for many businesses when [stalling](https://www.voiceflow.com/blog/how-mature-is-your-generative-ai) in their AI maturity became commonplace. Agents already had a pretty bad rap, and you didn’t want yours to be counted among them, so the risks of innovating were too high for many organizations. By 2024, we had established our baseline of what AI had the potential to do. But many teams, from big enterprises to small organizations, had questions about ROI, integrations with legacy systems, and how AI would affect existing workflows. How could AI lift the entire organization, rather than just a few teams? Today, treating your AI agent like you would a core product offering is the best way to ensure you’re creating real solutions to organizational goals—not just slapping AI onto an existing problem. **\[4 benefits of treating AI agents as core products, instead of one-off solutions** \ \ 1. Building a dedicated, collaborative team\ 2. Plugging your agent into useful business data \ 3. Starting and scaling across use cases \ 4. Tailoring your agent to your business needs with guardrails\] Mapping your AI product journey, from business case to AI roadmap ------------------------------------------------------------------ Getting your AI agent to production is not an easy step. Denys often runs into customers who have a cool proof of concept and even the backing of their leadership, but they’ve been slow to scale into production or release new use cases for their agents. That’s because, in an enterprise context, the list of contingencies is long—risk, security, budgets, and even the mood of your CEO. According to Denys, your AI agent needs a solid business case to move forward, “Propose bringing in stakeholders as early as possible to build, iterate, and deploy together. The strongest way to push past the obstacles to production is to create an unbeatable business case where the leadership teams cannot say no—and where security, risk, finance, or any other team’s critical questions are addressed as you build the agent together.” For example, your business case should answer the question of whether your AI agent will augment existing support or replace it altogether. [Roam’s AI agent](https://www.voiceflow.com/blog/how-roam-saved-30-hours-a-week-in-customer-support-hours) automates customer support, answering FAQs about their novel approach to the rental car industry. On the other hand, [Tico](https://www.voiceflow.com/blog/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) augments existing human support staff by pulling up relevant information and providing answers to customer questions. Both are valid approaches to AI agents, but vastly different, and knowing _how_ your AI agent solves organizational issues is just as important as knowing which issues it solves.  It should be clear by now that getting your agent to production is not a singular step. It’s more like five steps, comprised of building your: 1. Business case 2. Agent architecture 3. Proof of concept 4. Prototype for testing 5. Launch-ready AI agent “Ever seen a baby try to get up and run? They fall flat,” Denys explains, “Your agent is that baby, and your steps—from building your case to proof of concept and testing—are all in service of crawling in your AI maturity.” Once you’re comfortably crawling—an AI agent is live, meets a use case, and delivers value—you can begin the toddling steps towards walking, then eventually running. The crawl stage of AI maturity takes some effort, but it's required to deliver on that grand vision of walking or running with [complex AI agents](https://www.voiceflow.com/blog/crawl-walk-run-28-tactics-for-evolving-your-ai-agent) . Think you’re ready to run with your idea? Denys would advise you to slow down and work through it step by step, rather than skipping ahead. “You’ll end up with a wonderful POC that doesn’t leave testing. Build the muscles of a solid production run and start with the foundations first. Every organization can run eventually, but few start there.” Understand, decide, respond—your AI agent roadmap ------------------------------------------------- Now, let’s say you already have an agent in production (or are close to launching one) and you’re ready to think beyond your initial use cases. We often get questions about how to determine when it’s time to upgrade your AI agent. One helpful way of mapping the course of your agent is to ask how effectively it currently understands, decides, or responds (UDR), and how you wish to expand on those abilities. Depending on your answers, you can prioritize future use cases that align with those actions and focus on optimizations that solve current blind spots. Here are some UDR questions to help you create your AI agent roadmap:  By mapping your agent’s abilities across those three categories, you’ll also better understand who in the organization needs to be involved with production and iteration. Components like access to customer data or your agent’s ability to complete tasks using API integrations will require the expertise and buy-in of multiple team members. Their support will help keep your AI agent out of collaborative gridlock and reduce your time to production. Your applied AI team, from key roles to upskilling --------------------------------------------------- Any single person can spin up a decent proof of concept. It takes a collaborative team to create a useful AI agent that applies integrations, security, and conversation design. It’s why Denys doesn’t suggest building your AI agent without a team. For your organization to be successful you’ll need many different skills, aligned across functions. In order to do that, you need five roles represented on your AI team: 1. **Product:** They prioritize projects and have a comprehensive understanding of customers and what they need. 2. **UX/UI:** They improve and rebuild workflows. They focus on user research and understanding challenges with current workflows. 3. **Front end:** They integrate the AI agent with your organization’s overall customer experience. 4. **Back end:** They build your AI APIs and integrate your agent with your internal systems. 5. **Machine learning engineer:** They benchmark your current AI maturity, evaluate successes/setbacks, and choose the right AI models. But what happens when you lack the skills and expertise internally to fulfill these roles? Fear not, here’s a list of five ways to help upskill your current AI team. ### How to upskill your AI team 1. **Courses:** These help your team build a foundation with hands-on exercises. Courses equip the whole team with a baseline of AI experience, not just the ML engineer. 2. **Conferences:** Depending on the skill level of your team, conferences can offer new knowledge. These often go into deeper topics, so your team would benefit from having a foundation of knowledge so they know what to ask during discussions with peers. They're also great for networking. 3. **Hackathons:** After establishing a baseline of knowledge, these are great practices for brainstorming ideas, building a product, testing use cases, and validating POCs. You can also test cross-functional delivery. Hackathons often help teams build a POC that they can then move into production after the event. 4. **Proof of concepts:** As previously mentioned, many great AI agents start here. You can iterate through ideas and go through the product development process. But you should always test with customers before moving on to production (more on that in the next section). 5. **Hiring and mentorship:** Depending on your use cases, you might need additional skills. Bring on a mentor to coach existing teams with business and product context on new domains. Mentors or consultants can help with projects on a temporary or ad-hoc basis. You don’t have to reinvent product management—companies have been launching products for over 50 years—but every role is important to your AI agent project. If you’re missing important skills or roles, you run the risk your AI agent might have the same blind spots. Speaking of risks… Getting out of POC purgatory: Navigating risk and security ----------------------------------------------------------- There are a plethora of reasons why your AI agent could get stuck in POC purgatory. In fact, here’s a handy list of challenges you may face. _Thanks, Denys, I guess._  “Now, I don’t share this list to discourage you,” Denys says, “This is what you’ll face as you navigate the risks of building AI agents and treating them like core products.” Let's address a few of them, so you can too when they come up in your production cycle. Here’s a common situation. Your CEO says it’s time to build with AI—it’s now your organization’s top priority. But when you dig into how you’re supposed to implement it, your security team says you can’t use AI tools in customer-facing use cases. This may seem like an impassable obstacle, but let's go deeper. You might follow up with security with questions like: * What are your concerns? Model training with proprietary data? * Are there personal identifying information or intellectual property concerns? * Can we use AI tools internally? * Can we use an LLM to assist with classification tasks? * If we keep a human in the loop of the process, could we make exceptions? Once you investigate more, you realize that you can address some of these issues with straightforward solutions. If your security team is concerned about the API using your organization’s data to train the LLM model, bring in legal to review the contract. If security responds that a legal review isn’t enough, then you could suggest that the AI agent only access publicly available data, like data from your website. Other concerns may include compliance with SOC II or GDPR. Your solution could be to offer a clear business case with a POC that reflects lower-risk projects. Making that leap from POC purgatory to production is about de-risking your AI agent, and there are many ways to do that and still achieve your intended goals. Mapping out your production process (and the security steps you’re taking) will go a long way in gaining buy-in from your legal and security teams. For example, your production calendar could include checkpoints like: 1. **Review environment (UAT—User acceptance testing):** Rollout stakeholder and technical reviews. 2. **Production internal feature flag:** Release to a small set of users to test features before rolling out. 3. **Private beta review:** Validate with the community, select customers, power users and select enterprises. Ensure it’s solving problems and helping those who will be using it. 4. **General release rollout:** Add to core workflow for select general users before full rollout. 5. **Measurement, testing, iteration:** Each time you add a new feature or make a change, you repeat this process. Measure time to production as well, to ensure this process is efficient. If you take small leaps to get to production—in addition to understanding risks—when you return to your CEO with an update on your AI mandate, your answer will include a production plan with the support of your CIO, CPO, and security team. Mitigating risk allows you to expand use cases, build better workflows, revisit past decisions, or explore new models with the ability to adapt as things change in the AI space. Measuring your AI agent’s move and maintain metrics --------------------------------------------------- Most agents fall short because they aren’t being monitored and continually updated to reflect the needs of users. Luckily, Denys has created a handy [resource](https://www.voiceflow.com/blog/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy) for this—but generally, measuring the success of your agent usually falls under two categories: #### **1\. Move** Metrics your AI agent will directly affect. These metrics will also help inform your iteration process. For example, If you’re measuring: * **Time to first response:** You’re quickly acknowledging user issues that come up. If not, you may need to review your AI model and token usage. Focus on not making the workflow too complicated to avoid dropping user messages. * **Resolution rate/speed:** You want a faster resolution for customers. That means future iterations you make will prioritize efficiency. * **Ticket throughput:** You may realize that your tickets are increasing and your company needs to scale support to keep up, either human or AI. * **Repeat questions:** If you have an influx of repeat questions, you may need to focus on debugging your AI agent. #### **2\. Maintain** Metrics that your AI agent should have no impact upon. For example: * You’d like to use AI to augment your support team so you can maintain a flat team structure and size. * You don’t want your AI agent to impact customer satisfaction or net promoter score (NPS). Keeping a close eye on your move and maintain metrics helps guide the iteration conversation and, if your agent is working, proves the [ROI of your AI agent investment](https://docs.google.com/document/d/1VmW75dpHMmocgIF5k80URduvRWkozxHW2Nyg9FmPQOE/edit?usp=sharing) . But what happens when your agent is not working as planned? \[Don’t know how to measure the success of your AI agent? [We can help with that](https://www.voiceflow.com/blog/15-kpis-for-measuring-and-scaling-a-generative-ai-strategy)\ \] ### Let’s talk tokens—LLMs and the cost of AI agents First of all, Denys explains, it’s really easy to blame the shiny new AI agent when something goes awry. “Before everyone gets confused about metrics dropping, start with the cause. CSAT may have decreased because of a product change that coincides with the AI agent launch,” he says. Start with the cause of the problem and get more context, it might reveal the issue isn’t the agent, or if it is, the solution can be iterated and solved quickly. Let’s imagine another common scenario. You’d like to understand the cost per interaction of your LLM—you’re noticing the costs are higher than expected. Firstly, Denys warns against getting comfortable designing an AI agent that calls on your LLM too often. “There's a reason they're called large language models. They’re resource-intensive. If you stack a dollar per interaction and your agent is designed to make 30 LLM calls, you have an AI agent that costs $30 to do a task.” Needless to say, if your finance team hears about it, they’re unlikely to be pleased. Secondly, you should weigh the engineering costs of optimizing for LLM prompts. If the LLM Haiku costs $0.05 per call, but an engineer is working 10 hours or more on optimizing your AI agent, the cost of their time may not be worth the 900,000 Haiku calls you’d be able to make. “It may seem counterintuitive,” Deny says, “But depending on the model, it may be worth a first production launch with satisfactory accuracy and guardrails, without the hours of labor to optimize for the LLM.” The good news is that LLM costs are dropping all the time. Today, cheaper models often replace expensive ones in their abilities and can be easily swapped out. Denys advises you to weigh the costs thoughtfully—AI and human—when building and iterating your AI agent. Just like you would any other core product. \[Want to see what AI agent success looks like? Meet Trilogy, the team that automated [60% of their support tickets within 19 weeks](https://www.voiceflow.com/blog/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy)\ .\] AI agents are core products and they should be treated that way --------------------------------------------------------------- To go from ideas to useful AI agents takes effort and determination. You need to prioritize your use cases and align them closely with what customers want. You need to define success and always be thinking about ROI. And you need to align your AI team with the right skills while helping them collaborate across the organization by mitigating and addressing risk. But Denys is right, none of this is rocket science. We’re just applying commonly held product management principles to the AI space. The difference now is that we have the chance to make really cool, useful agents.  RECOMMENDED [\ \ ### AI automation on a budget: Getting started with high ROI use cases](https://www.voiceflow.com/pathways/ai-automation-on-a-budget-getting-started-with-high-roi-use-cases) PUBLISHED July 8, 2024 TOPICS IN THIS ARTICLE Elevating our work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/voiceflow-raises-3-5-million-to-build-the-future-of-voice-interface-design) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow raises $3.5 million to build the future of voice interface design](https://www.voiceflow.com/pathways/voiceflow-raises-3-5-million-to-build-the-future-of-voice-interface-design) [](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [](https://www.voiceflow.com/pathways/why-were-building-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Why we're building Voiceflow](https://www.voiceflow.com/pathways/why-were-building-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/building-your-ai-agents-like-products-a-blueprint-from-poc-purgatory-to-production#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Designing for voice assistants vs. chatbots | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Designing for voice assistants vs. chatbots =========================================== [\ \ Masha Guermonprez\ \ at](https://www.voiceflow.com/contributors/masha-guermonprez) _This article is by_ [_Masha Guermonprez_](https://www.linkedin.com/in/masha-guermonprez/) _, CX Lead for Voice Assistant at_ [_SPIX industry_](https://www.spix-industry.com/) _._ Marshall McLuhan coined the famous phrase “the medium is the message” in 1964—not long before [ELIZA](https://en.wikipedia.org/wiki/ELIZA) , the first ever chatbot therapist, appeared. Communication has certainly evolved since then, but the idea still stands: the channel we communicate through defines the perception of the informational content we receive or share. In the 1960s, the media channel environment was changing fast. The telephone was getting more and more popular (and mostly stable), people were witnessing the apparition of the first satellite TVs, and radios were in almost every house. However, people were still largely communicating by written message or face-to-face. Fast-forward to the 2020s, and we’re living in a world where we can communicate not only with other human beings without distance stopping us, but also with creations of a digital nature. ([Isaac Asimov](http://www.asimovonline.com/asimov_home_page.html) would totally enjoy that.) The chatbot took the world by surprise in the 2010s, though it had a long history before that. Suddenly we started seeing the little chat windows everywhere on the web and we asked ourselves peculiar questions like, _"Am I speaking to a human or... what am I speaking to?”_  In the mid 2010s, the voicebot tried to perform the same quiet coup d’etat as its big cousin. And though the conversations with the voicebot are very different from those with chatbots—the former being still very purpose-driven and not necessarily “natural”—we’re definitely heading somewhere here. Today we tend to design a lot for both mediums. You can develop an Alexa skill in a few hours without any particular experience, or create a Facebook bot just by following a YouTube tutorial. The online builders for creating voice or chatbots propose some appealing visual canvases with intuitive design, helping you quickly design great conversation flows. At the same time, this same visual canvas remains whether you’re designing for voice or chat—so we tend to design those experiences similarly. But one thing to keep in mind: **The experience, though designed similarly, is not the same. Different mediums should convey the information differently.** Be particularly cautious if you already have a chatbot and you want to add voice to it, or build a voice assistant based on this chatbot. The two mediums are not the same, and it’s not only due to the difference of our brain functioning while listening or reading. To understand why we design differently, we have to clearly distinguish different use cases surrounding chatbots and voicebots. While written conversation is more intimate and more engaging, it’s only natural that the “conversational” bots (designed to keep you company, like Mitsuki or Replica) are text-based. The “informational” bots like ChatGPT are also text-based, because they generate a big amount of text in their answers—and that’s not really adapted to voice, either. Voicebots, on the other hand, tend to be more about “action.” Voice interactions with bots are more command-based, like “turn on the lights” or “set a five-minute timer.” They are interesting when you can’t type because you’re in the middle of something else or not in front of the screen (e.g. Alexa or Google), or when you don’t have interface (e.g. industrial setting), or when the audio channel is the only one available (e.g. driving). As of today, it’s most useful when the user has to make a clear and short command and expects either an action or a short piece of information in return. _Note: there are certain voice assistants that are all about hospitality (think ElliQ) and keeping company, but those are more of an exception to the rules in actual context and have a very specific use case._ So, cutting to the point: What are the unique challenges of voice, and how is it different from chat? --------------------------------------------------------------------------- ### **1\. Engagement** The way I see it, one of the important distinctions between reading and listening is that reading is an action you take, while listening is something that happens to you. Reading requires engagement. You need to actively process the written information to make progress. A voice assistant makes progress regardless if you’re actively listening or not. Listening to a voice assistant can be a more passive experience, whereas chatbot requires more active participation. ### **2\. Cognitive load** Voice interactions never last. They are spoken, listened to, and then they disappear—and the only place they may remain for some time is someone’s memory, or the log file. In contrast, text-based chatbot interactions leave a written record, which can be referred to at a later time. This fundamental difference has several important implications, but the most significant one is that while designing a voice assistant, the cognitive load must be taken into consideration and addressed. The point that is not as crucial for text-based chatbot interactions. ### **3\. Timing** Another aspect to consider is the time frame. With a chatbot, you have the flexibility to read, ponder, research, and then respond to the bot. You have the luxury of taking your time. However, with voice interactions, there is less room for hesitation or long pauses, as it is necessary to respond promptly in order to maintain the flow of the conversation. In current voice assistants, for example, the user is given a short window of time, usually two to three seconds, to respond before the conversation ends. This can create an atmosphere of tension throughout the voice interaction, as the user find themself in a pressing “listen-get it-respond” situation. In addition, in text-based interactions, users can be provided with links, images, and other multimedia elements to reference while they are responding. This is not possible in voice-based interactions (unless there is a UI provided), where the only means of communication are the user’s voice and the voice of the assistant. Cognitively all those factors alter the ways we perceive information through different mediums of communication. Things to consider when designing for voice assistants ------------------------------------------------------ Through smart design choices, we have an opportunity to face and successfully address those points. There are certain things to take into consideration while designing for voice: how we handle errors, how we’re presenting lists and options, how long our messages are, how informative and how simple they are. Let’s see those in detail. ### 1\. Error handling Error handling and correction are more intricate and prevalent in voice interactions compared to text-based interactions. While one may make errors in typing or spelling while using text, if there are no typographical errors, the chatbot would understand the message correctly. However, in voice interactions, even if the words are pronounced correctly, there is still a possibility of being misunderstood. Some typical voice errors come from the ASR system, like the cases of “no match” because of the heavy background noise or an accent. Resolving these misunderstandings can be challenging, and it’s definitely something to think through with your voice assistant. **_Technique tip:_** _Implicit or explicit confirmation prompts can be used to verify the user’s intent and to ensure that the system understands the user’s request correctly. This can help to reduce the cognitive load on the user, by reducing the risk of errors and misunderstandings._ ### 2\. Limit or optimize lists This can help to reduce the cognitive load on the user, by making it easier for them to choose the right option. In a chatbot, you can easily give a user a list of choices like that: “Greetings! How can I assist you today? Please choose from the following options: 1. Book a flight 2. Make a hotel reservation 3. Rent a car 4. Purchase a vacation package 5. Find travel deals and discounts” And though I always preach for not more than three options in a chatbot, the voice assistant takes it to a very different level. The user needs to be highly focused to be able to process in auditory manner all the different options presented and respond accordingly. The voice assistant needs to make it easier for the user to make a choice, either by allowing the barge-in (so when the right option is heard, the user could say "Okay, yes, I want this"), or limiting the options presented to the user at one time. Be sure that you have different ways to select those options (“rent a car,” “option three,” “this one,” “the first one,” “the last one”). And the options should be as short in text as possible. **_Technique tip:_** _If you have a long list that you need to present by voice, don't give more than three choices at a time, with a possibility to say “next” to skip to the next selection._ This brings us logically to the next point: ### 3\. Limit the length With voice, you have to keep it short. The audio channel is more vulnerable to cognitive overload. Often the user forgets the beginning of the conversation by its end. That's mostly because their attention isn't solely focused on the virtual assistant—they're trying to do something else, and that’s the whole point of the vocal assistant. The real pitfall with voice interactions is over-informing the user. Remember that in vocal interaction, the user can’t usually skip the messages of the assistant, like in a chatbot. Keep it short and stick to what is needed to be said. **Technique tip 1:** _One breath test: speak your message out loud. out and if you can’t do it on one breath, it means it’s too long._ **Technique tip 2:** _Jenga technique aimed to reduce the volume of messages is a very useful thing in designing for chat and voice. It consists in taking out of your message, piece by piece, the information that doesn’t directly add any value to your message._ Or _Jenga technique consists of taking out the information that doesn’t add value to your message._ _See what I did there? ;)_ To this matter: ### 4\. Skip the unnecessary With voice, we can go even further—not every message has to have actual words. Sometimes sonar notifications are a more optimized way to respond. Don’t repeat back a message if a user has a clear understanding that something has been done. I mean, it’s only common sense that the voicebot doesn’t reply "I have switched off the light" if the user sees that the light is switched off, right? (For inclusivity purposes, we can always add a setting that allows the bot to be more verbal.) Sometimes a little sonar notification is enough, and sometimes even this is redundant. Think about which parts of your dialogues can be cut off or replaced by "success," "failure," or other sounds. ### 5\. Simplify Use simple and clear language when designing the voice assistant’s responses. This can help reduce the cognitive load on the user by making it easier for them to understand and process the information. Play Jenga with your scripts, then take them and play Jenga again. Replace long words with shorter synonyms. At the end, if you haven’t shortened your scripts by 50%, you’re not doing it right. Make the instruction as clear as possible. Review the context, and read the script OUT LOUD. You will quickly understand the parts of your bot that are too long or not worth mentioning at all. Remember: chatbots are way more about politeness and small talk than voicebots are. Voice equals agility. Skip the "Hello honey, how are you doing today?” part—unless the voice use case is specifically about hospitality. Doesn’t mean you shouldn’t include some miscellaneous jokes or easter eggs but only trigger them on a prompt. We’re still learning how to communicate with digital assistants. The use cases will evolve and the conversations will evolve at the same pace. One might say that all the steps mentioned above make a bot sound less human and more robotic. I say, let humans be humans and robots be robots. Robotic doesn’t necessarily rule out “user-friendly.” On the contrary, it optimized our interactions with the voice assistants, making it more about purpose and less about the noise. And that’s a wrap. _Editor's note: This story was originally published on_ [_Medium_](https://medium.com/design-bootcamp/a-little-less-conversation-a-little-more-action-designing-for-voice-assistants-versus-chatbots-b39b877a1c3d) _. Header image by_ [_Volodymyr Hryshchenko_](https://unsplash.com/@lunarts) _._ In the mid 2010s, the voicebot tried to perform the same quiet coup d’etat as its big cousin. And though the conversations with the voicebot are very different from those with chatbots—the former being still very purpose-driven and not necessarily “natural”—we’re definitely heading somewhere here. Today we tend to design a lot for both mediums. You can develop an Alexa skill in a few hours without any particular experience, or create a Facebook bot just by following a YouTube tutorial. The online builders for creating voice or chatbots propose some appealing visual canvases with intuitive design, helping you quickly design great conversation flows. At the same time, this same visual canvas remains whether you’re designing for voice or chat—so we tend to design those experiences similarly. But one thing to keep in mind: **The experience, though designed similarly, is not the same. Different mediums should convey the information differently.** Be particularly cautious if you already have a chatbot and you want to add voice to it, or build a voice assistant based on this chatbot. The two mediums are not the same, and it’s not only due to the difference of our brain functioning while listening or reading. To understand why we design differently, we have to clearly distinguish different use cases surrounding chatbots and voicebots. While written conversation is more intimate and more engaging, it’s only natural that the “conversational” bots (designed to keep you company, like Mitsuki or Replica) are text-based. The “informational” bots like ChatGPT are also text-based, because they generate a big amount of text in their answers—and that’s not really adapted to voice, either. Voicebots, on the other hand, tend to be more about “action.” Voice interactions with bots are more command-based, like “turn on the lights” or “set a five-minute timer.” They are interesting when you can’t type because you’re in the middle of something else or not in front of the screen (e.g. Alexa or Google), or when you don’t have interface (e.g. industrial setting), or when the audio channel is the only one available (e.g. driving). As of today, it’s most useful when the user has to make a clear and short command and expects either an action or a short piece of information in return. _Note: there are certain voice assistants that are all about hospitality (think ElliQ) and keeping company, but those are more of an exception to the rules in actual context and have a very specific use case._ So, cutting to the point: What are the unique challenges of voice, and how is it different from chat? --------------------------------------------------------------------------- ### **1\. Engagement** The way I see it, one of the important distinctions between reading and listening is that reading is an action you take, while listening is something that happens to you. Reading requires engagement. You need to actively process the written information to make progress. A voice assistant makes progress regardless if you’re actively listening or not. Listening to a voice assistant can be a more passive experience, whereas chatbot requires more active participation. ### **2\. Cognitive load** Voice interactions never last. They are spoken, listened to, and then they disappear—and the only place they may remain for some time is someone’s memory, or the log file. In contrast, text-based chatbot interactions leave a written record, which can be referred to at a later time. This fundamental difference has several important implications, but the most significant one is that while designing a voice assistant, the cognitive load must be taken into consideration and addressed. The point that is not as crucial for text-based chatbot interactions. ### **3\. Timing** Another aspect to consider is the time frame. With a chatbot, you have the flexibility to read, ponder, research, and then respond to the bot. You have the luxury of taking your time. However, with voice interactions, there is less room for hesitation or long pauses, as it is necessary to respond promptly in order to maintain the flow of the conversation. In current voice assistants, for example, the user is given a short window of time, usually two to three seconds, to respond before the conversation ends. This can create an atmosphere of tension throughout the voice interaction, as the user find themself in a pressing “listen-get it-respond” situation. In addition, in text-based interactions, users can be provided with links, images, and other multimedia elements to reference while they are responding. This is not possible in voice-based interactions (unless there is a UI provided), where the only means of communication are the user’s voice and the voice of the assistant. Cognitively all those factors alter the ways we perceive information through different mediums of communication. Things to consider when designing for voice assistants ------------------------------------------------------ Through smart design choices, we have an opportunity to face and successfully address those points. There are certain things to take into consideration while designing for voice: how we handle errors, how we’re presenting lists and options, how long our messages are, how informative and how simple they are. Let’s see those in detail. ### 1\. Error handling Error handling and correction are more intricate and prevalent in voice interactions compared to text-based interactions. While one may make errors in typing or spelling while using text, if there are no typographical errors, the chatbot would understand the message correctly. However, in voice interactions, even if the words are pronounced correctly, there is still a possibility of being misunderstood. Some typical voice errors come from the ASR system, like the cases of “no match” because of the heavy background noise or an accent. Resolving these misunderstandings can be challenging, and it’s definitely something to think through with your voice assistant. **_Technique tip:_** _Implicit or explicit confirmation prompts can be used to verify the user’s intent and to ensure that the system understands the user’s request correctly. This can help to reduce the cognitive load on the user, by reducing the risk of errors and misunderstandings._ ### 2\. Limit or optimize lists This can help to reduce the cognitive load on the user, by making it easier for them to choose the right option. In a chatbot, you can easily give a user a list of choices like that: “Greetings! How can I assist you today? Please choose from the following options: 1. Book a flight 2. Make a hotel reservation 3. Rent a car 4. Purchase a vacation package 5. Find travel deals and discounts” And though I always preach for not more than three options in a chatbot, the voice assistant takes it to a very different level. The user needs to be highly focused to be able to process in auditory manner all the different options presented and respond accordingly. The voice assistant needs to make it easier for the user to make a choice, either by allowing the barge-in (so when the right option is heard, the user could say "Okay, yes, I want this"), or limiting the options presented to the user at one time. Be sure that you have different ways to select those options (“rent a car,” “option three,” “this one,” “the first one,” “the last one”). And the options should be as short in text as possible. **_Technique tip:_** _If you have a long list that you need to present by voice, don't give more than three choices at a time, with a possibility to say “next” to skip to the next selection._ This brings us logically to the next point: ### 3\. Limit the length With voice, you have to keep it short. The audio channel is more vulnerable to cognitive overload. Often the user forgets the beginning of the conversation by its end. That's mostly because their attention isn't solely focused on the virtual assistant—they're trying to do something else, and that’s the whole point of the vocal assistant. The real pitfall with voice interactions is over-informing the user. Remember that in vocal interaction, the user can’t usually skip the messages of the assistant, like in a chatbot. Keep it short and stick to what is needed to be said. **Technique tip 1:** _One breath test: speak your message out loud. out and if you can’t do it on one breath, it means it’s too long._ **Technique tip 2:** _Jenga technique aimed to reduce the volume of messages is a very useful thing in designing for chat and voice. It consists in taking out of your message, piece by piece, the information that doesn’t directly add any value to your message._ Or _Jenga technique consists of taking out the information that doesn’t add value to your message._ _See what I did there? ;)_ To this matter: ### 4\. Skip the unnecessary With voice, we can go even further—not every message has to have actual words. Sometimes sonar notifications are a more optimized way to respond. Don’t repeat back a message if a user has a clear understanding that something has been done. I mean, it’s only common sense that the voicebot doesn’t reply "I have switched off the light" if the user sees that the light is switched off, right? (For inclusivity purposes, we can always add a setting that allows the bot to be more verbal.) Sometimes a little sonar notification is enough, and sometimes even this is redundant. Think about which parts of your dialogues can be cut off or replaced by "success," "failure," or other sounds. ### 5\. Simplify Use simple and clear language when designing the voice assistant’s responses. This can help reduce the cognitive load on the user by making it easier for them to understand and process the information. Play Jenga with your scripts, then take them and play Jenga again. Replace long words with shorter synonyms. At the end, if you haven’t shortened your scripts by 50%, you’re not doing it right. Make the instruction as clear as possible. Review the context, and read the script OUT LOUD. You will quickly understand the parts of your bot that are too long or not worth mentioning at all. Remember: chatbots are way more about politeness and small talk than voicebots are. Voice equals agility. Skip the "Hello honey, how are you doing today?” part—unless the voice use case is specifically about hospitality. Doesn’t mean you shouldn’t include some miscellaneous jokes or easter eggs but only trigger them on a prompt. We’re still learning how to communicate with digital assistants. The use cases will evolve and the conversations will evolve at the same pace. One might say that all the steps mentioned above make a bot sound less human and more robotic. I say, let humans be humans and robots be robots. Robotic doesn’t necessarily rule out “user-friendly.” On the contrary, it optimized our interactions with the voice assistants, making it more about purpose and less about the noise. And that’s a wrap. _Editor's note: This story was originally published on_ [_Medium_](https://medium.com/design-bootcamp/a-little-less-conversation-a-little-more-action-designing-for-voice-assistants-versus-chatbots-b39b877a1c3d) _. Header image by_ [_Volodymyr Hryshchenko_](https://unsplash.com/@lunarts) _._  RECOMMENDED [\ \ ### Breaking and rebuilding Nike's chatbot](https://www.voiceflow.com/pathways/breaking-nike-chatbot) PUBLISHED February 7, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Masha Guermonprez\ \ at\ \ Masha is CX Lead for Voice Assistant at SPIX industry.](https://www.voiceflow.com/contributors/masha-guermonprez) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Cathy Pearl on all things conversation design](https://www.voiceflow.com/pathways/cathy-pearl-talks-conversation-design) [](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Designing conversational bots that fix themselves](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves) [](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Chatbot glow-up: 3 reasons everyone hates your AI agent (and what you can do about it)](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Flowcharts to Voiceflow: How WooliesX Reinvented Their Conversation Design Workflow | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Flowcharts to Voiceflow: How WooliesX Reinvented Their Conversation Design Workflow =================================================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) > **Problem: The conversation design team at WooliesX needed to scale their workflow with the growth of the company.** > **Solution: The team turned to Voiceflow to reinvent their workflow, from design to launch.**  **30% in time savings, 15 new team members collaborating & 1 new workflow** --------------------------------------------------------------------------- The conversation design team at WooliesX was ready to grow. As the innovation arm of Woolworth’s Group - a portfolio of Australian & New Zealand consumer brands - the team actively designs and launches conversational AI experiences for IVR, chat, and more. Across food, and financial industries, this group of conversation designers at [WooliesX](https://wooliesx.com.au/) is responsible for the rapidly growing number of conversational channels customers use daily. As their business scaled, the team and its workflow looked to do the same. ### **The flowchart workflow** Flowcharts were at the foundation of the WooliesX conversation design process. “We had 3 skilled designers who all had different ways of expressing what needed to be created,” said [Peter Isaacs](https://www.linkedin.com/in/isaacspeter/) , conversation designer at WooliesX. “We worked in flowcharts to combine our styles and communicate with other teams, including development.” Inside a flowchart project, the entire conversational journey sprawled across one vast canvas. A key of colourful shapes represented different elements of the design - a blue diamond was a question, green circles were intents, yellow clouds were utterances, etc.  _A mockup example of a WooliesX flowchart project._ When looking for a specific fallback step, for example, Peter would change flowchart tabs, zoom out to 50%, and then meander around the flowchart design to find it. And while the team continued to design and launch top-notch conversational experiences, this workflow asked designers to spend creative energy scrolling the vast canvas and navigating the key. Stakeholders and developers also interacted with the conversation design team’s flowcharts. > **"I'd have hour-long one-on-one meetings with developers and stakeholders to walk them through a project,” remembered Peter.** After each meeting, developers would then start to build the experience from scratch. The current workflow asked Peter’s team to add fallbacks and error paths - both pillars of good conversational AI - into the design post-development. For any changes and additions (like adding a new policy offering option to the insurance IVR assistant) Peter and team went back to the drawing board. Stakeholders did find the visual blocks of the flowchart easy to follow in terms of the conversation’s pathways. However, as the design team would write any SSML (speech synthesis markup language) directly into the flowchart project for the developers to reference, the project became confusing for non-developers to understand. Peter knew the workflow needed to evolve to keep pace with the scaling company. The team realized that they needed a central, standardized system in their workflow for efficiency and communication. That’s when WooliesX decided to replace the team’s flowcharting tool with Voiceflow. ### A dynamic tool and team Using the collaborative conversation design tool, Peter and the team now have a powerful central source of truth for all of their projects. > **“We’ve standardized our design process,” remarked Peter. “Everyone on the team understands the organized system, which lets us each design in the way we’re best.”** No longer relying on the key and flowchart combination, the team now uses reusable components, organized topics, and line labelling on the Voiceflow canvas to communicate.  _A side-by-side mockup of a project in a flowchart and Voiceflow._ ### **And this communication boost is not limited to their team.** The hand-off process to development has been vastly improved. Using Voiceflow’s built-in steps and canvas markup, the conversation design team can now clearly express logic, APIs, data capture, and other conversational events that will impact the developer’s build. SSML is now even baked into the text and speak steps for the developers to reference without limiting the prototype experience for stakeholders. When a design is ready, the canvas and project code can both be shared - all from the WooliesX project workspace. If there’s a new policy offering that needs to be added to the insurance experience, for example, Peter can quickly build on top of the existing choice step in the project and the development team can easily redeploy. Nothing’s lost in translation between the two departments. Peter’s work with stakeholders is much more productive, too. Directly from the canvas, he shares high-fidelity prototypes of his designs with everyone involved in a project.  _A mockup of an IVR assistant prototype._ > **“I can share a hands-on, realistic version of the final product,” said Peter. “We’re even able to get real-time feedback through user testing, which was not possible before.”** Expectations for projects are set and then met - meaning time spent on revisions in the past is now allocated for improving the quality of the conversational experiences. **Bot Optimisation Specialists** are now actively involved in the workflow to maximize the testing tools in Voiceflow. After a design is built, these specialists spend time combing through utterances, conversation turns, and intents to find gaps or room for improvement. > **“I tell our specialists to go break my designs in testing,” laughed Peter.** Utilizing the Transcripts feature, the conversation designers can then review any and all tests of a design to seamlessly update content, including missing intents or suggestions from other teams. Even after launch, the conversation design workflow can now support continuous optimization. When looking at fallbacks and errors from real users, Peter can quickly find answers to the questions that make his conversations better: * why did this happen? * where in the conversation? * what was said before the error? * where can we extend the conversation? “There’s no more guesswork on our team,” continued Peter. “We can meet the user where they are in the conversation.” ### Bottom line results Across the organization, the conversational AI workflow is successfully scaling. The past assembly-line process of design, development, and training has been replaced with an increasingly collaborative approach. “We can now snowball our process,” mentioned Peter. “We’re seeing specialists from across the organization becoming engaged and adding to our experiences because they are now more a part of the conversation design workflow.”  In terms of efficiency, this new design process has saved the team **30% in terms of time spent on projects** compared to the past flowchart workflow. And that’s with the addition of **15 team members** from across the organization to the collaborative process - creating more conversations in far less time. “With less back-n-forth on the design, time spent on training the model, and stakeholder friendly versions of the prototype, Voiceflow has removed quite a bit of ambiguity,” said Peter. The result? The WooliesX team can efficiently build and launch more quality conversations for their customers. **30% in time savings, 15 new team members collaborating & 1 new workflow** --------------------------------------------------------------------------- The conversation design team at WooliesX was ready to grow. As the innovation arm of Woolworth’s Group - a portfolio of Australian & New Zealand consumer brands - the team actively designs and launches conversational AI experiences for IVR, chat, and more. Across food, and financial industries, this group of conversation designers at [WooliesX](https://wooliesx.com.au/) is responsible for the rapidly growing number of conversational channels customers use daily. As their business scaled, the team and its workflow looked to do the same. ### **The flowchart workflow** Flowcharts were at the foundation of the WooliesX conversation design process. “We had 3 skilled designers who all had different ways of expressing what needed to be created,” said [Peter Isaacs](https://www.linkedin.com/in/isaacspeter/) , conversation designer at WooliesX. “We worked in flowcharts to combine our styles and communicate with other teams, including development.” Inside a flowchart project, the entire conversational journey sprawled across one vast canvas. A key of colourful shapes represented different elements of the design - a blue diamond was a question, green circles were intents, yellow clouds were utterances, etc.  _A mockup example of a WooliesX flowchart project._ When looking for a specific fallback step, for example, Peter would change flowchart tabs, zoom out to 50%, and then meander around the flowchart design to find it. And while the team continued to design and launch top-notch conversational experiences, this workflow asked designers to spend creative energy scrolling the vast canvas and navigating the key. Stakeholders and developers also interacted with the conversation design team’s flowcharts. > **"I'd have hour-long one-on-one meetings with developers and stakeholders to walk them through a project,” remembered Peter.** After each meeting, developers would then start to build the experience from scratch. The current workflow asked Peter’s team to add fallbacks and error paths - both pillars of good conversational AI - into the design post-development. For any changes and additions (like adding a new policy offering option to the insurance IVR assistant) Peter and team went back to the drawing board. Stakeholders did find the visual blocks of the flowchart easy to follow in terms of the conversation’s pathways. However, as the design team would write any SSML (speech synthesis markup language) directly into the flowchart project for the developers to reference, the project became confusing for non-developers to understand. Peter knew the workflow needed to evolve to keep pace with the scaling company. The team realized that they needed a central, standardized system in their workflow for efficiency and communication. That’s when WooliesX decided to replace the team’s flowcharting tool with Voiceflow. ### A dynamic tool and team Using the collaborative conversation design tool, Peter and the team now have a powerful central source of truth for all of their projects. > **“We’ve standardized our design process,” remarked Peter. “Everyone on the team understands the organized system, which lets us each design in the way we’re best.”** No longer relying on the key and flowchart combination, the team now uses reusable components, organized topics, and line labelling on the Voiceflow canvas to communicate.  _A side-by-side mockup of a project in a flowchart and Voiceflow._ ### **And this communication boost is not limited to their team.** The hand-off process to development has been vastly improved. Using Voiceflow’s built-in steps and canvas markup, the conversation design team can now clearly express logic, APIs, data capture, and other conversational events that will impact the developer’s build. SSML is now even baked into the text and speak steps for the developers to reference without limiting the prototype experience for stakeholders. When a design is ready, the canvas and project code can both be shared - all from the WooliesX project workspace. If there’s a new policy offering that needs to be added to the insurance experience, for example, Peter can quickly build on top of the existing choice step in the project and the development team can easily redeploy. Nothing’s lost in translation between the two departments. Peter’s work with stakeholders is much more productive, too. Directly from the canvas, he shares high-fidelity prototypes of his designs with everyone involved in a project.  _A mockup of an IVR assistant prototype._ > **“I can share a hands-on, realistic version of the final product,” said Peter. “We’re even able to get real-time feedback through user testing, which was not possible before.”** Expectations for projects are set and then met - meaning time spent on revisions in the past is now allocated for improving the quality of the conversational experiences. **Bot Optimisation Specialists** are now actively involved in the workflow to maximize the testing tools in Voiceflow. After a design is built, these specialists spend time combing through utterances, conversation turns, and intents to find gaps or room for improvement. > **“I tell our specialists to go break my designs in testing,” laughed Peter.** Utilizing the Transcripts feature, the conversation designers can then review any and all tests of a design to seamlessly update content, including missing intents or suggestions from other teams. Even after launch, the conversation design workflow can now support continuous optimization. When looking at fallbacks and errors from real users, Peter can quickly find answers to the questions that make his conversations better: * why did this happen? * where in the conversation? * what was said before the error? * where can we extend the conversation? “There’s no more guesswork on our team,” continued Peter. “We can meet the user where they are in the conversation.” ### Bottom line results Across the organization, the conversational AI workflow is successfully scaling. The past assembly-line process of design, development, and training has been replaced with an increasingly collaborative approach. “We can now snowball our process,” mentioned Peter. “We’re seeing specialists from across the organization becoming engaged and adding to our experiences because they are now more a part of the conversation design workflow.”  In terms of efficiency, this new design process has saved the team **30% in terms of time spent on projects** compared to the past flowchart workflow. And that’s with the addition of **15 team members** from across the organization to the collaborative process - creating more conversations in far less time. “With less back-n-forth on the design, time spent on training the model, and stakeholder friendly versions of the prototype, Voiceflow has removed quite a bit of ambiguity,” said Peter. The result? The WooliesX team can efficiently build and launch more quality conversations for their customers.  RECOMMENDED [\ \ ### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) PUBLISHED May 4, 2023 TOPICS IN THIS ARTICLE Exceptional work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/using-voiceflow-to-run-more-efficient-inclusive-ai-design-reviews) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How a Global Tier 1 Investment Bank Uses Voiceflow to Run More Efficient, Inclusive Conversational AI Design Reviews](https://www.voiceflow.com/pathways/using-voiceflow-to-run-more-efficient-inclusive-ai-design-reviews) [](https://www.voiceflow.com/pathways/flowcharts-to-voiceflow-how-wooliesx-reinvented-their-conversation-design-workflow) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Flowcharts to Voiceflow: How WooliesX Reinvented Their Conversation Design Workflow](https://www.voiceflow.com/pathways/flowcharts-to-voiceflow-how-wooliesx-reinvented-their-conversation-design-workflow) [](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### From decision trees to AI: How Turo built a global, multilingual chatbot in two months](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/flowcharts-to-voiceflow-how-wooliesx-reinvented-their-conversation-design-workflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/flowcharts-to-voiceflow-how-wooliesx-reinvented-their-conversation-design-workflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate ============================================================================== [\ \ Anna Rosen\ \ Enterprise Solutions Engineer at Voiceflow. at](https://www.voiceflow.com/contributors/anna-rosen) Designers and data scientists using natural language understanding (NLU) platforms, like Dialogflow, to also design dialogue and response content need to reconsider their workflow. The demand on designers to ship complex, contextual assistants keeps growing. Navigational, FAQ bots with little personalization don't cut it for users anymore. They want assistants to remember key details and make it easy to speak naturally. They want actual conversation. Most teams consider a move from Dialogflow ES (DFES) to Voiceflow once their conversation design needs become this contextually mature. Designers use Voiceflow to clearly visualize, test and scale each experience while collaborating with product managers, data scientists and stakeholders all in one spot. Teams build and ship customer-focused chat and voice experiences rapidly. Before starting that **new source of truth**, it’s important to understand how core design principles are represented between Dialogflow and Voiceflow. Here we look at 3 core design pillars and how CAI teams would approach each in Voiceflow compared to Dialogflow. * Follow-up intents * Parameters * IF statements  1) Follow-up Intents -------------------- A **follow-up intent** adds a nested intent inside a parent intent. #### Dialogflow When using Dialogflow (a natural language understanding platform), you’d consider follow-up intents to be normal because they “hold” the context of the parent intent. There are many types of follow-up expressions, including repeat, confirm & cancel. **Let’s design a conversation in Dialogflow to illustrate.** **_Example 1:_** Your main intent was to order a caramel latte. A follow-up intent to this would be a secondary order confirmation. This seems simple enough. However, once we introduce complexity, building with nested context like this becomes quite cumbersome. For example, this same user wants a caramel latte, but before confirming, they add in, “I want to add whipped cream.”  Dialogflow ES custom intents **_Example 2:_** Now we have the main intent (order caramel latte), a follow-up intent to confirm, a second follow-up to add whipped cream, and an extra ‘yes’ confirmation intent to confirm the whipped cream.  Dialogflow ES nested intents As you can see, the more complex a conversation gets, the more CAI teams are required to nest contextual follow-ups. While follow-up intents seem straightforward when first designing, the larger and more complex a conversation gets, the harder this web of follow-up intents becomes to manage. Even more limiting, these nested intents are tied to their parent intent and don’t allow any flexible room for reusability. #### Voiceflow In Voiceflow, we still care about the context of the conversation, but we don’t need to keep track of follow-up intents in the same way. Every intent in Voiceflow is a parent level intent, which creates room for reusability. In order to account for contextual responses, Voiceflow also offers intent scoping. With a simple toggle, certain intents can only be accessible in a certain topic or globally available. It’s simple to reduce design bloat to a minimum. Designers have the ability to reference a reusable 'yes' intent wherever necessary and without having to create a net new 'yes' intent per main intent or node. The difference in mental models here is that instead of creating a net new yes intent for each main intent, one would re-use a system yes intent that has situational context baked in.  This applies to context specific yes intents that often have both an affirmation (such as “yes”) and a repetition of the main intent (”Yes, I would like to order coffee”). As you have the ability to select intent scoping based on each node, there is no concern about conflicting utterances. This all adds up to less latency, more efficiency and better reusability for designs. 2) Parameters ------------- **Parameters** are structured data that can be used to perform logic, a task, or generate a response. #### Dialogflow The concept of parameters in Dialogflow exists in order to provide the action value to your fulfillment web hook request or the API interaction response. It can be used, in one example, to trigger specific logic in your service. When an intent is matched at runtime, Dialogflow will give you the extracted value from the end user expression as a **_parameter_**. Each parameter has an entity type and that type dictates how the data is extracted. Parameters can also be used to reference an event. For example, if the parameter name is **duration,** the event name might be **alarm**. #### Voiceflow In Voiceflow, you can achieve the same outcome much more dynamically through variables. Whether capturing user responses or dynamically fulfilling content from web hooks or API calls, designers can use variables on canvas to quickly expand the assistant’s personalization and helpfulness. When considering how a Dialogflow ES parameter might be rebuilt within Voiceflow, we need to consider the two types: entity & static. **Entity parameters** are managed in our NLU Designer today. You’d set the Entity as ‘required’ and then create an entity re-prompt message in order to fill that entity as part of the conversational flow. This function would be useful for an {order} entity in which both **flavor** and **size** were required to move forward. **Static parameters** are best utilized as variables in Voiceflow. For example, you’d prompt the user for their coffee order and capture the user’s response. That captured response can then be mapped to a variable (e.g. {coffee}) and dynamically utilized throughout the conversation - at scale.  3) If Statements ---------------- #### Dialogflow Building IF statements in Dialogflow is not possible without writing code in the fulfillment. As an example, let’s say you need your Dialogflow Agent to answer differently when your business is open versus closed. If closed, we’d want to surface the next set of open hours and a way to get in touch. If open, we’d want to share the live ordering options. Below is an example of what this code block would look like as a function within the fulfillment section of your chatbot.  Similarly, IF statements are regularly used for new vs. returning customer designs. If we continued our coffees shop example in Dialogflow, we would need to write a function that checks if this user has opened a conversation previously or if they are new. The same objective is much faster to achieve using Voiceflow. #### Voiceflow **IF statements** in Voiceflow are used primarily as routers based on backend data. Each IF statement represents actual backend logic checks that need to happen throughout the conversation. This feature allows teams to build rich prototypes as the experience is as close to the production version as possible. We can take the same IF objective from above and build it in Voiceflow (below).  Here you can see we are using the IF condition block to determine what direction the conversation should go based on the value of the “sessions” variable. If **_sessions = 0,_** then the conversation will be routed to the _new user flow._ If **_sessions is > 0_**, then the conversation will be routed to the _return user flow_. For both production instances and prototyping purposes, Voiceflow’s method is much more efficient. Developer resources are not needed to hard code the design, while designers can work rapidly to create, test, and ship new flows. Better yet, when a project is ready for production, teams can publish directly to live agents (e.g. Dialogflow CX) or hand-off artifacts with all necessary code needed for implementation. It’s important for conversation designers, data scientists, and the entire conversational AI team to have a workflow that lets them ship contextual assistants at scale. As NLUs will continue to change, it’s even more essential to know that your team’s designs are agnostic - ready to plug into any NLU layer with nothing lost in translation. > Need to strategize your migration and CAI tech stack more? [Our team is here to chat.](https://voiceflow.com/demo) 1) Follow-up Intents -------------------- A **follow-up intent** adds a nested intent inside a parent intent. #### Dialogflow When using Dialogflow (a natural language understanding platform), you’d consider follow-up intents to be normal because they “hold” the context of the parent intent. There are many types of follow-up expressions, including repeat, confirm & cancel. **Let’s design a conversation in Dialogflow to illustrate.** **_Example 1:_** Your main intent was to order a caramel latte. A follow-up intent to this would be a secondary order confirmation. This seems simple enough. However, once we introduce complexity, building with nested context like this becomes quite cumbersome. For example, this same user wants a caramel latte, but before confirming, they add in, “I want to add whipped cream.”  Dialogflow ES custom intents **_Example 2:_** Now we have the main intent (order caramel latte), a follow-up intent to confirm, a second follow-up to add whipped cream, and an extra ‘yes’ confirmation intent to confirm the whipped cream.  Dialogflow ES nested intents As you can see, the more complex a conversation gets, the more CAI teams are required to nest contextual follow-ups. While follow-up intents seem straightforward when first designing, the larger and more complex a conversation gets, the harder this web of follow-up intents becomes to manage. Even more limiting, these nested intents are tied to their parent intent and don’t allow any flexible room for reusability. #### Voiceflow In Voiceflow, we still care about the context of the conversation, but we don’t need to keep track of follow-up intents in the same way. Every intent in Voiceflow is a parent level intent, which creates room for reusability. In order to account for contextual responses, Voiceflow also offers intent scoping. With a simple toggle, certain intents can only be accessible in a certain topic or globally available. It’s simple to reduce design bloat to a minimum. Designers have the ability to reference a reusable 'yes' intent wherever necessary and without having to create a net new 'yes' intent per main intent or node. The difference in mental models here is that instead of creating a net new yes intent for each main intent, one would re-use a system yes intent that has situational context baked in.  This applies to context specific yes intents that often have both an affirmation (such as “yes”) and a repetition of the main intent (”Yes, I would like to order coffee”). As you have the ability to select intent scoping based on each node, there is no concern about conflicting utterances. This all adds up to less latency, more efficiency and better reusability for designs. 2) Parameters ------------- **Parameters** are structured data that can be used to perform logic, a task, or generate a response. #### Dialogflow The concept of parameters in Dialogflow exists in order to provide the action value to your fulfillment web hook request or the API interaction response. It can be used, in one example, to trigger specific logic in your service. When an intent is matched at runtime, Dialogflow will give you the extracted value from the end user expression as a **_parameter_**. Each parameter has an entity type and that type dictates how the data is extracted. Parameters can also be used to reference an event. For example, if the parameter name is **duration,** the event name might be **alarm**. #### Voiceflow In Voiceflow, you can achieve the same outcome much more dynamically through variables. Whether capturing user responses or dynamically fulfilling content from web hooks or API calls, designers can use variables on canvas to quickly expand the assistant’s personalization and helpfulness. When considering how a Dialogflow ES parameter might be rebuilt within Voiceflow, we need to consider the two types: entity & static. **Entity parameters** are managed in our NLU Designer today. You’d set the Entity as ‘required’ and then create an entity re-prompt message in order to fill that entity as part of the conversational flow. This function would be useful for an {order} entity in which both **flavor** and **size** were required to move forward. **Static parameters** are best utilized as variables in Voiceflow. For example, you’d prompt the user for their coffee order and capture the user’s response. That captured response can then be mapped to a variable (e.g. {coffee}) and dynamically utilized throughout the conversation - at scale.  3) If Statements ---------------- #### Dialogflow Building IF statements in Dialogflow is not possible without writing code in the fulfillment. As an example, let’s say you need your Dialogflow Agent to answer differently when your business is open versus closed. If closed, we’d want to surface the next set of open hours and a way to get in touch. If open, we’d want to share the live ordering options. Below is an example of what this code block would look like as a function within the fulfillment section of your chatbot.  Similarly, IF statements are regularly used for new vs. returning customer designs. If we continued our coffees shop example in Dialogflow, we would need to write a function that checks if this user has opened a conversation previously or if they are new. The same objective is much faster to achieve using Voiceflow. #### Voiceflow **IF statements** in Voiceflow are used primarily as routers based on backend data. Each IF statement represents actual backend logic checks that need to happen throughout the conversation. This feature allows teams to build rich prototypes as the experience is as close to the production version as possible. We can take the same IF objective from above and build it in Voiceflow (below).  Here you can see we are using the IF condition block to determine what direction the conversation should go based on the value of the “sessions” variable. If **_sessions = 0,_** then the conversation will be routed to the _new user flow._ If **_sessions is > 0_**, then the conversation will be routed to the _return user flow_. For both production instances and prototyping purposes, Voiceflow’s method is much more efficient. Developer resources are not needed to hard code the design, while designers can work rapidly to create, test, and ship new flows. Better yet, when a project is ready for production, teams can publish directly to live agents (e.g. Dialogflow CX) or hand-off artifacts with all necessary code needed for implementation. It’s important for conversation designers, data scientists, and the entire conversational AI team to have a workflow that lets them ship contextual assistants at scale. As NLUs will continue to change, it’s even more essential to know that your team’s designs are agnostic - ready to plug into any NLU layer with nothing lost in translation. > Need to strategize your migration and CAI tech stack more? [Our team is here to chat.](https://voiceflow.com/demo)  RECOMMENDED [](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate#) PUBLISHED October 14, 2022 TOPICS IN THIS ARTICLE No items found. [\ \ Anna Rosen\ \ Enterprise Solutions Engineer at Voiceflow. at](https://www.voiceflow.com/contributors/anna-rosen) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### LLMs won’t replace NLUs. Here’s why](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) [](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [](https://www.voiceflow.com/pathways/the-ultimate-guide-to-vfv2-whats-new-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Voiceflow V2: The ultimate guide to what's new](https://www.voiceflow.com/pathways/the-ultimate-guide-to-vfv2-whats-new-with-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How To build a discord bot with Voiceflow | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) How To build a discord bot with Voiceflow ========================================= [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) Discord has evolved from a platform that was mostly popular with gamers and exploded into the mainstream as a place to house any kind of community. The platform has more than 150 million active users each month — [up from 56 million in 2019.](https://www.nytimes.com/2021/12/29/business/discord-server-social-media.html#:~:text=The%20platform%20has%20more%20than,young%20people%20are%20using%20Discord.) As with any community platform, automating the activation and engagement of users can help onboard new members to servers faster, help communicate updates quicker, and create a more inviting overall experience in each server you are a part of. Discord users are creating bots that welcome new users, bots that keep spammers out of servers, polling bots, music bots, security bots - anything you can think of, there is probably a Discord bot for that.... Or maybe there isn’t and you want to build it. Well you’re in luck, we’ll detail how to build a Discord bot for free using Voiceflow. Let’s dive in. 🚧 Before you start, make sure you've done these two things: 1\. Set up a [Discord bot on your Discord server. \ 2\.](https://www.freecodecamp.org/news/create-a-discord-bot-with-python/) Created a Voiceflow project. You need to first build a chat project on [Voiceflow.](https://creator.voiceflow.com/) .avif) Setup ----- To obtain your versionID you have to go to your Voiceflow Project:  ### _Find VersionID_ Then copy the VERSION\_ID from the URL in your address bar. When you are inside a Voiceflow project, your address bar should have a URL of the form: {% c-line %}https://creator.voiceflow.com/project/{VERSION\_ID}/...{% c-line-end %} ### apiKey To obtain the Workspace API Key we have to go to our workspace where we have created our General Project. After this, we have to append to the URL /api-keys:  Video Walkthrough Of Discord Integration ---------------------------------------- ### [Source Code From The Video](https://developer.voiceflow.com/docs/how-to-discord#source-code) Create Your Free Discord Bot Today ---------------------------------- Getting started with your first Discord bot is as simple as creating your free Voiceflow account and having a Discord server. Want to share your Discord bot with our community of developers? [Join the conversation here and share your Bot!](https://community.voiceflow.com/join?invitation_token=c3e1005bdc45460b4df56ccd5b9706020cc248bb-839e0042-e5a0-4ef0-80d0-a836269df56c) Want more developer documentation? Check out [our developer resources here!](https://www.voiceflow.com/developer) Setup ----- To obtain your versionID you have to go to your Voiceflow Project:  ### _Find VersionID_ Then copy the VERSION\_ID from the URL in your address bar. When you are inside a Voiceflow project, your address bar should have a URL of the form: {% c-line %}https://creator.voiceflow.com/project/{VERSION\_ID}/...{% c-line-end %} ### apiKey To obtain the Workspace API Key we have to go to our workspace where we have created our General Project. After this, we have to append to the URL /api-keys:  Video Walkthrough Of Discord Integration ---------------------------------------- ### [Source Code From The Video](https://developer.voiceflow.com/docs/how-to-discord#source-code) Create Your Free Discord Bot Today ---------------------------------- Getting started with your first Discord bot is as simple as creating your free Voiceflow account and having a Discord server. Want to share your Discord bot with our community of developers? [Join the conversation here and share your Bot!](https://community.voiceflow.com/join?invitation_token=c3e1005bdc45460b4df56ccd5b9706020cc248bb-839e0042-e5a0-4ef0-80d0-a836269df56c) Want more developer documentation? Check out [our developer resources here!](https://www.voiceflow.com/developer)  RECOMMENDED [](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow#) PUBLISHED March 10, 2022 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.avif)](https://www.voiceflow.com/pathways/how-to-leverage-topics-and-components-in-your-conversation-designs) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How to leverage topics and flows in your conversation designs](https://www.voiceflow.com/pathways/how-to-leverage-topics-and-components-in-your-conversation-designs) [](https://www.voiceflow.com/pathways/how-to-build-a-microsoft-teams-bot-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How to build a Microsoft Teams bot with Voiceflow](https://www.voiceflow.com/pathways/how-to-build-a-microsoft-teams-bot-with-voiceflow) [.avif)](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How To build a discord bot with Voiceflow](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Crawl, walk, run: 28+ tactics for evolving your AI agent | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Crawl, walk, run: 28+ tactics for evolving your AI agent ======================================================== [\ \ Daniel D'Souza\ \ Head of Growth at](https://www.voiceflow.com/contributors/daniel-dsouza) When it comes to AI agents, you've gotta learn to crawl before you can run. It can be so tempting to rush into things—you have big goals for your agent and a marathon of tasks. But it's critical to take a moment to envision your future path (Pathways pun unintended). Otherwise, you'll be stumbling around, trying to fix every single problem without making effective progress on your overall goal—to use AI to make things easier. At Voiceflow, I’ve worked with hundreds of companies—from Fortune 10 companies to medium-sized businesses—to help them find the right use cases for their AI agents. And depending on what stage of the journey they’re on, there are different areas of optimization. That’s why I’m sharing this massive list with you. In this article, we’ll define each stage of the AI support agent evolution and then share a list of use cases that will help you maximize your AI agent no matter if you’re crawling, walking, or running right now. You’ll learn how to get the most out of your simple AI agent, make your complex AI agent even more powerful—and, once you’ve achieved that—how to strive for industry-leading sophistication. Don’t forget to bookmark this one, because we’ve included loads of practical resources for how to implement the recommendations below.  Crawl—Your simple Q&A agent should be answering questions and automating basic tasks ------------------------------------------------------------------------------------ At the crawl phase, you already have or are in the process of implementing [a simple Q&A agent](https://www.voiceflow.com/blog/building-a-virtual-agent-from-scratch-start-here) . The best part of a simple Q&A agent is that it’s accessible to most businesses—even small and low-volume companies. Tools like [customGPT](https://customgpt.ai/) , [Microsoft CoPilo](https://copilot.microsoft.com/) t, [ChatBase](https://www.chatbase.co/) , and [GPT Builder](https://openai.com/blog/introducing-gpts) can help teams with minimal technical knowledge launch agents quickly. When done well, a simple Q&A agent can take a lot of tedious, manual tasks off your customer success (CS) team's hands and help customers get the answers they need quickly. ### How to get your Q&A agent to start answering questions To get started, [just upload a few documents and start asking questions](https://youtu.be/jWBkMZkDIYw?si=mpwG13MXEIgM2c9d&t=341) . Customers can use the agent on your website to find answers to frequently asked questions or to guide them through searches. In fact, I've noticed that [customers found more success](https://www.voiceflow.com/customer/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent) by embedding a simple Q&A agent within a customer's existing search flow using the [Voiceflow APIs](https://developer.voiceflow.com/docs) . Most AI tools have something called a Knowledge Base. This is how you upload your documents which are sourced by your AI assistant to answer user questions. It may seem like magic, but it is using a technology called [Retrieval Augmented Generation (RAG)](https://www.youtube.com/watch?v=Ui7XEI5CdgU&t=447s) . Understanding [how to format your data well](https://www.youtube.com/watch?v=oI9A84NBC3w) is the key to getting good responses.  How to add actions to your simple Q&A agent ------------------------------------------- There are loads of problems that a simple Q&A agent can solve for you. Automating tasks with a simple agent frees your CS, sales, and operations teams from menial tasks—expanding your simple AI agent has the potential to give everyone more of their valuable time back. Whether you're using an agent to route support tickets to the customer success team or even suggesting personalized content to users—here’s four areas we recommend focusing on to push a simple AI agent to its full potential. Remember, you don’t have to do everything on this list, but we recommend starting with customer support tasks (or those that will address high-volume requests) and moving down this list of priorities. ### 1\. Customer service and support * **Automating ticket submission**: Generate and submit support tickets automatically based on customer issues and queries. This could include categorizing queries, prioritizing the ticket based on urgency, and routing it to the right department or support agent. (Here’s a tutorial on how to [send a ticket to Zendesk](https://www.voiceflow.com/templates/zendesk-tickets) ). * **Collecting and processing feedback:** Use AI to [collect and categorize customer feedback](https://www.voiceflow.com/templates/ai-survey-bot) from your website or app. Automatically create follow-up actions or leads in your customer relationship management (CRM) based on user feedback with tools like [Zapier](https://www.voiceflow.com/templates/lead-gen-zapier) or using the API directly. ### 2\. Sales and marketing * **Generating, qualifying, and capturing leads:** Automatically create leads in a CRM from interactions on social media, email, website chatbots, and more. These can be based on specific triggers or interest shown by potential customers. Then capture basic customer information (e.g. name, email, interest) from a conversation and [add it as a new lead in your CRM](https://www.youtube.com/watch?v=nORaIEcD790) . * **Automating follow-ups and engagement**: Send personalized follow-up messages or emails to leads or customers based on their interaction history or behavior, encouraging them to take the next step in the sales funnel. Send a templated email to customers to confirm their inquiry or action (e.g. order confirmation, appointment booking, etc.) ### 3\. Operational efficiency * **Automating data entry tasks**: Use AI to update customer records in your database or CRM—from adding new contact details to existing user profiles or updating a customer's status based on their latest interaction. * **Automating routine tasks**: Automate scheduling appointments, send reminder emails, or update task statuses in project management tools through integrations with platforms like Zapier. ### 4\. Personalization and user engagement * **Personalizing content**: Suggest content, products, or services to users based on their previous interactions, queries, or preferences. **Automating event triggers**: Set up AI to trigger specific actions or notifications based on user behavior or milestones (e.g. sending a congratulatory message on a user's anniversary with the service, scheduling a reminder, or booking an appointment). Walk—Your complex Q&A agent should be leveraging customer data -------------------------------------------------------------- If you’re in the walking stage of AI maturity, you likely have more complex queries from users that require context and information. You already have a simple Q&A agent that you want to evolve to complete [more complex actions](https://www.youtube.com/watch?v=HHb1GJI7BtM) . You require integrations with existing databases, CRMs, and other third-party services to provide accurate and actionable responses from your agent. And you have the moderate technical ability on your teams to make it happen. This complex agent can complete simple actions ideal for mid-sized businesses or those with higher volume looking to automate a high percentage of their support queries.  **Here are three factors that set a complex agent apart from a simple one:** 1. **Deep data integration:** A complex assistant is adept at drawing from a wide array of data sources, including internal databases, CRM systems, and external APIs to gather all relevant customer information. That way, the agent understands the full context of each query, including past customer interactions and preferences. 2. **Intelligent query routing:** This assistant typically routes inquiries to the most appropriate pathway. Whether it requires specialized knowledge, access to specialized knowledge bases, or intervention by specific departments, a complex assistant ensures that each query is addressed by the most qualified entity. 3. **Ability to complete advanced actions:** Moving beyond basic actions, a complex assistant can perform sophisticated tasks that require multiple steps and verifications within business systems. Think multi-step account changes, or problem-solving support queries without human intervention.  How to optimize your complex AI agent based on your industry ------------------------------------------------------------ At this stage, you’ll want to tailor your approach based on your industry, since the areas of opportunity are so different. Below, we’ve honed in on software, telecommunications, retail, and financial services and shared a unique to-do list for each one. If you’re not in those industries, the use cases below still act as inspiration. Take what’s useful and leave what’s not. ### 1\. Software (SaaS) * **Diagnosing and troubleshooting user problems**: Use your AI assistant to pull up a customer's product usage or to provide personalized troubleshooting instructions based on their past issues logged in your CRM or product database. It can route the query to specialized support paths based on the product type or issue severity, using an internal knowledge base to guide the customer through a resolution. * **Managing tickets**: Generate detailed support tickets based on customer issues with relevant context pulled from various data sources. Use your AI agent to assign tickets to the most appropriate human support agents based on expertise or availability. * **Providing product information and updates**: Your AI agent can access up-to-date information to provide answers to product questions, suggest products that meet the customer's needs, or inform about the latest updates and how to apply them. * **Managing billing and subscriptions**: For questions or actions related to billing, surface specific billing details, explain charges, or modify subscription plans according to the customer's request.  ### 2\. Telecommunications * **Offering plan recommendations**: In a sophisticated Q&A system, your agent can suggest the most suitable mobile or broadband plans by analyzing user consumption patterns, available plans in the database, and current promotions. * **Resolving network issues**: By connecting your troubleshooting guide, network monitoring tools, and your AI agent, you can provide users with real-time solutions to connectivity issues, including personalized actions based on account information. * **Automating bill inquiries**: Answer complex billing questions by integrating your assistant with billing platforms to pull up individual customer histories, and explain charges or credits in detail. ### 3\. Retail * **Offering product recommendations**: Integrate your complex AI agent with your ecommerce platforms and customer purchase history databases to offer personalized product recommendations. * **Checking available inventory**: Allow customers to inquire about product availability in real-time by integrating with inventory management systems, including alternative suggestions if products are out of stock. * **Tracking order status**: Collect information on a customer's recent orders and generate a summary for them. Connect with tools like [Loop Returns](https://www.loopreturns.com/) and [Shopify](https://www.shopify.com/ca/orders) to get information on a customer's recent purchases and generate a status summary. ### 4\. Mid-sized banks and financial services * **Onboarding new digital accounts**: Guide customers through the process of submitting necessary documents, verifying their identity online, and setting up their new account without needing to visit a branch. Make it seamless for users to get started with your services. * **Offering financial health checkups**: Offer customers an automated financial health assessment tool that integrates with their account information and transaction history. By analyzing spending patterns, savings, investments, and financial goals, this system can provide personalized advice. Your AI assistant can answer complex questions about financial products that suit their needs, such as savings accounts with the best interest rates, credit cards with beneficial rewards, or investment products that match their risk tolerance. * **Monitoring and alerting users of fraud risk**: Integrate AI with your transaction processing systems and fraud detection algorithms. When potential fraud is detected, the system can automatically alert the customer via their preferred communication channel and then guide them through a verification process to confirm or deny the flagged transaction. If fraud is confirmed, the system can initiate a dispute resolution process, temporarily lock the account, or take other predefined actions to protect the customer's assets. Run—Your sophisticated AI agent should be completing increasingly complex actions --------------------------------------------------------------------------------- We all need something to strive for. A sophisticated agent goes beyond simple API calls and is able to receive and format data, as well as complete complex actions on behalf of the user. Examples of this include executing account changes on behalf of the user, triggering specific workflow within a product, and fixing user issues or executing a purchase on their behalf. Lately, our teams at Voiceflow have been experimenting with increasingly complex AI agents. We’re prototyping an assistant that downloads the user’s project file, reviews the code, identifies the issue, and provides a step-by-step report to fix the issue. The agent has also written code snippets for the user. We’re even exploring how the agent might execute those changes to the project itself on the Voiceflow platform. If you’re looking to create industry-leading AI agents, then the ideas below should be added to your product vision board. Needless to say, this kind of AI agent requires a high level of technical ability to build and iterate.  Dynamically and contextually solving problems with your AI agent ---------------------------------------------------------------- * **Dynamically offering services:** Tailoring service offerings based on the customer's usage patterns, preferences, or feedback. * **Managing complex accounts:** Your AI agent could handle multi-faceted requests such as merging accounts, adjusting service levels, or applying nuanced billing adjustments. * **Problem solving:** Troubleshooting and solving technical or service-related issues that require an understanding of the customer's specific setup, history, and requirements. * **Automating decision making:** Equipping your agent to make informed decisions on behalf of the business, such as approving discounts or customizing offers based on predefined criteria and customer data. * **Personalizing recommendations:** Generating tailored advice or suggestions based on a complex analysis of the customer's past interactions, preferences, and available options. * **Providing high-quality, contextual responses:** Consistently offer responses that are deeply informed by the customer's context. A level of service that mimics human understanding and intuition, thereby significantly enhancing customer satisfaction and engagement. Taking your AI agent from baby steps to marathons ------------------------------------------------- From simple to complex, AI agents have the potential to make our jobs easier by automating tasks and dynamically responding to customer requests. AI is evolving constantly, so our goal for this article is to help you envision how your agents could expand into the future. This isn’t an exhaustive list of possibilities. But regardless of which use cases you add to your wishlist, it’s always good practice to expand your agents with thoughtfulness and planning. Your agent can always do more, but AI teams are still made up of humans. Keep your capacity in mind as you expand. Currently, every AI tool still requires human oversight—we don’t advise letting your AI agent run off into the sunset just yet. Crawl—Your simple Q&A agent should be answering questions and automating basic tasks ------------------------------------------------------------------------------------ At the crawl phase, you already have or are in the process of implementing [a simple Q&A agent](https://www.voiceflow.com/blog/building-a-virtual-agent-from-scratch-start-here) . The best part of a simple Q&A agent is that it’s accessible to most businesses—even small and low-volume companies. Tools like [customGPT](https://customgpt.ai/) , [Microsoft CoPilo](https://copilot.microsoft.com/) t, [ChatBase](https://www.chatbase.co/) , and [GPT Builder](https://openai.com/blog/introducing-gpts) can help teams with minimal technical knowledge launch agents quickly. When done well, a simple Q&A agent can take a lot of tedious, manual tasks off your customer success (CS) team's hands and help customers get the answers they need quickly. ### How to get your Q&A agent to start answering questions To get started, [just upload a few documents and start asking questions](https://youtu.be/jWBkMZkDIYw?si=mpwG13MXEIgM2c9d&t=341) . Customers can use the agent on your website to find answers to frequently asked questions or to guide them through searches. In fact, I've noticed that [customers found more success](https://www.voiceflow.com/customer/how-esnipe-automated-70-of-their-help-center-tickets-with-an-ai-powered-search-agent) by embedding a simple Q&A agent within a customer's existing search flow using the [Voiceflow APIs](https://developer.voiceflow.com/docs) . Most AI tools have something called a Knowledge Base. This is how you upload your documents which are sourced by your AI assistant to answer user questions. It may seem like magic, but it is using a technology called [Retrieval Augmented Generation (RAG)](https://www.youtube.com/watch?v=Ui7XEI5CdgU&t=447s) . Understanding [how to format your data well](https://www.youtube.com/watch?v=oI9A84NBC3w) is the key to getting good responses.  How to add actions to your simple Q&A agent ------------------------------------------- There are loads of problems that a simple Q&A agent can solve for you. Automating tasks with a simple agent frees your CS, sales, and operations teams from menial tasks—expanding your simple AI agent has the potential to give everyone more of their valuable time back. Whether you're using an agent to route support tickets to the customer success team or even suggesting personalized content to users—here’s four areas we recommend focusing on to push a simple AI agent to its full potential. Remember, you don’t have to do everything on this list, but we recommend starting with customer support tasks (or those that will address high-volume requests) and moving down this list of priorities. ### 1\. Customer service and support * **Automating ticket submission**: Generate and submit support tickets automatically based on customer issues and queries. This could include categorizing queries, prioritizing the ticket based on urgency, and routing it to the right department or support agent. (Here’s a tutorial on how to [send a ticket to Zendesk](https://www.voiceflow.com/templates/zendesk-tickets) ). * **Collecting and processing feedback:** Use AI to [collect and categorize customer feedback](https://www.voiceflow.com/templates/ai-survey-bot) from your website or app. Automatically create follow-up actions or leads in your customer relationship management (CRM) based on user feedback with tools like [Zapier](https://www.voiceflow.com/templates/lead-gen-zapier) or using the API directly. ### 2\. Sales and marketing * **Generating, qualifying, and capturing leads:** Automatically create leads in a CRM from interactions on social media, email, website chatbots, and more. These can be based on specific triggers or interest shown by potential customers. Then capture basic customer information (e.g. name, email, interest) from a conversation and [add it as a new lead in your CRM](https://www.youtube.com/watch?v=nORaIEcD790) . * **Automating follow-ups and engagement**: Send personalized follow-up messages or emails to leads or customers based on their interaction history or behavior, encouraging them to take the next step in the sales funnel. Send a templated email to customers to confirm their inquiry or action (e.g. order confirmation, appointment booking, etc.) ### 3\. Operational efficiency * **Automating data entry tasks**: Use AI to update customer records in your database or CRM—from adding new contact details to existing user profiles or updating a customer's status based on their latest interaction. * **Automating routine tasks**: Automate scheduling appointments, send reminder emails, or update task statuses in project management tools through integrations with platforms like Zapier. ### 4\. Personalization and user engagement * **Personalizing content**: Suggest content, products, or services to users based on their previous interactions, queries, or preferences. **Automating event triggers**: Set up AI to trigger specific actions or notifications based on user behavior or milestones (e.g. sending a congratulatory message on a user's anniversary with the service, scheduling a reminder, or booking an appointment). Walk—Your complex Q&A agent should be leveraging customer data -------------------------------------------------------------- If you’re in the walking stage of AI maturity, you likely have more complex queries from users that require context and information. You already have a simple Q&A agent that you want to evolve to complete [more complex actions](https://www.youtube.com/watch?v=HHb1GJI7BtM) . You require integrations with existing databases, CRMs, and other third-party services to provide accurate and actionable responses from your agent. And you have the moderate technical ability on your teams to make it happen. This complex agent can complete simple actions ideal for mid-sized businesses or those with higher volume looking to automate a high percentage of their support queries.  **Here are three factors that set a complex agent apart from a simple one:** 1. **Deep data integration:** A complex assistant is adept at drawing from a wide array of data sources, including internal databases, CRM systems, and external APIs to gather all relevant customer information. That way, the agent understands the full context of each query, including past customer interactions and preferences. 2. **Intelligent query routing:** This assistant typically routes inquiries to the most appropriate pathway. Whether it requires specialized knowledge, access to specialized knowledge bases, or intervention by specific departments, a complex assistant ensures that each query is addressed by the most qualified entity. 3. **Ability to complete advanced actions:** Moving beyond basic actions, a complex assistant can perform sophisticated tasks that require multiple steps and verifications within business systems. Think multi-step account changes, or problem-solving support queries without human intervention.  How to optimize your complex AI agent based on your industry ------------------------------------------------------------ At this stage, you’ll want to tailor your approach based on your industry, since the areas of opportunity are so different. Below, we’ve honed in on software, telecommunications, retail, and financial services and shared a unique to-do list for each one. If you’re not in those industries, the use cases below still act as inspiration. Take what’s useful and leave what’s not. ### 1\. Software (SaaS) * **Diagnosing and troubleshooting user problems**: Use your AI assistant to pull up a customer's product usage or to provide personalized troubleshooting instructions based on their past issues logged in your CRM or product database. It can route the query to specialized support paths based on the product type or issue severity, using an internal knowledge base to guide the customer through a resolution. * **Managing tickets**: Generate detailed support tickets based on customer issues with relevant context pulled from various data sources. Use your AI agent to assign tickets to the most appropriate human support agents based on expertise or availability. * **Providing product information and updates**: Your AI agent can access up-to-date information to provide answers to product questions, suggest products that meet the customer's needs, or inform about the latest updates and how to apply them. * **Managing billing and subscriptions**: For questions or actions related to billing, surface specific billing details, explain charges, or modify subscription plans according to the customer's request.  ### 2\. Telecommunications * **Offering plan recommendations**: In a sophisticated Q&A system, your agent can suggest the most suitable mobile or broadband plans by analyzing user consumption patterns, available plans in the database, and current promotions. * **Resolving network issues**: By connecting your troubleshooting guide, network monitoring tools, and your AI agent, you can provide users with real-time solutions to connectivity issues, including personalized actions based on account information. * **Automating bill inquiries**: Answer complex billing questions by integrating your assistant with billing platforms to pull up individual customer histories, and explain charges or credits in detail. ### 3\. Retail * **Offering product recommendations**: Integrate your complex AI agent with your ecommerce platforms and customer purchase history databases to offer personalized product recommendations. * **Checking available inventory**: Allow customers to inquire about product availability in real-time by integrating with inventory management systems, including alternative suggestions if products are out of stock. * **Tracking order status**: Collect information on a customer's recent orders and generate a summary for them. Connect with tools like [Loop Returns](https://www.loopreturns.com/) and [Shopify](https://www.shopify.com/ca/orders) to get information on a customer's recent purchases and generate a status summary. ### 4\. Mid-sized banks and financial services * **Onboarding new digital accounts**: Guide customers through the process of submitting necessary documents, verifying their identity online, and setting up their new account without needing to visit a branch. Make it seamless for users to get started with your services. * **Offering financial health checkups**: Offer customers an automated financial health assessment tool that integrates with their account information and transaction history. By analyzing spending patterns, savings, investments, and financial goals, this system can provide personalized advice. Your AI assistant can answer complex questions about financial products that suit their needs, such as savings accounts with the best interest rates, credit cards with beneficial rewards, or investment products that match their risk tolerance. * **Monitoring and alerting users of fraud risk**: Integrate AI with your transaction processing systems and fraud detection algorithms. When potential fraud is detected, the system can automatically alert the customer via their preferred communication channel and then guide them through a verification process to confirm or deny the flagged transaction. If fraud is confirmed, the system can initiate a dispute resolution process, temporarily lock the account, or take other predefined actions to protect the customer's assets. Run—Your sophisticated AI agent should be completing increasingly complex actions --------------------------------------------------------------------------------- We all need something to strive for. A sophisticated agent goes beyond simple API calls and is able to receive and format data, as well as complete complex actions on behalf of the user. Examples of this include executing account changes on behalf of the user, triggering specific workflow within a product, and fixing user issues or executing a purchase on their behalf. Lately, our teams at Voiceflow have been experimenting with increasingly complex AI agents. We’re prototyping an assistant that downloads the user’s project file, reviews the code, identifies the issue, and provides a step-by-step report to fix the issue. The agent has also written code snippets for the user. We’re even exploring how the agent might execute those changes to the project itself on the Voiceflow platform. If you’re looking to create industry-leading AI agents, then the ideas below should be added to your product vision board. Needless to say, this kind of AI agent requires a high level of technical ability to build and iterate.  Dynamically and contextually solving problems with your AI agent ---------------------------------------------------------------- * **Dynamically offering services:** Tailoring service offerings based on the customer's usage patterns, preferences, or feedback. * **Managing complex accounts:** Your AI agent could handle multi-faceted requests such as merging accounts, adjusting service levels, or applying nuanced billing adjustments. * **Problem solving:** Troubleshooting and solving technical or service-related issues that require an understanding of the customer's specific setup, history, and requirements. * **Automating decision making:** Equipping your agent to make informed decisions on behalf of the business, such as approving discounts or customizing offers based on predefined criteria and customer data. * **Personalizing recommendations:** Generating tailored advice or suggestions based on a complex analysis of the customer's past interactions, preferences, and available options. * **Providing high-quality, contextual responses:** Consistently offer responses that are deeply informed by the customer's context. A level of service that mimics human understanding and intuition, thereby significantly enhancing customer satisfaction and engagement. Taking your AI agent from baby steps to marathons ------------------------------------------------- From simple to complex, AI agents have the potential to make our jobs easier by automating tasks and dynamically responding to customer requests. AI is evolving constantly, so our goal for this article is to help you envision how your agents could expand into the future. This isn’t an exhaustive list of possibilities. But regardless of which use cases you add to your wishlist, it’s always good practice to expand your agents with thoughtfulness and planning. Your agent can always do more, but AI teams are still made up of humans. Keep your capacity in mind as you expand. Currently, every AI tool still requires human oversight—we don’t advise letting your AI agent run off into the sunset just yet.  RECOMMENDED [\ \ ### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) PUBLISHED March 7, 2024 TOPICS IN THIS ARTICLE Exceptional work [\ \ Daniel D'Souza\ \ Head of Growth at\ \ Leading growth at Voiceflow.](https://www.voiceflow.com/contributors/daniel-dsouza) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/dominant-discourse-llms) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Why the dominant discourse surrounding LLMs needs to change](https://www.voiceflow.com/pathways/dominant-discourse-llms) [](https://www.voiceflow.com/pathways/using-machine-learning-to-analyze-customer-support-conversations) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Using machine learning to analyze customer support conversations](https://www.voiceflow.com/pathways/using-machine-learning-to-analyze-customer-support-conversations) [%25201.avif)](https://www.voiceflow.com/pathways/jumping-off-the-ai-hype-train-nlus-in-an-llm-dominated-world) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Jumping off the AI hype train: NLUs in an LLM-dominated world](https://www.voiceflow.com/pathways/jumping-off-the-ai-hype-train-nlus-in-an-llm-dominated-world) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/crawl-walk-run-28-tactics-for-evolving-your-ai-agent#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # From microbiology to mental health: In conversation with Ebere Wilson | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) From microbiology to mental health: In conversation with Ebere Wilson ===================================================================== [\ \ Kristin\ \ Editor-in-Chief at Pathways](https://www.voiceflow.com/contributors/kristin) In the ever-evolving world of conversation design (CxD), there’s no shortage of incredible people making incredible things. We've made it our mission to have more conversations with these folks—sitting down to understand what landed them in CxD, how they make sense of the chaos, and how their journeys could inspire your own. Our first conversation is with the self-taught, multidisciplinary savant Ebere Wilson, a product designer based in Nigeria who began her conversation design career only last year. After studying microbiology, Ebere planned to go into the brewery business. But a serendipitous turn of events led her to try her hand at product design and then design her conversation assistant, Ava. Created to be the main feature of a mental health app, through conversation prompts, Ava gives users the space to express what's on their minds and offers comfort and resources. In our conversation, Ebere recounts her winding path to conversation design (and why that's a good thing), the opportunities of cross-pollination between AI and healthcare tech, and her advice for emerging CxDs. So, grab a cup of your favorite warm beverage and enjoy the banter.  KH: Tell me about your journey to conversation design. -------------------------------------------------------- EW: Becoming a designer was never part of the plan. When I got into university my goal was to study microbiology with a focus on the brewery industry. I loved the whole concept of making alcohol with bacteria and fungi. While I was studying, I got a job as a music scheduler at a radio station. My job was to “clean” music of vulgar words before they aired. I got tired of how repetitive the work was, so I taught myself Python. Eventually, a software engineer friend asked me if I’d considered pursuing product design. I started looking into it and discovered how multi-faceted design could be (and much more fun than cleaning songs). Siri is what got me initially interested in CxD. I was interacting with her and began questioning how she works and who’s behind her. She also frustrated me at times—many chat assistants' responses are off-putting or frustratingly off-topic. When I began to research Siri, I got caught up in the web of conversation design—building realistic, useful assistants that could mimic real human conversation. That’s when I found the [Daily Conversation Design Challenge](https://uxcontent.com/14-days-daily-conversation-design-challenge/) with Hillary Black and decided to start on my own project. The rest is history. Why did you create a mental health conversation assistant? ----------------------------------------------------------- As humans, we don't like feeling like burdens to our loved ones. We don’t want to bring too many problems to them. Even our friends might get tired of hearing about the same issues over and over. But not a chatbot. One of the limitations of this technology is that it's not a diagnostic tool. It's not going to cure my mental issues. But if I need someone to talk to who will never tire of our conversations or make me feel like a bother—that’s what Ava can do. The goal of using chat assistants for people experiencing mental health issues is to help users understand and manage their mental state.   What was your process for building Ava? ---------------------------------------- In Nigeria we say ‘there's smoke coming out of my head’ to mean we’re reading incessantly. During that project, I was doing tons of research on chatbots and AI—going down one rabbit hole to the other. Because of my experience in product design, I did a competitive analysis of chat assistants and the role that conversation design plays in the mental health industry. I discovered that the main issue users have with mental health chatbots is that they lose the focus of a conversation. I would tell a bot what I'm going through—anxious feelings, for instance—and it would pick up on a phrase and start talking about something seemingly related, but not particularly helpful. I wanted Ava to be different. So I went to work on my sample flow and conversation scripts. I also created a personality for my bot. I didn't want it to be genderless, I wanted her to feel relatable. Because I was teaching myself everything, I had issues understanding how to build proper intents and account for utterances. But I watched [Voiceflow tutorials](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) and read a lot of Medium articles to get a better grasp on CxD concepts. After all my research, creating the actual prototype only took me a day. I'm very quick when it comes to learning new things. How could Ava support those experiencing mental health issues? How do users react to her? ------------------------------------------------------------------------------------------- Ava focuses on what you’re going through at that moment and gives you counsel. She will “check in” on users in three scenarios: 1. Ava drops a “fun fact”. Her prompt of a funny video or image can lead to a wider conversation about the user’s current mental state. 2. Ava prompts you by asking about your day, to which the user will respond with a reflection. 3. Ava checks in on you to continue a previous conversation you both had. She connects a user’s current mental state to previous utterances. This scenario is part of Ava’s development roadmap. The best part of this project has been hearing about how people feel when they talk to Ava. My friends, colleagues, and family have all loved using it. One friend couldn’t believe it was a prototype and he wanted to have more conversations with her. Another friend said that Ava was already better than the current mental health chatbot he downloaded from the app store. Even Hillary Black said it's a very valuable product. So, the positive responses have led me to begin building an MVP that will cater to Nigerians—for now. _\[An example of a conversation between a user and Ava:_\ \ * User: Hi Ava\ * Ava: Hi Blessing, how are you doing today?\ * User: I feel so bad. I'm having issues with work. \ * Ava: Okay, I'm sorry about that. What’s bothering you? \ * User: I’m feeling burnout at work, my boss doesn't listen to my ideas, and it’s really stressful \ * Ava: That sounds really difficult. Would you like some resources that might help you manage work stress? \ * User: Yes\ * _\[Ava shares mental health resources relevant to work stress\]_ \ * _\[The following day Ava checks in\]_ Hi, Blessing, I hope you're feeling better about work stress. How has today been?\]  How do you think conversation design will impact healthcare tech and vice versa? --------------------------------------------------------------------------------- Mental health is a very complex issue. It should be treated in several ways including medications and therapy. Conversation design can be an effective tool for healthcare professionals to offer their patients as they manage their mental health issues. Virtual assistants can be programmed to provide support, resources, and information. They will never be a complete solution, but they can offer support where our overburdened healthcare systems may not be able to in our everyday lives. It’s also true that assistants struggle to detect when a user needs to talk to a real human. That’s why on my app there will be an SOS button to get in touch with a therapist or crisis hotline. On the other hand, healthcare tech has the incredible potential to contribute to [large language models](https://www.voiceflow.com/blog/llms-wont-replace-nlus) (LLM) and personalization. With a language model, conversations can be used to generate empathetic responses. These responses help users to feel heard and understood. Using the anonymized mental health data that Ava collects, we could help these LLMs. Eventually, these conversations could be fed back to the model to make future health assistants better and better over time.  Do you have advice for emerging conversation designers? -------------------------------------------------------- Keep learning. I'm only two years into product design. Now imagine adding conversation design to it! I still feel there's a lot to learn. My second piece of advice is to bring your questions to industry leaders and mentors. I'm already in two communities, the [Daily Conversation Design Challenge](https://uxcontent.com/14-days-daily-conversation-design-challenge/) and [Women in Voice](https://womeninvoice.org/) . You'll find there are people who are willing to help. Nobody's withholding answers, you just have to ask questions. How did Voiceflow help you with building this project? ------------------------------------------------------ I started with a proposal—what Ava does, the project’s limitations, and goals. But later I realized it wasn't enough to capture the vision. There was a lot of functionality that I was missing without a working prototype. I started scribbling, writing, making sketches, and then went back to researching AI. That’s when I discovered Voiceflow and decided to build my prototype there. I couldn’t have completed the prototype without the Voiceflow platform. Because of that prototype I can share Ava with colleagues, receive instant feedback, and continue to iterate her conversational abilities until I build the app.  Voiceflow project Ebere and Ava—The future of mental health support -------------------------------------------------- For Ebere, there’s been no shortage of applications for conversation design in the healthcare space. As she continues to extend the flows of her Ava prototype, Ebere already has her sights set on her next project—an assistant that will support people fighting malaria. In the meantime, Ebere keeps endlessly occupied with freelancing, running her NGO supporting hospital patients, and writing poetry. It all goes back to her learning mindset, “Having varied interests makes you a better designer and a better person. You can see connections in the world that other people can’t. In a world where everyone wants me to specialize, I refuse.” Connect with Ebere Wilson on [LinkedIn.](https://www.linkedin.com/in/eberewilson/) KH: Tell me about your journey to conversation design. -------------------------------------------------------- EW: Becoming a designer was never part of the plan. When I got into university my goal was to study microbiology with a focus on the brewery industry. I loved the whole concept of making alcohol with bacteria and fungi. While I was studying, I got a job as a music scheduler at a radio station. My job was to “clean” music of vulgar words before they aired. I got tired of how repetitive the work was, so I taught myself Python. Eventually, a software engineer friend asked me if I’d considered pursuing product design. I started looking into it and discovered how multi-faceted design could be (and much more fun than cleaning songs). Siri is what got me initially interested in CxD. I was interacting with her and began questioning how she works and who’s behind her. She also frustrated me at times—many chat assistants' responses are off-putting or frustratingly off-topic. When I began to research Siri, I got caught up in the web of conversation design—building realistic, useful assistants that could mimic real human conversation. That’s when I found the [Daily Conversation Design Challenge](https://uxcontent.com/14-days-daily-conversation-design-challenge/) with Hillary Black and decided to start on my own project. The rest is history. Why did you create a mental health conversation assistant? ----------------------------------------------------------- As humans, we don't like feeling like burdens to our loved ones. We don’t want to bring too many problems to them. Even our friends might get tired of hearing about the same issues over and over. But not a chatbot. One of the limitations of this technology is that it's not a diagnostic tool. It's not going to cure my mental issues. But if I need someone to talk to who will never tire of our conversations or make me feel like a bother—that’s what Ava can do. The goal of using chat assistants for people experiencing mental health issues is to help users understand and manage their mental state.   What was your process for building Ava? ---------------------------------------- In Nigeria we say ‘there's smoke coming out of my head’ to mean we’re reading incessantly. During that project, I was doing tons of research on chatbots and AI—going down one rabbit hole to the other. Because of my experience in product design, I did a competitive analysis of chat assistants and the role that conversation design plays in the mental health industry. I discovered that the main issue users have with mental health chatbots is that they lose the focus of a conversation. I would tell a bot what I'm going through—anxious feelings, for instance—and it would pick up on a phrase and start talking about something seemingly related, but not particularly helpful. I wanted Ava to be different. So I went to work on my sample flow and conversation scripts. I also created a personality for my bot. I didn't want it to be genderless, I wanted her to feel relatable. Because I was teaching myself everything, I had issues understanding how to build proper intents and account for utterances. But I watched [Voiceflow tutorials](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) and read a lot of Medium articles to get a better grasp on CxD concepts. After all my research, creating the actual prototype only took me a day. I'm very quick when it comes to learning new things. How could Ava support those experiencing mental health issues? How do users react to her? ------------------------------------------------------------------------------------------- Ava focuses on what you’re going through at that moment and gives you counsel. She will “check in” on users in three scenarios: 1. Ava drops a “fun fact”. Her prompt of a funny video or image can lead to a wider conversation about the user’s current mental state. 2. Ava prompts you by asking about your day, to which the user will respond with a reflection. 3. Ava checks in on you to continue a previous conversation you both had. She connects a user’s current mental state to previous utterances. This scenario is part of Ava’s development roadmap. The best part of this project has been hearing about how people feel when they talk to Ava. My friends, colleagues, and family have all loved using it. One friend couldn’t believe it was a prototype and he wanted to have more conversations with her. Another friend said that Ava was already better than the current mental health chatbot he downloaded from the app store. Even Hillary Black said it's a very valuable product. So, the positive responses have led me to begin building an MVP that will cater to Nigerians—for now. _\[An example of a conversation between a user and Ava:_\ \ * User: Hi Ava\ * Ava: Hi Blessing, how are you doing today?\ * User: I feel so bad. I'm having issues with work. \ * Ava: Okay, I'm sorry about that. What’s bothering you? \ * User: I’m feeling burnout at work, my boss doesn't listen to my ideas, and it’s really stressful \ * Ava: That sounds really difficult. Would you like some resources that might help you manage work stress? \ * User: Yes\ * _\[Ava shares mental health resources relevant to work stress\]_ \ * _\[The following day Ava checks in\]_ Hi, Blessing, I hope you're feeling better about work stress. How has today been?\]  How do you think conversation design will impact healthcare tech and vice versa? --------------------------------------------------------------------------------- Mental health is a very complex issue. It should be treated in several ways including medications and therapy. Conversation design can be an effective tool for healthcare professionals to offer their patients as they manage their mental health issues. Virtual assistants can be programmed to provide support, resources, and information. They will never be a complete solution, but they can offer support where our overburdened healthcare systems may not be able to in our everyday lives. It’s also true that assistants struggle to detect when a user needs to talk to a real human. That’s why on my app there will be an SOS button to get in touch with a therapist or crisis hotline. On the other hand, healthcare tech has the incredible potential to contribute to [large language models](https://www.voiceflow.com/blog/llms-wont-replace-nlus) (LLM) and personalization. With a language model, conversations can be used to generate empathetic responses. These responses help users to feel heard and understood. Using the anonymized mental health data that Ava collects, we could help these LLMs. Eventually, these conversations could be fed back to the model to make future health assistants better and better over time.  Do you have advice for emerging conversation designers? -------------------------------------------------------- Keep learning. I'm only two years into product design. Now imagine adding conversation design to it! I still feel there's a lot to learn. My second piece of advice is to bring your questions to industry leaders and mentors. I'm already in two communities, the [Daily Conversation Design Challenge](https://uxcontent.com/14-days-daily-conversation-design-challenge/) and [Women in Voice](https://womeninvoice.org/) . You'll find there are people who are willing to help. Nobody's withholding answers, you just have to ask questions. How did Voiceflow help you with building this project? ------------------------------------------------------ I started with a proposal—what Ava does, the project’s limitations, and goals. But later I realized it wasn't enough to capture the vision. There was a lot of functionality that I was missing without a working prototype. I started scribbling, writing, making sketches, and then went back to researching AI. That’s when I discovered Voiceflow and decided to build my prototype there. I couldn’t have completed the prototype without the Voiceflow platform. Because of that prototype I can share Ava with colleagues, receive instant feedback, and continue to iterate her conversational abilities until I build the app.  Voiceflow project Ebere and Ava—The future of mental health support -------------------------------------------------- For Ebere, there’s been no shortage of applications for conversation design in the healthcare space. As she continues to extend the flows of her Ava prototype, Ebere already has her sights set on her next project—an assistant that will support people fighting malaria. In the meantime, Ebere keeps endlessly occupied with freelancing, running her NGO supporting hospital patients, and writing poetry. It all goes back to her learning mindset, “Having varied interests makes you a better designer and a better person. You can see connections in the world that other people can’t. In a world where everyone wants me to specialize, I refuse.” Connect with Ebere Wilson on [LinkedIn.](https://www.linkedin.com/in/eberewilson/)  RECOMMENDED [\ \ ### LLMs won’t replace NLUs. Here’s why](https://www.voiceflow.com/pathways/llms-wont-replace-nlus) PUBLISHED May 17, 2023 TOPICS IN THIS ARTICLE Exceptional people Exceptional work [\ \ Kristin\ \ Editor-in-Chief at Pathways\ \ As editor-in-chief, Kristin is the creative puppet master behind every story on Pathways. From InVision to OpenView, you can thank her for pushing the boundaries of how we talk about B2B tech, and now, how we propel conversational AI forward.](https://www.voiceflow.com/contributors/kristin) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/celene-osiecka-blog-post) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to design intelligent conversational interfaces](https://www.voiceflow.com/pathways/celene-osiecka-blog-post) [](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Experts share 3 expectations about LLMs to set with business stakeholders](https://www.voiceflow.com/pathways/expectations-llms-business-stakeholders) [](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### A bit of entity advice: Delight your overfillers](https://www.voiceflow.com/pathways/a-bit-of-entity-advice-delight-your-overfillers) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/future-mental-health-support#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/future-mental-health-support#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Voiceflow brings visual prototyping & enhanced test navigation to tool | Pathways [](https://www.voiceflow.com/pathways/manual-navigation-visuals#) Voiceflow brings visual prototyping & enhanced test navigation to tool ====================================================================== [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at](https://www.voiceflow.com/contributors/mark-ammendolia) Take your voice experience to the next level with extensive and efficient voice testing.  Dynamic prototyping at your fingertips -------------------------------------- Testing your voice experience represents an essential step when designing conversations. You can evaluate your voice design from different angles, pinpoint flaws and inefficiencies, and determine whether it functions as intended. Whether you're running user testing with customers or reviews with stakeholders, frequent testing gives you a clearer sense of your design's direction and scalability and ensures the reliability of the application you are building. It saves you time, money, and extensive maintenance after deployment. But above all else, it helps establish confidence in your experience so users keep returning for more. Outside of the value it promises, the success of any voice experience is gauged by its quality, ease-of-use, and customer reliability. At Voiceflow, we're determined to build upon our existing prototyping functionality to give you the toolkit you need to run extensive and efficient voice testing. That's why we're excited to introduce manual navigation and multimodal visuals within our test tool. Introducing Manual Navigation & Multimodal Visuals -------------------------------------------------- #### Manual Navigation Voice testing is a process. There may be multiple paths you want to test, different slot values you'd like to try, or different intents you'd like to match. These all lead to the inevitable: restarting your experience from the start to try and achieve the desired result. With the ability to start a test from any block, we've made it easier to test the part of the conversation you're focused on at any moment, so you can quickly jump in and test before continuing on with your design work. We want to empower designers, developers, and creators to prototype faster and more efficiently while getting the user experience right. With new manual controls, freely navigate forward and backward to specific parts of your voice design to give you more granular control over the entire test experience. This also opens the door for future enhancements such as moderated tests with customers. We want to give you the power to iterate faster, and so we're excited to release a robust prototyping process that will help push your designs from testing to deployment. #### Multimodal Visuals At Voiceflow, we want to stand behind a tool that encompasses all aspects of conversation design, including the rising demand for multimodal experiences. This is apparent now more than ever, with the wide-spread adoption and advancement of APL — Amazon's voice-first design language. APL allows you to create rich, interactive displays for Alexa skills so you can tailor your voice experience for all sorts of Alexa-enabled devices.  Voiceflow now makes it easy to prototype display elements in the test tool with the release of multimodal visuals. Now you can easily design and test for displays of varying sizes that help deepen the context of your voice experience. With multimodal visuals, get ready to experiment with graphics, images, and videos while building for a wide-array of Alexa devices like Amazon Echo Show and Fire TV. Multimodal visuals is currently only available within Amazon Alexa projects. This functionality will be accessible across all channels in the coming months - so stay tuned for more updates! What to Expect: --------------- #### Manual Navigation * Begin a test from any block on the canvas * Navigate backwards or forwards to key points in your design with ease * More granular control of tests * Boosts prototyping speed #### Multimodal Visuals * Unlocks visual multimodal testing for Alexa * Add graphics, images and videos using the display and card blocks * Build and test for popular Alexa-enabled devices like Amazon Echo Show * Examine visuals from a user's perspective with device mockup templates [](https://www.youtube.com/watch?v=NpUyIvg1dJs) Using this features ------------------- You can access these features by navigating to the test button in the top right-hand corner of the Creator Tool. > **Want to learn more?** > Please visit our [comprehensive changelog](https://www.notion.so/voiceflow/Voiceflow-Changelog-b5e32e269b204106b5b51014cd049346) > for more information on this release. Dynamic prototyping at your fingertips -------------------------------------- Testing your voice experience represents an essential step when designing conversations. You can evaluate your voice design from different angles, pinpoint flaws and inefficiencies, and determine whether it functions as intended. Whether you're running user testing with customers or reviews with stakeholders, frequent testing gives you a clearer sense of your design's direction and scalability and ensures the reliability of the application you are building. It saves you time, money, and extensive maintenance after deployment. But above all else, it helps establish confidence in your experience so users keep returning for more. Outside of the value it promises, the success of any voice experience is gauged by its quality, ease-of-use, and customer reliability. At Voiceflow, we're determined to build upon our existing prototyping functionality to give you the toolkit you need to run extensive and efficient voice testing. That's why we're excited to introduce manual navigation and multimodal visuals within our test tool. Introducing Manual Navigation & Multimodal Visuals -------------------------------------------------- #### Manual Navigation Voice testing is a process. There may be multiple paths you want to test, different slot values you'd like to try, or different intents you'd like to match. These all lead to the inevitable: restarting your experience from the start to try and achieve the desired result. With the ability to start a test from any block, we've made it easier to test the part of the conversation you're focused on at any moment, so you can quickly jump in and test before continuing on with your design work. We want to empower designers, developers, and creators to prototype faster and more efficiently while getting the user experience right. With new manual controls, freely navigate forward and backward to specific parts of your voice design to give you more granular control over the entire test experience. This also opens the door for future enhancements such as moderated tests with customers. We want to give you the power to iterate faster, and so we're excited to release a robust prototyping process that will help push your designs from testing to deployment. #### Multimodal Visuals At Voiceflow, we want to stand behind a tool that encompasses all aspects of conversation design, including the rising demand for multimodal experiences. This is apparent now more than ever, with the wide-spread adoption and advancement of APL — Amazon's voice-first design language. APL allows you to create rich, interactive displays for Alexa skills so you can tailor your voice experience for all sorts of Alexa-enabled devices.  Voiceflow now makes it easy to prototype display elements in the test tool with the release of multimodal visuals. Now you can easily design and test for displays of varying sizes that help deepen the context of your voice experience. With multimodal visuals, get ready to experiment with graphics, images, and videos while building for a wide-array of Alexa devices like Amazon Echo Show and Fire TV. Multimodal visuals is currently only available within Amazon Alexa projects. This functionality will be accessible across all channels in the coming months - so stay tuned for more updates! What to Expect: --------------- #### Manual Navigation * Begin a test from any block on the canvas * Navigate backwards or forwards to key points in your design with ease * More granular control of tests * Boosts prototyping speed #### Multimodal Visuals * Unlocks visual multimodal testing for Alexa * Add graphics, images and videos using the display and card blocks * Build and test for popular Alexa-enabled devices like Amazon Echo Show * Examine visuals from a user's perspective with device mockup templates [](https://www.youtube.com/watch?v=NpUyIvg1dJs) Using this features ------------------- You can access these features by navigating to the test button in the top right-hand corner of the Creator Tool. > **Want to learn more?** > Please visit our [comprehensive changelog](https://www.notion.so/voiceflow/Voiceflow-Changelog-b5e32e269b204106b5b51014cd049346) > for more information on this release.  RECOMMENDED [](https://www.voiceflow.com/pathways/manual-navigation-visuals#) PUBLISHED March 31, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Mark Ammendolia\ \ Senior Manager, Creative Marketing & Digital Strategy at\ \ Mark is a Pathways contributor.](https://www.voiceflow.com/contributors/mark-ammendolia) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH No items found. [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/manual-navigation-visuals#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/manual-navigation-visuals#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How Rocket Mortgage Accelerated their Team’s Conversation Design Workflow | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) How Rocket Mortgage Accelerated their Team’s Conversation Design Workflow ========================================================================= [\ \ Joanne Szeto\ \ Growth Co-Op, Voiceflow for Students at](https://www.voiceflow.com/contributors/joanne-szeto) > **⛔ Problem: The team started with a challenge that many other conversational AI teams do: _how do we manage our content better?_** > **✅ Solution: Working on the Voiceflow canvas today, Collin has a greater sense of freedom when it comes to scaling his team's work.** [Collin Carnahan](https://www.linkedin.com/in/ACoAABix5lQBJDIXiHVMgHZLArBfexCY5DIawcA) , Senior AI-ML Conversational UX Designer at Rocket Mortgage knew his team needed to scale their workflow. Providing exceptional customer service is a focus of the business and Collin’s team was integral in meeting those high expectations. He was looking for a way to scale his chatbot while making sure non-technical users could easily manage their content.  The team’s existing process caused 3 main pain points: ### Limited content management Collin was using Excel, Word, Lucid Chart, or Figma but quickly realized that these were not optimal for conversation design. He could "fake conversation design" but the tools were limiting the experiences that he wanted to design complex flows and create high-fidelity prototypes. For example, Figma didn’t have the ability for text entry and response, which is essential in prototyping. As a result, all of Collin’s content - flows, text, logic, and images - were forced to be scattered across various tools. ### Getting non-technical team members up to speed Designers were working within the NLU engine to design, which was difficult for non-technical team members - not to mention the lack of a quality user interface with other platforms. The team was forced to design within the rails of the existing tech stack, which limited the complexity and personalization of the end designs and experiences. ### No process for organizing or tracking feedback Stakeholders & other teams would provide changes to their designs via static files, and there was no process for organizing or tracking the feedback. At hand-off developers would leave feedback in both Figma and Excel. Some feedback would be missed by the design team and would result in countless back and forth conversation for approvals and edits that slowed the team's process down. ### How Rocket Mortgage Uses Voiceflow Working on the Voiceflow canvas today, Collin has a greater sense of freedom when it comes to scaling his team's work. ✅ the design process is accessible to everyone ✅ the workflow has been sped up drastically ✅ the approvals process is seamless for stakeholders ✅ the team has a CMS for dialogue data and conversational content Within the first week of using Voiceflow, Collin was able to completely reproduce his chatbot in Voiceflow, using it as a content management system from the beginning. Better yet, his team was now able to quickly send prototypes to business partners for feedback. This process became much more simplified as team members could easily leave notes and edits from the Voiceflow canvas. Both technical and non-technical team members, from product owners to developers, could finally collaborate effectively in an intuitive, central source of truth. ### “Voiceflow is how we make everything accessible to designers and non-designers alike.” #### \- COLLIN CARNAHAN, SENIOR AI-ML CONVERSATIONAL UX DESIGNER > **Ready to reinvent your own conversation design workflow?** [**Chat with our team**](https://www.voiceflow.com/demo) > **.** The team’s existing process caused 3 main pain points: ### Limited content management Collin was using Excel, Word, Lucid Chart, or Figma but quickly realized that these were not optimal for conversation design. He could "fake conversation design" but the tools were limiting the experiences that he wanted to design complex flows and create high-fidelity prototypes. For example, Figma didn’t have the ability for text entry and response, which is essential in prototyping. As a result, all of Collin’s content - flows, text, logic, and images - were forced to be scattered across various tools. ### Getting non-technical team members up to speed Designers were working within the NLU engine to design, which was difficult for non-technical team members - not to mention the lack of a quality user interface with other platforms. The team was forced to design within the rails of the existing tech stack, which limited the complexity and personalization of the end designs and experiences. ### No process for organizing or tracking feedback Stakeholders & other teams would provide changes to their designs via static files, and there was no process for organizing or tracking the feedback. At hand-off developers would leave feedback in both Figma and Excel. Some feedback would be missed by the design team and would result in countless back and forth conversation for approvals and edits that slowed the team's process down. ### How Rocket Mortgage Uses Voiceflow Working on the Voiceflow canvas today, Collin has a greater sense of freedom when it comes to scaling his team's work. ✅ the design process is accessible to everyone ✅ the workflow has been sped up drastically ✅ the approvals process is seamless for stakeholders ✅ the team has a CMS for dialogue data and conversational content Within the first week of using Voiceflow, Collin was able to completely reproduce his chatbot in Voiceflow, using it as a content management system from the beginning. Better yet, his team was now able to quickly send prototypes to business partners for feedback. This process became much more simplified as team members could easily leave notes and edits from the Voiceflow canvas. Both technical and non-technical team members, from product owners to developers, could finally collaborate effectively in an intuitive, central source of truth. ### “Voiceflow is how we make everything accessible to designers and non-designers alike.” #### \- COLLIN CARNAHAN, SENIOR AI-ML CONVERSATIONAL UX DESIGNER > **Ready to reinvent your own conversation design workflow?** [**Chat with our team**](https://www.voiceflow.com/demo) > **.**  RECOMMENDED [\ \ ### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) PUBLISHED September 24, 2023 TOPICS IN THIS ARTICLE No items found. [\ \ Joanne Szeto\ \ Growth Co-Op, Voiceflow for Students at\ \ Joanne is a contributor at Pathways.](https://www.voiceflow.com/contributors/joanne-szeto) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/using-voiceflow-to-run-more-efficient-inclusive-ai-design-reviews) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How a Global Tier 1 Investment Bank Uses Voiceflow to Run More Efficient, Inclusive Conversational AI Design Reviews](https://www.voiceflow.com/pathways/using-voiceflow-to-run-more-efficient-inclusive-ai-design-reviews) [](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### From decision trees to AI: How Turo built a global, multilingual chatbot in two months](https://www.voiceflow.com/pathways/how-turo-built-chatbot-in-two-months) [](https://www.voiceflow.com/pathways/how-rocket-mortgage-accelerated-their-teams-conversation-design-workflow) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How Rocket Mortgage Accelerated their Team’s Conversation Design Workflow](https://www.voiceflow.com/pathways/how-rocket-mortgage-accelerated-their-teams-conversation-design-workflow) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-rocket-mortgage-accelerated-their-teams-conversation-design-workflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-rocket-mortgage-accelerated-their-teams-conversation-design-workflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # The Five Stages of GenAI Maturity (and How Not to Stall at Stage Two) | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) How Mature is Your Generative AI? ================================= [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov) As a leader, you’re likely asking yourself the same two questions when it comes to AI maturity: “Where are we at?” and “What do we need to do?” In other words, how does your company’s use of generative AI (GenAI) stack up against competitors, and what could you be doing to make it better? It can be tough to know exactly where you stand in today’s GenAI race, and while there’s a lot of interest from executives and board members to implement it, too many companies stall out before they even enter the race. Today, I’ll introduce a definitive model for GenAI maturity, and offer some tips for how you can get yours past the demo stage and into production.  The five levels of AI maturity ------------------------------ As I see it, there are five levels of AI maturity. Let’s take a quick look at each, and then we’ll talk about why companies have trouble progressing from one stage to another. **Stage 1: Awareness **The team in this stage knows that generative AI tools like ChatGPT exist but the extent of their AI knowledge stops there. That kind of manager is also probably not reading this article, so let’s move on. **Stage 2: Demo **Managers in this stage have successfully built some kind of demo GenAI application, but haven’t deployed it. Developers in this stage got curious and built a demo chatbot for a specific use case, but never got brave enough to test it on customers. **Stage 3: Generative FAQ **At this stage, the company has deployed a generative FAQ chatbot to production that can answer customer questions. **Stage 4: Integrated agent **Teams at this stage have deployed an agent that can help users take actions and complete multiple tasks. The agent provides user support, checks systems status, and allows human escalation within its system. **Stage 5: High-complexity agent **At this stage, the team has created a complex agent that’s tailored to the customer experience and draws on multiple knowledge bases, models, and modalities.  Why most companies stall out at stage 2, the demo stage ------------------------------------------------------- It’s not hard to build an FAQ agent—in fact, many developers can build one in [five minutes](https://www.youtube.com/watch?v=HHb1GJI7BtM) —but most companies end up stalling their Gen-AI agent efforts at the demo stage (especially Fortune 500 and tech companies). Let’s figure out why. ### They fall off the data cliff Functioning FAQ bots need good data to function. Pre-existing FAQ answers and clean documents provide a strong foundation for a bot that provides customers with relevant, helpful answers. Without good data, demo GenAI agents tend to fall off a cliff. Too many companies fall off that cliff. Their data is disjointed, disorganized, and not actionable, which creates a hurdle that many companies never clear. ### They can’t deploy or test with confidence Some companies have great data, but lack the resources to properly deploy and test their GenAI agent. Being able to actually deploy a web chat requires a certain degree of expertise that some companies just can’t provide, due to silos slowing down and hindering the process. There can also be unnecessary bureaucracy when developers rely on other teams to validate and test a virtual agent. Often, these teams lack the tools they need and the way forward isn’t clear. The result is a muddy process that guarantees key testers get stuck in the weeds. Finally, some companies get cold feet before deploying an FAQ bot because they perceive security risks or simply lack confidence that they can keep it running well. In these cases, abandoning the project altogether can seem easier than having to face the unknown. How to get to stage 3, the GenAI sweet spot ------------------------------------------- Even if most companies stall out at stage two, that doesn’t mean they have to stay there. If bad data is the problem, good data is the solution. Being intentional about how you format your data can completely change how your FAQ bot will operate. If you’re stuck on how to improve your data, you may want to get in touch with a knowledgeable team that can help you organize and format it (like, oh I don’t know, Voiceflow, for example). You could also use a tool like [knowledge base](https://www.voiceflow.com/blog/knowledge-base) to action text data and turn it into a chatbot. If you’re running into hurdles at the deployment stage, you might want to look into a pre-built, customizable widget that allows you to set up and publish your bot in a matter of minutes. You can also avoid collaboration hurdles by using a platform that allows your company to collaborate cross-functionally between the large language model, team, back end, front end, design, and product teams. Finally, if apprehensions about testing and monitoring are keeping you from deploying, you may want to find a platform that provides a regression testing framework. With this in place, you’ll be able to validate answer quality easily and deploy with confidence. _Build an advanced AI agent right now._ To stage 4 and beyond --------------------- Deploying a stage 3 model to production is a huge achievement, but there’s also lots of room to grow afterward. You’ll find before long that while an FAQ bot can answer customer questions adequately, it can’t help customers take action nor does it allow for integrations. It also can’t handle multi-turn flows, which will limit you over time. If you can get your virtual agent to stage 4, you’ll enable it to integrate with existing systems and help users take actions. For example, in addition to answering product questions, it will also be able to help customers place and confirm orders. To get your agent to this stage, you’ll need to integrate it with existing systems, design the necessary multi-turn flows, and have a system in place to test and monitor the virtual agent’s performance. Most companies can serve customers very well by getting to stage 4, but you may have your sights set higher. If your goal is to get your virtual agent to stage 5, you’ll be able to provide a tailored customer experience. Voiceflow’s own chatbot, Tico, is an example of a stage 5 agent. It’s implemented on multiple channels, draws on many knowledge bases and models, and provides personalized answers to users. Stay tuned for future articles that dive further into how Tico works. Building a model this mature does require more time and resources. It can take days to build the initial model, and weeks to refine it. If your goal is to build a high-complexity agent, you’ll want to use a platform that will help you save time and avoid a tricky developer handoff. With the right tool, you’ll be able to design, build, and deploy your agent in the same place. Using a platform that has features like knowledge base, one-click deployment, an extendable set of APIs, the right integrations, and prototyping and testing will make this process much easier. Great GenAI agents are great at deflection ------------------------------------------ It can be easy to get consumed with up-leveling your GenAI agent. If you’re currently operating a passable FAQ bot, of course you’re thinking about what you can do to integrate it further. My advice would be to not worry too much about what your bot can do from a technical perspective, and instead keep deflection in mind as a key metric. How well does your virtual agent serve customers so they don’t have to speak to a live agent? One of our clients at Voiceflow has achieved a 70% deflection rate just by implementing a customer support chatbot in their help center. That’s an incredible improvement and says much more about the customer experience than any other metric might. GenAI models will continue to mature and improve, so constantly trying to keep up might feel like running a race with no finish line in sight. If you can get your virtual agent out of the demo stage and onto your site, you’re already ahead of most companies. The rest is a matter of incremental improvement. The five levels of AI maturity ------------------------------ As I see it, there are five levels of AI maturity. Let’s take a quick look at each, and then we’ll talk about why companies have trouble progressing from one stage to another. **Stage 1: Awareness **The team in this stage knows that generative AI tools like ChatGPT exist but the extent of their AI knowledge stops there. That kind of manager is also probably not reading this article, so let’s move on. **Stage 2: Demo **Managers in this stage have successfully built some kind of demo GenAI application, but haven’t deployed it. Developers in this stage got curious and built a demo chatbot for a specific use case, but never got brave enough to test it on customers. **Stage 3: Generative FAQ **At this stage, the company has deployed a generative FAQ chatbot to production that can answer customer questions. **Stage 4: Integrated agent **Teams at this stage have deployed an agent that can help users take actions and complete multiple tasks. The agent provides user support, checks systems status, and allows human escalation within its system. **Stage 5: High-complexity agent **At this stage, the team has created a complex agent that’s tailored to the customer experience and draws on multiple knowledge bases, models, and modalities.  Why most companies stall out at stage 2, the demo stage ------------------------------------------------------- It’s not hard to build an FAQ agent—in fact, many developers can build one in [five minutes](https://www.youtube.com/watch?v=HHb1GJI7BtM) —but most companies end up stalling their Gen-AI agent efforts at the demo stage (especially Fortune 500 and tech companies). Let’s figure out why. ### They fall off the data cliff Functioning FAQ bots need good data to function. Pre-existing FAQ answers and clean documents provide a strong foundation for a bot that provides customers with relevant, helpful answers. Without good data, demo GenAI agents tend to fall off a cliff. Too many companies fall off that cliff. Their data is disjointed, disorganized, and not actionable, which creates a hurdle that many companies never clear. ### They can’t deploy or test with confidence Some companies have great data, but lack the resources to properly deploy and test their GenAI agent. Being able to actually deploy a web chat requires a certain degree of expertise that some companies just can’t provide, due to silos slowing down and hindering the process. There can also be unnecessary bureaucracy when developers rely on other teams to validate and test a virtual agent. Often, these teams lack the tools they need and the way forward isn’t clear. The result is a muddy process that guarantees key testers get stuck in the weeds. Finally, some companies get cold feet before deploying an FAQ bot because they perceive security risks or simply lack confidence that they can keep it running well. In these cases, abandoning the project altogether can seem easier than having to face the unknown. How to get to stage 3, the GenAI sweet spot ------------------------------------------- Even if most companies stall out at stage two, that doesn’t mean they have to stay there. If bad data is the problem, good data is the solution. Being intentional about how you format your data can completely change how your FAQ bot will operate. If you’re stuck on how to improve your data, you may want to get in touch with a knowledgeable team that can help you organize and format it (like, oh I don’t know, Voiceflow, for example). You could also use a tool like [knowledge base](https://www.voiceflow.com/blog/knowledge-base) to action text data and turn it into a chatbot. If you’re running into hurdles at the deployment stage, you might want to look into a pre-built, customizable widget that allows you to set up and publish your bot in a matter of minutes. You can also avoid collaboration hurdles by using a platform that allows your company to collaborate cross-functionally between the large language model, team, back end, front end, design, and product teams. Finally, if apprehensions about testing and monitoring are keeping you from deploying, you may want to find a platform that provides a regression testing framework. With this in place, you’ll be able to validate answer quality easily and deploy with confidence. _Build an advanced AI agent right now._ To stage 4 and beyond --------------------- Deploying a stage 3 model to production is a huge achievement, but there’s also lots of room to grow afterward. You’ll find before long that while an FAQ bot can answer customer questions adequately, it can’t help customers take action nor does it allow for integrations. It also can’t handle multi-turn flows, which will limit you over time. If you can get your virtual agent to stage 4, you’ll enable it to integrate with existing systems and help users take actions. For example, in addition to answering product questions, it will also be able to help customers place and confirm orders. To get your agent to this stage, you’ll need to integrate it with existing systems, design the necessary multi-turn flows, and have a system in place to test and monitor the virtual agent’s performance. Most companies can serve customers very well by getting to stage 4, but you may have your sights set higher. If your goal is to get your virtual agent to stage 5, you’ll be able to provide a tailored customer experience. Voiceflow’s own chatbot, Tico, is an example of a stage 5 agent. It’s implemented on multiple channels, draws on many knowledge bases and models, and provides personalized answers to users. Stay tuned for future articles that dive further into how Tico works. Building a model this mature does require more time and resources. It can take days to build the initial model, and weeks to refine it. If your goal is to build a high-complexity agent, you’ll want to use a platform that will help you save time and avoid a tricky developer handoff. With the right tool, you’ll be able to design, build, and deploy your agent in the same place. Using a platform that has features like knowledge base, one-click deployment, an extendable set of APIs, the right integrations, and prototyping and testing will make this process much easier. Great GenAI agents are great at deflection ------------------------------------------ It can be easy to get consumed with up-leveling your GenAI agent. If you’re currently operating a passable FAQ bot, of course you’re thinking about what you can do to integrate it further. My advice would be to not worry too much about what your bot can do from a technical perspective, and instead keep deflection in mind as a key metric. How well does your virtual agent serve customers so they don’t have to speak to a live agent? One of our clients at Voiceflow has achieved a 70% deflection rate just by implementing a customer support chatbot in their help center. That’s an incredible improvement and says much more about the customer experience than any other metric might. GenAI models will continue to mature and improve, so constantly trying to keep up might feel like running a race with no finish line in sight. If you can get your virtual agent out of the demo stage and onto your site, you’re already ahead of most companies. The rest is a matter of incremental improvement.  RECOMMENDED [.avif)\ \ ### The AI Wild West—why you need a knowledge base for your AI agents](https://www.voiceflow.com/pathways/the-ai-wild-west-why-you-need-a-knowledge-base-for-your-ai-agents) PUBLISHED December 1, 2023 TOPICS IN THIS ARTICLE Learning and APIs [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/ai-automation-agency-templates) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### 5 powerful templates for AI automation agencies](https://www.voiceflow.com/pathways/ai-automation-agency-templates) [](https://www.voiceflow.com/pathways/how-mature-is-your-generative-ai) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How Mature is Your Generative AI?](https://www.voiceflow.com/pathways/how-mature-is-your-generative-ai) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-mature-is-your-generative-ai#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-mature-is-your-generative-ai#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # The future of conversational AI | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) The future of conversational AI =============================== [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow](https://www.voiceflow.com/contributors/peter) The [AI](https://www.voiceflow.com/blog-categories/ai) boom is here. And whether it makes you terrified or elated, the products, services, and experiences that come from the next few years of innovation are not unlike a modern-day industrial revolution. [Large language models (LLMs)](https://www.voiceflow.com/blog/dominant-discourse-llms) now have the ability to generate human-like responses in a way that is truly groundbreaking. Prior to this, we were in the most difficult stage of conversation design (CxD)—we had to design every little piece of logic, every utterance, and conversational flow. Everything just took longer. Though we are sure to save time as we integrate LLMs into our design process, we’re also inviting a new player into the game. This change should force us to reevaluate traditional CxD practices and what the future holds for conversation designers. Here are the four changes catalyzed by AI that I believe will push conversation designers to adapt and evolve:  1\. The shift from natural language (NLU) to natural language generation (NLG) design ------------------------------------------------------------------------------------- In the old world, we focused on the input—what utterances build out the structure of your [NLU model](https://www.voiceflow.com/blog/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) . In the AI future, we’ll see the concentration of design towards controlling the generated output. A lot of your models will be built on “few-shot learning” with LLMs. Instead of creating intents that can have anywhere between 20 to 100 different utterances, you’d create only a few training examples that outline the scope of the intent, then receive an output from the LLM and submit feedback to the model on its response. Over time, the AI will increase in accuracy. > "In the AI future, we’ll see the concentration of design towards controlling the generated output." How does your process change when the challenge is no longer whether the model understands your inputs, but instead, how it can generate the right outputs? And it’s already begun. Across industries we’re seeing everyone dip their toes in the water—integrating LLMs into business processes to interpret and, increasingly, generate data. As we become more confident with the outputs, that shift will accelerate. In this world, NLUs will fade into the background (but it won’t disappear). You’ll be asked to shift your attention away from narrow inputs and instead focus on crafting flexible prompts that can generate the right outputs for a wide array of user inputs. This is often called prompt engineering—the practice of designing and optimizing prompts to achieve specific outcomes and flows in AI-human conversations. But more on that later. 2\. The increasing importance of persona design ------------------------------------------------ Let’s face it: LLMs lack personality. They’re built to process the entirety of the English language—billions and sometimes trillions of [corpora](https://en.wikipedia.org/wiki/Text_corpus) —but sometimes those outputs leave something to be desired. Sure, I’ve asked [Claude](https://www.anthropic.com/index/introducing-claude) to summarize a podcast in the style of _The_ _New York Times_ columnist Kevin Roose_._ But unless the LLM your model relies on has been trained on your brand, it’ll struggle to nail your voice and assistant persona. > "In the future of AI-generated conversations, your assistant's personality will set it apart from a horde of charisma-less counterparts." I won’t wax poetic on the importance of designing the personality traits, tone, and style of your conversation assistants (plenty of folks have done a better job at [that](https://medium.com/ibm-watson/best-practices-designing-a-persona-for-your-assistant-c2a58666f3c) ), but in the future of AI-generated conversations, your assistant's personality will set it apart from a horde of charisma-less counterparts. And as it becomes easier and easier to design assistants, you’ll want to prioritize making yours cutting edge. It’s the human touch that will make your conversations come alive. 3\. The management of knowledge (and bias) ------------------------------------------- Conversation designers and developers alike will increasingly curate and maintain knowledge bases—those large swaths of data that feed into the model. It’s not unlike a call center that records conversations. A manager listens to calls and gives employees feedback. You'll be asked to do the same thing with AI. You’ll input data into the knowledge base—guides, marketing materials, FAQs, product specs—and the outputs will improve as you determine what worked and what didn’t. Obviously, issues of privacy, bias, and oversight come to mind. Anything you put through an open-source LLM model like ChatGPT can be used to continue to train the model. That means if you put sensitive emails, internal memos, or consumer data through an LLM, the model might use that data to answer other users’ prompts. Designers should be rightfully reticent to put any customer data through an LLM right now—the protections just aren't robust enough (though that is [changing over at OpenAI](https://openai.com/blog/new-ways-to-manage-your-data-in-chatgpt) ). On bias and oversight, there's no future in which you can just “set it and forget it.” CxD teams will be expected to handle misinformation and mitigate the risks of LLMs generating false or misleading information. So, if a recruitment bot is filtering through resumes, your human managers need to understand why certain people have been accepted or rejected, and shift those parameters at any time. And as robust AI models become more robust and interpretable, they can help us understand how they arrive at specific outputs. Just as with any new piece of technology, humans will need to adapt and learn to manage it. 4\. The art of prompt chaining ------------------------------ Most importantly, you’ll need to develop skills like [prompt chaining](https://www.voiceflow.com/blog/prompt-chaining-conversational-ai) , which involves connecting multiple prompts in a sequence, guiding the conversation and LLM-generated responses toward desired outcomes. You’ll need to follow a chain of prompts to work out how you got to an end result. And you’ll need to design processes to create repeatable outcomes. It’s no easy task. Everyone playing around with AI right now is managing the challenge of prompt chaining, especially because your inputs lead to outputs, but there’s little context from the AI on how it arrived at those conclusions. I suggest getting started by practicing reverse prompt chaining. I tell ChatGPT or Claude the task I'm trying to do and I get the AI to write the prompt for me. And as I receive outputs, I ask the AI to continue to suggest rewrites of the prompt until I achieve my desired results. Once you get the outcome that you're hoping for, you now understand the sequence of words that will help you get your desired outputs. The good news is that conversation designers have an advantage in this area—especially if you come from a copywriting background. You already have a good grasp of language. In its simplest form, prompt chaining is about understanding the simplest way to instruct AI to do something. And from there, trying over and over again until it starts doing the right thing. My advice? Learn by doing. The phenomenal future is here. Let’s protect it. -------------------------------------------------- Everyone’s excitement (and fear) about the future of AI is completely valid. There are lots of concerns—job security, societal value, upskilling workers. Each of these are worthwhile discussions we’ll be having in the coming months and years. Ultimately, my hope is that we gain regulatory restraints around this technology—protections that are intentionally created to protect consumers and companies but which don’t halt innovation. We didn’t have that with social media and we know that once the cat’s out of the bag, it’s out. The future of AI is interpretability, privacy, and oversight—and a lot of really thoughtful developers and designers pushing the conversation forward. 1\. The shift from natural language (NLU) to natural language generation (NLG) design ------------------------------------------------------------------------------------- In the old world, we focused on the input—what utterances build out the structure of your [NLU model](https://www.voiceflow.com/blog/nlu-design-how-to-train-and-use-a-natural-language-understanding-model) . In the AI future, we’ll see the concentration of design towards controlling the generated output. A lot of your models will be built on “few-shot learning” with LLMs. Instead of creating intents that can have anywhere between 20 to 100 different utterances, you’d create only a few training examples that outline the scope of the intent, then receive an output from the LLM and submit feedback to the model on its response. Over time, the AI will increase in accuracy. > "In the AI future, we’ll see the concentration of design towards controlling the generated output." How does your process change when the challenge is no longer whether the model understands your inputs, but instead, how it can generate the right outputs? And it’s already begun. Across industries we’re seeing everyone dip their toes in the water—integrating LLMs into business processes to interpret and, increasingly, generate data. As we become more confident with the outputs, that shift will accelerate. In this world, NLUs will fade into the background (but it won’t disappear). You’ll be asked to shift your attention away from narrow inputs and instead focus on crafting flexible prompts that can generate the right outputs for a wide array of user inputs. This is often called prompt engineering—the practice of designing and optimizing prompts to achieve specific outcomes and flows in AI-human conversations. But more on that later. 2\. The increasing importance of persona design ------------------------------------------------ Let’s face it: LLMs lack personality. They’re built to process the entirety of the English language—billions and sometimes trillions of [corpora](https://en.wikipedia.org/wiki/Text_corpus) —but sometimes those outputs leave something to be desired. Sure, I’ve asked [Claude](https://www.anthropic.com/index/introducing-claude) to summarize a podcast in the style of _The_ _New York Times_ columnist Kevin Roose_._ But unless the LLM your model relies on has been trained on your brand, it’ll struggle to nail your voice and assistant persona. > "In the future of AI-generated conversations, your assistant's personality will set it apart from a horde of charisma-less counterparts." I won’t wax poetic on the importance of designing the personality traits, tone, and style of your conversation assistants (plenty of folks have done a better job at [that](https://medium.com/ibm-watson/best-practices-designing-a-persona-for-your-assistant-c2a58666f3c) ), but in the future of AI-generated conversations, your assistant's personality will set it apart from a horde of charisma-less counterparts. And as it becomes easier and easier to design assistants, you’ll want to prioritize making yours cutting edge. It’s the human touch that will make your conversations come alive. 3\. The management of knowledge (and bias) ------------------------------------------- Conversation designers and developers alike will increasingly curate and maintain knowledge bases—those large swaths of data that feed into the model. It’s not unlike a call center that records conversations. A manager listens to calls and gives employees feedback. You'll be asked to do the same thing with AI. You’ll input data into the knowledge base—guides, marketing materials, FAQs, product specs—and the outputs will improve as you determine what worked and what didn’t. Obviously, issues of privacy, bias, and oversight come to mind. Anything you put through an open-source LLM model like ChatGPT can be used to continue to train the model. That means if you put sensitive emails, internal memos, or consumer data through an LLM, the model might use that data to answer other users’ prompts. Designers should be rightfully reticent to put any customer data through an LLM right now—the protections just aren't robust enough (though that is [changing over at OpenAI](https://openai.com/blog/new-ways-to-manage-your-data-in-chatgpt) ). On bias and oversight, there's no future in which you can just “set it and forget it.” CxD teams will be expected to handle misinformation and mitigate the risks of LLMs generating false or misleading information. So, if a recruitment bot is filtering through resumes, your human managers need to understand why certain people have been accepted or rejected, and shift those parameters at any time. And as robust AI models become more robust and interpretable, they can help us understand how they arrive at specific outputs. Just as with any new piece of technology, humans will need to adapt and learn to manage it. 4\. The art of prompt chaining ------------------------------ Most importantly, you’ll need to develop skills like [prompt chaining](https://www.voiceflow.com/blog/prompt-chaining-conversational-ai) , which involves connecting multiple prompts in a sequence, guiding the conversation and LLM-generated responses toward desired outcomes. You’ll need to follow a chain of prompts to work out how you got to an end result. And you’ll need to design processes to create repeatable outcomes. It’s no easy task. Everyone playing around with AI right now is managing the challenge of prompt chaining, especially because your inputs lead to outputs, but there’s little context from the AI on how it arrived at those conclusions. I suggest getting started by practicing reverse prompt chaining. I tell ChatGPT or Claude the task I'm trying to do and I get the AI to write the prompt for me. And as I receive outputs, I ask the AI to continue to suggest rewrites of the prompt until I achieve my desired results. Once you get the outcome that you're hoping for, you now understand the sequence of words that will help you get your desired outputs. The good news is that conversation designers have an advantage in this area—especially if you come from a copywriting background. You already have a good grasp of language. In its simplest form, prompt chaining is about understanding the simplest way to instruct AI to do something. And from there, trying over and over again until it starts doing the right thing. My advice? Learn by doing. The phenomenal future is here. Let’s protect it. -------------------------------------------------- Everyone’s excitement (and fear) about the future of AI is completely valid. There are lots of concerns—job security, societal value, upskilling workers. Each of these are worthwhile discussions we’ll be having in the coming months and years. Ultimately, my hope is that we gain regulatory restraints around this technology—protections that are intentionally created to protect consumers and companies but which don’t halt innovation. We didn’t have that with social media and we know that once the cat’s out of the bag, it’s out. The future of AI is interpretability, privacy, and oversight—and a lot of really thoughtful developers and designers pushing the conversation forward.  RECOMMENDED [\ \ ### Curiosity and conversation beat any best practice](https://www.voiceflow.com/pathways/best-practices-conversation-design) PUBLISHED May 30, 2023 TOPICS IN THIS ARTICLE Elevating our work [\ \ Peter Isaacs\ \ Senior Conversation Design Advocate at Voiceflow\ \ Peter’s current obsessions include LLMs and conversational AI. When Peter’s not writing furiously about saving the future of AI, you can find him solving a Rubik's Cube in under 60 seconds. (Note: may not result in single-colored sides.)](https://www.voiceflow.com/contributors/peter) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/5-strategies-to-transform-ai-powered-customer-experiences) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### 5 strategies product leaders use to transform AI-powered customer experiences \[Checklist\]](https://www.voiceflow.com/pathways/5-strategies-to-transform-ai-powered-customer-experiences) [.webp)](https://www.voiceflow.com/pathways/what-is-an-interaction-model-in-conversational-ai) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### What is an interaction model in conversational AI?](https://www.voiceflow.com/pathways/what-is-an-interaction-model-in-conversational-ai) [](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### Dialogflow vs. Voiceflow: 3 Core design differences to know before you migrate](https://www.voiceflow.com/pathways/dialogflow-vs-voiceflow-3-core-design-differences-to-know-before-you-migrate) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/future-of-conversational-ai#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/future-of-conversational-ai#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How I used conversation design to get a job | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) How I used conversation design to get a job =========================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) Hi, I'm Sam. I'm a new face on the Growth team at Voiceflow; but I'm no stranger to Voiceflow and no code tools. In fact, before I added "Growth Manager at Voiceflow" to my Twitter bio, I turned to the tool itself to build an experience that helped me land my job – here's the story. This post recaps what I learned in using Voiceflow for my application and why I think everyone, even non-designers, have a lot to gain with conversation design. **The Application Conversation** -------------------------------- ### **_"How can I stand out?"_** It's a feeling that any on-the-hunt job applicant has felt before. The copy-heavy resume and cover letter feel stale, while even that intro video you recorded hints towards being a bit overused. **It's not just you.** The hiring process relies heavily on the best communicators connecting with a team, yet the personal, human conversation of that initial application is very hard to find. If the end goal of the application is an interview with good conversation, why is the standard first impression so one-sided? Why are we doing all of the talking? To make my application stand out, I set out to create a voice experience that would help me have a natural conversation from square one.  **Introducing The Resume Assistant** ------------------------------------- Consider this applicant/company back-and-forth in a traditional setup. > **Company**: We're hiring! Please apply here. > **Applicant:** Here's my resume. I have relevant experience! > **Company**: ... > **Applicant:** Here's my cover letter. I can write and have human emotion! > **Company**: ... > **Applicant:** Go see my portfolio website. I have visual storytelling skills! > **Company**: ... > **Applicant:** Find me on Twitter. I'm tapped into industry conversations! > **Company**: Thank you for your interest in Company X. We've received your application materials for the role, but due to the high quality candidates for this position, we do not feel like you are a fit at this time. Now, take the resume assistant (see below) as a different approach to applying. The hiring manager or team has the chance to have a conversation from step one. Instead of reading a resume to learn about work experience or scroll through a Twitter account to understand personality, **they're prompted to ask.**  A welcome message for the assistant. The resume voice assistant injects creativity into your application, allowing the manager or team to ask about you, rather than just read.  Three intent options from the assistant. Having those questions answered on demand by the applicant's voice makes for a conversation before an interview even begins. This slight shift makes a mighty difference. **Build for impact, not complexity** ------------------------------------ My conversational experience was not built for complexity. Instead, its path contained just a few key steps: **_an introduction & three intents._** This framework gave me a place to then insert my personal, professional, and "why Voiceflow" stories directly into the conversation flow. Although having originally tested the tool in early 2019, my Voiceflow build skills were **_very_** green as I sat down to develop. The no-code solution, in my case, allowed for a quick, creative build.  A look at the project on the Voiceflow canvas. Designing each block with my story in mind, the .mp3 audio tool made prototyping a breeze. The most stressful part of process? Figuring out what I wanted to say. Despite the stage fright, I now had a chance to have a conversation where one typically didn't exist. Accessible as both an Alexa Skill and general assistant prototype, the Voiceflow team could ask Alexa, "Why Sam Should Work For Voiceflow," or simply interact on desktop or mobile via the shareable creator prototype link (above). The Resume Assistant Experience vs. Traditional Application ----------------------------------------------------------- Let's look at an example of that initial application, but with the resume assistant and good conversation design leading the way. > **Company**: We're hiring! Please apply here. > **Applicant**: Hello! I've sent in my resume and other application materials. In the meantime, you can get to know me better using my resume assistant. > Assistant: Hey, welcome to Why Sam Should Work at Voiceflow! You can learn about my career, personal life, and why I want to join your company. > **Company**: Tell me more about your professional experience to start. > Assistant: I've spent 5 years at an advertising agency working with more than 15 brands across a variety of industries. Content marketing, social media, and public relations are some of my areas of expertise. > **Company**: Why do you want to work here? > Assistant: Good question. I've followed your brand for more than 3 years and love where the company is headed. The culture seems to fit my core values, too. > **Company**: What about you personally? > Assistant: I live in the Midwest and love all things Wisconsin. Outside of work hours, I have a passion for the Green Bay Packers, craft beer, and running my family's foundation. > **Company**: Thanks! Let's wrap it up. > **Assistant:** Thank you for taking the time to listen to Why I Should Work at Voiceflow! If you want to see more, check out my portfolio website, social pages, and resume found in the application materials. For context, the shared prototype below is what I used to share the assistant with the Voiceflow team. In addition to supporting Amazon Alexa and Google Assistant, Voiceflow supports general assistants now, too. So, it's even easier for a job candidate to share a project link directly with the hiring team in an introduction email or application portal.  The resume assistant prototype in action. This type of conversation stands out. Sure, the fact that it's a new way to apply for a job is unique. That sets an applicant apart just like a having a website portfolio might have done 7 years ago. A flashy new medium is not the main point, though. ### **It's the quality conversation.** These "get to know you" questions that are typically asked in the first interview are now answered before the company even meets the applicant. The hiring team gets a sense of qualifications and culture fit that has more depth and personality than any cover letter could ever muster. For the first time, I wasn't doing all the talking. The conversation made all the difference. What's Next? ------------ Do you remember the story about the guy that secured [10 interviews with a box of donuts](https://abcnews.go.com/Business/man-scores-10-interviews-resume-delivered-box-doughnuts/story?id=42609704#:~:text=Lukas%20Yla%2C%2025%2C%20disguised%20himself%20as%20a%20Postmates%20delivery%20man,in%20the%20San%20Francisco%20area.&text=After%20delivering%20doughnut%20boxes%20to,Yla%20landed%2010%20job%20interviews.) ? Just like that eager, recent graduate creatively got his resume in front of the right people to start the conversation in 2016, building a simple general assistant using conversational ai can make an equally hefty (and innovative) impact for candidates in 2021. I'm confident that conversation design and voice user interface design can be used by any applicant in any industry. As communication tools and skills are must-haves for almost every career, voice and conversation design is the next step of that communication. The opportunity to create a healthy dialogue where there typically might not be one is so valuable. ### **Now, it's your turn to make a resume assistant.** You don't need design or coding skills to do it, either. To make it easy, I've put together a [template Voiceflow project](https://share.voiceflow.com/resumeassistant) that can be used to build your own resume/about me voice experience. After signing up for a creator account, follow the step by step instructions in the demo project to design and prototype your own resume assistant. Working from my sample audio, you can record your own story in the design process and then share the link as part of any job application. [You can create your own assistant using our template link here.](https://share.voiceflow.com/resumeassistant)  The template project in the Voiceflow canvas. Whether you've followed Voiceflow for years, or this post is your introduction, the recent rollout of V2 provides a fresh and more simple way for non-designers to start in the space. Whether that's an individual resume assistant or bringing voice to a business, I could not be more excited to explore the opportunities for ideating and sharing with this community. > Build your own resume assistant [using our template project.](https://share.voiceflow.com/resumeassistant) > Ideas or questions_? Send me an email at_ [_sam.burns@voiceflow.com_](mailto:sam.burns@voiceflow.com) > _._ **Introducing The Resume Assistant** ------------------------------------- Consider this applicant/company back-and-forth in a traditional setup. > **Company**: We're hiring! Please apply here. > **Applicant:** Here's my resume. I have relevant experience! > **Company**: ... > **Applicant:** Here's my cover letter. I can write and have human emotion! > **Company**: ... > **Applicant:** Go see my portfolio website. I have visual storytelling skills! > **Company**: ... > **Applicant:** Find me on Twitter. I'm tapped into industry conversations! > **Company**: Thank you for your interest in Company X. We've received your application materials for the role, but due to the high quality candidates for this position, we do not feel like you are a fit at this time. Now, take the resume assistant (see below) as a different approach to applying. The hiring manager or team has the chance to have a conversation from step one. Instead of reading a resume to learn about work experience or scroll through a Twitter account to understand personality, **they're prompted to ask.**  A welcome message for the assistant. The resume voice assistant injects creativity into your application, allowing the manager or team to ask about you, rather than just read.  Three intent options from the assistant. Having those questions answered on demand by the applicant's voice makes for a conversation before an interview even begins. This slight shift makes a mighty difference. **Build for impact, not complexity** ------------------------------------ My conversational experience was not built for complexity. Instead, its path contained just a few key steps: **_an introduction & three intents._** This framework gave me a place to then insert my personal, professional, and "why Voiceflow" stories directly into the conversation flow. Although having originally tested the tool in early 2019, my Voiceflow build skills were **_very_** green as I sat down to develop. The no-code solution, in my case, allowed for a quick, creative build.  A look at the project on the Voiceflow canvas. Designing each block with my story in mind, the .mp3 audio tool made prototyping a breeze. The most stressful part of process? Figuring out what I wanted to say. Despite the stage fright, I now had a chance to have a conversation where one typically didn't exist. Accessible as both an Alexa Skill and general assistant prototype, the Voiceflow team could ask Alexa, "Why Sam Should Work For Voiceflow," or simply interact on desktop or mobile via the shareable creator prototype link (above). The Resume Assistant Experience vs. Traditional Application ----------------------------------------------------------- Let's look at an example of that initial application, but with the resume assistant and good conversation design leading the way. > **Company**: We're hiring! Please apply here. > **Applicant**: Hello! I've sent in my resume and other application materials. In the meantime, you can get to know me better using my resume assistant. > Assistant: Hey, welcome to Why Sam Should Work at Voiceflow! You can learn about my career, personal life, and why I want to join your company. > **Company**: Tell me more about your professional experience to start. > Assistant: I've spent 5 years at an advertising agency working with more than 15 brands across a variety of industries. Content marketing, social media, and public relations are some of my areas of expertise. > **Company**: Why do you want to work here? > Assistant: Good question. I've followed your brand for more than 3 years and love where the company is headed. The culture seems to fit my core values, too. > **Company**: What about you personally? > Assistant: I live in the Midwest and love all things Wisconsin. Outside of work hours, I have a passion for the Green Bay Packers, craft beer, and running my family's foundation. > **Company**: Thanks! Let's wrap it up. > **Assistant:** Thank you for taking the time to listen to Why I Should Work at Voiceflow! If you want to see more, check out my portfolio website, social pages, and resume found in the application materials. For context, the shared prototype below is what I used to share the assistant with the Voiceflow team. In addition to supporting Amazon Alexa and Google Assistant, Voiceflow supports general assistants now, too. So, it's even easier for a job candidate to share a project link directly with the hiring team in an introduction email or application portal.  The resume assistant prototype in action. This type of conversation stands out. Sure, the fact that it's a new way to apply for a job is unique. That sets an applicant apart just like a having a website portfolio might have done 7 years ago. A flashy new medium is not the main point, though. ### **It's the quality conversation.** These "get to know you" questions that are typically asked in the first interview are now answered before the company even meets the applicant. The hiring team gets a sense of qualifications and culture fit that has more depth and personality than any cover letter could ever muster. For the first time, I wasn't doing all the talking. The conversation made all the difference. What's Next? ------------ Do you remember the story about the guy that secured [10 interviews with a box of donuts](https://abcnews.go.com/Business/man-scores-10-interviews-resume-delivered-box-doughnuts/story?id=42609704#:~:text=Lukas%20Yla%2C%2025%2C%20disguised%20himself%20as%20a%20Postmates%20delivery%20man,in%20the%20San%20Francisco%20area.&text=After%20delivering%20doughnut%20boxes%20to,Yla%20landed%2010%20job%20interviews.) ? Just like that eager, recent graduate creatively got his resume in front of the right people to start the conversation in 2016, building a simple general assistant using conversational ai can make an equally hefty (and innovative) impact for candidates in 2021. I'm confident that conversation design and voice user interface design can be used by any applicant in any industry. As communication tools and skills are must-haves for almost every career, voice and conversation design is the next step of that communication. The opportunity to create a healthy dialogue where there typically might not be one is so valuable. ### **Now, it's your turn to make a resume assistant.** You don't need design or coding skills to do it, either. To make it easy, I've put together a [template Voiceflow project](https://share.voiceflow.com/resumeassistant) that can be used to build your own resume/about me voice experience. After signing up for a creator account, follow the step by step instructions in the demo project to design and prototype your own resume assistant. Working from my sample audio, you can record your own story in the design process and then share the link as part of any job application. [You can create your own assistant using our template link here.](https://share.voiceflow.com/resumeassistant)  The template project in the Voiceflow canvas. Whether you've followed Voiceflow for years, or this post is your introduction, the recent rollout of V2 provides a fresh and more simple way for non-designers to start in the space. Whether that's an individual resume assistant or bringing voice to a business, I could not be more excited to explore the opportunities for ideating and sharing with this community. > Build your own resume assistant [using our template project.](https://share.voiceflow.com/resumeassistant) > Ideas or questions_? Send me an email at_ [_sam.burns@voiceflow.com_](mailto:sam.burns@voiceflow.com) > _._  RECOMMENDED [\ \ ### Expanding the definition of conversation design](https://www.voiceflow.com/pathways/expanding-the-definition-of-conversation-design) PUBLISHED March 19, 2021 TOPICS IN THIS ARTICLE Exceptional work [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### 5 tips for better multimodal design](https://www.voiceflow.com/pathways/5-tips-for-better-multimodal-design) [](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Chatbot glow-up: 3 reasons everyone hates your AI agent (and what you can do about it)](https://www.voiceflow.com/pathways/chatbot-glow-up-3-reasons-everyone-hates-your-ai-agent-and-what-you-can-do-about-it) [](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### How to prioritize requests for conversation design teams](https://www.voiceflow.com/pathways/how-to-prioritize-requests-for-conversation-design-teams) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-i-used-conversation-design-to-get-a-job#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-i-used-conversation-design-to-get-a-job#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How much do ChatGPT versions affect real world performance? | Pathways [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) How much do ChatGPT versions affect real-world performance? =========================================================== [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow](https://www.voiceflow.com/contributors/denys-linkov)  _Do versions change your performance by 1% or 10%? Let’s dive into a production-impacting LLM task and how model versions impacts agent performance._ If you’ve used the ChatGPT API before, you’ve likely noticed different model versions associated with them. The original ChatGPT API was released March 1st, leading to a naming convention of _gpt-3.5-turbo-0301_. Next came _gpt-3.5-turbo-0613_, _gpt-3.5-turbo-1106_ and as of this article’s writing, _gpt-3.5-turbo-0125_. After the initial _gpt-3.5-turbo-0301_ release, it was challenging to predict the impact of each of these versions; Open AI did not release new benchmarks for each and did not disclose major changes. LLMs are also non-deterministic, making it difficult to measure the impact of such a change. So, from a high-level perspective, _something_ shifted with these models due to a version upgrade, but it wasn’t clear what the consequences were. Once _gpt-3.5-turbo-0613_ was released, there were a few papers published looking into task-specific aspects of this model version change\[1\], but they mainly focused on small tasks like prime-number detection that are usually inconsequential to a real-world use case.   From _How Is ChatGPT’s Behaviour Changing over Time?_ Many of these tasks were also 0-shot based, rather than few-shot-based which is the technique that creates formatting and response stability that production applications look for. Over the course of 2023, more research was conducted on the overall sensitivity of prompts showing dramatic differences across models with small tweaks in prompt formats. \[2\]  LLM Drift for Conversational AI — Intent Classification ------------------------------------------------------- While ChatGPT and LLMs changed the narrative around building AI Agents, intent classification remains an important problem for directing user conversation and structuring agent actions. In late 2023, we were working with a customer who was using LLMs for intent classification and noticed that _gpt-3.5-turbo-0301_ was getting close to end of life. So, we upgraded this customer’s base model to _gpt-3.5-turbo-1106_, and it dramatically decreased the performance. After realizing the impact of our change (and reverting), we retroactively ran the benchmarks and saw the severe degradation on the newest model of ~10%\*.  With this degradation in mind, we decided to tweak the initial prompt slightly to improve model performance back to our initial benchmarked accuracy. At face value, the changes are not apparent, but make an impact. ### Methodology To improve our prompts and reduce version discrepancies, we iterated on one portion of our real-life dataset to improve our validation accuracy and find a prompt that brought \*gpt-3.5-turbo-1106’\*s accuracy to an acceptable range.  After the iterations, we ran our evaluation dataset\*\* once and confirmed that model results were much better for _gpt-3.5-turbo-1106_\*\*\* (and _gpt-3.5-turbo-0125_ a few months later for curiosities sake). ### Prompt changes Our final prompt had three main changes: 1. We changed the opening line to an indication of importance. 2. We changed from _descriptions and intents_ to _descriptions and actions._ 3. We added a one shot example to improve none intent classification.  To measure the impact of each of these changes, we ran a small ablation study to measure the impact across each change. ### Ablations Our goal with ablations was to gain a better insight into the magnitude of each change’s performance.  The ablations generally improved performance for later models and demonstrated impact of _\-10.66% to 13.08%._ Bolded results show the best performing ablation for each model. Interestingly enough, the ablations performed better on later models showing a higher impact across techniques. However, there was significant swings for seemingly small changes showing the brittle nature of prompts. ### Final results With our three prompt modifications, our intent classification task performed much better on newer models. The accuracy decreased for the initial _gpt-3.5-turbo_ versions, but increased across most recent versions.  Conclusion ---------- LLM versions can make a large difference! Even for a few shot approach. Having good, non benchmark datasets to validate performance is important, even when onboarding new use-cases. In the future we’ll be discussing some our techniques for internal prompt optimization! _The Voiceflow research section covers industry-relevant and applicable research on ML and LLM work in the conversational AI space._  From _How Is ChatGPT’s Behaviour Changing over Time?_ Many of these tasks were also 0-shot based, rather than few-shot-based which is the technique that creates formatting and response stability that production applications look for. Over the course of 2023, more research was conducted on the overall sensitivity of prompts showing dramatic differences across models with small tweaks in prompt formats. \[2\]  LLM Drift for Conversational AI — Intent Classification ------------------------------------------------------- While ChatGPT and LLMs changed the narrative around building AI Agents, intent classification remains an important problem for directing user conversation and structuring agent actions. In late 2023, we were working with a customer who was using LLMs for intent classification and noticed that _gpt-3.5-turbo-0301_ was getting close to end of life. So, we upgraded this customer’s base model to _gpt-3.5-turbo-1106_, and it dramatically decreased the performance. After realizing the impact of our change (and reverting), we retroactively ran the benchmarks and saw the severe degradation on the newest model of ~10%\*.  With this degradation in mind, we decided to tweak the initial prompt slightly to improve model performance back to our initial benchmarked accuracy. At face value, the changes are not apparent, but make an impact. ### Methodology To improve our prompts and reduce version discrepancies, we iterated on one portion of our real-life dataset to improve our validation accuracy and find a prompt that brought \*gpt-3.5-turbo-1106’\*s accuracy to an acceptable range.  After the iterations, we ran our evaluation dataset\*\* once and confirmed that model results were much better for _gpt-3.5-turbo-1106_\*\*\* (and _gpt-3.5-turbo-0125_ a few months later for curiosities sake). ### Prompt changes Our final prompt had three main changes: 1. We changed the opening line to an indication of importance. 2. We changed from _descriptions and intents_ to _descriptions and actions._ 3. We added a one shot example to improve none intent classification.  To measure the impact of each of these changes, we ran a small ablation study to measure the impact across each change. ### Ablations Our goal with ablations was to gain a better insight into the magnitude of each change’s performance.  The ablations generally improved performance for later models and demonstrated impact of _\-10.66% to 13.08%._ Bolded results show the best performing ablation for each model. Interestingly enough, the ablations performed better on later models showing a higher impact across techniques. However, there was significant swings for seemingly small changes showing the brittle nature of prompts. ### Final results With our three prompt modifications, our intent classification task performed much better on newer models. The accuracy decreased for the initial _gpt-3.5-turbo_ versions, but increased across most recent versions.  Conclusion ---------- LLM versions can make a large difference! Even for a few shot approach. Having good, non benchmark datasets to validate performance is important, even when onboarding new use-cases. In the future we’ll be discussing some our techniques for internal prompt optimization! _The Voiceflow research section covers industry-relevant and applicable research on ML and LLM work in the conversational AI space._  Notes ----- \* standard deviation across 5 runs was between 0.3-0.7% accuracy \*\* our validation set contained ~400 examples and our evaluation set contained ~200 \*\*\* our request parameters`` messages = [{"role": "user", "content": request}] response = openai.ChatCompletion.create( model=model_version, messages=messages, temperature=0.1, max_tokens=100 ) ` ` Citations: ---------- \[1\] L. Chen, M. Zaharia and J. Zou. 2023. How is ChatGPT’s behavior changing over time? ArXiv:2307.09009 \[cs\]. \[2\] M. Sclar, Y. Choi, Y. Tsvetkov, and A. Suhr. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324, 2023. Cite this Work -------------- @article{ IntentClassifcationChatGPTVersions, author = {Linkov, Denys}, title = {How much do ChatGPT versions affect real world performance?}, year = {2024}, month = {03}, howpublished = {\url{https://voiceflow.com}}, url = {https://www.voiceflow.com/blog/how-much-do-chatgpt-versions-affect-real-world-performance} } RECOMMENDED [\ \ ### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) PUBLISHED March 19, 2024 TOPICS IN THIS ARTICLE Learning and APIs Exceptional work [\ \ Denys Linkov\ \ Machine Learning Team Lead at Voiceflow\ \ Denys spends his days trying to understand how machine learning will impact our daily lives—whether it's building new models or diving into the latest generative AI tech. When he’s not leading courses on LLMs or expanding Voiceflow’s data science and ML capabilities, you can find him enjoying the outdoors on bike or on foot.](https://www.voiceflow.com/contributors/denys-linkov) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### 5 tips to optimize your LLM intent classification prompts](https://www.voiceflow.com/pathways/5-tips-to-optimize-your-llm-intent-classification-prompts) [](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### How much do ChatGPT versions affect real-world performance?](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance) [](https://www.voiceflow.com/pathways/benchmarking-hybrid-llm-classification-systems) [Applied research\ \ ](https://www.voiceflow.com/pathways-categories/research) [### Benchmarking hybrid LLM classification systems](https://www.voiceflow.com/pathways/benchmarking-hybrid-llm-classification-systems) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-much-do-chatgpt-versions-affect-real-world-performance#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Monorepos can help improve your productivity. Here's how | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) Monorepos can help improve your productivity. Here's how ======================================================== [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) What is a monorepo? Well, if you're tired of managing shared dependencies across multiple services/projects and tip-toeing around update sequencing, monorepos can save you from dependency hell by streamlining the release process of multiple dependencies. [Wikipedia's definition of monorepo](https://en.wikipedia.org/wiki/Monorepo) is this: "In version control systems, a monorepo ('mono' meaning 'single' and 'repo' being short for 'repository') is a software development strategy where code for many projects is stored in the same repository." It is true that when we have multiple libraries that have dependencies between them, the maintenance process turns into a complex task, especially when the libraries depend on each other. .avif) To resolve this problem, we have adopted the monorepo structure:  Image courtesy of [Toptal](https://www.toptal.com/) For Javascript/Typescript there are a lot of libraries that can help us build monorepos. The latest versions of the most common JS/TS package managers like {% c-line %}yarn{% c-line-end %} or {% c-line %}npm{% c-line-end %} accept the monorepo structure. For our monorepos we are going to use the {% c-line %}lerna{% c-line-end%} library and {% c-line %}yarn{% c-line-end %} workspaces. Monorepo structure ------------------ For the monorepo, we are going to look at the following structure: a) We are going to have a root {% c-line %}/package.json{% c-line-end %}where the common configurations will be. With this change, we will not have duplicate, obsolete or misconfiguration in our packages because it will be managed in the same place: 1. Lint-staged 2. Husky 3. Prettier 4. Eslint 5. IDEs configurations b) In the {% c-line %}/packages/{% c-line-end %} folder, we will have all the packages. Those packages will have only the necessary files, which means that only the specific configuration will be there. For example, there we can find the TS build configuration or the Mocha configuration for testing. This is a cleaner way to manage and maintain a monorepo. Find below the final structure of the monorepo:  Typical Repo Structure (Image courtesy of Sharath Holla) Conventional commits with Monorepo ---------------------------------- {% c-line %}lerna{% c-line-end %} supports [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) . By adding the flag {% c-line %}conventionalCommits {% c-line-end %} to the {% c-line %}lerna.json{% c-line-end %} file we can start committing using the conventional commit syntax. So how does it work if we have multiple packages in one repo? 1. {% c-line %}Lerna{% c-line-end %} will read all commits 2. Those commits will be checked 3. The files that are involved in each commit will be determined 4. After reading that information the new versions will be calculated Let's see an example that will clarify a lot about how it works. Imagine that we have two packages:{% c-line %}B{% c-line-end %} which has a dependency on {% c-line %}A {% c-line-end %}. Let's see some common scenarios: 1. If we make a change on package {% c-line %}A{% c-line-end %} and then we create a commit with the following commit message: {% c-line %}fix: some fixes done {% c-line-end %} Lerna will detect that there is a change in package {% c-line %}A{% c-line-end %} and it will update the minor version of that package. As package {% c-line %}A{% c-line-end %} is a dependency of package {% c-line %}B{% c-line-end %}, Lerna will update the minor version of that package too. 2. If we make a change on package {% c-line %}B{% c-line-end %}, and then we create a commit with the following commit message: fix: some fixes done. {% c-line %}Lerna{% c-line-end %} will update its minor version and as it does not have any dependencies with other packages, no more packages will be updated. 3. If we make a change on package {% c-line %}B{% c-line-end %} and package {% c-line %}A{% c-line-end %}: If we commit all the changes together and if the commit message has something like {% c-line %}fix: some fixes done{% c-line-end %} then both packages will be updated and package {% c-line %}B{% c-line-end %} will include the new version of package {% c-line %}A{% c-line-end %}. The next version calculation is generated when the CI process runs the command {% c-line %}lerna publish{% c-line-end %} command. SonarCloud and Monorepo ----------------------- As we have only one repository with independent packages we have to configure SoncarCloud accordingly to the monorepo structure. There are 2 ways to do that: 1\. The monorepo configuration is actually supported by SonarCloud natively. Not only for JS/TS process but also for other languages. You can see the instructions here: [https://sonarcloud.io/documentation/analysis/setup-monorepo/](https://sonarcloud.io/documentation/analysis/setup-monorepo/) . With this configuration, you have to create a new SonarCloud project per each package in the monorepo and then run multiple SonarCloud Scans. 2\. The other option is to create a single SonarCloud project and different SonarCloud sub-modules. For this configuration you need a {% c-line %}[sonar-project.properties](http://sonar-project.properties/) {% c-line-end %} file in the root folder and one per package as well. The important thing in this configuration is that we only have to run only one SonarCloud Scan. On the GUI you will find the monorepo project with the information of all its modules in a single place:  We recommend the second option because it is more optimal in terms of SonarCloud Scan executions. The yarn install process ------------------------ Using a monorepo structure will change the yarn install command due to the fact that we are using {% c-line %}yarn workspaces{% c-line-end %} and {% c-line %}lerna{% c-line-end %}. Starting with {% c-line %}yarn workspaces{% c-line-end %} we will see the following improvements: 1. The {% c-line %}node\_modules{% c-line-end %} will be installed only in one place and **ONLY** the needed **BINARIES** will be copied to the sub-modules 2. Only one {% c-line %}yarn.lock{% c-line-end %} file will be generated and is the only one we have to track, verify and store Following with {% c-line %}lerna{% c-line-end %}, the command {% c-line %}lerna bootstrap{% c-line-end %} will be executed immediately after the yarn install command. This command is a key because it will do the following tasks: 1. Link all shared dependencies creating symlinks 2. Run {% c-line %}yarn run prepublish{% c-line-end %} in all bootstrapped packages 3. Run {% c-line %}yarn run prepare{% c-line-end %} in all bootstrapped packages  Image courtesy of Brigad Engineering Blog/Thibault Malbranche Local package linking/unlinking flow during development ------------------------------------------------------- When a {% c-line %}yarn{% c-line-end %} install command is executed, a linking between all shared libraries will be created automatically. It is similar to the {% c-line %}yarn link{% c-line-end %} command but much more powerful. We have to specify the specific version of the packages in the dependencies so with that {% c-line %}lerna{% c-line-end %} knows that there is a dependency and it will create a symlink automatically:  Image courtesy of Naresh Bhatia Continuous integration using CircleCI ------------------------------------- Since we introduced our first monorepo, there were some improvements on the pipelines that had to be applied in order to reduce the build times and do things more intelligently. Because of that, we needed to develop some improvements in the monorepos that we have currently. There are monorepos that have microservices as a {% c-line %}package{% c-line-end %} and there are monorepos that only contain libraries and there are others that have a mix. This is why we introduced the following improvements depending on the monorepo composition. The main change introduced for the monorepos is that not all {% c-line %}packages{% c-line-end %} have changes in every commit or in every PR. Due to this, we need to implement a multi-level change detection that allows us to determine which job or workflow we need to execute. ### Changes detection from the master branch Thanks to the new [dynamic configurations](https://circleci.com/docs/2.0/dynamic-config/) from CircleCI we can trigger a configuration file with some toggles depending on the changes of the current branch and comparing it with {% c-line %}master{% c-line-end %} branch. So if we take a look at the monorepo and its CircleCI config, we will see the pattern explained above:  Here, you can see that depending on the changes, and more specifically, depending on the packages that have changed, we will set some {% c-line %}vars{% c-line-end %} to {% c-line %}true{% c-line-end %}. After specifying the {% c-line %}env{% c-line-end %} vars, we will trigger the {% c-line %}continue-config.yaml{% c-line-end %} configuration file with those values properly set.  The new CircleCI configuration will trigger the workflows and the jobs required depending on the changes from {% c-line %}master{% c-line-end %} branch. This is a great improvement, but must move forward and improve. Why is this pattern not enough? Well, it's because it only looks at the changes from the {% c-line %}master{% c-line-end %} branch instead of the last commit. Because of this, we can make an addition: determine the packages that have changed in every single commit and execute the jobs and commands in those specific packages along with the common ones. ### Changes detection from the last commit To determine if a pipeline has to be executed or not, we have implemented two mechanisms that detect the changes from the last commit. 1\. Determine if a job has to be executed: a) In the last chapter, we saw that the **dynamic configurations** detects changes from {% c-line %}master{% c-line-end %}, which means that when we change a file from a {% c-line %}package{% c-line-end %}, the toggle will be always be set as true, so all the jobs and workflows that have that toggle will be executed. We have also added a detection in all the jobs that checks if a job has to be executed or not depending on the changes in the last commit. For this, we have created the {% c-line %}command stop\_if\_no\_changes{% c-line-end %}:  2\. Determine if a command has to be executed: there are some jobs that are common to all the packages like unit tests, integration tests, lint, etc. For that, we have created a command called {% c-line %}exec\_command\_monorepo{% c-line-end %} that only executes those tasks in the {% c-line %}packages{% c-line-end %} that have changed from the last commits:  These commands will detect the changes against the last commit and will determine the package that has to run that command. With that, we have a smarter pipeline! Code ---- Here you can find a Proof of Concept with everything explained in this blog post: [https://github.com/voiceflow/poc-monorepo-ci](https://github.com/voiceflow/poc-monorepo-ci) Monerepos are awesome --------------------- As you can see, Monorepos can help us a lot during the development process and during the release process. With the setup and structure explained above, we can focus on developing new features and/or fixing bugs instead of upgrading a number of repositories. With Monorepos - everything is automated. To resolve this problem, we have adopted the monorepo structure:  Image courtesy of [Toptal](https://www.toptal.com/) For Javascript/Typescript there are a lot of libraries that can help us build monorepos. The latest versions of the most common JS/TS package managers like {% c-line %}yarn{% c-line-end %} or {% c-line %}npm{% c-line-end %} accept the monorepo structure. For our monorepos we are going to use the {% c-line %}lerna{% c-line-end%} library and {% c-line %}yarn{% c-line-end %} workspaces. Monorepo structure ------------------ For the monorepo, we are going to look at the following structure: a) We are going to have a root {% c-line %}/package.json{% c-line-end %}where the common configurations will be. With this change, we will not have duplicate, obsolete or misconfiguration in our packages because it will be managed in the same place: 1. Lint-staged 2. Husky 3. Prettier 4. Eslint 5. IDEs configurations b) In the {% c-line %}/packages/{% c-line-end %} folder, we will have all the packages. Those packages will have only the necessary files, which means that only the specific configuration will be there. For example, there we can find the TS build configuration or the Mocha configuration for testing. This is a cleaner way to manage and maintain a monorepo. Find below the final structure of the monorepo:  Typical Repo Structure (Image courtesy of Sharath Holla) Conventional commits with Monorepo ---------------------------------- {% c-line %}lerna{% c-line-end %} supports [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) . By adding the flag {% c-line %}conventionalCommits {% c-line-end %} to the {% c-line %}lerna.json{% c-line-end %} file we can start committing using the conventional commit syntax. So how does it work if we have multiple packages in one repo? 1. {% c-line %}Lerna{% c-line-end %} will read all commits 2. Those commits will be checked 3. The files that are involved in each commit will be determined 4. After reading that information the new versions will be calculated Let's see an example that will clarify a lot about how it works. Imagine that we have two packages:{% c-line %}B{% c-line-end %} which has a dependency on {% c-line %}A {% c-line-end %}. Let's see some common scenarios: 1. If we make a change on package {% c-line %}A{% c-line-end %} and then we create a commit with the following commit message: {% c-line %}fix: some fixes done {% c-line-end %} Lerna will detect that there is a change in package {% c-line %}A{% c-line-end %} and it will update the minor version of that package. As package {% c-line %}A{% c-line-end %} is a dependency of package {% c-line %}B{% c-line-end %}, Lerna will update the minor version of that package too. 2. If we make a change on package {% c-line %}B{% c-line-end %}, and then we create a commit with the following commit message: fix: some fixes done. {% c-line %}Lerna{% c-line-end %} will update its minor version and as it does not have any dependencies with other packages, no more packages will be updated. 3. If we make a change on package {% c-line %}B{% c-line-end %} and package {% c-line %}A{% c-line-end %}: If we commit all the changes together and if the commit message has something like {% c-line %}fix: some fixes done{% c-line-end %} then both packages will be updated and package {% c-line %}B{% c-line-end %} will include the new version of package {% c-line %}A{% c-line-end %}. The next version calculation is generated when the CI process runs the command {% c-line %}lerna publish{% c-line-end %} command. SonarCloud and Monorepo ----------------------- As we have only one repository with independent packages we have to configure SoncarCloud accordingly to the monorepo structure. There are 2 ways to do that: 1\. The monorepo configuration is actually supported by SonarCloud natively. Not only for JS/TS process but also for other languages. You can see the instructions here: [https://sonarcloud.io/documentation/analysis/setup-monorepo/](https://sonarcloud.io/documentation/analysis/setup-monorepo/) . With this configuration, you have to create a new SonarCloud project per each package in the monorepo and then run multiple SonarCloud Scans. 2\. The other option is to create a single SonarCloud project and different SonarCloud sub-modules. For this configuration you need a {% c-line %}[sonar-project.properties](http://sonar-project.properties/) {% c-line-end %} file in the root folder and one per package as well. The important thing in this configuration is that we only have to run only one SonarCloud Scan. On the GUI you will find the monorepo project with the information of all its modules in a single place:  We recommend the second option because it is more optimal in terms of SonarCloud Scan executions. The yarn install process ------------------------ Using a monorepo structure will change the yarn install command due to the fact that we are using {% c-line %}yarn workspaces{% c-line-end %} and {% c-line %}lerna{% c-line-end %}. Starting with {% c-line %}yarn workspaces{% c-line-end %} we will see the following improvements: 1. The {% c-line %}node\_modules{% c-line-end %} will be installed only in one place and **ONLY** the needed **BINARIES** will be copied to the sub-modules 2. Only one {% c-line %}yarn.lock{% c-line-end %} file will be generated and is the only one we have to track, verify and store Following with {% c-line %}lerna{% c-line-end %}, the command {% c-line %}lerna bootstrap{% c-line-end %} will be executed immediately after the yarn install command. This command is a key because it will do the following tasks: 1. Link all shared dependencies creating symlinks 2. Run {% c-line %}yarn run prepublish{% c-line-end %} in all bootstrapped packages 3. Run {% c-line %}yarn run prepare{% c-line-end %} in all bootstrapped packages  Image courtesy of Brigad Engineering Blog/Thibault Malbranche Local package linking/unlinking flow during development ------------------------------------------------------- When a {% c-line %}yarn{% c-line-end %} install command is executed, a linking between all shared libraries will be created automatically. It is similar to the {% c-line %}yarn link{% c-line-end %} command but much more powerful. We have to specify the specific version of the packages in the dependencies so with that {% c-line %}lerna{% c-line-end %} knows that there is a dependency and it will create a symlink automatically:  Image courtesy of Naresh Bhatia Continuous integration using CircleCI ------------------------------------- Since we introduced our first monorepo, there were some improvements on the pipelines that had to be applied in order to reduce the build times and do things more intelligently. Because of that, we needed to develop some improvements in the monorepos that we have currently. There are monorepos that have microservices as a {% c-line %}package{% c-line-end %} and there are monorepos that only contain libraries and there are others that have a mix. This is why we introduced the following improvements depending on the monorepo composition. The main change introduced for the monorepos is that not all {% c-line %}packages{% c-line-end %} have changes in every commit or in every PR. Due to this, we need to implement a multi-level change detection that allows us to determine which job or workflow we need to execute. ### Changes detection from the master branch Thanks to the new [dynamic configurations](https://circleci.com/docs/2.0/dynamic-config/) from CircleCI we can trigger a configuration file with some toggles depending on the changes of the current branch and comparing it with {% c-line %}master{% c-line-end %} branch. So if we take a look at the monorepo and its CircleCI config, we will see the pattern explained above:  Here, you can see that depending on the changes, and more specifically, depending on the packages that have changed, we will set some {% c-line %}vars{% c-line-end %} to {% c-line %}true{% c-line-end %}. After specifying the {% c-line %}env{% c-line-end %} vars, we will trigger the {% c-line %}continue-config.yaml{% c-line-end %} configuration file with those values properly set.  The new CircleCI configuration will trigger the workflows and the jobs required depending on the changes from {% c-line %}master{% c-line-end %} branch. This is a great improvement, but must move forward and improve. Why is this pattern not enough? Well, it's because it only looks at the changes from the {% c-line %}master{% c-line-end %} branch instead of the last commit. Because of this, we can make an addition: determine the packages that have changed in every single commit and execute the jobs and commands in those specific packages along with the common ones. ### Changes detection from the last commit To determine if a pipeline has to be executed or not, we have implemented two mechanisms that detect the changes from the last commit. 1\. Determine if a job has to be executed: a) In the last chapter, we saw that the **dynamic configurations** detects changes from {% c-line %}master{% c-line-end %}, which means that when we change a file from a {% c-line %}package{% c-line-end %}, the toggle will be always be set as true, so all the jobs and workflows that have that toggle will be executed. We have also added a detection in all the jobs that checks if a job has to be executed or not depending on the changes in the last commit. For this, we have created the {% c-line %}command stop\_if\_no\_changes{% c-line-end %}:  2\. Determine if a command has to be executed: there are some jobs that are common to all the packages like unit tests, integration tests, lint, etc. For that, we have created a command called {% c-line %}exec\_command\_monorepo{% c-line-end %} that only executes those tasks in the {% c-line %}packages{% c-line-end %} that have changed from the last commits:  These commands will detect the changes against the last commit and will determine the package that has to run that command. With that, we have a smarter pipeline! Code ---- Here you can find a Proof of Concept with everything explained in this blog post: [https://github.com/voiceflow/poc-monorepo-ci](https://github.com/voiceflow/poc-monorepo-ci) Monerepos are awesome --------------------- As you can see, Monorepos can help us a lot during the development process and during the release process. With the setup and structure explained above, we can focus on developing new features and/or fixing bugs instead of upgrading a number of repositories. With Monorepos - everything is automated.  RECOMMENDED [](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity#) PUBLISHED September 12, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Building a virtual agent from scratch? Start here.](https://www.voiceflow.com/pathways/building-a-virtual-agent-from-scratch-start-here) [](https://www.voiceflow.com/pathways/webpage-assistant-voiceflow-dialog-api) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Create a webpage to talk to your assistant using Voiceflow Dialog API](https://www.voiceflow.com/pathways/webpage-assistant-voiceflow-dialog-api) [.avif)](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Monorepos can help improve your productivity. Here's how](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-monorepos-can-help-improve-your-productivity#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Every app will be a voice app | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) Every app will be a voice app ============================= [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) When raising money for Voiceflow in 2018, the most common question I heard from investors was _“what is the killer voice app?”_. I hadn’t heard the term “killer app” before, but it quickly became clear they were referring to the _blockbuster billion dollar apps_ that now define the mobile app ecosystem — think the Angry Birds, Instagrams, and UBERs of the world. Flash forward a year later and I’m still working toward finding a _true_ answer, so I’ve gone ahead and written this post. We’ve been living in the era of mobile for more than a decade now and **we are quick to compare voice assistants in their infancy against mobile in its maturity.** Platforms follow the curve of the [innovator’s dilemma](https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma) where today’s niche technologies, like voice, will become tomorrow’s dominant platforms more slowly than expected when measured in months, but faster when measured in years. This post is a combination of a couple different ideas we’ve been thinking about at Voiceflow. It discusses: \- What “voice” and voice apps are \- Why we haven’t had a blockbuster voice app yet, and \- ultimately the reasoning behind why I think every app will be a voice app in the near future.  What is a “voice app”? ------------------------- If you asked the average person on the street today what an “app” is they would more often than not point to one of the square icons on their phone. When asked what a “voice app” is, they’d likely guess and land somewhere around a mobile app with optional voice commands. There isn’t a good, or even remotely agreed upon definition of what a “voice app” is — but the average guess isn’t far off. > "A voice app is an application that utilizes voice as a primary input interface." This definition is simple and encompasses voice apps across _all platforms_. The most important part of this definition is that it allows for apps that have touch screens, keyboards and other interfaces to still be [“voice apps”](https://www.voiceflow.com/blog/6-differences-between-mobile-apps-and-voice-apps) . **The only criteria that matters for an app to be a voice app is it having voice interaction capabilities that are a _primary interface_** , not an afterthought, gimmick, or shortlist of commands. A simple test for whether an app is a voice app is if you could use the app successfully and wholly with only your voice. If so, it’s a voice app irregardless of whether users choose to even use their voice. ### Voice is not a platform, and it won’t replace mobile It’s become common for people to say voice is the next “platform shift”, meaning the next major way for people to interact with technology on a daily, if not hourly basis. The previous platform shift was desktop to mobile, and many now say mobile to voice. The problem with this is that “voice” is an interface shift, not a platform shift. Platforms in the sense of a platform shift are the foundational technologies that facilitate app makers to connect with app consumers, think smartphones, computers and your voice assistant like Alexa or Google Assistant. Interfaces are the technologies that allow us to interact with platforms, such as keyboards, touch screens and your voice. An interface shift has occurred when the way we expect to interact with mainstream technology changes. It’s unlikely today you’ll see a phone with a keyboard instead of a touchscreen or a computer with only a command line instead of graphics — these were interface shifts. I believe [Conversational AIs](https://www.voiceflow.com/glossary/conversational-artificial-intelligence-cai) , like Alexa and Google Assistant, are the next platform shift while voice interfaces represent the next interface shift. When shifts happen the previous technology doesn’t go away but settles into use cases where it’s best suited with its unique abilities whilst the dominant platform or interface becomes the default. We still use keyboards and desktops alongside our touchscreens and mobile phones — it’s the technologies we rely on most and that capture our collective imagination that have shifted. Voice will not replace mobile because voice is not a platform, it’s an interface. Mobile is likely to continue being the largest platform for voice interface use both in the present and near future. What will dethrone mobile however, are omni-channel voice assistants, but that’s a topic for another post. ### Platforms are always written off at the start Platforms, like most technologies, follow Clayton Christensen’s famous theory of “The Innovator’s Dilemma”. New platforms are created to solve a particular use case that is impossible with pre-existing platforms, such as mobile and personal computing. At launch these new platforms solve only for their niche use case and thus have far less functionality than their current mainstream versions. Because of this apparent lack in functionality compared to the mature dominant platforms of the day, they are quickly written off as being non-useful. Personal computers were written off as being only useful for computation, smartphones for mobile email, and voice assistants for questions, weather and music. Over time, the dominant platform saturates in functionality and has diminishing performance returns for every new version (sounds a lot like mobile phones). Meanwhile, the new platform is able to continue adding functionality at rapid pace, expanding its capabilities and range of use cases. Soon, the new platform is at parity with the previous platform for many use cases, like commerce and entertainment, and then suddenly the new platform shifts into the foreground as dominant. Platforms rarely go away and it’s likely you still have a desktop as well as a mobile phone. However the platform that is top-of-mind for the public changes hands when the new platform and previous platforms fortify their hold on platform-specific “killer apps” — think desktop for photo editing, or mobile and Snapchat.  The Innovator’s Dilemma developed by Clay Christensen ### So, where are the “killer apps” for Alexa and Google? It’s too early to tell what the killer apps will be for voice because voice assistants are still adding functionality at a rapid pace. As features are added additional use cases for “killer apps” will be unlocked in a similar fashion to how GPS on mobile unlocked UBER, cameras unlocked Snapchat, or vector graphics on computers unlocked Adobe Illustrator. We suffer from a strong recency bias when searching for killer apps on voice assistants against the backdrop of today’s mobile apps which have had a decade to mature. Voice assistants are still limited in functionality and have tremendous room to grow as “killer apps” are unlocked through future platform additions. An interesting note is that many of the best early use cases for voice assistant apps are business-facing and thus have less visibility despite enormous impact. Many businesses are creating simple voice apps to perform database queries or run calculations where they’re benefitting from the speed of input and lack of physical touch. These aren’t the “Angry Birds” that defined an entire platform, but they’re making a tangible difference across a wide number of industries. ### How to find a killer voice app use case It’s hard to imagine exactly what the killer apps will be, and when and where we’ll find see them emerge. If it was easy, we’d all be rich or too late. We know from investor track records with previous platforms that no one can predict with perfect accuracy what the killer apps will be for a new platform every time. Indeed, predicting successful use cases is likely better served by the phrase “I’ll know it when I see it”. What we can do however is think through a simple framework to know where to look. The best place to start looking for killer use cases within a new platform is to think about the limitations of previous platforms and explore what the new platform makes possible. It’s not enough for the new platform to make a previous platform’s task only slightly better. A killer use case for a new platform does what previous platforms could not do, or it does it at least 10x better (the rule of magnitude). In the case of voice assistants, what’s now possible with the current functionality is use cases where you can’t use your hands or have a line of sight — scenarios such as driving, cooking, or manual labor. **Tasks that are 10x better with voice assistants are those which are highly repetitive where you know exactly what you want to do** — such as asking a question, changing a song, or making a routine purchase like coffee for your morning commute. ### The opportunity for voice app startups There’s a popular, simple idea that there will be a similar successful startup category present for every platform — ie “we’re the X” for “platform”. I don’t believe this is true as every platform has unique challenges and opportunities, however it makes for easy startup pitches and is justified by hindsight bias and pattern recognition. When a voice app company pitches themselves as an existing mobile/web company “for voice”, it’s clear that voice does not make this idea possible as evident by the existence of the company they’re mimicking. With this in mind we have to think long and hard as to whether the service being a voice app delivers a true 10x user experience over the existing platforms. If there is a performance increase through adding a voice interface but it’s not an order of magnitude improvement, it’s likely the startup opportunity is small and the original company will acquire a new voice app or build it themselves eventually. If the startup doesn’t deliver a 10x improvement, they’ll have a hard time winning customers from other platforms and thus won’t be able to build a large company fast enough to fend off the original company when they do launch their voice interface. The platform shift from static to mobile computing was enormous but it unlocked millions of use cases and opportunities. Mobile to voice assistants is less of an upfront dramatic shift because voice assistants, without additional functionality, already help users move technology from our hands to our pockets, whereas mobile moved technology from the office to anywhere. Voice assistants today are still a profound change, but it won’t unlock the same scale of opportunities as quickly as mobile computing did. Voice assistants will begin to have a more profound impact through the addition of further features, notably visuals, authentication, and true proactivity. Today voice assistants are mostly reactive systems similar to mobile/desktop operating systems, with the only major difference being they’ve adopted a voice interface. Longer term, voice assistants have the potential and proper foundation to become the first proactive operating systems that will truly live up to their “assistant” name and deliver massive impact. These conversational AIs are cloud-based operating systems and can thus live on any device with a wifi connection to become the foundational platform we interact with on a daily basis through a proactive, truly helpful assistant. These omni-channel assistants will allow companies to carry consistent and contextually intelligent conversations with consumers across channels (SMS, Messenger, IVR, etc) as they go about their day. This future is far away, but closer than we think. The opportunity for a voice app startup to compete with existing solutions today comes through great conversation design and leveraging the unique functionalities of being a voice app living on a voice assistant. Features such as non-linear app structures, being entirely cloud based, and the omnipresent nature of being accessible across millions of device types are all strengths that when combined with strong design can yield 10x improvements. An immediate example that comes to mind is [DriveTime](https://www.drivetime.fm/) which is building the HQ Trivia of voice. Their team will be able to build-up voice specific content and great conversation design practices for millions of commuters whom cannot reasonably use a visual-only HQ Trivia on the go, thus a 10x improvement. By the time HQ Trivia or other mobile visual content producers become voice apps, they’ll be far behind DriveTime whom will have developed an experienced team with great conversation design. ### Visuals changes everything An interface is a way to input and output data between user and computer. Each interface has different pros and cons — voice has a fast input speed, and slow output speed. Imagine for a second you’re choosing a Netflix movie with a visual interface. You would have to input using that visual keyboard everyone hates, but then receive a visual list of all the movie results. This experience had a slow input, and fast output. Conversely, the same experience with only voice would have you say the movie you want to search quickly, but then have Netflix list out verbally all the movie results — fast input, slow output. The slow output limitation of voice interfaces restricts use cases where voice interfaces are usable as giving users results of more than a sentence creates cognitive load where the user need remember their options. Further, voice interfaces inhibit browsing which is key to enabling great user experiences. For example, you wouldn’t order an UBER through voice unless you knew exactly where you needed to go because voice interfaces have a built-in turn timer that cuts us off when we’re not speaking. If you had to check a map, ask a friend where to go, or any activity that takes more than a few seconds during a conversation— the interface times out. **If our app has both a screen and voice interface, we get the best of both worlds** — a fast interface and fast output with a flexible conversation timer. The user can go as fast as they want and use only the voice interface, also allowing the user to not touch the device, or slow the conversation down and visually see their options. This kind of app interface is a “voice-first” interface where the first interaction is voice (thus “voice first”), and the second is visual. Many voice-first devices pair a touchscreen with a voice interface allowing for the optionality of a completely visual interface should the user not want to use voice, perhaps because they’re in a public setting. ### The rise of multi-interface apps As voice interfaces grow in prominence we’ll see the rise of multi-interface apps which are able to handle voice-only interactions, voice-first interactions, and visual-only interactions. These multi-interface apps will provide a superior user-experience that is flexible to the user’s situation and will become the new standard set for app development. Designing multi-interface apps requires teams with both an understanding of traditional UI/UX development in addition to CxD (Conversation Design). Creating apps that are truly multi-interfaced will require a deep understanding of conversation design as the development of the app will have to begin with conversation design, and layer on visuals later. The current approach to voice app design is to design the visual app first and layer the voice interface on later. This approach will not work for multi-interface apps as visual app design is linear whereas conversations are not. To create a good multi-interface design, product teams will need to first design the non-linear conversation paths that could manage voice-only interaction, then layer in visuals for voice-first interaction, and finally create visual interaction paths for visual-only interaction. ### Final thoughts In the near future, every app will become a voice app, and eventually a multi-interface app. As the world emerges from a global pandemic users will demand better user experiences that are flexible to how we want to interact with technology at any given moment/context. People won’t want to touch dirty elevator buttons, step out of the shower to change a song, or pull out their phone to order a coffee — in these cases and many more, it’s easier to talk than type. We can’t easily predict today what the billion dollar voice apps will look like, or when they’ll come. All we know is that there are use cases today that voice makes possible, and that as the platforms grow in functionality so will the number of “killer” apps. What is a “voice app”? ------------------------- If you asked the average person on the street today what an “app” is they would more often than not point to one of the square icons on their phone. When asked what a “voice app” is, they’d likely guess and land somewhere around a mobile app with optional voice commands. There isn’t a good, or even remotely agreed upon definition of what a “voice app” is — but the average guess isn’t far off. > "A voice app is an application that utilizes voice as a primary input interface." This definition is simple and encompasses voice apps across _all platforms_. The most important part of this definition is that it allows for apps that have touch screens, keyboards and other interfaces to still be [“voice apps”](https://www.voiceflow.com/blog/6-differences-between-mobile-apps-and-voice-apps) . **The only criteria that matters for an app to be a voice app is it having voice interaction capabilities that are a _primary interface_** , not an afterthought, gimmick, or shortlist of commands. A simple test for whether an app is a voice app is if you could use the app successfully and wholly with only your voice. If so, it’s a voice app irregardless of whether users choose to even use their voice. ### Voice is not a platform, and it won’t replace mobile It’s become common for people to say voice is the next “platform shift”, meaning the next major way for people to interact with technology on a daily, if not hourly basis. The previous platform shift was desktop to mobile, and many now say mobile to voice. The problem with this is that “voice” is an interface shift, not a platform shift. Platforms in the sense of a platform shift are the foundational technologies that facilitate app makers to connect with app consumers, think smartphones, computers and your voice assistant like Alexa or Google Assistant. Interfaces are the technologies that allow us to interact with platforms, such as keyboards, touch screens and your voice. An interface shift has occurred when the way we expect to interact with mainstream technology changes. It’s unlikely today you’ll see a phone with a keyboard instead of a touchscreen or a computer with only a command line instead of graphics — these were interface shifts. I believe [Conversational AIs](https://www.voiceflow.com/glossary/conversational-artificial-intelligence-cai) , like Alexa and Google Assistant, are the next platform shift while voice interfaces represent the next interface shift. When shifts happen the previous technology doesn’t go away but settles into use cases where it’s best suited with its unique abilities whilst the dominant platform or interface becomes the default. We still use keyboards and desktops alongside our touchscreens and mobile phones — it’s the technologies we rely on most and that capture our collective imagination that have shifted. Voice will not replace mobile because voice is not a platform, it’s an interface. Mobile is likely to continue being the largest platform for voice interface use both in the present and near future. What will dethrone mobile however, are omni-channel voice assistants, but that’s a topic for another post. ### Platforms are always written off at the start Platforms, like most technologies, follow Clayton Christensen’s famous theory of “The Innovator’s Dilemma”. New platforms are created to solve a particular use case that is impossible with pre-existing platforms, such as mobile and personal computing. At launch these new platforms solve only for their niche use case and thus have far less functionality than their current mainstream versions. Because of this apparent lack in functionality compared to the mature dominant platforms of the day, they are quickly written off as being non-useful. Personal computers were written off as being only useful for computation, smartphones for mobile email, and voice assistants for questions, weather and music. Over time, the dominant platform saturates in functionality and has diminishing performance returns for every new version (sounds a lot like mobile phones). Meanwhile, the new platform is able to continue adding functionality at rapid pace, expanding its capabilities and range of use cases. Soon, the new platform is at parity with the previous platform for many use cases, like commerce and entertainment, and then suddenly the new platform shifts into the foreground as dominant. Platforms rarely go away and it’s likely you still have a desktop as well as a mobile phone. However the platform that is top-of-mind for the public changes hands when the new platform and previous platforms fortify their hold on platform-specific “killer apps” — think desktop for photo editing, or mobile and Snapchat.  The Innovator’s Dilemma developed by Clay Christensen ### So, where are the “killer apps” for Alexa and Google? It’s too early to tell what the killer apps will be for voice because voice assistants are still adding functionality at a rapid pace. As features are added additional use cases for “killer apps” will be unlocked in a similar fashion to how GPS on mobile unlocked UBER, cameras unlocked Snapchat, or vector graphics on computers unlocked Adobe Illustrator. We suffer from a strong recency bias when searching for killer apps on voice assistants against the backdrop of today’s mobile apps which have had a decade to mature. Voice assistants are still limited in functionality and have tremendous room to grow as “killer apps” are unlocked through future platform additions. An interesting note is that many of the best early use cases for voice assistant apps are business-facing and thus have less visibility despite enormous impact. Many businesses are creating simple voice apps to perform database queries or run calculations where they’re benefitting from the speed of input and lack of physical touch. These aren’t the “Angry Birds” that defined an entire platform, but they’re making a tangible difference across a wide number of industries. ### How to find a killer voice app use case It’s hard to imagine exactly what the killer apps will be, and when and where we’ll find see them emerge. If it was easy, we’d all be rich or too late. We know from investor track records with previous platforms that no one can predict with perfect accuracy what the killer apps will be for a new platform every time. Indeed, predicting successful use cases is likely better served by the phrase “I’ll know it when I see it”. What we can do however is think through a simple framework to know where to look. The best place to start looking for killer use cases within a new platform is to think about the limitations of previous platforms and explore what the new platform makes possible. It’s not enough for the new platform to make a previous platform’s task only slightly better. A killer use case for a new platform does what previous platforms could not do, or it does it at least 10x better (the rule of magnitude). In the case of voice assistants, what’s now possible with the current functionality is use cases where you can’t use your hands or have a line of sight — scenarios such as driving, cooking, or manual labor. **Tasks that are 10x better with voice assistants are those which are highly repetitive where you know exactly what you want to do** — such as asking a question, changing a song, or making a routine purchase like coffee for your morning commute. ### The opportunity for voice app startups There’s a popular, simple idea that there will be a similar successful startup category present for every platform — ie “we’re the X” for “platform”. I don’t believe this is true as every platform has unique challenges and opportunities, however it makes for easy startup pitches and is justified by hindsight bias and pattern recognition. When a voice app company pitches themselves as an existing mobile/web company “for voice”, it’s clear that voice does not make this idea possible as evident by the existence of the company they’re mimicking. With this in mind we have to think long and hard as to whether the service being a voice app delivers a true 10x user experience over the existing platforms. If there is a performance increase through adding a voice interface but it’s not an order of magnitude improvement, it’s likely the startup opportunity is small and the original company will acquire a new voice app or build it themselves eventually. If the startup doesn’t deliver a 10x improvement, they’ll have a hard time winning customers from other platforms and thus won’t be able to build a large company fast enough to fend off the original company when they do launch their voice interface. The platform shift from static to mobile computing was enormous but it unlocked millions of use cases and opportunities. Mobile to voice assistants is less of an upfront dramatic shift because voice assistants, without additional functionality, already help users move technology from our hands to our pockets, whereas mobile moved technology from the office to anywhere. Voice assistants today are still a profound change, but it won’t unlock the same scale of opportunities as quickly as mobile computing did. Voice assistants will begin to have a more profound impact through the addition of further features, notably visuals, authentication, and true proactivity. Today voice assistants are mostly reactive systems similar to mobile/desktop operating systems, with the only major difference being they’ve adopted a voice interface. Longer term, voice assistants have the potential and proper foundation to become the first proactive operating systems that will truly live up to their “assistant” name and deliver massive impact. These conversational AIs are cloud-based operating systems and can thus live on any device with a wifi connection to become the foundational platform we interact with on a daily basis through a proactive, truly helpful assistant. These omni-channel assistants will allow companies to carry consistent and contextually intelligent conversations with consumers across channels (SMS, Messenger, IVR, etc) as they go about their day. This future is far away, but closer than we think. The opportunity for a voice app startup to compete with existing solutions today comes through great conversation design and leveraging the unique functionalities of being a voice app living on a voice assistant. Features such as non-linear app structures, being entirely cloud based, and the omnipresent nature of being accessible across millions of device types are all strengths that when combined with strong design can yield 10x improvements. An immediate example that comes to mind is [DriveTime](https://www.drivetime.fm/) which is building the HQ Trivia of voice. Their team will be able to build-up voice specific content and great conversation design practices for millions of commuters whom cannot reasonably use a visual-only HQ Trivia on the go, thus a 10x improvement. By the time HQ Trivia or other mobile visual content producers become voice apps, they’ll be far behind DriveTime whom will have developed an experienced team with great conversation design. ### Visuals changes everything An interface is a way to input and output data between user and computer. Each interface has different pros and cons — voice has a fast input speed, and slow output speed. Imagine for a second you’re choosing a Netflix movie with a visual interface. You would have to input using that visual keyboard everyone hates, but then receive a visual list of all the movie results. This experience had a slow input, and fast output. Conversely, the same experience with only voice would have you say the movie you want to search quickly, but then have Netflix list out verbally all the movie results — fast input, slow output. The slow output limitation of voice interfaces restricts use cases where voice interfaces are usable as giving users results of more than a sentence creates cognitive load where the user need remember their options. Further, voice interfaces inhibit browsing which is key to enabling great user experiences. For example, you wouldn’t order an UBER through voice unless you knew exactly where you needed to go because voice interfaces have a built-in turn timer that cuts us off when we’re not speaking. If you had to check a map, ask a friend where to go, or any activity that takes more than a few seconds during a conversation— the interface times out. **If our app has both a screen and voice interface, we get the best of both worlds** — a fast interface and fast output with a flexible conversation timer. The user can go as fast as they want and use only the voice interface, also allowing the user to not touch the device, or slow the conversation down and visually see their options. This kind of app interface is a “voice-first” interface where the first interaction is voice (thus “voice first”), and the second is visual. Many voice-first devices pair a touchscreen with a voice interface allowing for the optionality of a completely visual interface should the user not want to use voice, perhaps because they’re in a public setting. ### The rise of multi-interface apps As voice interfaces grow in prominence we’ll see the rise of multi-interface apps which are able to handle voice-only interactions, voice-first interactions, and visual-only interactions. These multi-interface apps will provide a superior user-experience that is flexible to the user’s situation and will become the new standard set for app development. Designing multi-interface apps requires teams with both an understanding of traditional UI/UX development in addition to CxD (Conversation Design). Creating apps that are truly multi-interfaced will require a deep understanding of conversation design as the development of the app will have to begin with conversation design, and layer on visuals later. The current approach to voice app design is to design the visual app first and layer the voice interface on later. This approach will not work for multi-interface apps as visual app design is linear whereas conversations are not. To create a good multi-interface design, product teams will need to first design the non-linear conversation paths that could manage voice-only interaction, then layer in visuals for voice-first interaction, and finally create visual interaction paths for visual-only interaction. ### Final thoughts In the near future, every app will become a voice app, and eventually a multi-interface app. As the world emerges from a global pandemic users will demand better user experiences that are flexible to how we want to interact with technology at any given moment/context. People won’t want to touch dirty elevator buttons, step out of the shower to change a song, or pull out their phone to order a coffee — in these cases and many more, it’s easier to talk than type. We can’t easily predict today what the billion dollar voice apps will look like, or when they’ll come. All we know is that there are use cases today that voice makes possible, and that as the platforms grow in functionality so will the number of “killer” apps.  RECOMMENDED [\ \ ### Designing for voice assistants vs. chatbots](https://www.voiceflow.com/pathways/designing-for-voice-assistants-vs-chatbots) PUBLISHED April 20, 2020 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/conversation-design-best-practices) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The best practices of conversation design](https://www.voiceflow.com/pathways/conversation-design-best-practices) [](https://www.voiceflow.com/pathways/the-5-best-practices-when-designing-for-voice-vs-chat-experiences) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### The 5 best practices when designing for voice vs. chat experiences](https://www.voiceflow.com/pathways/the-5-best-practices-when-designing-for-voice-vs-chat-experiences) [](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Designing conversational bots that fix themselves](https://www.voiceflow.com/pathways/designing-conversational-bots-that-fix-themselves) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/every-app-will-be-a-voice-app#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/every-app-will-be-a-voice-app#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How to build a Microsoft Teams bot with Voiceflow | Pathways [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) How to build a Microsoft Teams bot with Voiceflow ================================================= [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) Microsoft Teams is one of the most important applications for businesses and offers functionality like channels, apps, and chatbots. In this article, we will show you how to build a Microsoft Teams bot using [Bot Builder](https://www.npmjs.com/package/botbuilder) and the [Voiceflow Dialog Manager API](https://www.voiceflow.com/api/dialog-manager) . Prerequisites ------------- The technologies used in this project are listed below: 1. Voiceflow Account 2. VS Code 3. Microsoft 365 developer account. [Sign up if you don't have one already!](https://developer.microsoft.com/en-us/microsoft-365/dev-program) 4. App Studio - look for the app in Teams desktop client and install  **Overview** ------------ Bots are conversational apps that a user can send and/or receive messages to/from the bot. 1. **index.ts:** The TypeScript code that listens to the incoming requests. Using Bot Builder and Express.JS 2. **bot.ts:** Sends a message to Voiceflow using the Dialog Manager API. 3. **teams-app:** Manifest package to be installed in Teams client. 4. **http-client.ts:** Abstract Class that has the common methods of the HTTP Client. 5. **dialog-manager-api.ts:** Class that extends the abstract class explained above and adds all the methods required to interact with the Dialog Manager API. See the instruction below to see how you can create your chatbot package using **App Studio** Microsoft Teams: App Configuration ---------------------------------- Here's the step-by-step how to set up the chatbot and install it on Teams. ### Creating **App Manifest with App Studio** #### **App Details** Open **App Studio** in the Teams client. Click **Create a new app** and fill out all the required fields including the Bot names, descriptions, etc. Generate an App ID.  In the **App URLs** section, include your privacy and TOU webpages. In this example, I am using the placeholder URL, {% c-line %}https://example.com{% c-line-end %}. ### **Bots config** From the left menu, select Capabilities > **Bots**. Then, click **Set up** to configure a new bot. Fill out the bot name, and let's select the Personal scope for now. Next, click **Generate new password**. At the prompt, copy the password. You will need to paste it in your **.env** file in the next step. ### **App Credentials** Copy the ID under your bot name (something looks like {% c-line %}123xx567-123x-...{% c-line-end %}) and paste it as an environment variable in your {% c-line %}.env{% c-line-end %} file. Under **App Passwords**, generate a new password, and copy it. Then paste it in your {% c-line %}.env{% c-line-end %} file. These credentials are used to initialize your bot adapter. (See index.ts). For **Messaging Endpoint**, we will use [ngrok](https://ngrok.com/) . The URL should be something like: {% c-line %}https://.ngrok.io/api/messages{% c-line-end %}. ### **Finish creating the app manifest package** Go to Finish > **Test and distribute**.  If you get any errors, you'll need to fix it. Otherwise, click **Install**. You can also download the zip file that contains {% c-line %}manifest.json{% c-line-end %}, and two icon images to install later or distribute. Voiceflow Configuration ----------------------- **NOTE:** Before we continue, it's important we create a general project on Voiceflow. Let's create the Voiceflow client to work with Voiceflow's cloud using its Dialog Manager API:  The {% c-line %}getclient(){% c-line-end %} function calls the {% c-line %}DialogManagerApi{% c-line-end %} This is the initialization of the API client. As you can see, there are some values that will be added to our {% c-line %}.env file{% c-line-end %}. Let's explain how to obtain these variables. #### **1\. VersionID** To obtain your VersionID, you have to go to your Voiceflow Project:  Then copy the {% c-line %}VERSION\_ID{% c-line-end %} from the URL in your address bar. When you are inside a Voiceflow project, your address bar should have a URL: {% c-line %}https://creator.voiceflow.com/project/{VERSION\_ID}/...{% c-line-end %} #### **2\. apiKey** To obtain the API Key, we need to go to our workspace where we created our General Project. After this, go to the settings of this workspace and click on the developer tab:  Next, click the {% c-line %}Create new API Key{% c-line-end %} button to create a new one:  You'll need to add a name for the new API Key, for example, {% c-line %}ms-teams-bot{% c-line-end %}. Once this is completed, we can click the {% c-line %}Confirm button{% c-line-end %}:  Finally, we have to add these variables to our final {% c-line %}.env{% c-line-end %} file. It should look like this:  Test your bot ------------- In your Microsoft Teams client, we can now try out our personal bot. You can access the bot you installed from the sidebar on your left. Now you have a 1:1 chat interface with the bot. Let's type sending a message.  Deployment ---------- Please note: [Ngrock](https://ngrok.com/) is a great tool; however, this does not work for production environments. To deploy your chatbot, view the following: [Deploy your bot to Azure](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-tutorial-basic-deploy) . **Resources** * [Official Bot Builder Documentation](https://www.npmjs.com/package/botbuilder) - The Official Bot Builder Documentation * [Official Voiceflow's Dialog Manager API Documentation](https://www.voiceflow.com/api/dialog-manager) - Official Voiceflow's Dialog Manager API Documentation * ⚙️ To access all of the code mentioned above, click [here.](https://github.com/xavidop/ms-teams-voiceflow-bot) Conclusion ---------- And so with less than 200 lines of code we now have a Microsoft Teams bot connected to Voiceflow. I hope this gives you a better understanding of the how you can use Voicefow to bring conversational experiences into your very own interfaces and products. Happy coding! **Overview** ------------ Bots are conversational apps that a user can send and/or receive messages to/from the bot. 1. **index.ts:** The TypeScript code that listens to the incoming requests. Using Bot Builder and Express.JS 2. **bot.ts:** Sends a message to Voiceflow using the Dialog Manager API. 3. **teams-app:** Manifest package to be installed in Teams client. 4. **http-client.ts:** Abstract Class that has the common methods of the HTTP Client. 5. **dialog-manager-api.ts:** Class that extends the abstract class explained above and adds all the methods required to interact with the Dialog Manager API. See the instruction below to see how you can create your chatbot package using **App Studio** Microsoft Teams: App Configuration ---------------------------------- Here's the step-by-step how to set up the chatbot and install it on Teams. ### Creating **App Manifest with App Studio** #### **App Details** Open **App Studio** in the Teams client. Click **Create a new app** and fill out all the required fields including the Bot names, descriptions, etc. Generate an App ID.  In the **App URLs** section, include your privacy and TOU webpages. In this example, I am using the placeholder URL, {% c-line %}https://example.com{% c-line-end %}. ### **Bots config** From the left menu, select Capabilities > **Bots**. Then, click **Set up** to configure a new bot. Fill out the bot name, and let's select the Personal scope for now. Next, click **Generate new password**. At the prompt, copy the password. You will need to paste it in your **.env** file in the next step. ### **App Credentials** Copy the ID under your bot name (something looks like {% c-line %}123xx567-123x-...{% c-line-end %}) and paste it as an environment variable in your {% c-line %}.env{% c-line-end %} file. Under **App Passwords**, generate a new password, and copy it. Then paste it in your {% c-line %}.env{% c-line-end %} file. These credentials are used to initialize your bot adapter. (See index.ts). For **Messaging Endpoint**, we will use [ngrok](https://ngrok.com/) . The URL should be something like: {% c-line %}https://.ngrok.io/api/messages{% c-line-end %}. ### **Finish creating the app manifest package** Go to Finish > **Test and distribute**.  If you get any errors, you'll need to fix it. Otherwise, click **Install**. You can also download the zip file that contains {% c-line %}manifest.json{% c-line-end %}, and two icon images to install later or distribute. Voiceflow Configuration ----------------------- **NOTE:** Before we continue, it's important we create a general project on Voiceflow. Let's create the Voiceflow client to work with Voiceflow's cloud using its Dialog Manager API:  The {% c-line %}getclient(){% c-line-end %} function calls the {% c-line %}DialogManagerApi{% c-line-end %} This is the initialization of the API client. As you can see, there are some values that will be added to our {% c-line %}.env file{% c-line-end %}. Let's explain how to obtain these variables. #### **1\. VersionID** To obtain your VersionID, you have to go to your Voiceflow Project:  Then copy the {% c-line %}VERSION\_ID{% c-line-end %} from the URL in your address bar. When you are inside a Voiceflow project, your address bar should have a URL: {% c-line %}https://creator.voiceflow.com/project/{VERSION\_ID}/...{% c-line-end %} #### **2\. apiKey** To obtain the API Key, we need to go to our workspace where we created our General Project. After this, go to the settings of this workspace and click on the developer tab:  Next, click the {% c-line %}Create new API Key{% c-line-end %} button to create a new one:  You'll need to add a name for the new API Key, for example, {% c-line %}ms-teams-bot{% c-line-end %}. Once this is completed, we can click the {% c-line %}Confirm button{% c-line-end %}:  Finally, we have to add these variables to our final {% c-line %}.env{% c-line-end %} file. It should look like this:  Test your bot ------------- In your Microsoft Teams client, we can now try out our personal bot. You can access the bot you installed from the sidebar on your left. Now you have a 1:1 chat interface with the bot. Let's type sending a message.  Deployment ---------- Please note: [Ngrock](https://ngrok.com/) is a great tool; however, this does not work for production environments. To deploy your chatbot, view the following: [Deploy your bot to Azure](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-tutorial-basic-deploy) . **Resources** * [Official Bot Builder Documentation](https://www.npmjs.com/package/botbuilder) - The Official Bot Builder Documentation * [Official Voiceflow's Dialog Manager API Documentation](https://www.voiceflow.com/api/dialog-manager) - Official Voiceflow's Dialog Manager API Documentation * ⚙️ To access all of the code mentioned above, click [here.](https://github.com/xavidop/ms-teams-voiceflow-bot) Conclusion ---------- And so with less than 200 lines of code we now have a Microsoft Teams bot connected to Voiceflow. I hope this gives you a better understanding of the how you can use Voicefow to bring conversational experiences into your very own interfaces and products. Happy coding!  RECOMMENDED [\ \ ### Introducing Voiceflow's GPT-powered AI Builder](https://www.voiceflow.com/pathways/introducing-voiceflows-gpt-powered-ai-builder) PUBLISHED June 30, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [.avif)](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How To build a discord bot with Voiceflow](https://www.voiceflow.com/pathways/how-to-build-a-discord-bot-with-voiceflow) [](https://www.voiceflow.com/pathways/how-to-build-a-microsoft-teams-bot-with-voiceflow) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How to build a Microsoft Teams bot with Voiceflow](https://www.voiceflow.com/pathways/how-to-build-a-microsoft-teams-bot-with-voiceflow) [.avif)](https://www.voiceflow.com/pathways/how-to-leverage-topics-and-components-in-your-conversation-designs) [Industry\ \ ](https://www.voiceflow.com/pathways-categories/industry) [### How to leverage topics and flows in your conversation designs](https://www.voiceflow.com/pathways/how-to-leverage-topics-and-components-in-your-conversation-designs) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-to-build-a-microsoft-teams-bot-with-voiceflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-to-build-a-microsoft-teams-bot-with-voiceflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How to prove the ROI of conversational AI through buyer journey mapping: framework | Pathways [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) How to prove the ROI of conversational AI through buyer journey mapping: Framework ================================================================================== [\ \ Sam Burns\ \ Head of Marketing at Voiceflow](https://www.voiceflow.com/contributors/sam-burns) With the rapid growth of conversational AI teams and channels, proving ROI is a must-have tool for conversation design managers. In the [expert conversations workshop](https://www.youtube.com/watch?v=33HbEu0nHHY) last month, [**Ayesha Saleem**](https://www.linkedin.com/in/ayeshasa?trk=public_post_share-update_update-text) **, c**onversational AI manager at **Instacart**, walked through a simple guide that teams can use to reframe their work, understand high intent buying signals, and prove ROI. I broke that guide down further here for managers to easily reference - especially heading into 2023 planning. ### The goal: map the buying and conversational journeys. Mapping these two journeys allows for the entire organization to have clarity between the customer and conversational journeys. Great assistants are extensions of a brand - which means they impact the business and bottom line. When the two journeys are not aligned, conversational AI channels become a nice-to-have instead of essential to accelerating revenue & customer experiences. Once the high intent moments on both journeys are set, conversation design teams can then move forward with designing, testing and measuring to show how their work clearly moves the needle and generates ROI.  The framework ------------- The framework includes three key parts: business journey milestones, assistant journey milestones, and map out business and assistant journeys. Here’s an example of using the framework for a retail shopping experience focused on activating new sign-ups towards a first purchase. **Step 1** Write down the business journey milestones that the overall team cares about and tracks. For the retail business, a few examples might look like: * New audience / user - high Intent * Browsed product catalogue * Added item to cart * Complete purchase * Returning customer **Step 2** Identify the assistant journey milestones in relation to the business milestones from Step 1. The conversational journey milestone might look like this: * New user = 0 sessions that clicks start button * Reached product carousel in the assistant * Added item from product carousel to cart * Finished checkout flow via component * Returning user, > 1 sessions **Step 3** Now that you have both the business and conversational journeys that matter plotted out to analyze, it’s time to map the both. You’ll notice that your assistant either 1) has many high intent buying moment that you haven’t recognized or 2) needs additional design work to better map to the customer buying journey. The framework then would ideally look something like this:  The established buying journey now has a tangible touchpoint within the assistant which is measurable and trackable. Product, revenue, and conversational AI teams can then all rally behind creating a great customer experience that is trackable and mapped back to revenue at key points. Best of all, conversation design teams will start to see how much of a true impact their assistant has on customer experience and buying behavior in comparison to other business channels. Ready to make these ROI design decisions easier? [Chat with our team.](https://www.voiceflow.com/demo) The framework ------------- The framework includes three key parts: business journey milestones, assistant journey milestones, and map out business and assistant journeys. Here’s an example of using the framework for a retail shopping experience focused on activating new sign-ups towards a first purchase. **Step 1** Write down the business journey milestones that the overall team cares about and tracks. For the retail business, a few examples might look like: * New audience / user - high Intent * Browsed product catalogue * Added item to cart * Complete purchase * Returning customer **Step 2** Identify the assistant journey milestones in relation to the business milestones from Step 1. The conversational journey milestone might look like this: * New user = 0 sessions that clicks start button * Reached product carousel in the assistant * Added item from product carousel to cart * Finished checkout flow via component * Returning user, > 1 sessions **Step 3** Now that you have both the business and conversational journeys that matter plotted out to analyze, it’s time to map the both. You’ll notice that your assistant either 1) has many high intent buying moment that you haven’t recognized or 2) needs additional design work to better map to the customer buying journey. The framework then would ideally look something like this:  The established buying journey now has a tangible touchpoint within the assistant which is measurable and trackable. Product, revenue, and conversational AI teams can then all rally behind creating a great customer experience that is trackable and mapped back to revenue at key points. Best of all, conversation design teams will start to see how much of a true impact their assistant has on customer experience and buying behavior in comparison to other business channels. Ready to make these ROI design decisions easier? [Chat with our team.](https://www.voiceflow.com/demo)  RECOMMENDED [](https://www.voiceflow.com/pathways/how-to-prove-the-roi-of-conversational-ai-through-buyer-journey-mapping-framework#) PUBLISHED TOPICS IN THIS ARTICLE No items found. [\ \ Sam Burns\ \ Head of Marketing at Voiceflow\ \ A Wisconsin fan first, everything else second. His expertise ranges from breaking bots and ad campaigns, to social strategy and events. Sam can also give you the undisputed ranking of every cheese curd option in the Midwest.](https://www.voiceflow.com/contributors/sam-burns) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/conversation-design-needs-a-universal-language) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Conversation design needs a universal language](https://www.voiceflow.com/pathways/conversation-design-needs-a-universal-language) [](https://www.voiceflow.com/pathways/every-app-will-be-a-voice-app) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Every app will be a voice app](https://www.voiceflow.com/pathways/every-app-will-be-a-voice-app) [](https://www.voiceflow.com/pathways/why-you-need-to-consider-designing-for-multimodal-interfaces) [Conversation design\ \ ](https://www.voiceflow.com/pathways-categories/conversation-design) [### Why you need to consider designing for multimodal interfaces](https://www.voiceflow.com/pathways/why-you-need-to-consider-designing-for-multimodal-interfaces) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-to-prove-the-roi-of-conversational-ai-through-buyer-journey-mapping-framework#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-to-prove-the-roi-of-conversational-ai-through-buyer-journey-mapping-framework#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # How to create a simple Telegram bot with Voiceflow | Pathways [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) How to create a simple Telegram bot with Voiceflow ================================================== [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) Telegram is becoming one of the most important messaging applications in the world today. The free and open-source instant messaging software offers many useful features that make it suitable for implementing sophisticated bots that can assist us in a variety of ways. Since its inception, there have been several ways to develop bots for Telegram, however only a has been focused on CxD and providing rich conversational experiences. With Voiceflow's new Dialog Manager API, the conversation design platform stands as the best choice for designing, developing, and integrating a Telegram chatbot.  Prerequisites ------------- Here is what you'll need to create your very own Telegram bot using Voiceflow 1. Telegram Account 2. Voiceflow Account 3. Visual Studio Code Architecture ------------ In this example, we are going to use the Telegraf library for NodeJS projects to interact with our Telegram Bot. As mentioned above, every user interaction in the Telegram bot will be sent to Voiceflow using our Dialog Manager API.  The architecture used to create this experience Create your own bot with BotFather ---------------------------------- First, we need to use [botfather](https://core.telegram.org/bots#) to create our very own bot inside Telegram. BotFather allows you to create new bot accounts and manage existing ones in a few simple clicks. To start the process of creating your bot, you just need to talk to the botfather while following the steps outlined below[.](https://core.telegram.org/bots#) To open a chat with botfather, click on the "**start**" button. We'll need to create a new bot by clicking /newbot command. Next, you can name your bot. In this example, we have named ours "Voiceflow bot". It's that easy. The Telegram setup is now complete! Remember to add your Telegram token to your {% c-line %}.env file{% c-line-end %} in the property {% c-line %}TELEGRAM\_TOKEN{% c-line-end %}.**** Setting up the project ---------------------- Install and run the project: 1\. Clone this repo: {% c-line %}git clone {% c-line-end %} 2\. Install dependencies: {% c-line %}yarn install{% c-line-end %} 3\. Launch project: {% c-line %}yarn start{% c-line-end %} ** Writing the bot’s code** ----------------------------- ### **Telegraf setup** We can create bot by the following code lines:  ### **Voiceflow setup** **NOTE:** It's important to note that you will need to create a General Project type on Voiceflow for this experience to work. Now, let's implement the Voiceflow client to work with Voiceflow's cloud using its Dialog Manager API.  The {% c-line %}getclient(){% c-line-end %} function calls the {% c-line %}DialogManagerApi.getInstance{% c-line-end %} a REST API client that will interact directly with the Dialog Manager API. As you can see there are some values that will be added to our {% c-line %}.env{% c-line-end %} file. Let's explain how to obtain those variables. #### ** 1\. Version ID** To obtain your VersionID you have to go to your Voiceflow Project:  Then copy the {% c-line %}VOICEFLOW\_VERSION\_ID{% c-line-END %} from the URL in your address bar. When you are inside a Voiceflow project, your address bar should have a URL of the form: {% c-line %}https://creator.voiceflow.com/project/{VOICEFLOW\_VERSION\_ID}/...{% c-line-end %} #### **2\. apiKey** To obtain the API Key we have to go to our workspace where we have created our General Project. After this, head to workplace settings and click on the developer tab.  Then we have to click to {% c-line %}Create new API Key{% c-line-end %} button to create a new one:  You need to add a name to the new API Key, for example, {% c-line %}telegram-bot{% c-line-end %}. Once we have filled it, we can click the {% c-line %}Confirm{% c-line-end %} button:  Finally, we have to add these variables to our final {% c-line %}.env file{% c-line-end %}. It should look like this:  Everything is ready. Let's continue with our Telegram bot code. Let's replace the standard start replay for the one below, getting the correct replay from Voiceflow:  Then we replace the {% c-line %}hi{% c-line-end %} utterance for a regex like {% c-line %}(.+){% c-line-end %}. This means that the bot will hear for everything. All the text received we will pass directly to Voiceflow and we manage the state of the conversation: if it is ended or if it is not ended yet:  The response method is in charge of sending the user input in Telegram to Voiceflow using the runtime client SDK. This allows us to process the response:  In the example above you'll notice different types of responses including {% c-line %}SPEAK{% c-line-end %}, {% c-line %}VISUAL{% c-line-end %} and {% c-line %}AUDIO{% c-line-end %}responses. **Running the Telegram Bot** ----------------------------  You're all set to test this experience in Telegram ### **Help us help you** Interested in helping us improve this experience? You can submit a PR by following the steps below. 1. Fork this repo 2. Clone your fork 3. Code 4. 🤓 5. Test your changes 6. Submit a PR! ### **Resources** * [Official Telegraf Documentation](https://www.npmjs.com/package/ask-sdk) - The Official Telegraf Documentation * [Official Voiceflow's Runtime Dialog Manager Documentation](https://www.voiceflow.com/api/dialog-manager) - Official Voiceflow Dialog Manager API * To access all of the code mentioned above, click [here](https://github.com/xavidop/telegram-voiceflow-api-bot) ⚙️ ### **Conclusion** As you can see, with just 60 lines of code you can quickly bring your chat experiences to life. I hope this gives you a better understanding of how you can use Voicefow to bring conversational experiences into your very own interfaces and products. Happy coding! ### Voiceflow is Hiring! 📢 > Voiceflow is looking for more people who are excited to build the future of interactive conversations. [You can visit this link to apply](https://www.voiceflow.com/about#Jobs) > . Prerequisites ------------- Here is what you'll need to create your very own Telegram bot using Voiceflow 1. Telegram Account 2. Voiceflow Account 3. Visual Studio Code Architecture ------------ In this example, we are going to use the Telegraf library for NodeJS projects to interact with our Telegram Bot. As mentioned above, every user interaction in the Telegram bot will be sent to Voiceflow using our Dialog Manager API.  The architecture used to create this experience Create your own bot with BotFather ---------------------------------- First, we need to use [botfather](https://core.telegram.org/bots#) to create our very own bot inside Telegram. BotFather allows you to create new bot accounts and manage existing ones in a few simple clicks. To start the process of creating your bot, you just need to talk to the botfather while following the steps outlined below[.](https://core.telegram.org/bots#) To open a chat with botfather, click on the "**start**" button. We'll need to create a new bot by clicking /newbot command. Next, you can name your bot. In this example, we have named ours "Voiceflow bot". It's that easy. The Telegram setup is now complete! Remember to add your Telegram token to your {% c-line %}.env file{% c-line-end %} in the property {% c-line %}TELEGRAM\_TOKEN{% c-line-end %}.**** Setting up the project ---------------------- Install and run the project: 1\. Clone this repo: {% c-line %}git clone {% c-line-end %} 2\. Install dependencies: {% c-line %}yarn install{% c-line-end %} 3\. Launch project: {% c-line %}yarn start{% c-line-end %} ** Writing the bot’s code** ----------------------------- ### **Telegraf setup** We can create bot by the following code lines:  ### **Voiceflow setup** **NOTE:** It's important to note that you will need to create a General Project type on Voiceflow for this experience to work. Now, let's implement the Voiceflow client to work with Voiceflow's cloud using its Dialog Manager API.  The {% c-line %}getclient(){% c-line-end %} function calls the {% c-line %}DialogManagerApi.getInstance{% c-line-end %} a REST API client that will interact directly with the Dialog Manager API. As you can see there are some values that will be added to our {% c-line %}.env{% c-line-end %} file. Let's explain how to obtain those variables. #### ** 1\. Version ID** To obtain your VersionID you have to go to your Voiceflow Project:  Then copy the {% c-line %}VOICEFLOW\_VERSION\_ID{% c-line-END %} from the URL in your address bar. When you are inside a Voiceflow project, your address bar should have a URL of the form: {% c-line %}https://creator.voiceflow.com/project/{VOICEFLOW\_VERSION\_ID}/...{% c-line-end %} #### **2\. apiKey** To obtain the API Key we have to go to our workspace where we have created our General Project. After this, head to workplace settings and click on the developer tab.  Then we have to click to {% c-line %}Create new API Key{% c-line-end %} button to create a new one:  You need to add a name to the new API Key, for example, {% c-line %}telegram-bot{% c-line-end %}. Once we have filled it, we can click the {% c-line %}Confirm{% c-line-end %} button:  Finally, we have to add these variables to our final {% c-line %}.env file{% c-line-end %}. It should look like this:  Everything is ready. Let's continue with our Telegram bot code. Let's replace the standard start replay for the one below, getting the correct replay from Voiceflow:  Then we replace the {% c-line %}hi{% c-line-end %} utterance for a regex like {% c-line %}(.+){% c-line-end %}. This means that the bot will hear for everything. All the text received we will pass directly to Voiceflow and we manage the state of the conversation: if it is ended or if it is not ended yet:  The response method is in charge of sending the user input in Telegram to Voiceflow using the runtime client SDK. This allows us to process the response:  In the example above you'll notice different types of responses including {% c-line %}SPEAK{% c-line-end %}, {% c-line %}VISUAL{% c-line-end %} and {% c-line %}AUDIO{% c-line-end %}responses. **Running the Telegram Bot** ----------------------------  You're all set to test this experience in Telegram ### **Help us help you** Interested in helping us improve this experience? You can submit a PR by following the steps below. 1. Fork this repo 2. Clone your fork 3. Code 4. 🤓 5. Test your changes 6. Submit a PR! ### **Resources** * [Official Telegraf Documentation](https://www.npmjs.com/package/ask-sdk) - The Official Telegraf Documentation * [Official Voiceflow's Runtime Dialog Manager Documentation](https://www.voiceflow.com/api/dialog-manager) - Official Voiceflow Dialog Manager API * To access all of the code mentioned above, click [here](https://github.com/xavidop/telegram-voiceflow-api-bot) ⚙️ ### **Conclusion** As you can see, with just 60 lines of code you can quickly bring your chat experiences to life. I hope this gives you a better understanding of how you can use Voicefow to bring conversational experiences into your very own interfaces and products. Happy coding! ### Voiceflow is Hiring! 📢 > Voiceflow is looking for more people who are excited to build the future of interactive conversations. [You can visit this link to apply](https://www.voiceflow.com/about#Jobs) > .  RECOMMENDED [\ \ ### Harness the power of OpenAI’s Whisper model for ASR with Voiceflow](https://www.voiceflow.com/pathways/openai-whisper-model-asr) PUBLISHED June 18, 2021 TOPICS IN THIS ARTICLE No items found. [\ \ Xavi Portilla Edo\ \ Head of Cloud Infrastructure at Voiceflow](https://www.voiceflow.com/contributors/xavi-portilla-edo) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/alexa-skill-chat-gpt-4) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### How to create an Alexa skill with GPT-4 and Voiceflow](https://www.voiceflow.com/pathways/alexa-skill-chat-gpt-4) [](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Harness the power of OpenAI’s Whisper model for ASR with Voiceflow](https://www.voiceflow.com/pathways/openai-whisper-model-asr) [](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [Devs\ \ ](https://www.voiceflow.com/pathways-categories/developers) [### Want to try Claude in your Voiceflow assistant? Here's how](https://www.voiceflow.com/pathways/claude-llm-ai-assistant) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-to-create-a-simple-telegram-bot-with-voiceflow#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-to-create-a-simple-telegram-bot-with-voiceflow#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # From Intercom to Voiceflow: How Roam saved 30 hours a week in customer support hours | Pathways [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) How Roam saved 30 hours a week in customer support hours ======================================================== [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow](https://www.voiceflow.com/contributors/kim-lu) The problem—An influx of inbound calls --------------------------------------- The team at Roam was overwhelmed with the influx of inbound calls and support tickets. As a startup in a new, exciting industry—flexible, subscription-based car rentals—they get a lot of questions from prospects and curious customers about how their services work. They tried to launch a DIY chatbot solution and even used Intercom, but found their lack of customizability and inaccessible pricing model left their customer success team just as overwhelmed as when they began. As a result, Nick Coatsworth, Head of Customer Success, and the team at Roam was inundated with up to 50 hours of customer support conversations a month and spent additional hours of support time managing email and inbound calls—mostly to educate prospects on their services. The team needed an easy way to educate their prospects so they could spend more time addressing urgent customer needs, like collision support or supporting high-value sales conversations. > _“I can literally jump into Voiceflow without any training, use a template, and create a powerful AI agent—it’s that simple.”_ Jump in at 2:38 on the full interview to learn more about pain points of running Customer Success.  Before Voiceflow—How Intercom fell short ---------------------------------------- Before Voiceflow, Roam used Intercom’s AI chat tool. It fell short for three core reasons: 1. **Lack of customizability:** The out-of-box AI tool never took over the conversation properly. Instead, the user had to go through many layers of conversation before they were offered a solution. This didn’t achieve their goal of reducing tickets in their support inbox. 2. **The pricing model:** Intercom charged based on resolutions provided by their AI chat. So even if the team found that the agent gave answers that weren’t the best or correct, Roam was still getting charged for them. 3. **Workflow mismatch:** Nick’s team found that Intercom was hard to test and iterate quickly. “I would set something up and then I'd realize it didn't connect to a piece of conversation,” Nick explains, “Unless everything was connected, you couldn't publish it. So I would make these concessions to get it live, but the agent ended up a bit half-baked.” Jump in at 5:16 on the full interview to learn more about Nick's transition from Intercom to Voiceflow. The solutions—A template for an AI agent and a trained knowledge base ---------------------------------------------------------------------- Nick and his team used Voiceflow to launch an AI chat agent onto their homepage in a matter of days. Here's how they did it: 1. **Curated a knowledge base (KB):** Using Roam’s collection of help articles and FAQs, Nick uploaded resources to a knowledge base for the AI agent to pull relevant answers to common questions. Using the KB, they were able to completely automate their Level 1 support. 2. **Built an AI agent using a Voiceflow template:** Nick got his AI agent up and running quickly by personalizing existing conversational flows from the Voiceflow template library. 3. **Added chat bubble to homepage:** Using a simple copy-and-paste of Voiceflow’s web chat snippet, Nick added the AI agent’s source code to Roam’s homepage. 4. **Iterated his assistant with rapid testing and transcript reviews:** Nick found success with his agent through prototyping his designs, adding new prompts based on live user transcripts, and tweaking his FAQ knowledge base. “We often have an idea of how we think customers perceive our business and the questions they might ask. But the better way is trial by fire,” says Nick. 5. **Leveraged the analytics dashboard:** The team used the top intents view in the Voiceflow analytics dashboard to inform product decisions and gauge where they needed to double down. Nick found that _booking your pickup_ and _cancellation_ were top intents for his agent so quickly brought back to his team, “Are we doing a good enough job onboarding customers to make them understand what the cancellation or the return requirements are or where they can go to access that information?” > _“We don't have a team dedicated to this, but it's already working. The ROI on Voiceflow is ridiculously good. And the best part is, we’re just getting started.”_  The impact—Customer support time, saved ----------------------------------------- Since deploying Voiceflow, Nick’s team has saved invaluable time and effort educating prospects on their services. Since the launch of their digital AI agent, the team at Roam has seen: * **A drastic decrease in inbound calls:** They’ve saved over 30 hours of customer support hours per week, freeing up time to address more urgent customer needs without additional CS staff hours. “I can’t put a price tag on the brain power and processing that Voiceflow saves us. We get overwhelmed sometimes but It's nice to know that now there's a ‘person’ on the team who will never run out of steam,” says Nick. * **An increase of sales calls booked asynchronously:** Nick’s team has experienced a dramatic increase in async sales meetings booked through their digital agent in the short few months it's been live. Roam’s chat agent provides the transcript of the conversation, which offers helpful context to help them prepare for high-value meetings. * **Accurate and comprehensive answers powered by AI:** Whether users are inquiring about leasing a car, signing up for a service, learning more about Roam’s offerings, or moving through the sales funnel, Roam’s digital agent answers user questions and shares useful information. By using a knowledge base, Nick’s team trusts that their digital agent will only provide approved and accurate outputs based on their resources, FAQs, and website. * **Efficient launch and easy maintenance:** “Voiceflow has a very simple interface and when you use a template, 80% of the work is done for you,” Nick explains, “This is huge as a small business. I'm not making sacrifices in other areas because I can trust the output of our digital agent and simply make changes or tweak things.” * **Insight into customer needs:** Before**,** Nick’s team struggled to understand why users would call into customer support and abandon their requests. Now, Nick has insight into what users are asking about, when they fall off the sales funnel, and what their common questions are. Using this data, Roam is able to pivot their marketing based on users’ pain points, or onboard customers better to address common questions. > _“Voiceflow is elastic to each customer. We started with level 1 customer support inquiries and we have the tools to scale our AI agent to handle more advanced customer support tasks.”_ #### **What’s next? Nick and the Roam team will expand their digital AI agent to authenticate users with Voiceflow’s API steps, so customers can inquire about their account and receive accurate, personalized information about their car subscription.** Before Voiceflow—How Intercom fell short ---------------------------------------- Before Voiceflow, Roam used Intercom’s AI chat tool. It fell short for three core reasons: 1. **Lack of customizability:** The out-of-box AI tool never took over the conversation properly. Instead, the user had to go through many layers of conversation before they were offered a solution. This didn’t achieve their goal of reducing tickets in their support inbox. 2. **The pricing model:** Intercom charged based on resolutions provided by their AI chat. So even if the team found that the agent gave answers that weren’t the best or correct, Roam was still getting charged for them. 3. **Workflow mismatch:** Nick’s team found that Intercom was hard to test and iterate quickly. “I would set something up and then I'd realize it didn't connect to a piece of conversation,” Nick explains, “Unless everything was connected, you couldn't publish it. So I would make these concessions to get it live, but the agent ended up a bit half-baked.” Jump in at 5:16 on the full interview to learn more about Nick's transition from Intercom to Voiceflow. The solutions—A template for an AI agent and a trained knowledge base ---------------------------------------------------------------------- Nick and his team used Voiceflow to launch an AI chat agent onto their homepage in a matter of days. Here's how they did it: 1. **Curated a knowledge base (KB):** Using Roam’s collection of help articles and FAQs, Nick uploaded resources to a knowledge base for the AI agent to pull relevant answers to common questions. Using the KB, they were able to completely automate their Level 1 support. 2. **Built an AI agent using a Voiceflow template:** Nick got his AI agent up and running quickly by personalizing existing conversational flows from the Voiceflow template library. 3. **Added chat bubble to homepage:** Using a simple copy-and-paste of Voiceflow’s web chat snippet, Nick added the AI agent’s source code to Roam’s homepage. 4. **Iterated his assistant with rapid testing and transcript reviews:** Nick found success with his agent through prototyping his designs, adding new prompts based on live user transcripts, and tweaking his FAQ knowledge base. “We often have an idea of how we think customers perceive our business and the questions they might ask. But the better way is trial by fire,” says Nick. 5. **Leveraged the analytics dashboard:** The team used the top intents view in the Voiceflow analytics dashboard to inform product decisions and gauge where they needed to double down. Nick found that _booking your pickup_ and _cancellation_ were top intents for his agent so quickly brought back to his team, “Are we doing a good enough job onboarding customers to make them understand what the cancellation or the return requirements are or where they can go to access that information?” > _“We don't have a team dedicated to this, but it's already working. The ROI on Voiceflow is ridiculously good. And the best part is, we’re just getting started.”_  The impact—Customer support time, saved ----------------------------------------- Since deploying Voiceflow, Nick’s team has saved invaluable time and effort educating prospects on their services. Since the launch of their digital AI agent, the team at Roam has seen: * **A drastic decrease in inbound calls:** They’ve saved over 30 hours of customer support hours per week, freeing up time to address more urgent customer needs without additional CS staff hours. “I can’t put a price tag on the brain power and processing that Voiceflow saves us. We get overwhelmed sometimes but It's nice to know that now there's a ‘person’ on the team who will never run out of steam,” says Nick. * **An increase of sales calls booked asynchronously:** Nick’s team has experienced a dramatic increase in async sales meetings booked through their digital agent in the short few months it's been live. Roam’s chat agent provides the transcript of the conversation, which offers helpful context to help them prepare for high-value meetings. * **Accurate and comprehensive answers powered by AI:** Whether users are inquiring about leasing a car, signing up for a service, learning more about Roam’s offerings, or moving through the sales funnel, Roam’s digital agent answers user questions and shares useful information. By using a knowledge base, Nick’s team trusts that their digital agent will only provide approved and accurate outputs based on their resources, FAQs, and website. * **Efficient launch and easy maintenance:** “Voiceflow has a very simple interface and when you use a template, 80% of the work is done for you,” Nick explains, “This is huge as a small business. I'm not making sacrifices in other areas because I can trust the output of our digital agent and simply make changes or tweak things.” * **Insight into customer needs:** Before**,** Nick’s team struggled to understand why users would call into customer support and abandon their requests. Now, Nick has insight into what users are asking about, when they fall off the sales funnel, and what their common questions are. Using this data, Roam is able to pivot their marketing based on users’ pain points, or onboard customers better to address common questions. > _“Voiceflow is elastic to each customer. We started with level 1 customer support inquiries and we have the tools to scale our AI agent to handle more advanced customer support tasks.”_ #### **What’s next? Nick and the Roam team will expand their digital AI agent to authenticate users with Voiceflow’s API steps, so customers can inquire about their account and receive accurate, personalized information about their car subscription.**  RECOMMENDED [\ \ ### How to integrate OpenAI GPT and your knowledge base into a Voiceflow Assistant](https://www.voiceflow.com/pathways/voiceflow-assistant-openai-gpt) PUBLISHED December 7, 2023 TOPICS IN THIS ARTICLE Exceptional work Exceptional people [\ \ Kim Lu\ \ Growth and Product Marketing at Voiceflow\ \ A growth and product marketer at Voiceflow who loves to design, build, and grow things. Equipped with an endless curiosity and bad eyesight.](https://www.voiceflow.com/contributors/kim-lu) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### How Roam saved 30 hours a week in customer support hours](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours) [](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### How Trilogy automated 70% of their customer support](https://www.voiceflow.com/pathways/automating-60-of-customer-support-for-90-products-in-12-weeks-how-ai-automation-transformed-trilogy) [](https://www.voiceflow.com/pathways/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) [Customer Experience\ \ ](https://www.voiceflow.com/pathways-categories/customer-experience) [### Introducing Tico: The AI customer support agent resolving 97% of tickets](https://www.voiceflow.com/pathways/introducing-tico-the-ai-customer-support-agent-resolving-97-of-tickets) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/how-roam-saved-30-hours-a-week-in-customer-support-hours#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) --- # Introducing the Agent step in Voiceflow | Pathways [Voiceflow news\ \ ](https://www.voiceflow.com/pathways-categories/voiceflow-news) Introducing the Agent step in Voiceflow ======================================= [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow](https://www.voiceflow.com/contributors/braden-ream) Today we released the Agent step—a new and more natural way to build AI Agents that allow for AI to determine the flow of your conversation. Previously complex multi-step workflows can now be condensed into a single step. This makes it faster and easier than ever to build Agents in Voiceflow, allowing your team to focus on other aspects of improving conversational chat experience. The Agent step is a new block in Voiceflow that introduces autonomous AI conversation flow, tool use, and decision making. The Agent step can decide when to use tools you connect it to, when to access the knowledge base for information, or even when to call other Agent steps to pass the conversation over to another Agent.  **How the Agent step works** ---------------------------- With the Agent step, you can combine hard business logic with Agent networks layered on top, allowing for both risk mitigation and Agent control, as well as conversational flexibility and capability.  This component allows developers to designate specific parts of a conversation where the AI can operate independently while maintaining structured flows elsewhere in the experience. When a user interaction reaches an Agent step, AI takes over with the ability to: * Plan and execute appropriate responses based on user needs * Access relevant knowledge bases when necessary * Connect with external tools and APIs to retrieve information * Determine when to exit the Agent step or hand off to another Agent step Voiceflow gives businesses a complete toolkit from Agentic networks to human-written responses and business logic to ensure you always have the right tool for the job. **Use Case: Customer Support Agent** ------------------------------------ Consider a customer support scenario: When a user initiates a return request, the deterministic flow handles initial validation and account verification. Once basic information is collected, the Agent step activates, allowing the AI to autonomously: 1. Assess the return reason and determine eligibility based on policies 2. Access order history to verify purchase details 3. Generate a return label if appropriate or escalate to a specialized Agent step for exceptions 4. Exit back to the deterministic flow for confirmation messaging This hybrid approach ensures compliance with return policies while providing intelligent handling of complex cases. You can also create specific exit scenarios that automatically exit the Agent back to the safety of deterministic logic. These exit scenarios include situations such as an upset customer, requesting to speak with a human, or the agent detecting malicious usage outside its scope.  **Tying it all together** ------------------------- The addition of an Agent step eliminates the trade-off between reliability and flexibility. Deterministic flows provide consistency for critical processes, while Agent steps enable adaptability for complex scenarios—all within a unified development environment. Voiceflow provides a practical solution for building AI Agents that balance structure with intelligence, giving you control over exactly where and when autonomous capabilities are deployed. **Next up: Agent Frameworks!** ------------------------------ The Agent step's modular design supports advanced implementation patterns. Developers can connect multiple Agent steps to create sophisticated frameworks, including the widely-used Supervisor pattern where specialized agents handle different conversation aspects. We are excited to see the novel Agent frameworks people develop using the Agent step! **How the Agent step works** ---------------------------- With the Agent step, you can combine hard business logic with Agent networks layered on top, allowing for both risk mitigation and Agent control, as well as conversational flexibility and capability.  This component allows developers to designate specific parts of a conversation where the AI can operate independently while maintaining structured flows elsewhere in the experience. When a user interaction reaches an Agent step, AI takes over with the ability to: * Plan and execute appropriate responses based on user needs * Access relevant knowledge bases when necessary * Connect with external tools and APIs to retrieve information * Determine when to exit the Agent step or hand off to another Agent step Voiceflow gives businesses a complete toolkit from Agentic networks to human-written responses and business logic to ensure you always have the right tool for the job. **Use Case: Customer Support Agent** ------------------------------------ Consider a customer support scenario: When a user initiates a return request, the deterministic flow handles initial validation and account verification. Once basic information is collected, the Agent step activates, allowing the AI to autonomously: 1. Assess the return reason and determine eligibility based on policies 2. Access order history to verify purchase details 3. Generate a return label if appropriate or escalate to a specialized Agent step for exceptions 4. Exit back to the deterministic flow for confirmation messaging This hybrid approach ensures compliance with return policies while providing intelligent handling of complex cases. You can also create specific exit scenarios that automatically exit the Agent back to the safety of deterministic logic. These exit scenarios include situations such as an upset customer, requesting to speak with a human, or the agent detecting malicious usage outside its scope.  **Tying it all together** ------------------------- The addition of an Agent step eliminates the trade-off between reliability and flexibility. Deterministic flows provide consistency for critical processes, while Agent steps enable adaptability for complex scenarios—all within a unified development environment. Voiceflow provides a practical solution for building AI Agents that balance structure with intelligence, giving you control over exactly where and when autonomous capabilities are deployed. **Next up: Agent Frameworks!** ------------------------------ The Agent step's modular design supports advanced implementation patterns. Developers can connect multiple Agent steps to create sophisticated frameworks, including the widely-used Supervisor pattern where specialized agents handle different conversation aspects. We are excited to see the novel Agent frameworks people develop using the Agent step!  RECOMMENDED [](https://www.voiceflow.com/pathways/introducing-the-agent-step#) PUBLISHED March 20, 2025 TOPICS IN THIS ARTICLE Elevating our work [\ \ Braden Ream\ \ CEO & Co-founder at Voiceflow\ \ Braden founded Voiceflow to help teams collaborate and build better assistants, together. He splits his time between San Francisco and his native Toronto, where you can find him writing and speaking on the future of conversational AI.](https://www.voiceflow.com/contributors/braden-ream) RECOMMENDED RESOURCES No items found. ALONG THE SAME PATH [](https://www.voiceflow.com/pathways/no-impact-to-voiceflow-customers-in-supply-chain-security-incident-sha1-hulud) [Voiceflow news\ \ ](https://www.voiceflow.com/pathways-categories/voiceflow-news) [### No impact to Voiceflow customers in supply-chain security incident (sha1-hulud)](https://www.voiceflow.com/pathways/no-impact-to-voiceflow-customers-in-supply-chain-security-incident-sha1-hulud) [](https://www.voiceflow.com/pathways/turbocharging-voice-agents) [Voiceflow news\ \ ](https://www.voiceflow.com/pathways-categories/voiceflow-news) [### Turbocharging Voice Agents: Speed, Quality, and Powerful New Features](https://www.voiceflow.com/pathways/turbocharging-voice-agents) [](https://www.voiceflow.com/pathways/introducing-voiceflow-credits) [Voiceflow news\ \ ](https://www.voiceflow.com/pathways-categories/voiceflow-news) [### Introducing Voiceflow Credits: a simpler, more transparent billing approach](https://www.voiceflow.com/pathways/introducing-voiceflow-credits) [Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**\ \ Too many words? **Explore our YouTube channel.**](https://www.youtube.com/channel/UCbqUIYQ7J2rS6C_nk4cNTxQ) Cross paths with us Our best conversations, updates, tips, and more delivered straight to your inbox. Sign up Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. [.svg)](https://www.voiceflow.com/pathways) The publication for conversational AI trailblazers and tinkerers. [Try Voiceflow](https://creator.voiceflow.com/signup) [See all the contributors](https://www.voiceflow.com/pathways/introducing-the-agent-step#) [](https://twitter.com/voiceflowhq) [](https://link.voiceflow.com/community) [](https://www.youtube.com/@Voiceflow) [](https://github.com/voiceflow) [](https://www.linkedin.com/company/voiceflowhq/) Browse by category [Applied research](https://www.voiceflow.com/blog-categories/research) [Conversation Design](https://www.voiceflow.com/blog-categories/conversation-design) [Developers](https://www.voiceflow.com/blog-categories/developers) [Industry](https://www.voiceflow.com/blog-categories/industry) [Customer Experience](https://www.voiceflow.com/blog-categories/customer-experience) EXPLORE BY [Articles](https://www.voiceflow.com/content-type/articles) [Workshops](https://www.voiceflow.com/content-type/workshops) [Resources](https://www.voiceflow.com/pathways/introducing-the-agent-step#) [Tutorials](https://www.youtube.com/@Voiceflow) © 2024 Voiceflow, Inc. All rights reserved. | [Terms](https://www.voiceflow.com/legal/terms) | [Privacy](https://www.voiceflow.com/privacy) ---
Ready to explore how Voiceflow can help your team? Let’s talk.