# Table of Contents - [Introduction | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#introduction-langchain) - [Community navigator | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#community-navigator-langchain) - [External guides | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#external-guides-langchain) - [Providers | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#providers-langchain) - [Error reference | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#error-reference-langchain) - [People | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#people-langchain) - [Welcome Contributors | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#welcome-contributors-langchain) - [Introduction | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#introduction-langchain) - [Tutorials | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#tutorials-langchain) - [Introduction | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#introduction-langchain) - [How-to guides | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-guides-langchain) - [Build a simple LLM application with chat models and prompt templates | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-simple-llm-application-with-chat-models-and-prompt-templates-langchain) - [Build a Question Answering application over a Graph Database | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-question-answering-application-over-a-graph-database-langchain) - [Summarize Text | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#summarize-text-langchain) - [Build an Extraction Chain | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-an-extraction-chain-langchain) - [Build a Question/Answering system over SQL data | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-question-answering-system-over-sql-data-langchain) - [Build a Chatbot | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-chatbot-langchain) - [Tagging | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#tagging-langchain) - [How to use example selectors | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-use-example-selectors-langchain) - [Build a Retrieval Augmented Generation (RAG) App: Part 2 | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-retrieval-augmented-generation-rag-app-part-2-langchain) - [How to stream responses from an LLM | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-stream-responses-from-an-llm-langchain) - [How to add memory to chatbots | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-add-memory-to-chatbots-langchain) - [Build a semantic search engine | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-semantic-search-engine-langchain) - [Installation | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#installation-langchain) - [How to cache chat model responses | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-cache-chat-model-responses-langchain) - [How to stream chat model responses | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-stream-chat-model-responses-langchain) - [How to embed text data | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-embed-text-data-langchain) - [How to use few shot examples in chat models | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-use-few-shot-examples-in-chat-models-langchain) - [How to cache model responses | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-cache-model-responses-langchain) - [Richer outputs | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#richer-outputs-langchain) - [How to use few shot examples | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-use-few-shot-examples-langchain) - [Build a Retrieval Augmented Generation (RAG) App: Part 1 | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#build-a-retrieval-augmented-generation-rag-app-part-1-langchain) - [How to use output parsers to parse an LLM response into structured format | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain](#how-to-use-output-parsers-to-parse-an-llm-response-into-structured-format-langchain) --- # Introduction | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Introduction ============ **LangChain** is a framework for developing applications powered by large language models (LLMs). LangChain simplifies every stage of the LLM application lifecycle: * **Development**: Build your applications using LangChain's open-source [building blocks](/docs/concepts/lcel) , [components](/docs/concepts) , and [third-party integrations](/docs/integrations/platforms/) . Use [LangGraph.js](/docs/concepts/architecture#langgraph) to build stateful agents with first-class streaming and human-in-the-loop support. * **Productionization**: Use [LangSmith](https://docs.smith.langchain.com/) to inspect, monitor and evaluate your chains, so that you can continuously optimize and deploy with confidence. * **Deployment**: Turn your LangGraph applications into production-ready APIs and Assistants with [LangGraph Cloud](https://langchain-ai.github.io/langgraph/cloud/) . ![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](/svg/langchain_stack_062024.svg "LangChain Framework Overview")![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](/svg/langchain_stack_062024_dark.svg "LangChain Framework Overview") Concretely, the framework consists of the following open-source libraries: * **`@langchain/core`**: Base abstractions and LangChain Expression Language. * **`@langchain/community`**: Third party integrations. * Partner packages (e.g. **`@langchain/openai`**, **`@langchain/anthropic`**, etc.): Some integrations have been further split into their own lightweight packages that only depend on **`@langchain/core`**. * **`langchain`**: Chains, agents, and retrieval strategies that make up an application's cognitive architecture. * **[LangGraph.js](https://langchain-ai.github.io/langgraphjs/) **: Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. * **[LangSmith](https://docs.smith.langchain.com) **: A developer platform that lets you debug, test, evaluate, and monitor LLM applications. note These docs focus on the JavaScript LangChain library. [Head here](https://python.langchain.com) for docs on the Python LangChain library. [Tutorials](/docs/tutorials) [โ€‹](#tutorials "Direct link to tutorials") ------------------------------------------------------------------------ If you're looking to build something specific or are more of a hands-on learner, check out our [tutorials](/docs/tutorials) . This is the best place to get started. These are the best ones to get started with: * [Build a Simple LLM Application](/docs/tutorials/llm_chain) * [Build a Chatbot](/docs/tutorials/chatbot) * [Build an Agent](https://langchain-ai.github.io/langgraphjs/tutorials/quickstart/) * [LangGraph.js quickstart](https://langchain-ai.github.io/langgraphjs/tutorials/quickstart/) Explore the full list of LangChain tutorials [here](/docs/tutorials) , and check out other [LangGraph tutorials here](https://langchain-ai.github.io/langgraphjs/tutorials/) . [How-To Guides](/docs/how_to/) [โ€‹](#how-to-guides "Direct link to how-to-guides") ---------------------------------------------------------------------------------- [Here](/docs/how_to/) you'll find short answers to โ€œHow do Iโ€ฆ.?โ€ types of questions. These how-to guides don't cover topics in depth - you'll find that material in the [Tutorials](/docs/tutorials) and the [API Reference](https://api.js.langchain.com) . However, these guides will help you quickly accomplish common tasks. Check out [LangGraph-specific how-tos here](https://langchain-ai.github.io/langgraphjs/how-tos/) . [Conceptual Guide](/docs/concepts) [โ€‹](#conceptual-guide "Direct link to conceptual-guide") -------------------------------------------------------------------------------------------- Introductions to all the key parts of LangChain you'll need to know! [Here](/docs/concepts) you'll find high level explanations of all LangChain concepts. For a deeper dive into LangGraph concepts, check out [this page](https://langchain-ai.github.io/langgraph/concepts/) . [API reference](https://api.js.langchain.com) [โ€‹](#api-reference "Direct link to api-reference") ------------------------------------------------------------------------------------------------- Head to the reference section for full documentation of all classes and methods in the LangChain JavaScript packages. Ecosystem[โ€‹](#ecosystem "Direct link to Ecosystem") ---------------------------------------------------- ### [๐Ÿฆœ๐Ÿ› ๏ธ LangSmith](https://docs.smith.langchain.com) [โ€‹](#๏ธ-langsmith "Direct link to ๏ธ-langsmith") Trace and evaluate your language model applications and intelligent agents to help you move from prototype to production. ### [๐Ÿฆœ๐Ÿ•ธ๏ธ LangGraph](https://langchain-ai.github.io/langgraphjs/) [โ€‹](#๏ธ-langgraph "Direct link to ๏ธ-langgraph") Build stateful, multi-actor applications with LLMs, built on top of (and intended to be used with) LangChain primitives. Additional resources[โ€‹](#additional-resources "Direct link to Additional resources") ------------------------------------------------------------------------------------- ### [Security](/docs/security) [โ€‹](#security "Direct link to security") Read up on our [Security](/docs/security) best practices to make sure you're developing safely with LangChain. ### [Integrations](/docs/integrations/platforms/) [โ€‹](#integrations "Direct link to integrations") LangChain is part of a rich ecosystem of tools that integrate with our framework and build on top of it. Check out our growing list of [integrations](/docs/integrations/platforms/) . ### [Contributing](/docs/contributing) [โ€‹](#contributing "Direct link to contributing") Check out the developer's guide for guidelines on contributing and help getting your dev environment set up. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Tutorials](#tutorials) * [How-To Guides](#how-to-guides) * [Conceptual Guide](#conceptual-guide) * [API reference](#api-reference) * [Ecosystem](#ecosystem) * [๐Ÿฆœ๐Ÿ› ๏ธ LangSmith](#๏ธ-langsmith) * [๐Ÿฆœ๐Ÿ•ธ๏ธ LangGraph](#๏ธ-langgraph) * [Additional resources](#additional-resources) * [Security](#security) * [Integrations](#integrations) * [Contributing](#contributing) --- # Community navigator | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) Community navigator =================== Hi! Thanks for being here. We're lucky to have a community of so many passionate developers building with LangChainโ€“we have so much to teach and learn from each other. Community members contribute code, host meetups, write blog posts, amplify each other's work, become each other's customers and collaborators, and so much more. Whether you're new to LangChain, looking to go deeper, or just want to get more exposure to the world of building with LLMs, this page can point you in the right direction. * **๐Ÿฆœ Contribute to LangChain** * **๐ŸŒ Meetups, Events, and Hackathons** * **๐Ÿ“ฃ Help Us Amplify Your Work** * **๐Ÿ’ฌ Stay in the loop** ๐Ÿฆœ Contribute to LangChain ========================== LangChain is the product of over 5,000+ contributions by 1,500+ contributors, and there is **still** so much to do together. Here are some ways to get involved: * **[Open a pull request](https://github.com/langchain-ai/langchainjs/issues) :** we'd appreciate all forms of contributionsโ€“new features, infrastructure improvements, better documentation, bug fixes, etc. If you have an improvement or an idea, we'd love to work on it with you. * **[Read our contributor guidelines:](https://github.com/langchain-ai/langchainjs/blob/main/CONTRIBUTING.md) ** We ask contributors to follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow, run a few local checks for formatting, linting, and testing before submitting, and follow certain documentation and testing conventions. * **Become an expert:** our experts help the community by answering product questions in Discord. If that's a role you'd like to play, we'd be so grateful! (And we have some special experts-only goodies/perks we can tell you more about). Send us an email to introduce yourself at [hello@langchain.dev](mailto:hello@langchain.dev) and we'll take it from there! * **Integrate with LangChain:** if your product integrates with LangChainโ€“or aspires toโ€“we want to help make sure the experience is as smooth as possible for you and end users. Send us an email at [hello@langchain.dev](mailto:hello@langchain.dev) and tell us what you're working on. * **Become an Integration Maintainer:** Partner with our team to ensure your integration stays up-to-date and talk directly with users (and answer their inquiries) in our Discord. Introduce yourself at [hello@langchain.dev](mailto:hello@langchain.dev) if you'd like to explore this role. ๐ŸŒ Meetups, Events, and Hackathons ================================== One of our favorite things about working in AI is how much enthusiasm there is for building together. We want to help make that as easy and impactful for you as possible! * **Find a meetup, hackathon, or webinar:** you can find the one for you on on our [global events calendar](https://mirror-feeling-d80.notion.site/0bc81da76a184297b86ca8fc782ee9a3?v=0d80342540df465396546976a50cfb3f) . * **Submit an event to our calendar:** email us at [events@langchain.dev](mailto:events@langchain.dev) with a link to your event page! We can also help you spread the word with our local communities. * **Host a meetup:** If you want to bring a group of builders together, we want to help! We can publicize your event on our event calendar/Twitter, share with our local communities in Discord, send swag, or potentially hook you up with a sponsor. Email us at [events@langchain.dev](mailto:events@langchain.dev) to tell us about your event! * **Become a meetup sponsor:** we often hear from groups of builders that want to get together, but are blocked or limited on some dimension (space to host, budget for snacks, prizes to distribute, etc.). If you'd like to help, send us an email to [events@langchain.dev](mailto:events@langchain.dev) we can share more about how it works! * **Speak at an event:** meetup hosts are always looking for great speakers, presenters, and panelists. If you'd like to do that at an event, send us an email to [hello@langchain.dev](mailto:hello@langchain.dev) with more information about yourself, what you want to talk about, and what city you're based in and we'll try to match you with an upcoming event! * **Tell us about your LLM community:** If you host or participate in a community that would welcome support from LangChain and/or our team, send us an email at [hello@langchain.dev](mailto:hello@langchain.dev) and let us know how we can help. ๐Ÿ“ฃ Help Us Amplify Your Work ============================ If you're working on something you're proud of, and think the LangChain community would benefit from knowing about it, we want to help you show it off. * **Post about your work and mention us:** we love hanging out on Twitter to see what people in the space are talking about and working on. If you tag [@langchainai](https://twitter.com/LangChainAI) , we'll almost certainly see it and can show you some love. * **Publish something on our blog:** if you're writing about your experience building with LangChain, we'd love to post (or crosspost) it on our blog! E-mail [hello@langchain.dev](mailto:hello@langchain.dev) with a draft of your post! Or even an idea for something you want to write about. * **Get your product onto our [integrations hub](https://integrations.langchain.com/) :** Many developers take advantage of our seamless integrations with other products, and come to our integrations hub to find out who those are. If you want to get your product up there, tell us about it (and how it works with LangChain) at [hello@langchain.dev](mailto:hello@langchain.dev) . โ˜€๏ธ Stay in the loop =================== Here's where our team hangs out, talks shop, spotlights cool work, and shares what we're up to. We'd love to see you there too. * **[Twitter](https://twitter.com/LangChainAI) :** we post about what we're working on and what cool things we're seeing in the space. If you tag @langchainai in your post, we'll almost certainly see it, and can snow you some love! * **[GitHub](https://github.com/langchain-ai/langchainjs) :** open pull requests, contribute to a discussion, and/or contribute * **[Subscribe to our bi-weekly Release Notes](https://6w1pwbss0py.typeform.com/to/KjZB1auB) :** a twice/month email roundup of the coolest things going on in our orbit * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . --- # External guides | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page External guides =============== Below are links to external tutorials and courses on LangChain.js. For other written guides on common use cases for LangChain.js, check out the [tutorials](/docs/tutorials/) and [how to](/docs/how_to/) sections. * * * Deeplearning.ai[โ€‹](#deeplearningai "Direct link to Deeplearning.ai") --------------------------------------------------------------------- We've partnered with [Deeplearning.ai](https://deeplearning.ai) and [Andrew Ng](https://en.wikipedia.org/wiki/Andrew_Ng) on a LangChain.js short course. It covers LCEL and other building blocks you can combine to build more complex chains, as well as fundamentals around loading data for retrieval augmented generation (RAG). Try it for free below: * [Build LLM Apps with LangChain.js](https://www.deeplearning.ai/short-courses/build-llm-apps-with-langchain-js) Scrimba interactive guides[โ€‹](#scrimba-interactive-guides "Direct link to Scrimba interactive guides") ------------------------------------------------------------------------------------------------------- [Scrimba](https://scrimba.com) is a code-learning platform that allows you to interactively edit and run code while watching a video walkthrough. We've partnered with Scrimba on course materials (called "scrims") that teach the fundamentals of building with LangChain.js - check them out below, and check back for more as they become available! ### Learn LangChain.js[โ€‹](#learn-langchainjs "Direct link to Learn LangChain.js") * [Learn LangChain.js on Scrimba](https://scrimba.com/learn/langchain) An full end-to-end course that walks through how to build a chatbot that can answer questions about a provided document. A great introduction to LangChain and a great first project for learning how to use LangChain Expression Language primitives to perform retrieval! ### LangChain Expression Language (LCEL)[โ€‹](#langchain-expression-language-lcel "Direct link to LangChain Expression Language (LCEL)") * [The basics (PromptTemplate + LLM)](https://v2.scrimba.com/s05iemh) * [Adding an output parser](https://scrimba.com/scrim/co6ae44248eacc1abd87ae3dc) * [Attaching function calls to a model](https://scrimba.com/scrim/cof5449f5bc972f8c90be6a82) * [Composing multiple chains](https://scrimba.com/scrim/co14344c29595bfb29c41f12a) * [Retrieval chains](https://scrimba.com/scrim/co0e040d09941b4000244db46) * [Conversational retrieval chains ("Chat with Docs")](https://scrimba.com/scrim/co3ed4a9eb4c6c6d0361a507c) ### Deeper dives[โ€‹](#deeper-dives "Direct link to Deeper dives") * [Setting up a new `PromptTemplate`](https://scrimba.com/scrim/cbGwRwuV) * [Setting up `ChatOpenAI` parameters](https://scrimba.com/scrim/cEgbBBUw) * [Attaching stop sequences](https://scrimba.com/scrim/co9704e389428fe2193eb955c) Neo4j GraphAcademy[โ€‹](#neo4j-graphacademy "Direct link to Neo4j GraphAcademy") ------------------------------------------------------------------------------- [Neo4j](https://neo4j.com) has put together a hands-on, practical course that shows how to build a movie-recommending chatbot in Next.js. It covers retrieval-augmented generation (RAG), tracking history, and more. Check it out below: * [Build a Neo4j-backed Chatbot with TypeScript](https://graphacademy.neo4j.com/courses/llm-chatbot-typescript/?ref=langchainjs) LangChain.js x AI SDK[โ€‹](#langchainjs-x-ai-sdk "Direct link to LangChain.js x AI SDK") --------------------------------------------------------------------------------------- How to use LangChain.js with AI SDK and React Server Components. * [Streaming agentic data to the client](https://github.com/langchain-ai/langchain-nextjs-template/blob/main/app/ai_sdk/agent/README.md) * [Streaming tool responses to the client](https://github.com/langchain-ai/langchain-nextjs-template/blob/main/app/ai_sdk/tools/README.md) * * * * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Deeplearning.ai](#deeplearningai) * [Scrimba interactive guides](#scrimba-interactive-guides) * [Learn LangChain.js](#learn-langchainjs) * [LangChain Expression Language (LCEL)](#langchain-expression-language-lcel) * [Deeper dives](#deeper-dives) * [Neo4j GraphAcademy](#neo4j-graphacademy) * [LangChain.js x AI SDK](#langchainjs-x-ai-sdk) --- # Providers | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Providers ========= LangChain integrates with many providers. Partner Packages[โ€‹](#partner-packages "Direct link to Partner Packages") ------------------------------------------------------------------------- These providers have standalone `@langchain/{provider}` packages for improved versioning, dependency management and testing. For specifics on how to use each package, look for their pages in the appropriate component docs section (e.g. [chat models](/docs/integrations/chat/) ). * [Anthropic](https://www.npmjs.com/package/@langchain/anthropic) * [Cerebras](https://www.npmjs.com/package/@langchain/cerebras) * [Cloudflare](https://www.npmjs.com/package/@langchain/cloudflare) * [Cohere](https://www.npmjs.com/package/@langchain/cohere) * [Exa](https://www.npmjs.com/package/@langchain/exa) * [Google GenAI](https://www.npmjs.com/package/@langchain/google-genai) * [Google VertexAI](https://www.npmjs.com/package/@langchain/google-vertexai) * [Google VertexAI (Web Environments)](https://www.npmjs.com/package/@langchain/google-vertexai-web) * [Groq](https://www.npmjs.com/package/@langchain/groq) * [MistralAI](https://www.npmjs.com/package/@langchain/mistralai) * [MongoDB](https://www.npmjs.com/package/@langchain/mongodb) * [Nomic](https://www.npmjs.com/package/@langchain/nomic) * [OpenAI](https://www.npmjs.com/package/@langchain/openai) * [Pinecone](https://www.npmjs.com/package/@langchain/pinecone) * [Qdrant](https://www.npmjs.com/package/@langchain/qdrant) * [Redis](https://www.npmjs.com/package/@langchain/redis) * [Weaviate](https://www.npmjs.com/package/@langchain/weaviate) * [Yandex](https://www.npmjs.com/package/@langchain/yandex) * [Azure CosmosDB](https://www.npmjs.com/package/@langchain/azure-cosmosdb) * [xAI](https://www.npmjs.com/package/@langchain/xai) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Partner Packages](#partner-packages) --- # Error reference | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) Error reference =============== This page contains guides around resolving common errors you may find while building with LangChain. Errors referenced below will have an `lc_error_code` property corresponding to one of the below codes when they are thrown in code. * [INVALID\_PROMPT\_INPUT](/docs/troubleshooting/errors/INVALID_PROMPT_INPUT) * [INVALID\_TOOL\_RESULTS](/docs/troubleshooting/errors/INVALID_TOOL_RESULTS) * [MESSAGE\_COERCION\_FAILURE](/docs/troubleshooting/errors/MESSAGE_COERCION_FAILURE) * [MODEL\_AUTHENTICATION](/docs/troubleshooting/errors/MODEL_AUTHENTICATION) * [MODEL\_NOT\_FOUND](/docs/troubleshooting/errors/MODEL_NOT_FOUND) * [MODEL\_RATE\_LIMIT](/docs/troubleshooting/errors/MODEL_RATE_LIMIT) * [OUTPUT\_PARSING\_FAILURE](/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . --- # People | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) People ====== There are some incredible humans from all over the world who have been instrumental in helping the LangChain community flourish ๐ŸŒ! This page highlights a few of those folks who have dedicated their time to the open-source repo in the form of direct contributions and reviews. Top reviewers[โ€‹](#top-reviewers "Direct link to Top reviewers") ---------------------------------------------------------------- As LangChain has grown, the amount of surface area that maintainers cover has grown as well. Thank you to the following folks who have gone above and beyond in reviewing incoming PRs ๐Ÿ™! [![Avatar for afirstenberg](https://avatars.githubusercontent.com/u/3507578?v=4)](https://github.com/afirstenberg) [@afirstenberg](https://github.com/afirstenberg) [![Avatar for sullivan-sean](https://avatars.githubusercontent.com/u/22581534?u=8f88473db2f929a965b6371733efda28e3fa1948&v=4)](https://github.com/sullivan-sean) [@sullivan-sean](https://github.com/sullivan-sean) [![Avatar for tomasonjo](https://avatars.githubusercontent.com/u/19948365?v=4)](https://github.com/tomasonjo) [@tomasonjo](https://github.com/tomasonjo) [![Avatar for ppramesi](https://avatars.githubusercontent.com/u/6775031?v=4)](https://github.com/ppramesi) [@ppramesi](https://github.com/ppramesi) [![Avatar for jacobrosenthal](https://avatars.githubusercontent.com/u/455796?v=4)](https://github.com/jacobrosenthal) [@jacobrosenthal](https://github.com/jacobrosenthal) [![Avatar for sinedied](https://avatars.githubusercontent.com/u/593151?u=08557bbdd96221813b8aec932dd7de895ac040ea&v=4)](https://github.com/sinedied) [@sinedied](https://github.com/sinedied) [![Avatar for mieslep](https://avatars.githubusercontent.com/u/5420540?u=8f038c002fbce42427999eb715dc9f868cef1c84&v=4)](https://github.com/mieslep) [@mieslep](https://github.com/mieslep) Top recent contributors[โ€‹](#top-recent-contributors "Direct link to Top recent contributors") ---------------------------------------------------------------------------------------------- The list below contains contributors who have had the most PRs merged in the last three months, weighted (imperfectly) by impact. Thank you all so much for your time and efforts in making LangChain better โค๏ธ! [![Avatar for dl102306](https://avatars.githubusercontent.com/u/2979960?v=4)](https://github.com/dl102306) [@dl102306](https://github.com/dl102306) [![Avatar for Anirudh31415926535](https://avatars.githubusercontent.com/u/171019460?v=4)](https://github.com/Anirudh31415926535) [@Anirudh31415926535](https://github.com/Anirudh31415926535) [![Avatar for dependabot](https://avatars.githubusercontent.com/in/29110?v=4)](https://github.com/apps/dependabot) [@dependabot](https://github.com/apps/dependabot) [![Avatar for chentschel](https://avatars.githubusercontent.com/u/319227?u=db20ce1d424f10d7760665ab693791ebc580131a&v=4)](https://github.com/chentschel) [@chentschel](https://github.com/chentschel) [![Avatar for sinedied](https://avatars.githubusercontent.com/u/593151?u=08557bbdd96221813b8aec932dd7de895ac040ea&v=4)](https://github.com/sinedied) [@sinedied](https://github.com/sinedied) [![Avatar for miloradvojnovic](https://avatars.githubusercontent.com/u/11798350?u=a9b931a1a3319787bef5e2d16e1fdec0888cdad5&v=4)](https://github.com/miloradvojnovic) [@miloradvojnovic](https://github.com/miloradvojnovic) [![Avatar for anthonychu](https://avatars.githubusercontent.com/u/3982077?u=8bbebac42cb84a25c629f83f212b2d099ffa3964&v=4)](https://github.com/anthonychu) [@anthonychu](https://github.com/anthonychu) [![Avatar for josemussa](https://avatars.githubusercontent.com/u/4422500?u=d676ede0cec8ee5df6879ebf9d8b72d51ea1eb7f&v=4)](https://github.com/josemussa) [@josemussa](https://github.com/josemussa) [![Avatar for ovuruska](https://avatars.githubusercontent.com/u/75265893?u=7f11152d07f1719da22084388c09b5fc64ab6c89&v=4)](https://github.com/ovuruska) [@ovuruska](https://github.com/ovuruska) [![Avatar for tofuliang](https://avatars.githubusercontent.com/u/1814685?v=4)](https://github.com/tofuliang) [@tofuliang](https://github.com/tofuliang) [![Avatar for AvaterClasher](https://avatars.githubusercontent.com/u/116944847?u=102a870b3efed7f30f0a57123391a293eb6f5b08&v=4)](https://github.com/AvaterClasher) [@AvaterClasher](https://github.com/AvaterClasher) [![Avatar for jl4nz](https://avatars.githubusercontent.com/u/94814971?u=266358610eeb54c3393dc127718dd6a997fdbf52&v=4)](https://github.com/jl4nz) [@jl4nz](https://github.com/jl4nz) [![Avatar for volodymyr-memsql](https://avatars.githubusercontent.com/u/57520563?v=4)](https://github.com/volodymyr-memsql) [@volodymyr-memsql](https://github.com/volodymyr-memsql) Core maintainers[โ€‹](#core-maintainers "Direct link to Core maintainers") ------------------------------------------------------------------------- Hello there ๐Ÿ‘‹! We're LangChain's core maintainers. If you've spent time in the community, you've probably crossed paths with at least one of us already. [![Avatar for nfcampos](https://avatars.githubusercontent.com/u/56902?u=fdb30e802c68bc338dd9c0820f713e4fdac75db7&v=4)](https://github.com/nfcampos) [@nfcampos](https://github.com/nfcampos) [![Avatar for bracesproul](https://avatars.githubusercontent.com/u/46789226?u=83f467441c4b542b900fe2bb8fe45e26bf918da0&v=4)](https://github.com/bracesproul) [@bracesproul](https://github.com/bracesproul) [![Avatar for dqbd](https://avatars.githubusercontent.com/u/1443449?u=fe32372ae8f497065ef0a1c54194d9dff36fb81d&v=4)](https://github.com/dqbd) [@dqbd](https://github.com/dqbd) [![Avatar for jacoblee93](https://avatars.githubusercontent.com/u/6952323?u=d785f9406c5a78ebd75922567b2693fb643c3bb0&v=4)](https://github.com/jacoblee93) [@jacoblee93](https://github.com/jacoblee93) [![Avatar for hwchase17](https://avatars.githubusercontent.com/u/11986836?u=f4c4f21a82b2af6c9f91e1f1d99ea40062f7a101&v=4)](https://github.com/hwchase17) [@hwchase17](https://github.com/hwchase17) Top all-time contributors[โ€‹](#top-all-time-contributors "Direct link to Top all-time contributors") ---------------------------------------------------------------------------------------------------- And finally, this is an all-time list of all-stars who have made significant contributions to the framework ๐ŸŒŸ: [![Avatar for afirstenberg](https://avatars.githubusercontent.com/u/3507578?v=4)](https://github.com/afirstenberg) [@afirstenberg](https://github.com/afirstenberg) [![Avatar for ppramesi](https://avatars.githubusercontent.com/u/6775031?v=4)](https://github.com/ppramesi) [@ppramesi](https://github.com/ppramesi) [![Avatar for jacobrosenthal](https://avatars.githubusercontent.com/u/455796?v=4)](https://github.com/jacobrosenthal) [@jacobrosenthal](https://github.com/jacobrosenthal) [![Avatar for sullivan-sean](https://avatars.githubusercontent.com/u/22581534?u=8f88473db2f929a965b6371733efda28e3fa1948&v=4)](https://github.com/sullivan-sean) [@sullivan-sean](https://github.com/sullivan-sean) [![Avatar for sinedied](https://avatars.githubusercontent.com/u/593151?u=08557bbdd96221813b8aec932dd7de895ac040ea&v=4)](https://github.com/sinedied) [@sinedied](https://github.com/sinedied) [![Avatar for tomasonjo](https://avatars.githubusercontent.com/u/19948365?v=4)](https://github.com/tomasonjo) [@tomasonjo](https://github.com/tomasonjo) [![Avatar for skarard](https://avatars.githubusercontent.com/u/602085?u=f8a9736cfa9fe8875d19861b0276e24de8f3d0a0&v=4)](https://github.com/skarard) [@skarard](https://github.com/skarard) [![Avatar for chasemcdo](https://avatars.githubusercontent.com/u/74692158?u=9c25a170d24cc30f10eafc4d44a38067cdf5eed8&v=4)](https://github.com/chasemcdo) [@chasemcdo](https://github.com/chasemcdo) [![Avatar for MaximeThoonsen](https://avatars.githubusercontent.com/u/4814551?u=efb35c6a7dc1ce99dfa8ac8f0f1314cdb4fddfe1&v=4)](https://github.com/MaximeThoonsen) [@MaximeThoonsen](https://github.com/MaximeThoonsen) [![Avatar for easwee](https://avatars.githubusercontent.com/u/2518825?u=a24026bc5ed35688174b1a36f3c29eda594d38d7&v=4)](https://github.com/easwee) [@easwee](https://github.com/easwee) [![Avatar for mieslep](https://avatars.githubusercontent.com/u/5420540?u=8f038c002fbce42427999eb715dc9f868cef1c84&v=4)](https://github.com/mieslep) [@mieslep](https://github.com/mieslep) [![Avatar for ysnows](https://avatars.githubusercontent.com/u/11255869?u=b0b519b6565c43d01795ba092521c8677f30134c&v=4)](https://github.com/ysnows) [@ysnows](https://github.com/ysnows) [![Avatar for tyumentsev4](https://avatars.githubusercontent.com/u/56769451?u=088102b6160822bc68c25a2a5df170080d0b16a2&v=4)](https://github.com/tyumentsev4) [@tyumentsev4](https://github.com/tyumentsev4) [![Avatar for nickscamara](https://avatars.githubusercontent.com/u/20311743?u=29bf2391ae34297a12a88d813731b0bdf289e4a5&v=4)](https://github.com/nickscamara) [@nickscamara](https://github.com/nickscamara) [![Avatar for nigel-daniels](https://avatars.githubusercontent.com/u/4641452?v=4)](https://github.com/nigel-daniels) [@nigel-daniels](https://github.com/nigel-daniels) [![Avatar for MJDeligan](https://avatars.githubusercontent.com/u/48515433?v=4)](https://github.com/MJDeligan) [@MJDeligan](https://github.com/MJDeligan) [![Avatar for jeasonnow](https://avatars.githubusercontent.com/u/16950207?u=ab2d0d4f1574398ac842e6bb3c2ba020ab7711eb&v=4)](https://github.com/jeasonnow) [@jeasonnow](https://github.com/jeasonnow) [![Avatar for malandis](https://avatars.githubusercontent.com/u/3690240?v=4)](https://github.com/malandis) [@malandis](https://github.com/malandis) [![Avatar for danielchalef](https://avatars.githubusercontent.com/u/131175?u=332fe36f12d9ffe9e4414dc776b381fe801a9c53&v=4)](https://github.com/danielchalef) [@danielchalef](https://github.com/danielchalef) [![Avatar for Swimburger](https://avatars.githubusercontent.com/u/3382717?u=5a84a173b0e80effc9161502c0848bf06c84bde9&v=4)](https://github.com/Swimburger) [@Swimburger](https://github.com/Swimburger) [![Avatar for Anush008](https://avatars.githubusercontent.com/u/46051506?u=026f5f140e8b7ba4744bf971f9ebdea9ebab67ca&v=4)](https://github.com/Anush008) [@Anush008](https://github.com/Anush008) [![Avatar for mfortman11](https://avatars.githubusercontent.com/u/6100513?u=c758a02fc05dc36315fcfadfccd6208883436cb8&v=4)](https://github.com/mfortman11) [@mfortman11](https://github.com/mfortman11) [![Avatar for kwkr](https://avatars.githubusercontent.com/u/20127759?v=4)](https://github.com/kwkr) [@kwkr](https://github.com/kwkr) [![Avatar for sarangan12](https://avatars.githubusercontent.com/u/602456?u=d39962c60b0ac5fea4e97cb67433a42c736c3c5b&v=4)](https://github.com/sarangan12) [@sarangan12](https://github.com/sarangan12) [![Avatar for fahreddinozcan](https://avatars.githubusercontent.com/u/88107904?v=4)](https://github.com/fahreddinozcan) [@fahreddinozcan](https://github.com/fahreddinozcan) [![Avatar for ewfian](https://avatars.githubusercontent.com/u/12423122?u=681de0c470e9b349963ee935ddfd6b2e097e7181&v=4)](https://github.com/ewfian) [@ewfian](https://github.com/ewfian) [![Avatar for jl4nz](https://avatars.githubusercontent.com/u/94814971?u=266358610eeb54c3393dc127718dd6a997fdbf52&v=4)](https://github.com/jl4nz) [@jl4nz](https://github.com/jl4nz) [![Avatar for volodymyr-memsql](https://avatars.githubusercontent.com/u/57520563?v=4)](https://github.com/volodymyr-memsql) [@volodymyr-memsql](https://github.com/volodymyr-memsql) [![Avatar for jasondotparse](https://avatars.githubusercontent.com/u/13938372?u=0e3f80aa515c41b7d9084b73d761cad378ebdc7a&v=4)](https://github.com/jasondotparse) [@jasondotparse](https://github.com/jasondotparse) [![Avatar for mishushakov](https://avatars.githubusercontent.com/u/10400064?u=52b50611d587317f397a96f898753099d65931f1&v=4)](https://github.com/mishushakov) [@mishushakov](https://github.com/mishushakov) [![Avatar for kristianfreeman](https://avatars.githubusercontent.com/u/922353?u=212a67ff65d67d39e41c3cb58cd7a7b8b2f89f3e&v=4)](https://github.com/kristianfreeman) [@kristianfreeman](https://github.com/kristianfreeman) [![Avatar for neebdev](https://avatars.githubusercontent.com/u/94310799?u=b6f604bc6c3a6380f0b83025ca94e2e22179ac2a&v=4)](https://github.com/neebdev) [@neebdev](https://github.com/neebdev) [![Avatar for tsg](https://avatars.githubusercontent.com/u/101817?u=39f31ff29d2589046148c6ed1c1c923982d86b1a&v=4)](https://github.com/tsg) [@tsg](https://github.com/tsg) [![Avatar for lokesh-couchbase](https://avatars.githubusercontent.com/u/113521973?v=4)](https://github.com/lokesh-couchbase) [@lokesh-couchbase](https://github.com/lokesh-couchbase) [![Avatar for nicoloboschi](https://avatars.githubusercontent.com/u/23314389?u=2014e20e246530fa89bd902fe703b6f9e6ecf833&v=4)](https://github.com/nicoloboschi) [@nicoloboschi](https://github.com/nicoloboschi) [![Avatar for zackproser](https://avatars.githubusercontent.com/u/1769996?u=67913e5af19c6ea2df87f33db0ddd2b6cb805eb5&v=4)](https://github.com/zackproser) [@zackproser](https://github.com/zackproser) [![Avatar for justindra](https://avatars.githubusercontent.com/u/4289486?v=4)](https://github.com/justindra) [@justindra](https://github.com/justindra) [![Avatar for vincelwt](https://avatars.githubusercontent.com/u/5092466?u=713f9947e4315b6f0ef62ec5cccd978133006783&v=4)](https://github.com/vincelwt) [@vincelwt](https://github.com/vincelwt) [![Avatar for cwoolum](https://avatars.githubusercontent.com/u/942415?u=8210ef711d1666ec234db9a0c4a9b32fd9f36593&v=4)](https://github.com/cwoolum) [@cwoolum](https://github.com/cwoolum) [![Avatar for sunner](https://avatars.githubusercontent.com/u/255413?v=4)](https://github.com/sunner) [@sunner](https://github.com/sunner) [![Avatar for dl102306](https://avatars.githubusercontent.com/u/2979960?v=4)](https://github.com/dl102306) [@dl102306](https://github.com/dl102306) [![Avatar for rahilvora](https://avatars.githubusercontent.com/u/5127548?u=0cd74312c28da39646785409fb0a37a9b3d3420a&v=4)](https://github.com/rahilvora) [@rahilvora](https://github.com/rahilvora) [![Avatar for lukywong](https://avatars.githubusercontent.com/u/1433871?v=4)](https://github.com/lukywong) [@lukywong](https://github.com/lukywong) [![Avatar for mayooear](https://avatars.githubusercontent.com/u/107035552?u=708ca9b002559f6175803a80a1e47f3e84ba91e2&v=4)](https://github.com/mayooear) [@mayooear](https://github.com/mayooear) [![Avatar for chitalian](https://avatars.githubusercontent.com/u/26822232?u=accedd106a5e9d8335cb631c1bfe84b8cc494083&v=4)](https://github.com/chitalian) [@chitalian](https://github.com/chitalian) [![Avatar for paaatrrrick](https://avatars.githubusercontent.com/u/88113528?u=23275c7b8928a38b34195358ea9f4d057fe1e171&v=4)](https://github.com/paaatrrrick) [@paaatrrrick](https://github.com/paaatrrrick) [![Avatar for alexleventer](https://avatars.githubusercontent.com/u/3254549?u=794d178a761379e162a1092c556e98a9ec5c2410&v=4)](https://github.com/alexleventer) [@alexleventer](https://github.com/alexleventer) [![Avatar for Anirudh31415926535](https://avatars.githubusercontent.com/u/171019460?v=4)](https://github.com/Anirudh31415926535) [@Anirudh31415926535](https://github.com/Anirudh31415926535) [![Avatar for 3eif](https://avatars.githubusercontent.com/u/29833473?u=37b8f7a25883ee98bc6b6bd6029c6d5479724e2f&v=4)](https://github.com/3eif) [@3eif](https://github.com/3eif) [![Avatar for BitVoyagerMan](https://avatars.githubusercontent.com/u/121993229?u=717ed7012c040d5bf3a8ff1fd695a6a4f1ff0626&v=4)](https://github.com/BitVoyagerMan) [@BitVoyagerMan](https://github.com/BitVoyagerMan) [![Avatar for xixixao](https://avatars.githubusercontent.com/u/1473433?u=c4bf1cf9f8699c8647894cd226c0bf9124bdad58&v=4)](https://github.com/xixixao) [@xixixao](https://github.com/xixixao) [![Avatar for ovuruska](https://avatars.githubusercontent.com/u/75265893?u=7f11152d07f1719da22084388c09b5fc64ab6c89&v=4)](https://github.com/ovuruska) [@ovuruska](https://github.com/ovuruska) [![Avatar for jo32](https://avatars.githubusercontent.com/u/501632?u=a714d65c000d8f489f9fc2363f9a372b0dba05e3&v=4)](https://github.com/jo32) [@jo32](https://github.com/jo32) [![Avatar for RohitMidha23](https://avatars.githubusercontent.com/u/38888530?u=5c4b99eff970e551e5b756f270aa5234bc666316&v=4)](https://github.com/RohitMidha23) [@RohitMidha23](https://github.com/RohitMidha23) [![Avatar for karol-f](https://avatars.githubusercontent.com/u/893082?u=0cda88d40a24ee696580f2e62f5569f49117cf40&v=4)](https://github.com/karol-f) [@karol-f](https://github.com/karol-f) [![Avatar for konstantinov-raft](https://avatars.githubusercontent.com/u/105433902?v=4)](https://github.com/konstantinov-raft) [@konstantinov-raft](https://github.com/konstantinov-raft) [![Avatar for jameshfisher](https://avatars.githubusercontent.com/u/166966?u=b78059abca798fbce8c9da4f6ddfb72ea03b20bb&v=4)](https://github.com/jameshfisher) [@jameshfisher](https://github.com/jameshfisher) [![Avatar for the-powerpointer](https://avatars.githubusercontent.com/u/134403026?u=ddd77b62b35c5497ae3d846f8917bdd81e5ef19e&v=4)](https://github.com/the-powerpointer) [@the-powerpointer](https://github.com/the-powerpointer) [![Avatar for davidfant](https://avatars.githubusercontent.com/u/17096641?u=9b935c68c077d53642c1b4aff62f04d08e2ffac7&v=4)](https://github.com/davidfant) [@davidfant](https://github.com/davidfant) [![Avatar for dependabot](https://avatars.githubusercontent.com/in/29110?v=4)](https://github.com/apps/dependabot) [@dependabot](https://github.com/apps/dependabot) [![Avatar for MthwRobinson](https://avatars.githubusercontent.com/u/1635179?u=0631cb84ca580089198114f94d9c27efe730220e&v=4)](https://github.com/MthwRobinson) [@MthwRobinson](https://github.com/MthwRobinson) [![Avatar for SimonPrammer](https://avatars.githubusercontent.com/u/44960995?u=a513117a60e9f1aa09247ec916018ee272897169&v=4)](https://github.com/SimonPrammer) [@SimonPrammer](https://github.com/SimonPrammer) [![Avatar for munkhorgil](https://avatars.githubusercontent.com/u/978987?u=eff77a6f7bc4edbace4929731638d4727923013f&v=4)](https://github.com/munkhorgil) [@munkhorgil](https://github.com/munkhorgil) [![Avatar for alx13](https://avatars.githubusercontent.com/u/1572864?v=4)](https://github.com/alx13) [@alx13](https://github.com/alx13) [![Avatar for castroCrea](https://avatars.githubusercontent.com/u/20707343?u=25e872c764bd31b71148f2dec896f64be5e034ff&v=4)](https://github.com/castroCrea) [@castroCrea](https://github.com/castroCrea) [![Avatar for samheutmaker](https://avatars.githubusercontent.com/u/1767032?u=a50f2b3b339eb965b9c812977aa10d64202e2e95&v=4)](https://github.com/samheutmaker) [@samheutmaker](https://github.com/samheutmaker) [![Avatar for archie-swif](https://avatars.githubusercontent.com/u/2158707?u=8a0aeee45e93ba575321804a7b709bf8897941de&v=4)](https://github.com/archie-swif) [@archie-swif](https://github.com/archie-swif) [![Avatar for valdo99](https://avatars.githubusercontent.com/u/41517614?u=ba37c9a21db3068953ae50d90c1cd07c3dec3abd&v=4)](https://github.com/valdo99) [@valdo99](https://github.com/valdo99) [![Avatar for chentschel](https://avatars.githubusercontent.com/u/319227?u=db20ce1d424f10d7760665ab693791ebc580131a&v=4)](https://github.com/chentschel) [@chentschel](https://github.com/chentschel) [![Avatar for gmpetrov](https://avatars.githubusercontent.com/u/4693180?u=8cf781d9099d6e2f2d2caf7612a5c2811ba13ef8&v=4)](https://github.com/gmpetrov) [@gmpetrov](https://github.com/gmpetrov) [![Avatar for mattzcarey](https://avatars.githubusercontent.com/u/77928207?u=fc8febe2a4b67384046eb4041b325bb34665d59c&v=4)](https://github.com/mattzcarey) [@mattzcarey](https://github.com/mattzcarey) [![Avatar for albertpurnama](https://avatars.githubusercontent.com/u/14824254?u=b3acdfc46d3d26d44f66a7312b102172c7ff9722&v=4)](https://github.com/albertpurnama) [@albertpurnama](https://github.com/albertpurnama) [![Avatar for CahidArda](https://avatars.githubusercontent.com/u/57228345?v=4)](https://github.com/CahidArda) [@CahidArda](https://github.com/CahidArda) [![Avatar for yroc92](https://avatars.githubusercontent.com/u/17517541?u=7405432fa828c094e130e8193be3cae04ac96d11&v=4)](https://github.com/yroc92) [@yroc92](https://github.com/yroc92) [![Avatar for Basti-an](https://avatars.githubusercontent.com/u/42387209?u=43ac44545861ce4adec99f973aeea3e6cf9a1bc0&v=4)](https://github.com/Basti-an) [@Basti-an](https://github.com/Basti-an) [![Avatar for CarlosZiegler](https://avatars.githubusercontent.com/u/38855507?u=65c19ae772581fb7367f646ed90be44311e60e70&v=4)](https://github.com/CarlosZiegler) [@CarlosZiegler](https://github.com/CarlosZiegler) [![Avatar for iloveitaly](https://avatars.githubusercontent.com/u/150855?v=4)](https://github.com/iloveitaly) [@iloveitaly](https://github.com/iloveitaly) [![Avatar for dilling](https://avatars.githubusercontent.com/u/5846912?v=4)](https://github.com/dilling) [@dilling](https://github.com/dilling) [![Avatar for anselm94](https://avatars.githubusercontent.com/u/9033201?u=e5f657c3a1657c089d7cb88121e544ae7212e6f1&v=4)](https://github.com/anselm94) [@anselm94](https://github.com/anselm94) [![Avatar for aixgeek](https://avatars.githubusercontent.com/u/9697715?u=d139c5568375c2472ac6142325e6856cd766d88d&v=4)](https://github.com/aixgeek) [@aixgeek](https://github.com/aixgeek) [![Avatar for gramliu](https://avatars.githubusercontent.com/u/24856195?u=9f55337506cdcac3146772c56b4634e6b46a5e46&v=4)](https://github.com/gramliu) [@gramliu](https://github.com/gramliu) [![Avatar for jeffchuber](https://avatars.githubusercontent.com/u/891664?u=722172a0061f68ab22819fa88a354ec973f70a63&v=4)](https://github.com/jeffchuber) [@jeffchuber](https://github.com/jeffchuber) [![Avatar for ywkim](https://avatars.githubusercontent.com/u/588581?u=df702e5b817a56476cb0cd8e7587b9be844d2850&v=4)](https://github.com/ywkim) [@ywkim](https://github.com/ywkim) [![Avatar for jirimoravcik](https://avatars.githubusercontent.com/u/951187?u=e80c215810058f57145042d12360d463e3a53443&v=4)](https://github.com/jirimoravcik) [@jirimoravcik](https://github.com/jirimoravcik) [![Avatar for miloradvojnovic](https://avatars.githubusercontent.com/u/11798350?u=a9b931a1a3319787bef5e2d16e1fdec0888cdad5&v=4)](https://github.com/miloradvojnovic) [@miloradvojnovic](https://github.com/miloradvojnovic) [![Avatar for janvi-kalra](https://avatars.githubusercontent.com/u/119091286?u=ed9e9d72bbf9964b80f81e5ba8d1d5b2f860c23f&v=4)](https://github.com/janvi-kalra) [@janvi-kalra](https://github.com/janvi-kalra) [![Avatar for yuku](https://avatars.githubusercontent.com/u/96157?v=4)](https://github.com/yuku) [@yuku](https://github.com/yuku) [![Avatar for conroywhitney](https://avatars.githubusercontent.com/u/249891?u=36703ce68261be59109622877012be08fbc090da&v=4)](https://github.com/conroywhitney) [@conroywhitney](https://github.com/conroywhitney) [![Avatar for seuha516](https://avatars.githubusercontent.com/u/79067549?u=de7a2688cb44010afafd055d707f3463585494df&v=4)](https://github.com/seuha516) [@seuha516](https://github.com/seuha516) [![Avatar for Czechh](https://avatars.githubusercontent.com/u/4779936?u=ab072503433effc18c071b31adda307988877d5e&v=4)](https://github.com/Czechh) [@Czechh](https://github.com/Czechh) [![Avatar for adam101](https://avatars.githubusercontent.com/u/1535782?v=4)](https://github.com/adam101) [@adam101](https://github.com/adam101) [![Avatar for OlegIvaniv](https://avatars.githubusercontent.com/u/12657221?v=4)](https://github.com/OlegIvaniv) [@OlegIvaniv](https://github.com/OlegIvaniv) [![Avatar for jaclar](https://avatars.githubusercontent.com/u/362704?u=52d868cc75c793fa895ef7035ae45516bd915e84&v=4)](https://github.com/jaclar) [@jaclar](https://github.com/jaclar) [![Avatar for TeCHiScy](https://avatars.githubusercontent.com/u/741195?u=e5937011ef84ff8a4b4b62ac1926a291c04f5d8b&v=4)](https://github.com/TeCHiScy) [@TeCHiScy](https://github.com/TeCHiScy) [![Avatar for anthonychu](https://avatars.githubusercontent.com/u/3982077?u=8bbebac42cb84a25c629f83f212b2d099ffa3964&v=4)](https://github.com/anthonychu) [@anthonychu](https://github.com/anthonychu) [![Avatar for josemussa](https://avatars.githubusercontent.com/u/4422500?u=d676ede0cec8ee5df6879ebf9d8b72d51ea1eb7f&v=4)](https://github.com/josemussa) [@josemussa](https://github.com/josemussa) [![Avatar for ivoneijr](https://avatars.githubusercontent.com/u/6401435?u=96c11b6333636bd784ffbff72998591f3b3f087b&v=4)](https://github.com/ivoneijr) [@ivoneijr](https://github.com/ivoneijr) [![Avatar for tonisives](https://avatars.githubusercontent.com/u/1083534?v=4)](https://github.com/tonisives) [@tonisives](https://github.com/tonisives) [![Avatar for Njuelle](https://avatars.githubusercontent.com/u/3192870?u=e126aae39f36565450ebc854b35c6e890b705e71&v=4)](https://github.com/Njuelle) [@Njuelle](https://github.com/Njuelle) [![Avatar for Roland0511](https://avatars.githubusercontent.com/u/588050?u=3c91917389117ee84843d961252ab7a2b9097e0e&v=4)](https://github.com/Roland0511) [@Roland0511](https://github.com/Roland0511) [![Avatar for SebastjanPrachovskij](https://avatars.githubusercontent.com/u/86522260?u=66898c89771c7b8ff38958e9fb9563a1cf7f8004&v=4)](https://github.com/SebastjanPrachovskij) [@SebastjanPrachovskij](https://github.com/SebastjanPrachovskij) [![Avatar for cinqisap](https://avatars.githubusercontent.com/u/158295355?v=4)](https://github.com/cinqisap) [@cinqisap](https://github.com/cinqisap) [![Avatar for dylanintech](https://avatars.githubusercontent.com/u/86082012?u=6516bbf39c5af198123d8ed2e35fff5d200f4d2e&v=4)](https://github.com/dylanintech) [@dylanintech](https://github.com/dylanintech) [![Avatar for andrewnguonly](https://avatars.githubusercontent.com/u/7654246?u=b8599019655adaada3cdc3c3006798df42c44494&v=4)](https://github.com/andrewnguonly) [@andrewnguonly](https://github.com/andrewnguonly) [![Avatar for clemenspeters](https://avatars.githubusercontent.com/u/13015002?u=059c556d90a2e5639dee42123077d51223c190f0&v=4)](https://github.com/clemenspeters) [@clemenspeters](https://github.com/clemenspeters) [![Avatar for ShaunBaker](https://avatars.githubusercontent.com/u/1176557?u=c2e8ecfb45b736fc4d3bbfe182e26936bd519fd3&v=4)](https://github.com/ShaunBaker) [@ShaunBaker](https://github.com/ShaunBaker) [![Avatar for machulav](https://avatars.githubusercontent.com/u/2857712?u=6809bef8bf07c46b39cd2fcd6027ed86e76372cd&v=4)](https://github.com/machulav) [@machulav](https://github.com/machulav) [![Avatar for dersia](https://avatars.githubusercontent.com/u/1537958?u=5da46ca1cd93c6fed927c612fc454ba51d0a36b1&v=4)](https://github.com/dersia) [@dersia](https://github.com/dersia) [![Avatar for joshsny](https://avatars.githubusercontent.com/u/7135900?u=109e43c5e906a8ecc1a2d465c4457f5cf29328a5&v=4)](https://github.com/joshsny) [@joshsny](https://github.com/joshsny) [![Avatar for eactisgrosso](https://avatars.githubusercontent.com/u/2279003?u=d122874eedb211359d4bf0119877d74ea7d5bcab&v=4)](https://github.com/eactisgrosso) [@eactisgrosso](https://github.com/eactisgrosso) [![Avatar for frankolson](https://avatars.githubusercontent.com/u/6773706?u=738775762205a07fd7de297297c99f781e957c58&v=4)](https://github.com/frankolson) [@frankolson](https://github.com/frankolson) [![Avatar for uthmanmoh](https://avatars.githubusercontent.com/u/83053931?u=5c715d2d4f6786fa749276de8eced710be8bfa99&v=4)](https://github.com/uthmanmoh) [@uthmanmoh](https://github.com/uthmanmoh) [![Avatar for Jordan-Gilliam](https://avatars.githubusercontent.com/u/25993686?u=319a6ed2119197d4d11301614a104ae686f9fc70&v=4)](https://github.com/Jordan-Gilliam) [@Jordan-Gilliam](https://github.com/Jordan-Gilliam) [![Avatar for winor30](https://avatars.githubusercontent.com/u/12413150?u=691a5e076bdd8c9e9fd637a41496b29e11b0c82f&v=4)](https://github.com/winor30) [@winor30](https://github.com/winor30) [![Avatar for willemmulder](https://avatars.githubusercontent.com/u/70933?u=206fafc72fd14b4291cb29269c5e1cc8081d043b&v=4)](https://github.com/willemmulder) [@willemmulder](https://github.com/willemmulder) [![Avatar for mhart](https://avatars.githubusercontent.com/u/367936?v=4)](https://github.com/mhart) [@mhart](https://github.com/mhart) [![Avatar for mvaker](https://avatars.githubusercontent.com/u/5671913?u=2e237cb1dd51f9d0dd01f0deb80003163641fc49&v=4)](https://github.com/mvaker) [@mvaker](https://github.com/mvaker) [![Avatar for vitaly-ps](https://avatars.githubusercontent.com/u/141448200?u=a3902a9c11399c916f1af2bf0ead901e7afe1a67&v=4)](https://github.com/vitaly-ps) [@vitaly-ps](https://github.com/vitaly-ps) [![Avatar for cbh123](https://avatars.githubusercontent.com/u/14149230?u=ca710ca2a64391470163ddef6b5ea7633ab26872&v=4)](https://github.com/cbh123) [@cbh123](https://github.com/cbh123) [![Avatar for Neverland3124](https://avatars.githubusercontent.com/u/52025513?u=865e861a1abb0d78be587f685d28fe8a00aee8fe&v=4)](https://github.com/Neverland3124) [@Neverland3124](https://github.com/Neverland3124) [![Avatar for jasonnathan](https://avatars.githubusercontent.com/u/780157?u=006e0deda897eb1a4abcc459adcd7242dcbe8fee&v=4)](https://github.com/jasonnathan) [@jasonnathan](https://github.com/jasonnathan) [![Avatar for Maanethdesilva](https://avatars.githubusercontent.com/u/94875583?v=4)](https://github.com/Maanethdesilva) [@Maanethdesilva](https://github.com/Maanethdesilva) [![Avatar for fuleinist](https://avatars.githubusercontent.com/u/1163738?v=4)](https://github.com/fuleinist) [@fuleinist](https://github.com/fuleinist) [![Avatar for kwadhwa18](https://avatars.githubusercontent.com/u/6015244?u=a127081404b8dc16ac0e84a869dfff4ac82bbab2&v=4)](https://github.com/kwadhwa18) [@kwadhwa18](https://github.com/kwadhwa18) [![Avatar for sousousore1](https://avatars.githubusercontent.com/u/624438?v=4)](https://github.com/sousousore1) [@sousousore1](https://github.com/sousousore1) [![Avatar for seth-25](https://avatars.githubusercontent.com/u/49222652?u=203c2bef6cbb77668a289b8272aea4fb654558d5&v=4)](https://github.com/seth-25) [@seth-25](https://github.com/seth-25) [![Avatar for tomi-mercado](https://avatars.githubusercontent.com/u/60221771?u=f8c1214535e402b0ff5c3428bfe98b586b517106&v=4)](https://github.com/tomi-mercado) [@tomi-mercado](https://github.com/tomi-mercado) [![Avatar for JHeidinga](https://avatars.githubusercontent.com/u/1702015?u=fa33fb709707e2429f10fbb824abead61628d50c&v=4)](https://github.com/JHeidinga) [@JHeidinga](https://github.com/JHeidinga) [![Avatar for niklas-lohmann](https://avatars.githubusercontent.com/u/68230177?v=4)](https://github.com/niklas-lohmann) [@niklas-lohmann](https://github.com/niklas-lohmann) [![Avatar for Durisvk](https://avatars.githubusercontent.com/u/8467003?u=f07b8c070eaed3ad8972be4f4ca91afb1ae6e2c0&v=4)](https://github.com/Durisvk) [@Durisvk](https://github.com/Durisvk) [![Avatar for BjoernRave](https://avatars.githubusercontent.com/u/36173920?u=c3acae11221a037c16254e2187555ea6259d89c3&v=4)](https://github.com/BjoernRave) [@BjoernRave](https://github.com/BjoernRave) [![Avatar for LordMsz](https://avatars.githubusercontent.com/u/33070601?u=ddc6c16156f6397198692c547324e51f94c70ca7&v=4)](https://github.com/LordMsz) [@LordMsz](https://github.com/LordMsz) [![Avatar for tanyaasharma](https://avatars.githubusercontent.com/u/140478067?v=4)](https://github.com/tanyaasharma) [@tanyaasharma](https://github.com/tanyaasharma) [![Avatar for crazyurus](https://avatars.githubusercontent.com/u/2209055?u=b39f7e70f137ff3d1785d261cb15067f0d91ae05&v=4)](https://github.com/crazyurus) [@crazyurus](https://github.com/crazyurus) [![Avatar for qalqi](https://avatars.githubusercontent.com/u/1781048?u=837879a7e62c6b3736dc39a31ff42873bee2c532&v=4)](https://github.com/qalqi) [@qalqi](https://github.com/qalqi) [![Avatar for katarinasupe](https://avatars.githubusercontent.com/u/61758502?u=20cdcb0bae81b9eb330c94f7cfae462327785219&v=4)](https://github.com/katarinasupe) [@katarinasupe](https://github.com/katarinasupe) [![Avatar for paul-paliychuk](https://avatars.githubusercontent.com/u/26054637?u=edd1e4f54e91b549f2edb525d43210f4f04d7367&v=4)](https://github.com/paul-paliychuk) [@paul-paliychuk](https://github.com/paul-paliychuk) [![Avatar for andrewlei](https://avatars.githubusercontent.com/u/1158058?v=4)](https://github.com/andrewlei) [@andrewlei](https://github.com/andrewlei) [![Avatar for floomby](https://avatars.githubusercontent.com/u/3113021?v=4)](https://github.com/floomby) [@floomby](https://github.com/floomby) [![Avatar for milanjrodd](https://avatars.githubusercontent.com/u/121220673?u=55636f26ea48e77e0372008089ff2c38691eaa0a&v=4)](https://github.com/milanjrodd) [@milanjrodd](https://github.com/milanjrodd) [![Avatar for NickMandylas](https://avatars.githubusercontent.com/u/19514618?u=95f8c29ed06696260722c2c6aa7bac3a1136d7a2&v=4)](https://github.com/NickMandylas) [@NickMandylas](https://github.com/NickMandylas) [![Avatar for DravenCat](https://avatars.githubusercontent.com/u/55412122?v=4)](https://github.com/DravenCat) [@DravenCat](https://github.com/DravenCat) [![Avatar for Alireza29675](https://avatars.githubusercontent.com/u/2771377?u=65ec71f9860ac2610e1cb5028173f67713a174d7&v=4)](https://github.com/Alireza29675) [@Alireza29675](https://github.com/Alireza29675) [![Avatar for zhengxs2018](https://avatars.githubusercontent.com/u/7506913?u=42c32ca59ae2e44532cd45027e5b62d2712cf2a2&v=4)](https://github.com/zhengxs2018) [@zhengxs2018](https://github.com/zhengxs2018) [![Avatar for tofuliang](https://avatars.githubusercontent.com/u/1814685?v=4)](https://github.com/tofuliang) [@tofuliang](https://github.com/tofuliang) [![Avatar for cmtoomey](https://avatars.githubusercontent.com/u/12201602?u=ea5cbb8d158980f6050dd41ae41b7f72e0a47337&v=4)](https://github.com/cmtoomey) [@cmtoomey](https://github.com/cmtoomey) [![Avatar for igorshapiro](https://avatars.githubusercontent.com/u/1085209?u=16b60724316a7ed8e8b52af576c121215461922a&v=4)](https://github.com/igorshapiro) [@igorshapiro](https://github.com/igorshapiro) [![Avatar for ezynda3](https://avatars.githubusercontent.com/u/5308871?v=4)](https://github.com/ezynda3) [@ezynda3](https://github.com/ezynda3) [![Avatar for more-by-more](https://avatars.githubusercontent.com/u/67614844?u=d3d818efb3e3e2ddda589d6157f853922a460f5b&v=4)](https://github.com/more-by-more) [@more-by-more](https://github.com/more-by-more) [![Avatar for noble-varghese](https://avatars.githubusercontent.com/u/109506617?u=c1d2a1813c51bff89bfa85d533633ed4c201ba2e&v=4)](https://github.com/noble-varghese) [@noble-varghese](https://github.com/noble-varghese) [![Avatar for SananR](https://avatars.githubusercontent.com/u/14956384?u=538ff9bf09497059b312067333f68eba75594802&v=4)](https://github.com/SananR) [@SananR](https://github.com/SananR) [![Avatar for fraserxu](https://avatars.githubusercontent.com/u/1183541?v=4)](https://github.com/fraserxu) [@fraserxu](https://github.com/fraserxu) [![Avatar for ashvardanian](https://avatars.githubusercontent.com/u/1983160?u=536f2558c6ac33b74a6d89520dcb27ba46954070&v=4)](https://github.com/ashvardanian) [@ashvardanian](https://github.com/ashvardanian) [![Avatar for adeelehsan](https://avatars.githubusercontent.com/u/8156837?u=99cacfbd962ff58885bdf68e5fc640fc0d3cb87c&v=4)](https://github.com/adeelehsan) [@adeelehsan](https://github.com/adeelehsan) [![Avatar for henriquegdantas](https://avatars.githubusercontent.com/u/12974790?u=80d76f256a7854da6ae441b6ee078119877398e7&v=4)](https://github.com/henriquegdantas) [@henriquegdantas](https://github.com/henriquegdantas) [![Avatar for evad1n](https://avatars.githubusercontent.com/u/50718218?u=ee35784971ef8dcdfdb25cfe0a8284ca48724938&v=4)](https://github.com/evad1n) [@evad1n](https://github.com/evad1n) [![Avatar for benjibc](https://avatars.githubusercontent.com/u/1585539?u=654a21985c875f78a20eda7e4884e8d64de86fba&v=4)](https://github.com/benjibc) [@benjibc](https://github.com/benjibc) [![Avatar for P-E-B](https://avatars.githubusercontent.com/u/38215315?u=3985b6a3ecb0e8338c5912ea9e20787152d0ad7a&v=4)](https://github.com/P-E-B) [@P-E-B](https://github.com/P-E-B) [![Avatar for omikader](https://avatars.githubusercontent.com/u/16735699?u=29fc7c7c777c3cabc22449b68bbb01fe2fa0b574&v=4)](https://github.com/omikader) [@omikader](https://github.com/omikader) [![Avatar for jasongill](https://avatars.githubusercontent.com/u/241711?v=4)](https://github.com/jasongill) [@jasongill](https://github.com/jasongill) [![Avatar for Luisotee](https://avatars.githubusercontent.com/u/50471205?u=059d6ab166e5a32c496ff50ef6e3fb0ca04a50ad&v=4)](https://github.com/Luisotee) [@Luisotee](https://github.com/Luisotee) [![Avatar for puigde](https://avatars.githubusercontent.com/u/83642160?u=7e76b13b7484e4601bea47dc6e238c89d453a24d&v=4)](https://github.com/puigde) [@puigde](https://github.com/puigde) [![Avatar for Adrastopoulos](https://avatars.githubusercontent.com/u/76796897?u=0bd50d301b4c7025f29396af44c8e1829eff1db6&v=4)](https://github.com/Adrastopoulos) [@Adrastopoulos](https://github.com/Adrastopoulos) [![Avatar for chase-crumbaugh](https://avatars.githubusercontent.com/u/90289500?u=0129550ecfbb4a92922fff7a406566a47a23dfb0&v=4)](https://github.com/chase-crumbaugh) [@chase-crumbaugh](https://github.com/chase-crumbaugh) [![Avatar for Zeneos](https://avatars.githubusercontent.com/u/95008961?v=4)](https://github.com/Zeneos) [@Zeneos](https://github.com/Zeneos) [![Avatar for joseanu](https://avatars.githubusercontent.com/u/2730127?u=9fe1d593bd63c7f116b9c46e9cbd359a2e4304f0&v=4)](https://github.com/joseanu) [@joseanu](https://github.com/joseanu) [![Avatar for JackFener](https://avatars.githubusercontent.com/u/20380671?u=b51d10b71850203e6360655fa59cc679c5a498e6&v=4)](https://github.com/JackFener) [@JackFener](https://github.com/JackFener) [![Avatar for swyxio](https://avatars.githubusercontent.com/u/6764957?u=97ad815028595b73b06ee4b0510e66bbe391228d&v=4)](https://github.com/swyxio) [@swyxio](https://github.com/swyxio) [![Avatar for pczekaj](https://avatars.githubusercontent.com/u/1460539?u=24c2db4a29757f608a54a062340a466cad843825&v=4)](https://github.com/pczekaj) [@pczekaj](https://github.com/pczekaj) [![Avatar for devinburnette](https://avatars.githubusercontent.com/u/13012689?u=7b68c67ea1bbc272c35be7c0bcf1c66a04554179&v=4)](https://github.com/devinburnette) [@devinburnette](https://github.com/devinburnette) [![Avatar for ananis25](https://avatars.githubusercontent.com/u/16446513?u=5026326ed39bfee8325c30cdbd24ac20519d21b8&v=4)](https://github.com/ananis25) [@ananis25](https://github.com/ananis25) [![Avatar for joaopcm](https://avatars.githubusercontent.com/u/58827242?u=3e03812a1074f2ce888b751c48e78a849c7e0aff&v=4)](https://github.com/joaopcm) [@joaopcm](https://github.com/joaopcm) [![Avatar for SalehHindi](https://avatars.githubusercontent.com/u/15721377?u=37fadd6a7bf9dfa63ceb866bda23ca44a7b2c0c2&v=4)](https://github.com/SalehHindi) [@SalehHindi](https://github.com/SalehHindi) [![Avatar for AvaterClasher](https://avatars.githubusercontent.com/u/116944847?u=102a870b3efed7f30f0a57123391a293eb6f5b08&v=4)](https://github.com/AvaterClasher) [@AvaterClasher](https://github.com/AvaterClasher) [![Avatar for JamsheedMistri](https://avatars.githubusercontent.com/u/13024750?u=6ae631199ec7c0bb34eb8d56200023cdd94720d3&v=4)](https://github.com/JamsheedMistri) [@JamsheedMistri](https://github.com/JamsheedMistri) [![Avatar for cmanou](https://avatars.githubusercontent.com/u/683160?u=e9050e4341c2c9d46b035ea17ea94234634e1b2c&v=4)](https://github.com/cmanou) [@cmanou](https://github.com/cmanou) [![Avatar for micahriggan](https://avatars.githubusercontent.com/u/3626473?u=508e8c831d8eb804e95985d5191a08c761544fad&v=4)](https://github.com/micahriggan) [@micahriggan](https://github.com/micahriggan) [![Avatar for w00ing](https://avatars.githubusercontent.com/u/29723695?u=563d4a628c9af35f827f476e38635310f1cec114&v=4)](https://github.com/w00ing) [@w00ing](https://github.com/w00ing) [![Avatar for madmed88](https://avatars.githubusercontent.com/u/1579388?u=62ca1bfe7c271b5fd1d77abc470aa5e535b1ed83&v=4)](https://github.com/madmed88) [@madmed88](https://github.com/madmed88) [![Avatar for ardsh](https://avatars.githubusercontent.com/u/23664687?u=158ef7e156a7881b8647ece63683aca2c28f132e&v=4)](https://github.com/ardsh) [@ardsh](https://github.com/ardsh) [![Avatar for JoeABCDEF](https://avatars.githubusercontent.com/u/39638510?u=f5fac0a3578572817b37a6dfc00adacb705ec7d0&v=4)](https://github.com/JoeABCDEF) [@JoeABCDEF](https://github.com/JoeABCDEF) [![Avatar for saul-jb](https://avatars.githubusercontent.com/u/2025187?v=4)](https://github.com/saul-jb) [@saul-jb](https://github.com/saul-jb) [![Avatar for JTCorrin](https://avatars.githubusercontent.com/u/73115680?v=4)](https://github.com/JTCorrin) [@JTCorrin](https://github.com/JTCorrin) [![Avatar for zandko](https://avatars.githubusercontent.com/u/37948383?u=04ccf6e060b27e39c931c2608381351cf236a28f&v=4)](https://github.com/zandko) [@zandko](https://github.com/zandko) [![Avatar for federicoestevez](https://avatars.githubusercontent.com/u/10424147?v=4)](https://github.com/federicoestevez) [@federicoestevez](https://github.com/federicoestevez) [![Avatar for martinseanhunt](https://avatars.githubusercontent.com/u/65744?u=ddac1e773828d8058a40bca680cf549e955f69ae&v=4)](https://github.com/martinseanhunt) [@martinseanhunt](https://github.com/martinseanhunt) [![Avatar for functorism](https://avatars.githubusercontent.com/u/17207277?u=4df9bc30a55b4da4b3d6fd20a2956afd722bde24&v=4)](https://github.com/functorism) [@functorism](https://github.com/functorism) [![Avatar for erictt](https://avatars.githubusercontent.com/u/9592198?u=567fa49c73e824525d33eefd836ece16ab9964c8&v=4)](https://github.com/erictt) [@erictt](https://github.com/erictt) [![Avatar for WilliamEspegren](https://avatars.githubusercontent.com/u/131612909?v=4)](https://github.com/WilliamEspegren) [@WilliamEspegren](https://github.com/WilliamEspegren) [![Avatar for lesters](https://avatars.githubusercontent.com/u/5798036?u=4eba31d63c3818d17fb8f9aa923599ac63ebfea8&v=4)](https://github.com/lesters) [@lesters](https://github.com/lesters) [![Avatar for my8bit](https://avatars.githubusercontent.com/u/782268?u=d83da3e6269d53a828bbeb6d661049a1ed185cb0&v=4)](https://github.com/my8bit) [@my8bit](https://github.com/my8bit) [![Avatar for erhant](https://avatars.githubusercontent.com/u/16037166?u=9d056a2f5059684620e22aa4d880e38183309b51&v=4)](https://github.com/erhant) [@erhant](https://github.com/erhant) We're so thankful for your support! And one more thank you to [@tiangolo](https://github.com/tiangolo) for inspiration via FastAPI's [excellent people page](https://fastapi.tiangolo.com/fastapi-people) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . --- # Welcome Contributors | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Welcome Contributors ==================== Hi there! Thank you for even being interested in contributing to LangChain. As an open-source project in a rapidly developing field, we are extremely open to contributions, whether they involve new features, improved infrastructure, better documentation, or bug fixes. ๐Ÿ—บ๏ธ Guidelines[โ€‹](#๏ธ-guidelines "Direct link to ๐Ÿ—บ๏ธ Guidelines") ----------------------------------------------------------------- ### ๐Ÿ‘ฉโ€๐Ÿ’ป Ways to contribute[โ€‹](#-ways-to-contribute "Direct link to ๐Ÿ‘ฉโ€๐Ÿ’ป Ways to contribute") There are many ways to contribute to LangChain. Here are some common ways people contribute: * [**Documentation**](/docs/contributing/documentation/style_guide) : Help improve our docs, including this one! * [**Code**](/docs/contributing/code) : Help us write code, fix bugs, or improve our infrastructure. * [**Integrations**](/docs/contributing/integrations) : Help us integrate with your favorite vendors and tools. * [**Discussions**](https://github.com/langchain-ai/langchainjs/discussions) : Help answer usage questions and discuss issues with users. ### ๐Ÿšฉ GitHub Issues[โ€‹](#-github-issues "Direct link to ๐Ÿšฉ GitHub Issues") Our [issues](https://github.com/langchain-ai/langchainjs/issues) page is kept up to date with bugs, improvements, and feature requests. There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help organize issues. If you start working on an issue, please assign it to yourself. If you are adding an issue, please try to keep it focused on a single, modular bug/improvement/feature. If two issues are related, or blocking, please link them rather than combining them. We will try to keep these issues as up-to-date as possible, though with the rapid rate of development in this field some may get out of date. If you notice this happening, please let us know. ### ๐Ÿ’ญ GitHub Discussions[โ€‹](#-github-discussions "Direct link to ๐Ÿ’ญ GitHub Discussions") We have a [discussions](https://github.com/langchain-ai/langchainjs/discussions) page where users can ask usage questions, discuss design decisions, and propose new features. If you are able to help answer questions, please do so! This will allow the maintainers to spend more time focused on development and bug fixing. ### ๐Ÿ™‹ Getting Help[โ€‹](#-getting-help "Direct link to ๐Ÿ™‹ Getting Help") Our goal is to have the simplest developer setup possible. Should you experience any difficulty getting setup, please contact a maintainer! Not only do we want to help get you unblocked, but we also want to make sure that the process is smooth for future contributors. In a similar vein, we do enforce certain linting, formatting, and documentation standards in the codebase. If you are finding these difficult (or even just annoying) to work with, feel free to contact a maintainer for help - we do not want these to get in the way of getting good code into the codebase. ๐ŸŒŸ Recognition ============== If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)! If you have a Twitter account you would like us to mention, please let us know in the PR or through another means. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [๐Ÿ—บ๏ธ Guidelines](#๏ธ-guidelines) * [๐Ÿ‘ฉโ€๐Ÿ’ป Ways to contribute](#-ways-to-contribute) * [๐Ÿšฉ GitHub Issues](#-github-issues) * [๐Ÿ’ญ GitHub Discussions](#-github-discussions) * [๐Ÿ™‹ Getting Help](#-getting-help) --- # Introduction | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) A newer LangChain version is out! Check out the [latest version](https://js.langchain.com/docs/introduction) . This is documentation for LangChain **v0.2**, which is no longer actively maintained. For the current stable version, see **[this version](https://js.langchain.com/docs/introduction) ** (Latest). On this page Introduction ============ **LangChain** is a framework for developing applications powered by large language models (LLMs). LangChain simplifies every stage of the LLM application lifecycle: * **Development**: Build your applications using LangChain's open-source [building blocks](/v0.2/docs/concepts#langchain-expression-language) , [components](/v0.2/docs/concepts) , and [third-party integrations](/v0.2/docs/integrations/platforms/) . Use [LangGraph.js](/v0.2/docs/concepts/#langgraphjs) to build stateful agents with first-class streaming and human-in-the-loop support. * **Productionization**: Use [LangSmith](https://docs.smith.langchain.com/) to inspect, monitor and evaluate your chains, so that you can continuously optimize and deploy with confidence. * **Deployment**: Turn your LangGraph applications into production-ready APIs and Assistants with [LangGraph Cloud](https://langchain-ai.github.io/langgraph/cloud/) (currently Python-only). ![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](/v0.2/svg/langchain_stack_062024.svg "LangChain Framework Overview")![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](/v0.2/svg/langchain_stack_062024_dark.svg "LangChain Framework Overview") Concretely, the framework consists of the following open-source libraries: * **`@langchain/core`**: Base abstractions and LangChain Expression Language. * **`@langchain/community`**: Third party integrations. * Partner packages (e.g. **`@langchain/openai`**, **`@langchain/anthropic`**, etc.): Some integrations have been further split into their own lightweight packages that only depend on **`@langchain/core`**. * **`langchain`**: Chains, agents, and retrieval strategies that make up an application's cognitive architecture. * **[LangGraph.js](https://langchain-ai.github.io/langgraphjs/) **: Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. * **[LangSmith](https://docs.smith.langchain.com) **: A developer platform that lets you debug, test, evaluate, and monitor LLM applications. note These docs focus on the JavaScript LangChain library. [Head here](https://python.langchain.com) for docs on the Python LangChain library. [Tutorials](/v0.2/docs/tutorials) [โ€‹](#tutorials "Direct link to tutorials") ----------------------------------------------------------------------------- If you're looking to build something specific or are more of a hands-on learner, check out our [tutorials](/v0.2/docs/tutorials) . This is the best place to get started. These are the best ones to get started with: * [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain) * [Build a Chatbot](/v0.2/docs/tutorials/chatbot) * [Build an Agent](/v0.2/docs/tutorials/agents) * [LangGraph.js quickstart](https://langchain-ai.github.io/langgraphjs/tutorials/quickstart/) Explore the full list of LangChain tutorials [here](/v0.2/docs/tutorials) , and check out other [LangGraph tutorials here](https://langchain-ai.github.io/langgraphjs/tutorials/) . [How-To Guides](/v0.2/docs/how_to/) [โ€‹](#how-to-guides "Direct link to how-to-guides") --------------------------------------------------------------------------------------- [Here](/v0.2/docs/how_to/) you'll find short answers to โ€œHow do Iโ€ฆ.?โ€ types of questions. These how-to guides don't cover topics in depth - you'll find that material in the [Tutorials](/v0.2/docs/tutorials) and the [API Reference](https://v02.api.js.langchain.com) . However, these guides will help you quickly accomplish common tasks. Check out [LangGraph-specific how-tos here](https://langchain-ai.github.io/langgraphjs/how-tos/) . [Conceptual Guide](/v0.2/docs/concepts) [โ€‹](#conceptual-guide "Direct link to conceptual-guide") ------------------------------------------------------------------------------------------------- Introductions to all the key parts of LangChain you'll need to know! [Here](/v0.2/docs/concepts) you'll find high level explanations of all LangChain concepts. For a deeper dive into LangGraph concepts, check out [this page](https://langchain-ai.github.io/langgraph/concepts/) . [API reference](https://api.js.langchain.com) [โ€‹](#api-reference "Direct link to api-reference") ------------------------------------------------------------------------------------------------- Head to the reference section for full documentation of all classes and methods in the LangChain Python packages. Ecosystem[โ€‹](#ecosystem "Direct link to Ecosystem") ---------------------------------------------------- ### [๐Ÿฆœ๐Ÿ› ๏ธ LangSmith](https://docs.smith.langchain.com) [โ€‹](#๏ธ-langsmith "Direct link to ๏ธ-langsmith") Trace and evaluate your language model applications and intelligent agents to help you move from prototype to production. ### [๐Ÿฆœ๐Ÿ•ธ๏ธ LangGraph](https://langchain-ai.github.io/langgraphjs/) [โ€‹](#๏ธ-langgraph "Direct link to ๏ธ-langgraph") Build stateful, multi-actor applications with LLMs, built on top of (and intended to be used with) LangChain primitives. Additional resources[โ€‹](#additional-resources "Direct link to Additional resources") ------------------------------------------------------------------------------------- ### [Security](/v0.2/docs/security) [โ€‹](#security "Direct link to security") Read up on our [Security](/v0.2/docs/security) best practices to make sure you're developing safely with LangChain. ### [Integrations](/v0.2/docs/integrations/platforms/) [โ€‹](#integrations "Direct link to integrations") LangChain is part of a rich ecosystem of tools that integrate with our framework and build on top of it. Check out our growing list of [integrations](/v0.2/docs/integrations/platforms/) . ### [Contributing](/v0.2/docs/contributing) [โ€‹](#contributing "Direct link to contributing") Check out the developer's guide for guidelines on contributing and help getting your dev environment set up. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Tutorials](#tutorials) * [How-To Guides](#how-to-guides) * [Conceptual Guide](#conceptual-guide) * [API reference](#api-reference) * [Ecosystem](#ecosystem) * [๐Ÿฆœ๐Ÿ› ๏ธ LangSmith](#๏ธ-langsmith) * [๐Ÿฆœ๐Ÿ•ธ๏ธ LangGraph](#๏ธ-langgraph) * [Additional resources](#additional-resources) * [Security](#security) * [Integrations](#integrations) * [Contributing](#contributing) --- # Tutorials | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Tutorials ========= New to LangChain or LLM app development in general? Read this material to quickly get up and running building your first applications. Get started[โ€‹](#get-started "Direct link to Get started") ---------------------------------------------------------- Familiarize yourself with LangChain's open-source components by building simple applications. If you're looking to get started with [chat models](/docs/integrations/chat/) , [vector stores](/docs/integrations/vectorstores/) , or other LangChain components from a specific provider, check out our supported [integrations](/docs/integrations/platforms/) . * [Chat models and prompts](/docs/tutorials/llm_chain) : Build a simple LLM application with [prompt templates](/docs/concepts/prompt_templates) and [chat models](/docs/concepts/chat_models) . * [Semantic search](/docs/tutorials/retrievers) : Build a semantic search engine over a PDF with [document loaders](/docs/concepts/document_loaders) , [embedding models](/docs/concepts/embedding_models/) , and [vector stores](/docs/concepts/vectorstores/) . * [Classification](/docs/tutorials/classification) : Classify text into categories or labels using [chat models](/docs/concepts/chat_models) with [structured outputs](/docs/concepts/structured_outputs/) . * [Extraction](/docs/tutorials/extraction) : Extract structured data from text and other unstructured media using [chat models](/docs/concepts/chat_models) and [few-shot examples](/docs/concepts/few_shot_prompting/) . Refer to the [how-to guides](/docs/how_to) for more detail on using all LangChain components. Orchestration[โ€‹](#orchestration "Direct link to Orchestration") ---------------------------------------------------------------- Get started using [LangGraph](https://langchain-ai.github.io/langgraphjs/) to assemble LangChain components into full-featured applications. * [Chatbots](/docs/tutorials/chatbot) : Build a chatbot that incorporates memory. * [Agents](https://langchain-ai.github.io/langgraphjs/tutorials/quickstart/) : Build an agent with LangGraph.js that interacts with external tools. * [Retrieval Augmented Generation (RAG) Part 1](/docs/tutorials/rag) : Build an application that uses your own documents to inform its responses. * [Retrieval Augmented Generation (RAG) Part 2](/docs/tutorials/qa_chat_history) : Build a RAG application that incorporates a memory of its user interactions and multi-step retrieval. * [Question-Answering with SQL](/docs/tutorials/sql_qa) : Build a question-answering system that executes SQL queries to inform its responses. * [Summarization](/docs/tutorials/summarization) : Generate summaries of (potentially long) texts. * [Question-Answering with Graph Databases](/docs/tutorials/graph) : Build a question-answering system that queries a graph database to inform its responses. LangSmith[โ€‹](#langsmith "Direct link to LangSmith") ---------------------------------------------------- LangSmith allows you to closely trace, monitor and evaluate your LLM application. It seamlessly integrates with LangChain, and you can use it to inspect and debug individual steps of your chains as you build. LangSmith documentation is hosted on a separate site. You can peruse [LangSmith tutorials here](https://docs.smith.langchain.com/tutorials/) . ### Evaluation[โ€‹](#evaluation "Direct link to Evaluation") LangSmith helps you evaluate the performance of your LLM applications. The below tutorial is a great way to get started: * [Evaluate your LLM application](https://docs.smith.langchain.com/tutorials/Developers/evaluation) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Get started](#get-started) * [Orchestration](#orchestration) * [LangSmith](#langsmith) * [Evaluation](#evaluation) --- # Introduction | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) Newer LangChain version out! You are currently viewing the old v0.1 docs. View the latest docs [here](/docs/introduction/) . This is documentation for LangChain **v0.1**, which is no longer actively maintained. For the current stable version, see **[this version](https://js.langchain.com/docs/tutorials/) ** (Latest). On this page Introduction ============ **LangChain** is a framework for developing applications powered by language models. It enables applications that: * **Are context-aware**: connect a language model to sources of context (prompt instructions, few shot examples, content to ground its response in, etc.) * **Reason**: rely on a language model to reason (about how to answer based on provided context, what actions to take, etc.) This framework consists of several parts. * **LangChain Libraries**: The Python and JavaScript libraries. Contains interfaces and integrations for a myriad of components, a basic run time for combining these components into chains and agents, and off-the-shelf implementations of chains and agents. * **[LangChain Templates](https://python.langchain.com/docs/templates) **: A collection of easily deployable reference architectures for a wide variety of tasks. (_Python only_) * **[LangServe](https://python.langchain.com/docs/langserve) **: A library for deploying LangChain chains as a REST API. (_Python only_) * **[LangSmith](https://smith.langchain.com/) **: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain. ![LangChain Diagram](/v0.1/assets/images/langchain_stack_feb_2024-101939844004a99c1b676723fc0ee5e9.webp) Together, these products simplify the entire application lifecycle: * **Develop**: Write your applications in LangChain/LangChain.js. Hit the ground running using Templates for reference. * **Productionize**: Use LangSmith to inspect, test and monitor your chains, so that you can constantly improve and deploy with confidence. * **Deploy**: Turn any chain into an API with LangServe. LangChain Libraries[โ€‹](#langchain-libraries "Direct link to LangChain Libraries") ---------------------------------------------------------------------------------- The main value props of the LangChain packages are: 1. **Components**: composable tools and integrations for working with language models. Components are modular and easy-to-use, whether you are using the rest of the LangChain framework or not 2. **Off-the-shelf chains**: built-in assemblages of components for accomplishing higher-level tasks Off-the-shelf chains make it easy to get started. Components make it easy to customize existing chains and build new ones. Get started[โ€‹](#get-started "Direct link to Get started") ---------------------------------------------------------- [Here's](/v0.1/docs/get_started/installation/) how to install LangChain, set up your environment, and start building. We recommend following our [Quickstart](/v0.1/docs/get_started/quickstart/) guide to familiarize yourself with the framework by building your first LangChain application. Read up on our [Security](/v0.1/docs/security/) best practices to make sure you're developing safely with LangChain. note These docs focus on the JS/TS LangChain library. [Head here](https://python.langchain.com) for docs on the Python LangChain library. LangChain Expression Language (LCEL)[โ€‹](#langchain-expression-language-lcel "Direct link to LangChain Expression Language (LCEL)") ----------------------------------------------------------------------------------------------------------------------------------- LCEL is a declarative way to compose chains. LCEL was designed from day 1 to support putting prototypes in production, with no code changes, from the simplest โ€œprompt + LLMโ€ chain to the most complex chains. * **[Overview](/v0.1/docs/expression_language/) **: LCEL and its benefits * **[Interface](/v0.1/docs/expression_language/interface/) **: The standard interface for LCEL objects * **[How-to](/v0.1/docs/expression_language/how_to/routing/) **: Key features of LCEL * **[Cookbook](/v0.1/docs/expression_language/cookbook/) **: Example code for accomplishing common tasks Modules[โ€‹](#modules "Direct link to Modules") ---------------------------------------------- LangChain provides standard, extendable interfaces and integrations for the following modules: #### [Model I/O](/v0.1/docs/modules/model_io/) [โ€‹](#model-io "Direct link to model-io") Interface with language models #### [Retrieval](/v0.1/docs/modules/data_connection/) [โ€‹](#retrieval "Direct link to retrieval") Interface with application-specific data #### [Agents](/v0.1/docs/modules/agents/) [โ€‹](#agents "Direct link to agents") Let models choose which tools to use given high-level directives Examples, ecosystem, and resources[โ€‹](#examples-ecosystem-and-resources "Direct link to Examples, ecosystem, and resources") ----------------------------------------------------------------------------------------------------------------------------- ### [Use cases](/v0.1/docs/use_cases/) [โ€‹](#use-cases "Direct link to use-cases") Walkthroughs and techniques for common end-to-end use cases, like: * [Document question answering](/v0.1/docs/use_cases/question_answering/) * [RAG](/v0.1/docs/use_cases/question_answering/) * [Agents](/v0.1/docs/use_cases/autonomous_agents/) * and much more... ### [Integrations](/v0.1/docs/integrations/platforms/) [โ€‹](#integrations "Direct link to integrations") LangChain is part of a rich ecosystem of tools that integrate with our framework and build on top of it. Check out our growing list of [integrations](/v0.1/docs/integrations/platforms/) . ### [API reference](https://api.js.langchain.com) [โ€‹](#api-reference "Direct link to api-reference") Head to the reference section for full documentation of all classes and methods in the LangChain and LangChain Experimental packages. ### [Developer's guide](/v0.1/docs/contributing/) [โ€‹](#developers-guide "Direct link to developers-guide") Check out the developer's guide for guidelines on contributing and help getting your dev environment set up. ### [Community](/v0.1/docs/community/) [โ€‹](#community "Direct link to community") Head to the [Community navigator](/v0.1/docs/community/) to find places to ask questions, share feedback, meet other developers, and dream about the future of LLM's. * * * #### Help us out by providing feedback on this documentation page: * [LangChain Libraries](#langchain-libraries) * [Get started](#get-started) * [LangChain Expression Language (LCEL)](#langchain-expression-language-lcel) * [Modules](#modules) * [Examples, ecosystem, and resources](#examples-ecosystem-and-resources) * [Use cases](#use-cases) * [Integrations](#integrations) * [API reference](#api-reference) * [Developer's guide](#developers-guide) * [Community](#community) --- # How-to guides | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page How-to guides ============= Here you'll find answers to โ€œHow do Iโ€ฆ.?โ€ types of questions. These guides are _goal-oriented_ and _concrete_; they're meant to help you complete a specific task. For conceptual explanations see [Conceptual Guides](/docs/concepts/) . For end-to-end walkthroughs see [Tutorials](/docs/tutorials) . For comprehensive descriptions of every class and function see [API Reference](https://api.js.langchain.com/) . Installation[โ€‹](#installation "Direct link to Installation") ------------------------------------------------------------- * [How to: install LangChain packages](/docs/how_to/installation/) Key features[โ€‹](#key-features "Direct link to Key features") ------------------------------------------------------------- This highlights functionality that is core to using LangChain. * [How to: return structured data from an LLM](/docs/how_to/structured_output/) * [How to: use a chat model to call tools](/docs/how_to/tool_calling/) * [How to: stream runnables](/docs/how_to/streaming) * [How to: debug your LLM apps](/docs/how_to/debugging/) LangChain Expression Language (LCEL)[โ€‹](#langchain-expression-language-lcel "Direct link to LangChain Expression Language (LCEL)") ----------------------------------------------------------------------------------------------------------------------------------- LangChain Expression Language is a way to create arbitrary custom chains. It is built on the [`Runnable`](https://api.js.langchain.com/classes/langchain_core.runnables.Runnable.html) protocol. [**LCEL cheatsheet**](/docs/how_to/lcel_cheatsheet/) : For a quick overview of how to use the main LCEL primitives. * [How to: chain runnables](/docs/how_to/sequence) * [How to: stream runnables](/docs/how_to/streaming) * [How to: invoke runnables in parallel](/docs/how_to/parallel/) * [How to: attach runtime arguments to a runnable](/docs/how_to/binding/) * [How to: run custom functions](/docs/how_to/functions) * [How to: pass through arguments from one step to the next](/docs/how_to/passthrough) * [How to: add values to a chain's state](/docs/how_to/assign) * [How to: add message history](/docs/how_to/message_history) * [How to: route execution within a chain](/docs/how_to/routing) * [How to: add fallbacks](/docs/how_to/fallbacks) * [How to: cancel execution](/docs/how_to/cancel_execution/) Components[โ€‹](#components "Direct link to Components") ------------------------------------------------------- These are the core building blocks you can use when building applications. ### Prompt templates[โ€‹](#prompt-templates "Direct link to Prompt templates") [Prompt Templates](/docs/concepts/prompt_templates) are responsible for formatting user input into a format that can be passed to a language model. * [How to: use few shot examples](/docs/how_to/few_shot_examples) * [How to: use few shot examples in chat models](/docs/how_to/few_shot_examples_chat/) * [How to: partially format prompt templates](/docs/how_to/prompts_partial) * [How to: compose prompts together](/docs/how_to/prompts_composition) ### Example selectors[โ€‹](#example-selectors "Direct link to Example selectors") [Example Selectors](/docs/concepts/example_selectors) are responsible for selecting the correct few shot examples to pass to the prompt. * [How to: use example selectors](/docs/how_to/example_selectors) * [How to: select examples by length](/docs/how_to/example_selectors_length_based) * [How to: select examples by semantic similarity](/docs/how_to/example_selectors_similarity) * [How to: select examples from LangSmith few-shot datasets](/docs/how_to/example_selectors_langsmith) ### Chat models[โ€‹](#chat-models "Direct link to Chat models") [Chat Models](/docs/concepts/chat_models) are newer forms of language models that take messages in and output a message. * [How to: do function/tool calling](/docs/how_to/tool_calling) * [How to: get models to return structured output](/docs/how_to/structured_output) * [How to: cache model responses](/docs/how_to/chat_model_caching) * [How to: create a custom chat model class](/docs/how_to/custom_chat) * [How to: get log probabilities](/docs/how_to/logprobs) * [How to: stream a response back](/docs/how_to/chat_streaming) * [How to: track token usage](/docs/how_to/chat_token_usage_tracking) * [How to: pass tool outputs to chat models](/docs/how_to/tool_results_pass_to_model/) * [How to: stream tool calls](/docs/how_to/tool_streaming) * [How to: few shot prompt tool behavior](/docs/how_to/tools_few_shot) * [How to: force a specific tool call](/docs/how_to/tool_choice) * [How to: disable parallel tool calling](/docs/how_to/tool_calling_parallel/) * [How to: init any model in one line](/docs/how_to/chat_models_universal_init/) ### Messages[โ€‹](#messages "Direct link to Messages") [Messages](/docs/concepts/##message-types) are the input and output of chat models. They have some `content` and a `role`, which describes the source of the message. * [How to: trim messages](/docs/how_to/trim_messages/) * [How to: filter messages](/docs/how_to/filter_messages/) * [How to: merge consecutive messages of the same type](/docs/how_to/merge_message_runs/) ### LLMs[โ€‹](#llms "Direct link to LLMs") What LangChain calls [LLMs](/docs/concepts/text_llms) are older forms of language models that take a string in and output a string. * [How to: cache model responses](/docs/how_to/llm_caching) * [How to: create a custom LLM class](/docs/how_to/custom_llm) * [How to: stream a response back](/docs/how_to/streaming_llm) * [How to: track token usage](/docs/how_to/llm_token_usage_tracking) ### Output parsers[โ€‹](#output-parsers "Direct link to Output parsers") [Output Parsers](/docs/concepts/output_parsers) are responsible for taking the output of an LLM and parsing into more structured format. * [How to: use output parsers to parse an LLM response into structured format](/docs/how_to/output_parser_structured) * [How to: parse JSON output](/docs/how_to/output_parser_json) * [How to: parse XML output](/docs/how_to/output_parser_xml) * [How to: try to fix errors in output parsing](/docs/how_to/output_parser_fixing/) ### Document loaders[โ€‹](#document-loaders "Direct link to Document loaders") [Document Loaders](/docs/concepts/document_loaders) are responsible for loading documents from a variety of sources. * [How to: load CSV data](/docs/how_to/document_loader_csv) * [How to: load data from a directory](/docs/how_to/document_loader_directory) * [How to: load PDF files](/docs/how_to/document_loader_pdf) * [How to: write a custom document loader](/docs/how_to/document_loader_custom) * [How to: load HTML data](/docs/how_to/document_loader_html) * [How to: load Markdown data](/docs/how_to/document_loader_markdown) ### Text splitters[โ€‹](#text-splitters "Direct link to Text splitters") [Text Splitters](/docs/concepts/text_splitters) take a document and split into chunks that can be used for retrieval. * [How to: recursively split text](/docs/how_to/recursive_text_splitter) * [How to: split by character](/docs/how_to/character_text_splitter) * [How to: split code](/docs/how_to/code_splitter) * [How to: split by tokens](/docs/how_to/split_by_token) ### Embedding models[โ€‹](#embedding-models "Direct link to Embedding models") [Embedding Models](/docs/concepts/embedding_models) take a piece of text and create a numerical representation of it. * [How to: embed text data](/docs/how_to/embed_text) * [How to: cache embedding results](/docs/how_to/caching_embeddings) ### Vector stores[โ€‹](#vector-stores "Direct link to Vector stores") [Vector stores](/docs/concepts/#vectorstores) are databases that can efficiently store and retrieve embeddings. * [How to: create and query vector stores](/docs/how_to/vectorstores) ### Retrievers[โ€‹](#retrievers "Direct link to Retrievers") [Retrievers](/docs/concepts/retrievers) are responsible for taking a query and returning relevant documents. * [How to: use a vector store to retrieve data](/docs/how_to/vectorstore_retriever) * [How to: generate multiple queries to retrieve data for](/docs/how_to/multiple_queries) * [How to: use contextual compression to compress the data retrieved](/docs/how_to/contextual_compression) * [How to: write a custom retriever class](/docs/how_to/custom_retriever) * [How to: combine the results from multiple retrievers](/docs/how_to/ensemble_retriever) * [How to: generate multiple embeddings per document](/docs/how_to/multi_vector) * [How to: retrieve the whole document for a chunk](/docs/how_to/parent_document_retriever) * [How to: generate metadata filters](/docs/how_to/self_query) * [How to: create a time-weighted retriever](/docs/how_to/time_weighted_vectorstore) * [How to: reduce retrieval latency](/docs/how_to/reduce_retrieval_latency) ### Indexing[โ€‹](#indexing "Direct link to Indexing") Indexing is the process of keeping your vectorstore in-sync with the underlying data source. * [How to: reindex data to keep your vectorstore in-sync with the underlying data source](/docs/how_to/indexing) ### Tools[โ€‹](#tools "Direct link to Tools") LangChain [Tools](/docs/concepts/tools) contain a description of the tool (to pass to the language model) as well as the implementation of the function to call. * [How to: create tools](/docs/how_to/custom_tools) * [How to: use built-in tools and toolkits](/docs/how_to/tools_builtin) * [How to: use chat models to call tools](/docs/how_to/tool_calling/) * [How to: pass tool outputs to chat models](/docs/how_to/tool_results_pass_to_model/) * [How to: few shot prompt tool behavior](/docs/how_to/tools_few_shot) * [How to: pass run time values to tools](/docs/how_to/tool_runtime) * [How to: handle tool errors](/docs/how_to/tools_error) * [How to: force a specific tool call](/docs/how_to/tool_choice/) * [How to: disable parallel tool calling](/docs/how_to/tool_calling_parallel/) * [How to: access the `RunnableConfig` object within a custom tool](/docs/how_to/tool_configure) * [How to: stream events from child runs within a custom tool](/docs/how_to/tool_stream_events) * [How to: return artifacts from a tool](/docs/how_to/tool_artifacts) * [How to: convert Runnables to tools](/docs/how_to/convert_runnable_to_tool) * [How to: add ad-hoc tool calling capability to models](/docs/how_to/tools_prompting) ### Agents[โ€‹](#agents "Direct link to Agents") note For in depth how-to guides for agents, please check out [LangGraph](https://langchain-ai.github.io/langgraphjs/) documentation. * [How to: use legacy LangChain Agents (AgentExecutor)](/docs/how_to/agent_executor) * [How to: migrate from legacy LangChain agents to LangGraph](/docs/how_to/migrate_agent) ### Callbacks[โ€‹](#callbacks "Direct link to Callbacks") [Callbacks](/docs/concepts/callbacks) allow you to hook into the various stages of your LLM application's execution. * [How to: pass in callbacks at runtime](/docs/how_to/callbacks_runtime) * [How to: attach callbacks to a module](/docs/how_to/callbacks_attach) * [How to: pass callbacks into a module constructor](/docs/how_to/callbacks_constructor) * [How to: create custom callback handlers](/docs/how_to/custom_callbacks) * [How to: await callbacks in serverless environments](/docs/how_to/callbacks_serverless) * [How to: dispatch custom callback events](/docs/how_to/callbacks_custom_events) ### Custom[โ€‹](#custom "Direct link to Custom") All of LangChain components can easily be extended to support your own versions. * [How to: create a custom chat model class](/docs/how_to/custom_chat) * [How to: create a custom LLM class](/docs/how_to/custom_llm) * [How to: write a custom retriever class](/docs/how_to/custom_retriever) * [How to: write a custom document loader](/docs/how_to/document_loader_custom) * [How to: create custom callback handlers](/docs/how_to/custom_callbacks) * [How to: define a custom tool](/docs/how_to/custom_tools) * [How to: dispatch custom callback events](/docs/how_to/callbacks_custom_events) ### Generative UI[โ€‹](#generative-ui "Direct link to Generative UI") * [How to: build an LLM generated UI](/docs/how_to/generative_ui) * [How to: stream agentic data to the client](/docs/how_to/stream_agent_client) * [How to: stream structured output to the client](/docs/how_to/stream_tool_client) ### Multimodal[โ€‹](#multimodal "Direct link to Multimodal") * [How to: pass multimodal data directly to models](/docs/how_to/multimodal_inputs/) * [How to: use multimodal prompts](/docs/how_to/multimodal_prompts/) * [How to: call tools with multimodal data](/docs/how_to/tool_calls_multimodal/) Use cases[โ€‹](#use-cases "Direct link to Use cases") ---------------------------------------------------- These guides cover use-case specific details. ### Q&A with RAG[โ€‹](#qa-with-rag "Direct link to Q&A with RAG") Retrieval Augmented Generation (RAG) is a way to connect LLMs to external sources of data. For a high-level tutorial on RAG, check out [this guide](/docs/tutorials/rag/) . * [How to: add chat history](/docs/how_to/qa_chat_history_how_to/) * [How to: stream](/docs/how_to/qa_streaming/) * [How to: return sources](/docs/how_to/qa_sources/) * [How to: return citations](/docs/how_to/qa_citations/) * [How to: do per-user retrieval](/docs/how_to/qa_per_user/) ### Extraction[โ€‹](#extraction "Direct link to Extraction") Extraction is when you use LLMs to extract structured information from unstructured text. For a high level tutorial on extraction, check out [this guide](/docs/tutorials/extraction/) . * [How to: use reference examples](/docs/how_to/extraction_examples/) * [How to: handle long text](/docs/how_to/extraction_long_text/) * [How to: do extraction without using function calling](/docs/how_to/extraction_parse) ### Chatbots[โ€‹](#chatbots "Direct link to Chatbots") Chatbots involve using an LLM to have a conversation. For a high-level tutorial on building chatbots, check out [this guide](/docs/tutorials/chatbot/) . * [How to: manage memory](/docs/how_to/chatbots_memory) * [How to: do retrieval](/docs/how_to/chatbots_retrieval) * [How to: use tools](/docs/how_to/chatbots_tools) ### Query analysis[โ€‹](#query-analysis "Direct link to Query analysis") Query Analysis is the task of using an LLM to generate a query to send to a retriever. For a high-level tutorial on query analysis, check out [this guide](/docs/tutorials/rag#query-analysis) . * [How to: add examples to the prompt](/docs/how_to/query_few_shot) * [How to: handle cases where no queries are generated](/docs/how_to/query_no_queries) * [How to: handle multiple queries](/docs/how_to/query_multiple_queries) * [How to: handle multiple retrievers](/docs/how_to/query_multiple_retrievers) * [How to: construct filters](/docs/how_to/query_constructing_filters) * [How to: deal with high cardinality categorical variables](/docs/how_to/query_high_cardinality) ### Q&A over SQL + CSV[โ€‹](#qa-over-sql--csv "Direct link to Q&A over SQL + CSV") You can use LLMs to do question answering over tabular data. For a high-level tutorial, check out [this guide](/docs/tutorials/sql_qa/) . * [How to: use prompting to improve results](/docs/how_to/sql_prompting) * [How to: do query validation](/docs/how_to/sql_query_checking) * [How to: deal with large databases](/docs/how_to/sql_large_db) ### Q&A over graph databases[โ€‹](#qa-over-graph-databases "Direct link to Q&A over graph databases") You can use an LLM to do question answering over graph databases. For a high-level tutorial, check out [this guide](/docs/tutorials/graph/) . * [How to: map values to a database](/docs/how_to/graph_mapping) * [How to: add a semantic layer over the database](/docs/how_to/graph_semantic) * [How to: improve results with prompting](/docs/how_to/graph_prompting) * [How to: construct knowledge graphs](/docs/how_to/graph_constructing) [LangGraph.js](https://langchain-ai.github.io/langgraphjs) [โ€‹](#langgraphjs "Direct link to langgraphjs") ---------------------------------------------------------------------------------------------------------- LangGraph.js is an extension of LangChain aimed at building robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. LangGraph.js documentation is currently hosted on a separate site. You can peruse [LangGraph.js how-to guides here](https://langchain-ai.github.io/langgraphjs/how-tos/) . [LangSmith](https://docs.smith.langchain.com/) [โ€‹](#langsmith "Direct link to langsmith") ------------------------------------------------------------------------------------------ LangSmith allows you to closely trace, monitor and evaluate your LLM application. It seamlessly integrates with LangChain and LangGraph.js, and you can use it to inspect and debug individual steps of your chains as you build. LangSmith documentation is hosted on a separate site. You can peruse [LangSmith how-to guides here](https://docs.smith.langchain.com/how_to_guides/) , but we'll highlight a few sections that are particularly relevant to LangChain below: ### Evaluation[โ€‹](#evaluation "Direct link to Evaluation") Evaluating performance is a vital part of building LLM-powered applications. LangSmith helps with every step of the process from creating a dataset to defining metrics to running evaluators. To learn more, check out the [LangSmith evaluation how-to guides](https://docs.smith.langchain.com/how_to_guides#evaluation) . ### Tracing[โ€‹](#tracing "Direct link to Tracing") Tracing gives you observability inside your chains and agents, and is vital in diagnosing issues. * [How to: trace with LangChain](https://docs.smith.langchain.com/how_to_guides/tracing/trace_with_langchain) * [How to: add metadata and tags to traces](https://docs.smith.langchain.com/how_to_guides/tracing/trace_with_langchain#add-metadata-and-tags-to-traces) You can see general tracing-related how-tos [in this section of the LangSmith docs](https://docs.smith.langchain.com/how_to_guides/tracing) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Installation](#installation) * [Key features](#key-features) * [LangChain Expression Language (LCEL)](#langchain-expression-language-lcel) * [Components](#components) * [Prompt templates](#prompt-templates) * [Example selectors](#example-selectors) * [Chat models](#chat-models) * [Messages](#messages) * [LLMs](#llms) * [Output parsers](#output-parsers) * [Document loaders](#document-loaders) * [Text splitters](#text-splitters) * [Embedding models](#embedding-models) * [Vector stores](#vector-stores) * [Retrievers](#retrievers) * [Indexing](#indexing) * [Tools](#tools) * [Agents](#agents) * [Callbacks](#callbacks) * [Custom](#custom) * [Generative UI](#generative-ui) * [Multimodal](#multimodal) * [Use cases](#use-cases) * [Q&A with RAG](#qa-with-rag) * [Extraction](#extraction) * [Chatbots](#chatbots) * [Query analysis](#query-analysis) * [Q&A over SQL + CSV](#qa-over-sql--csv) * [Q&A over graph databases](#qa-over-graph-databases) * [LangGraph.js](#langgraphjs) * [LangSmith](#langsmith) * [Evaluation](#evaluation) * [Tracing](#tracing) --- # Build a simple LLM application with chat models and prompt templates | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page In this quickstart weโ€™ll show you how to build a simple LLM application with LangChain. This application will translate text from English into another language. This is a relatively simple LLM application - itโ€™s just a single LLM call plus some prompting. Still, this is a great way to get started with LangChain - a lot of features can be built with just some prompting and an LLM call! After reading this tutorial, youโ€™ll have a high level overview of: * Using [language models](/docs/concepts/chat_models) * Using [prompt templates](/docs/concepts/prompt_templates) * Debugging and tracing your application using [LangSmith](https://docs.smith.langchain.com/) Letโ€™s dive in! Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Installation[โ€‹](#installation "Direct link to Installation") To install LangChain run: * npm * yarn * pnpm npm i langchain @langchain/core yarn add langchain @langchain/core pnpm add langchain @langchain/core For more details, see our [Installation guide](/docs/how_to/installation/) . ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com) . After you sign up at the link above, make sure to set your environment variables to start logging traces: export LANGSMITH_TRACING="true"export LANGSMITH_API_KEY="..."# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true Using Language Models[โ€‹](#using-language-models "Direct link to Using Language Models") ---------------------------------------------------------------------------------------- First up, letโ€™s learn how to use a language model by itself. LangChain supports many different language models that you can use interchangeably. For details on getting started with a specific model, refer to [supported integrations](/docs/integrations/chat/) . ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({ model: "gpt-4" }); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); Letโ€™s first use the model directly. [ChatModels](/docs/concepts/chat_models) are instances of LangChain [Runnables](/docs/concepts/runnables/) , which means they expose a standard interface for interacting with them. To simply call the model, we can pass in a list of [messages](/docs/concepts/messages/) to the `.invoke` method. import { HumanMessage, SystemMessage } from "@langchain/core/messages";const messages = [ new SystemMessage("Translate the following from English into Italian"), new HumanMessage("hi!"),];await model.invoke(messages); AIMessage { "id": "chatcmpl-AekSfJkg3QIOsk42BH6Qom4Gt159j", "content": "Ciao!", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 20, "completionTokens": 3, "totalTokens": 23 }, "finish_reason": "stop", "usage": { "prompt_tokens": 20, "completion_tokens": 3, "total_tokens": 23, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 3, "input_tokens": 20, "total_tokens": 23, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} tip If we've enabled LangSmith, we can see that this run is logged to LangSmith, and can see the [LangSmith trace](https://smith.langchain.com/public/45f1a650-38fb-41e1-9b61-becc0684f2ce/r) . The LangSmith trace reports [token](/docs/concepts/tokens/) usage information, latency, [standard model parameters](/docs/concepts/chat_models/#standard-parameters) (such as temperature), and other information. Note that ChatModels receive [message](/docs/concepts/messages/) objects as input and generate message objects as output. In addition to text content, message objects convey conversational [roles](/docs/concepts/messages/#role) and hold important data, such as [tool calls](/docs/concepts/tool_calling/) and token usage counts. LangChain also supports chat model inputs via strings or [OpenAI format](/docs/concepts/messages/#openai-format) . The following are equivalent: await model.invoke("Hello");await model.invoke([{ role: "user", content: "Hello" }]);await model.invoke([new HumanMessage("hi!")]); ### Streaming[โ€‹](#streaming "Direct link to Streaming") Because chat models are [Runnables](/docs/concepts/runnables/) , they expose a standard interface that includes async and streaming modes of invocation. This allows us to stream individual tokens from a chat model: const stream = await model.stream(messages);const chunks = [];for await (const chunk of stream) { chunks.push(chunk); console.log(`${chunk.content}|`);} |C|iao|!||| Prompt Templates[โ€‹](#prompt-templates "Direct link to Prompt Templates") ------------------------------------------------------------------------- Right now we are passing a list of messages directly into the language model. Where does this list of messages come from? Usually, it is constructed from a combination of user input and application logic. This application logic usually takes the raw user input and transforms it into a list of messages ready to pass to the language model. Common transformations include adding a system message or formatting a template with the user input. [Prompt templates](/docs/concepts/prompt_templates/) are a concept in LangChain designed to assist with this transformation. They take in raw user input and return data (a prompt) that is ready to pass into a language model. Letโ€™s create a prompt template here. It will take in two user variables: * `language`: The language to translate text into * `text`: The text to translate import { ChatPromptTemplate } from "@langchain/core/prompts"; First, letโ€™s create a string that we will format to be the system message: const systemTemplate = "Translate the following from English into {language}"; Next, we can create the PromptTemplate. This will be a combination of the `systemTemplate` as well as a simpler template for where to put the text const promptTemplate = ChatPromptTemplate.fromMessages([ ["system", systemTemplate], ["user", "{text}"],]); Note that `ChatPromptTemplate` supports multiple [message roles](/docs/concepts/messages/#role) in a single template. We format the `language` parameter into the system message, and the user `text` into a user message. The input to this prompt template is a dictionary. We can play around with this prompt template by itself to see what it does by itself const promptValue = await promptTemplate.invoke({ language: "italian", text: "hi!",});promptValue; ChatPromptValue { lc_serializable: true, lc_kwargs: { messages: [ SystemMessage { "content": "Translate the following from English into italian", "additional_kwargs": {}, "response_metadata": {} }, HumanMessage { "content": "hi!", "additional_kwargs": {}, "response_metadata": {} } ] }, lc_namespace: [ 'langchain_core', 'prompt_values' ], messages: [ SystemMessage { "content": "Translate the following from English into italian", "additional_kwargs": {}, "response_metadata": {} }, HumanMessage { "content": "hi!", "additional_kwargs": {}, "response_metadata": {} } ]} We can see that it returns a `ChatPromptValue` that consists of two messages. If we want to access the messages directly we do: promptValue.toChatMessages(); [ SystemMessage { "content": "Translate the following from English into italian", "additional_kwargs": {}, "response_metadata": {} }, HumanMessage { "content": "hi!", "additional_kwargs": {}, "response_metadata": {} }] Finally, we can invoke the chat model on the formatted prompt: const response = await model.invoke(promptValue);console.log(`${response.content}`); Ciao! If we take a look at the LangSmith trace, we can see all three components show up in the [LangSmith trace](https://smith.langchain.com/public/6529d912-8564-4686-8df8-999c427621a7/r) . Conclusion[โ€‹](#conclusion "Direct link to Conclusion") ------------------------------------------------------- Thatโ€™s it! In this tutorial youโ€™ve learned how to create your first simple LLM application. Youโ€™ve learned how to work with language models, how to create a prompt template, and how to get great observability into applications you create with LangSmith. This just scratches the surface of what you will want to learn to become a proficient AI Engineer. Luckily - weโ€™ve got a lot of other resources! For further reading on the core concepts of LangChain, weโ€™ve got detailed [Conceptual Guides](/docs/concepts) . If you have more specific questions on these concepts, check out the following sections of the how-to guides: * [Chat models](/docs/how_to/#chat-models) * [Prompt templates](/docs/how_to/#prompt-templates) And the LangSmith docs: * [LangSmith](https://docs.smith.langchain.com) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Setup](#setup) * [Installation](#installation) * [LangSmith](#langsmith) * [Using Language Models](#using-language-models) * [Streaming](#streaming) * [Prompt Templates](#prompt-templates) * [Conclusion](#conclusion) --- # Build a Question Answering application over a Graph Database | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page In this guide weโ€™ll go over the basic ways to create a Q&A chain over a graph database. These systems will allow us to ask a question about the data in a graph database and get back a natural language answer. โš ๏ธ Security note โš ๏ธ[โ€‹](#security-note "Direct link to โš ๏ธ Security note โš ๏ธ") ---------------------------------------------------------------------------- Building Q&A systems of graph databases requires executing model-generated graph queries. There are inherent risks in doing this. Make sure that your database connection permissions are always scoped as narrowly as possible for your chain/agentโ€™s needs. This will mitigate though not eliminate the risks of building a model-driven system. For more on general security best practices, [see here](/docs/security) . Architecture[โ€‹](#architecture "Direct link to Architecture") ------------------------------------------------------------- At a high-level, the steps of most graph chains are: 1. **Convert question to a graph database query**: Model converts user input to a graph database query (e.g.ย Cypher). 2. **Execute graph database query**: Execute the graph database query. 3. **Answer the question**: Model responds to user input using the query results. ![sql_usecase.png](/assets/images/graph_usecase-34d891523e6284bb6230b38c5f8392e5.png) Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- #### Install dependencies[โ€‹](#install-dependencies "Direct link to Install dependencies") tip See [this section for general instructions on installing integration packages](/docs/how_to/installation#installing-integration-packages) . * npm * yarn * pnpm npm i langchain @langchain/community @langchain/openai @langchain/core neo4j-driver yarn add langchain @langchain/community @langchain/openai @langchain/core neo4j-driver pnpm add langchain @langchain/community @langchain/openai @langchain/core neo4j-driver #### Set environment variables[โ€‹](#set-environment-variables "Direct link to Set environment variables") Weโ€™ll use OpenAI in this example: OPENAI_API_KEY=your-api-key# Optional, use LangSmith for best-in-class observabilityLANGSMITH_API_KEY=your-api-keyLANGSMITH_TRACING=true# Reduce tracing latency if you are not in a serverless environment# LANGCHAIN_CALLBACKS_BACKGROUND=true Next, we need to define Neo4j credentials. Follow [these installation steps](https://neo4j.com/docs/operations-manual/current/installation/) to set up a Neo4j database. NEO4J_URI="bolt://localhost:7687"NEO4J_USERNAME="neo4j"NEO4J_PASSWORD="password" The below example will create a connection with a Neo4j database and will populate it with example data about movies and their actors. import "neo4j-driver";import { Neo4jGraph } from "@langchain/community/graphs/neo4j_graph";const url = process.env.NEO4J_URI;const username = process.env.NEO4J_USER;const password = process.env.NEO4J_PASSWORD;const graph = await Neo4jGraph.initialize({ url, username, password });// Import movie informationconst moviesQuery = `LOAD CSV WITH HEADERS FROM 'https://raw.githubusercontent.com/tomasonjo/blog-datasets/main/movies/movies_small.csv'AS rowMERGE (m:Movie {id:row.movieId})SET m.released = date(row.released), m.title = row.title, m.imdbRating = toFloat(row.imdbRating)FOREACH (director in split(row.director, '|') | MERGE (p:Person {name:trim(director)}) MERGE (p)-[:DIRECTED]->(m))FOREACH (actor in split(row.actors, '|') | MERGE (p:Person {name:trim(actor)}) MERGE (p)-[:ACTED_IN]->(m))FOREACH (genre in split(row.genres, '|') | MERGE (g:Genre {name:trim(genre)}) MERGE (m)-[:IN_GENRE]->(g))`;await graph.query(moviesQuery); Schema refreshed successfully. [] Graph schema[โ€‹](#graph-schema "Direct link to Graph schema") ------------------------------------------------------------- In order for an LLM to be able to generate a Cypher statement, it needs information about the graph schema. When you instantiate a graph object, it retrieves the information about the graph schema. If you later make any changes to the graph, you can run the `refreshSchema` method to refresh the schema information. await graph.refreshSchema();console.log(graph.getSchema()); Node properties are the following:Movie {imdbRating: FLOAT, id: STRING, released: DATE, title: STRING}, Person {name: STRING}, Genre {name: STRING}Relationship properties are the following:The relationships are the following:(:Movie)-[:IN_GENRE]->(:Genre), (:Person)-[:DIRECTED]->(:Movie), (:Person)-[:ACTED_IN]->(:Movie) Great! Weโ€™ve got a graph database that we can query. Now letโ€™s try hooking it up to an LLM. Chain[โ€‹](#chain "Direct link to Chain") ---------------------------------------- Letโ€™s use a simple chain that takes a question, turns it into a Cypher query, executes the query, and uses the result to answer the original question. ![graph_chain.webp](/assets/images/graph_chain-6379941793e0fa985e51e4bda0329403.webp) LangChain comes with a built-in chain for this workflow that is designed to work with Neo4j: `GraphCypherQAChain`. danger The `GraphCypherQAChain` used in this guide will execute Cypher statements against the provided database. For production, make sure that the database connection uses credentials that are narrowly-scoped to only include necessary permissions. Failure to do so may result in data corruption or loss, since the calling code may attempt commands that would result in deletion, mutation of data if appropriately prompted or reading sensitive data if such data is present in the database. import { GraphCypherQAChain } from "langchain/chains/graph_qa/cypher";import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-3.5-turbo", temperature: 0 });const chain = GraphCypherQAChain.fromLLM({ llm, graph,});const response = await chain.invoke({ query: "What was the cast of the Casino?",});console.log(response); { result: "James Woods, Joe Pesci, Robert De Niro, Sharon Stone" } ### Next steps[โ€‹](#next-steps "Direct link to Next steps") For more complex query-generation, we may want to create few-shot prompts or add query-checking steps. For advanced techniques like this and more check out: * [Prompting strategies](/docs/how_to/graph_prompting) : Advanced prompt engineering techniques. * [Mapping values](/docs/how_to/graph_mapping/) : Techniques for mapping values from questions to database. * [Semantic layer](/docs/how_to/graph_semantic) : Techniques for working implementing semantic layers. * [Constructing graphs](/docs/how_to/graph_constructing) : Techniques for constructing knowledge graphs. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [โš ๏ธ Security note โš ๏ธ](#security-note) * [Architecture](#architecture) * [Setup](#setup) * [Graph schema](#graph-schema) * [Chain](#chain) * [Next steps](#next-steps) --- # Summarize Text | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Summarize Text ============== info This tutorial demonstrates text summarization using built-in chains and [LangGraph](https://langchain-ai.github.io/langgraphjs/) . See [here](https://js.langchain.com/v0.2/docs/tutorials/summarization/) for a previous version of this page, which showcased the legacy chain [RefineDocumentsChain](https://api.js.langchain.com/classes/langchain.chains.RefineDocumentsChain.html) . Suppose you have a set of documents (PDFs, Notion pages, customer questions, etc.) and you want to summarize the content. LLMs are a great tool for this given their proficiency in understanding and synthesizing text. In the context of [retrieval-augmented generation](/docs/tutorials/rag) , summarizing text can help distill the information in a large number of retrieved documents to provide context for a LLM. In this walkthrough weโ€™ll go over how to summarize content from multiple documents using LLMs. Concepts[โ€‹](#concepts "Direct link to Concepts") ------------------------------------------------- Concepts we will cover are: * Using [language models](/docs/concepts/chat_models) . * Using [document loaders](/docs/concepts/document_loaders) , specifically the [CheerioWebBaseLoader](https://api.js.langchain.com/classes/langchain.document_loaders_web_cheerio.CheerioWebBaseLoader.html) to load content from an HTML webpage. * Two ways to summarize or otherwise combine documents. 1. [Stuff](/docs/tutorials/summarization#stuff) , which simply concatenates documents into a prompt; 2. [Map-reduce](/docs/tutorials/summarization#map-reduce) , for larger sets of documents. This splits documents into batches, summarizes those, and then summarizes the summaries. Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Jupyter Notebook[โ€‹](#jupyter-notebook "Direct link to Jupyter Notebook") This and other tutorials are perhaps most conveniently run in a [Jupyter notebooks](https://jupyter.org/) . Going through guides in an interactive environment is a great way to better understand them. See [here](https://jupyter.org/install) for instructions on how to install. ### Installation[โ€‹](#installation "Direct link to Installation") To install LangChain run: `bash npm2yarn npm i langchain @langchain/core` For more details, see our [Installation guide](/docs/how_to/installation) . ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com) . After you sign up at the link above, make sure to set your environment variables to start logging traces: export LANGSMITH_TRACING="true"export LANGSMITH_API_KEY="..."# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true Overview[โ€‹](#overview "Direct link to Overview") ------------------------------------------------- A central question for building a summarizer is how to pass your documents into the LLMโ€™s context window. Two common approaches for this are: 1. `Stuff`: Simply โ€œstuffโ€ all your documents into a single prompt. This is the simplest approach. 2. `Map-reduce`: Summarize each document on its own in a โ€œmapโ€ step and then โ€œreduceโ€ the summaries into a final summary. Note that map-reduce is especially effective when understanding of a sub-document does not rely on preceding context. For example, when summarizing a corpus of many, shorter documents. In other cases, such as summarizing a novel or body of text with an inherent sequence, [iterative refinement](https://js.langchain.com/v0.2/docs/tutorials/summarization/) may be more effective. First we load in our documents. We will use [WebBaseLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.web_base.WebBaseLoader.html) to load a blog post: import "cheerio";import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";const pTagSelector = "p";const cheerioLoader = new CheerioWebBaseLoader( "https://lilianweng.github.io/posts/2023-06-23-agent/", { selector: pTagSelector, });const docs = await cheerioLoader.load(); Letโ€™s next select a [chat model](/docs/integrations/chat/) : ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); Stuff: summarize in a single LLM call[โ€‹](#stuff "Direct link to Stuff: summarize in a single LLM call") -------------------------------------------------------------------------------------------------------- We can use [createStuffDocumentsChain](https://api.js.langchain.com/functions/langchain.chains_combine_documents.createStuffDocumentsChain.html) , especially if using larger context window models such as: * 128k token OpenAI `gpt-4o` * 200k token Anthropic `claude-3-5-sonnet-20240620` The chain will take a list of documents, insert them all into a prompt, and pass that prompt to an LLM: import { createStuffDocumentsChain } from "langchain/chains/combine_documents";import { StringOutputParser } from "@langchain/core/output_parsers";import { PromptTemplate } from "@langchain/core/prompts";// Define promptconst prompt = PromptTemplate.fromTemplate( "Summarize the main themes in these retrieved docs: {context}");// Instantiateconst chain = await createStuffDocumentsChain({ llm: llm, outputParser: new StringOutputParser(), prompt,});// Invokeconst result = await chain.invoke({ context: docs });console.log(result); The retrieved documents discuss the development and capabilities of autonomous agents powered by large language models (LLMs). Here are the main themes:1. **LLM as a Core Controller**: LLMs are positioned as the central intelligence in autonomous agent systems, capable of performing complex tasks beyond simple text generation. They can be framed as general problem solvers, with various implementations like AutoGPT, GPT-Engineer, and BabyAGI serving as proof-of-concept demonstrations.2. **Task Decomposition and Planning**: Effective task management is crucial for LLMs. Techniques like Chain of Thought (CoT) and Tree of Thoughts (ToT) are highlighted for breaking down complex tasks into manageable steps. CoT encourages step-by-step reasoning, while ToT explores multiple reasoning paths, enhancing the agent's problem-solving capabilities.3. **Integration of External Tools**: The use of external tools significantly enhances LLM capabilities. Frameworks like MRKL and Toolformer allow LLMs to interact with various APIs and tools, improving their performance in specific tasks. This modular approach enables LLMs to route inquiries to specialized modules, combining neural and symbolic reasoning.4. **Self-Reflection and Learning**: Self-reflection mechanisms are essential for agents to learn from past actions and improve over time. Approaches like ReAct and Reflexion integrate reasoning with action, allowing agents to evaluate their performance and adjust strategies based on feedback.5. **Memory and Context Management**: The documents discuss different types of memory (sensory, short-term, long-term) and their relevance to LLMs. The challenge of finite context length in LLMs is emphasized, as it limits the ability to retain and utilize historical information effectively. Techniques like external memory storage and vector databases are suggested to mitigate these limitations.6. **Challenges and Limitations**: Several challenges are identified, including the reliability of natural language interfaces, difficulties in long-term planning, and the need for robust task decomposition. The documents note that LLMs may struggle with unexpected errors and formatting issues, which can hinder their performance in real-world applications.7. **Emerging Applications**: The potential applications of LLM-powered agents are explored, including scientific discovery, autonomous design, and interactive simulations (e.g., generative agents mimicking human behavior). These applications demonstrate the versatility and innovative possibilities of LLMs in various domains.Overall, the documents present a comprehensive overview of the current state of LLM-powered autonomous agents, highlighting their capabilities, methodologies, and the challenges they face in practical implementations. ### Streaming[โ€‹](#streaming "Direct link to Streaming") Note that we can also stream the result token-by-token: const stream = await chain.stream({ context: docs });for await (const token of stream) { process.stdout.write(token + "|");} |The| retrieved| documents| discuss| the| development| and| capabilities| of| autonomous| agents| powered| by| large| language| models| (|LL|Ms|).| Here| are| the| main| themes|:|1|.| **|LL|M| as| a| Core| Controller|**|:| L|LM|s| are| positioned| as| the| central| intelligence| in| autonomous| agent| systems|,| capable| of| performing| complex| tasks| beyond| simple| text| generation|.| They| can| be| framed| as| general| problem| sol|vers|,| with| various| implementations| like| Auto|GPT|,| GPT|-|Engineer|,| and| Baby|AG|I| serving| as| proof|-of|-con|cept| demonstrations|.|2|.| **|Task| De|composition| and| Planning|**|:| Effective| task| management| is| crucial| for| L|LM|s| to| handle| complicated| tasks|.| Techniques| like| Chain| of| Thought| (|Co|T|)| and| Tree| of| Thoughts| (|To|T|)| are| highlighted| for| breaking| down| tasks| into| manageable| steps| and| exploring| multiple| reasoning| paths|.| Additionally|,| L|LM|+|P| integrates| classical| planning| methods| to| enhance| long|-term| planning| capabilities|.|3|.| **|Self|-|Reflection| and| Learning|**|:| Self|-ref|lection| mechanisms| are| essential| for| agents| to| learn| from| past| actions| and| improve| their| decision|-making| processes|.| Framework|s| like| Re|Act| and| Reflex|ion| incorporate| dynamic| memory| and| self|-ref|lection| to| refine| reasoning| skills| and| enhance| performance| through| iterative| learning|.|4|.| **|Tool| Util|ization|**|:| The| integration| of| external| tools| significantly| extends| the| capabilities| of| L|LM|s|.| Appro|aches| like| MR|KL| and| Tool|former| demonstrate| how| L|LM|s| can| be| augmented| with| various| APIs| to| perform| specialized| tasks|,| enhancing| their| functionality| in| real|-world| applications|.|5|.| **|Memory| and| Context| Management|**|:| The| documents| discuss| different| types| of| memory| (|sens|ory|,| short|-term|,| long|-term|)| and| their| relevance| to| L|LM|s|.| The| challenge| of| finite| context| length| is| emphasized|,| as| it| limits| the| model|'s| ability| to| retain| and| utilize| historical| information| effectively|.| Techniques| like| vector| stores| and| approximate| nearest| neighbors| (|ANN|)| are| suggested| to| improve| retrieval| speed| and| memory| management|.|6|.| **|Challenges| and| Limit|ations|**|:| Several| limitations| of| current| L|LM|-powered| agents| are| identified|,| including| issues| with| the| reliability| of| natural| language| interfaces|,| difficulties| in| long|-term| planning|,| and| the| need| for| improved| efficiency| in| task| execution|.| The| documents| also| highlight| the| importance| of| human| feedback| in| refining| model| outputs| and| addressing| potential| biases|.|7|.| **|Emer|ging| Applications|**|:| The| potential| applications| of| L|LM|-powered| agents| are| explored|,| including| scientific| discovery|,| autonomous| design|,| and| interactive| simulations| (|e|.g|.,| gener|ative| agents|).| These| applications| showcase| the| versatility| of| L|LM|s| in| various| domains|,| from| drug| discovery| to| social| behavior| simulations|.|Overall|,| the| documents| present| a| comprehensive| overview| of| the| current| state| of| L|LM|-powered| autonomous| agents|,| their| capabilities|,| methodologies| for| improvement|,| and| the| challenges| they| face| in| practical| applications|.||| ### Go deeper[โ€‹](#go-deeper "Direct link to Go deeper") * You can easily customize the prompt. * You can easily try different LLMs, (e.g., [Claude](/docs/integrations/chat/anthropic) ) via the `llm` parameter. Map-Reduce: summarize long texts via parallelization[โ€‹](#map-reduce "Direct link to Map-Reduce: summarize long texts via parallelization") ------------------------------------------------------------------------------------------------------------------------------------------- Letโ€™s unpack the map reduce approach. For this, weโ€™ll first map each document to an individual summary using an LLM. Then weโ€™ll reduce or consolidate those summaries into a single global summary. Note that the map step is typically parallelized over the input documents. [LangGraph](https://langchain-ai.github.io/langgraphjs/) , built on top of [@langchain/core](/docs/concepts/architecture#langchaincore) , supports [map-reduce](https://langchain-ai.github.io/langgraphjs/how-tos/map-reduce/) workflows and is well-suited to this problem: * LangGraph allows for individual steps (such as successive summarizations) to be streamed, allowing for greater control of execution; * LangGraphโ€™s [checkpointing](https://langchain-ai.github.io/langgraphjs/how-tos/persistence/) supports error recovery, extending with human-in-the-loop workflows, and easier incorporation into conversational applications. * The LangGraph implementation is straightforward to modify and extend, as we will see below. ### Map[โ€‹](#map "Direct link to Map") Letโ€™s first define the prompt associated with the map step. We can use the same summarization prompt as in the `stuff` approach, above: import { ChatPromptTemplate } from "@langchain/core/prompts";const mapPrompt = ChatPromptTemplate.fromMessages([ ["user", "Write a concise summary of the following: \n\n{context}"],]); We can also use the Prompt Hub to store and fetch prompts. This will work with your [LangSmith API key](https://docs.smith.langchain.com/) . For example, see the map prompt [here](https://smith.langchain.com/hub/rlm/map-prompt) . import { pull } from "langchain/hub";import { ChatPromptTemplate } from "@langchain/core/prompts";const mapPrompt = (await pull) < ChatPromptTemplate > "rlm/map-prompt"; ### Reduce[โ€‹](#reduce "Direct link to Reduce") We also define a prompt that takes the document mapping results and reduces them into a single output. // Also available via the hub at `rlm/reduce-prompt`let reduceTemplate = `The following is a set of summaries:{docs}Take these and distill it into a final, consolidated summaryof the main themes.`;const reducePrompt = ChatPromptTemplate.fromMessages([ ["user", reduceTemplate],]); ### Orchestration via LangGraph[โ€‹](#orchestration-via-langgraph "Direct link to Orchestration via LangGraph") Below we implement a simple application that maps the summarization step on a list of documents, then reduces them using the above prompts. Map-reduce flows are particularly useful when texts are long compared to the context window of a LLM. For long texts, we need a mechanism that ensures that the context to be summarized in the reduce step does not exceed a modelโ€™s context window size. Here we implement a recursive โ€œcollapsingโ€ of the summaries: the inputs are partitioned based on a token limit, and summaries are generated of the partitions. This step is repeated until the total length of the summaries is within a desired limit, allowing for the summarization of arbitrary-length text. First we chunk the blog post into smaller โ€œsub documentsโ€ to be mapped: import { TokenTextSplitter } from "@langchain/textsplitters";const textSplitter = new TokenTextSplitter({ chunkSize: 1000, chunkOverlap: 0,});const splitDocs = await textSplitter.splitDocuments(docs);console.log(`Generated ${splitDocs.length} documents.`); Generated 6 documents. Next, we define our graph. Note that we define an artificially low maximum token length of 1,000 tokens to illustrate the โ€œcollapsingโ€ step. import { collapseDocs, splitListOfDocs,} from "langchain/chains/combine_documents/reduce";import { Document } from "@langchain/core/documents";import { StateGraph, Annotation, Send } from "@langchain/langgraph";let tokenMax = 1000;async function lengthFunction(documents) { const tokenCounts = await Promise.all( documents.map(async (doc) => { return llm.getNumTokens(doc.pageContent); }) ); return tokenCounts.reduce((sum, count) => sum + count, 0);}const OverallState = Annotation.Root({ contents: Annotation, // Notice here we pass a reducer function. // This is because we want combine all the summaries we generate // from individual nodes back into one list. - this is essentially // the "reduce" part summaries: Annotation({ reducer: (state, update) => state.concat(update), }), collapsedSummaries: Annotation, finalSummary: Annotation,});// This will be the state of the node that we will "map" all// documents to in order to generate summariesinterface SummaryState { content: string;}// Here we generate a summary, given a documentconst generateSummary = async ( state: SummaryState): Promise<{ summaries: string[] }> => { const prompt = await mapPrompt.invoke({ context: state.content }); const response = await llm.invoke(prompt); return { summaries: [String(response.content)] };};// Here we define the logic to map out over the documents// We will use this an edge in the graphconst mapSummaries = (state: typeof OverallState.State) => { // We will return a list of `Send` objects // Each `Send` object consists of the name of a node in the graph // as well as the state to send to that node return state.contents.map( (content) => new Send("generateSummary", { content }) );};const collectSummaries = async (state: typeof OverallState.State) => { return { collapsedSummaries: state.summaries.map( (summary) => new Document({ pageContent: summary }) ), };};async function _reduce(input) { const prompt = await reducePrompt.invoke({ docs: input }); const response = await llm.invoke(prompt); return String(response.content);}// Add node to collapse summariesconst collapseSummaries = async (state: typeof OverallState.State) => { const docLists = splitListOfDocs( state.collapsedSummaries, lengthFunction, tokenMax ); const results = []; for (const docList of docLists) { results.push(await collapseDocs(docList, _reduce)); } return { collapsedSummaries: results };};// This represents a conditional edge in the graph that determines// if we should collapse the summaries or notasync function shouldCollapse(state: typeof OverallState.State) { let numTokens = await lengthFunction(state.collapsedSummaries); if (numTokens > tokenMax) { return "collapseSummaries"; } else { return "generateFinalSummary"; }}// Here we will generate the final summaryconst generateFinalSummary = async (state: typeof OverallState.State) => { const response = await _reduce(state.collapsedSummaries); return { finalSummary: response };};// Construct the graphconst graph = new StateGraph(OverallState) .addNode("generateSummary", generateSummary) .addNode("collectSummaries", collectSummaries) .addNode("collapseSummaries", collapseSummaries) .addNode("generateFinalSummary", generateFinalSummary) .addConditionalEdges("__start__", mapSummaries, ["generateSummary"]) .addEdge("generateSummary", "collectSummaries") .addConditionalEdges("collectSummaries", shouldCollapse, [ "collapseSummaries", "generateFinalSummary", ]) .addConditionalEdges("collapseSummaries", shouldCollapse, [ "collapseSummaries", "generateFinalSummary", ]) .addEdge("generateFinalSummary", "__end__");const app = graph.compile(); LangGraph allows the graph structure to be plotted to help visualize its function: // Note: tslab only works inside a jupyter notebook. Don't worry about running this code yourself!import * as tslab from "tslab";const image = await app.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_summarization](/assets/images/graph_img_summarization-2cb471f28eeb4eac82e47a6604b12a06.png) When running the application, we can stream the graph to observe its sequence of steps. Below, we will simply print out the name of the step. Note that because we have a loop in the graph, it can be helpful to specify a [recursion\_limit](https://langchain-ai.github.io/langgraphjs/reference/classes/langgraph.GraphRecursionError.html) on its execution. This will raise a specific error when the specified limit is exceeded. let finalSummary = null;for await (const step of await app.stream( { contents: splitDocs.map((doc) => doc.pageContent) }, { recursionLimit: 10 })) { console.log(Object.keys(step)); if (step.hasOwnProperty("generateFinalSummary")) { finalSummary = step.generateFinalSummary; }} [ 'generateSummary' ][ 'generateSummary' ][ 'generateSummary' ][ 'generateSummary' ][ 'generateSummary' ][ 'generateSummary' ][ 'collectSummaries' ][ 'generateFinalSummary' ] finalSummary; { finalSummary: 'The summaries highlight the evolving landscape of large language models (LLMs) and their integration into autonomous agents and various applications. Key themes include:\n' + '\n' + '1. **Autonomous Agents and LLMs**: Projects like AutoGPT and GPT-Engineer demonstrate the potential of LLMs as core controllers in autonomous systems, utilizing techniques such as Chain of Thought (CoT) and Tree of Thoughts (ToT) for task management and reasoning. These agents can learn from past actions through self-reflection mechanisms, enhancing their problem-solving capabilities.\n' + '\n' + '2. **Supervised Fine-Tuning and Human Feedback**: The importance of human feedback in fine-tuning models is emphasized, with methods like Algorithm Distillation (AD) showing promise in improving model performance while preventing overfitting. The integration of various memory types and external memory systems is suggested to enhance cognitive capabilities.\n' + '\n' + '3. **Integration of External Tools**: The incorporation of external tools and APIs significantly extends LLM capabilities, particularly in specialized tasks like maximum inner-product search (MIPS) and domain-specific applications such as ChemCrow for drug discovery. Frameworks like MRKL and HuggingGPT illustrate the potential for LLMs to effectively utilize these tools.\n' + '\n' + '4. **Evaluation Discrepancies**: There are notable discrepancies between LLM-based assessments and expert evaluations, indicating that LLMs may struggle with specialized knowledge. This raises concerns about their reliability in critical applications, such as scientific discovery.\n' + '\n' + '5. **Limitations of LLMs**: Despite advancements, LLMs face limitations, including finite context lengths, challenges in long-term planning, and difficulties in adapting to unexpected errors. These constraints hinder their robustness compared to human capabilities.\n' + '\n' + 'Overall, the advancements in LLMs and their applications reveal both their potential and limitations, emphasizing the need for ongoing research and development to enhance their effectiveness in various domains.'} In the corresponding [LangSmith trace](https://smith.langchain.com/public/467d535b-1732-46ee-8d3b-f44d9cea7efa/r) we can see the individual LLM calls, grouped under their respective nodes. ### Go deeper[โ€‹](#go-deeper-1 "Direct link to Go deeper") **Customization** * As shown above, you can customize the LLMs and prompts for map and reduce stages. **Real-world use-case** * See [this blog post](https://blog.langchain.dev/llms-to-improve-documentation/) case-study on analyzing user interactions (questions about LangChain documentation)! * The blog post and associated [repo](https://github.com/mendableai/QA_clustering) also introduce clustering as a means of summarization. * This opens up another path beyond the `stuff` or `map-reduce` approaches that is worth considering. Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- We encourage you to check out the [how-to guides](/docs/how_to) for more detail on: * Built-in [document loaders](/docs/how_to/#document-loaders) and [text-splitters](/docs/how_to/#text-splitters) * Integrating various combine-document chains into a [RAG application](/docs/tutorials/rag/) * Incorporating retrieval into a [chatbot](/docs/how_to/chatbots_retrieval/) and other concepts. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Concepts](#concepts) * [Setup](#setup) * [Jupyter Notebook](#jupyter-notebook) * [Installation](#installation) * [LangSmith](#langsmith) * [Overview](#overview) * [Stuff: summarize in a single LLM call](#stuff) * [Streaming](#streaming) * [Go deeper](#go-deeper) * [Map-Reduce: summarize long texts via parallelization](#map-reduce) * [Map](#map) * [Reduce](#reduce) * [Orchestration via LangGraph](#orchestration-via-langgraph) * [Go deeper](#go-deeper-1) * [Next steps](#next-steps) --- # Build an Extraction Chain | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Prerequisites This guide assumes familiarity with the following concepts: * [Chat Models](/docs/concepts/chat_models) * [Tools](/docs/concepts/tools) * [Tool calling](/docs/concepts/tool_calling) In this tutorial, we will build a chain to extract structured information from unstructured text. info This tutorial will only work with models that support **function/tool calling** Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Installation[โ€‹](#installation "Direct link to Installation") To install LangChain run: * npm * yarn * pnpm npm i langchain @langchain/core yarn add langchain @langchain/core pnpm add langchain @langchain/core For more details, see our [Installation guide](/docs/how_to/installation/) . ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com) . After you sign up at the link above, make sure to set your environment variables to start logging traces: export LANGSMITH_TRACING="true"export LANGSMITH_API_KEY="..."# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true The Schema[โ€‹](#the-schema "Direct link to The Schema") ------------------------------------------------------- First, we need to describe what information we want to extract from the text. Weโ€™ll use [Zod](https://zod.dev) to define an example schema that extracts personal information. * npm * yarn * pnpm npm i zod @langchain/core yarn add zod @langchain/core pnpm add zod @langchain/core import { z } from "zod";const personSchema = z.object({ name: z.optional(z.string()).describe("The name of the person"), hair_color: z .optional(z.string()) .describe("The color of the person's hair if known"), height_in_meters: z .optional(z.string()) .describe("Height measured in meters"),}); There are two best practices when defining schema: 1. Document the **attributes** and the **schema** itself: This information is sent to the LLM and is used to improve the quality of information extraction. 2. Do not force the LLM to make up information! Above we used `.nullish()` for the attributes allowing the LLM to output `null` or `undefined` if it doesnโ€™t know the answer. info For best performance, document the schema well and make sure the model isnโ€™t force to return results if thereโ€™s no information to be extracted in the text. The Extractor[โ€‹](#the-extractor "Direct link to The Extractor") ---------------------------------------------------------------- Letโ€™s create an information extractor using the schema we defined above. import { ChatPromptTemplate } from "@langchain/core/prompts";// Define a custom prompt to provide instructions and any additional context.// 1) You can add examples into the prompt template to improve extraction quality// 2) Introduce additional parameters to take context into account (e.g., include metadata// about the document from which the text was extracted.)const promptTemplate = ChatPromptTemplate.fromMessages([ [ "system", `You are an expert extraction algorithm.Only extract relevant information from the text.If you do not know the value of an attribute asked to extract,return null for the attribute's value.`, ], // Please see the how-to about improving performance with // reference examples. // ["placeholder", "{examples}"], ["human", "{text}"],]); We need to use a model that supports function/tool calling. Please review [the documentation](/docs/integrations/chat) for list of some models that can be used with this API. ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); We enable structured output by creating a new object with the `.withStructuredOutput` method: const structured_llm = llm.withStructuredOutput(personSchema); We can then invoke it normally: const prompt = await promptTemplate.invoke({ text: "Alan Smith is 6 feet tall and has blond hair.",});await structured_llm.invoke(prompt); { name: 'Alan Smith', hair_color: 'blond', height_in_meters: '1.83' } info Extraction is Generative ๐Ÿคฏ LLMs are generative models, so they can do some pretty cool things like correctly extract the height of the person in meters even though it was provided in feet! We can see the LangSmith trace [here](https://smith.langchain.com/public/3d44b7e8-e7ca-4e02-951d-3290ccc89d64/r) . Even though we defined our schema with the variable name `personSchema`, Zod is unable to infer this name and therefore does not pass it along to the model. To help give the LLM more clues as to what your provided schema represents, you can also give the schema you pass to `withStructuredOutput()` a name: const structured_llm2 = llm.withStructuredOutput(personSchema, { name: "person",});const prompt2 = await promptTemplate.invoke({ text: "Alan Smith is 6 feet tall and has blond hair.",});await structured_llm2.invoke(prompt2); { name: 'Alan Smith', hair_color: 'blond', height_in_meters: '1.83' } This can improve performance in many cases. Multiple Entities[โ€‹](#multiple-entities "Direct link to Multiple Entities") ---------------------------------------------------------------------------- In **most cases**, you should be extracting a list of entities rather than a single entity. This can be easily achieved using Zod by nesting models inside one another. import { z } from "zod";const person = z.object({ name: z.optional(z.string()).describe("The name of the person"), hair_color: z .optional(z.string()) .describe("The color of the person's hair if known"), height_in_meters: z.number().nullish().describe("Height measured in meters"),});const dataSchema = z.object({ people: z.array(person).describe("Extracted data about people"),}); info Extraction might not be perfect here. Please continue to see how to use **Reference Examples** to improve the quality of extraction, and see the **guidelines** section! const structured_llm3 = llm.withStructuredOutput(dataSchema);const prompt3 = await promptTemplate.invoke({ text: "My name is Jeff, my hair is black and i am 6 feet tall. Anna has the same color hair as me.",});await structured_llm3.invoke(prompt3); { people: [ { name: 'Jeff', hair_color: 'black', height_in_meters: 1.83 }, { name: 'Anna', hair_color: 'black', height_in_meters: null } ]} tip When the schema accommodates the extraction of **multiple entities**, it also allows the model to extract **no entities** if no relevant information is in the text by providing an empty list. This is usually a **good** thing! It allows specifying **required** attributes on an entity without necessarily forcing the model to detect this entity. We can see the LangSmith trace [here](https://smith.langchain.com/public/272096ab-9ac5-43f9-aa00-3b8443477d17/r) Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Now that you understand the basics of extraction with LangChain, youโ€™re ready to proceed to the rest of the how-to guides: * [Add Examples](/docs/how_to/extraction_examples) : Learn how to use **reference examples** to improve performance. * [Handle Long Text](/docs/how_to/extraction_long_text) : What should you do if the text does not fit into the context window of the LLM? * [Use a Parsing Approach](/docs/how_to/extraction_parse) : Use a prompt based approach to extract with models that do not support **tool/function calling**. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Setup](#setup) * [Installation](#installation) * [LangSmith](#langsmith) * [The Schema](#the-schema) * [The Extractor](#the-extractor) * [Multiple Entities](#multiple-entities) * [Next steps](#next-steps) --- # Build a Question/Answering system over SQL data | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Prerequisites This guide assumes familiarity with the following concepts: * [Chat models](/docs/concepts/chat_models) * [Tools](/docs/concepts/tools) * [Agents](/docs/concepts/agents) * [LangGraph](/docs/concepts/architecture/#langgraph) Enabling a LLM system to query structured data can be qualitatively different from unstructured text data. Whereas in the latter it is common to generate text that can be searched against a vector database, the approach for structured data is often for the LLM to write and execute queries in a DSL, such as SQL. In this guide weโ€™ll go over the basic ways to create a Q&A system over tabular data in databases. We will cover implementations using both [chains](/docs/tutorials/sql_qa#chains) and [agents](/docs/tutorials/sql_qa#agents) . These systems will allow us to ask a question about the data in a database and get back a natural language answer. The main difference between the two is that our agent can query the database in a loop as many times as it needs to answer the question. โš ๏ธ Security note โš ๏ธ[โ€‹](#security-note "Direct link to โš ๏ธ Security note โš ๏ธ") ---------------------------------------------------------------------------- Building Q&A systems of SQL databases requires executing model-generated SQL queries. There are inherent risks in doing this. Make sure that your database connection permissions are always scoped as narrowly as possible for your chain/agentโ€™s needs. This will mitigate though not eliminate the risks of building a model-driven system. For more on general security best practices, [see here](/docs/security) . Architecture[โ€‹](#architecture "Direct link to Architecture") ------------------------------------------------------------- At a high-level, the steps of these systems are: 1. **Convert question to SQL query**: Model converts user input to a SQL query. 2. **Execute SQL query**: Execute the query. 3. **Answer the question**: Model responds to user input using the query results. ![sql_usecase.png](/assets/images/sql_usecase-d432701261f05ab69b38576093718cf3.png) Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- First, get required packages and set environment variables: `bash npm2yarn npm i langchain @langchain/community @langchain/langgraph` # Uncomment the below to use LangSmith. Not required, but recommended for debugging and observability.# export LANGSMITH_API_KEY=# export LANGSMITH_TRACING=true# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true ### Sample data[โ€‹](#sample-data "Direct link to Sample data") The below example will use a SQLite connection with the Chinook database, which is a sample database that represents a digital media store. Follow [these installation steps](https://database.guide/2-sample-databases-sqlite/) to create `Chinook.db` in the same directory as this notebook. You can also download and build the database via the command line: curl -s https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql | sqlite3 Chinook.db Now, `Chinook.db` is in our directory and we can interface with it using the [SqlDatabase](https://api.js.langchain.com/classes/langchain.sql_db.SqlDatabase.html) class: import { SqlDatabase } from "langchain/sql_db";import { DataSource } from "typeorm";const datasource = new DataSource({ type: "sqlite", database: "Chinook.db",});const db = await SqlDatabase.fromDataSourceParams({ appDataSource: datasource,});await db.run("SELECT * FROM Artist LIMIT 10;"); [{"ArtistId":1,"Name":"AC/DC"},{"ArtistId":2,"Name":"Accept"},{"ArtistId":3,"Name":"Aerosmith"},{"ArtistId":4,"Name":"Alanis Morissette"},{"ArtistId":5,"Name":"Alice In Chains"},{"ArtistId":6,"Name":"Antรดnio Carlos Jobim"},{"ArtistId":7,"Name":"Apocalyptica"},{"ArtistId":8,"Name":"Audioslave"},{"ArtistId":9,"Name":"BackBeat"},{"ArtistId":10,"Name":"Billy Cobham"}] Great! Weโ€™ve got a SQL database that we can query. Now letโ€™s try hooking it up to an LLM. Chains[โ€‹](#chains "Direct link to Chains") ------------------------------------------- Chains are compositions of predictable steps. In [LangGraph](/docs/concepts/architecture#langchainlanggraph) , we can represent a chain via simple sequence of nodes. Letโ€™s create a sequence of steps that, given a question, does the following: - converts the question into a SQL query; - executes the query; - uses the result to answer the original question. There are scenarios not supported by this arrangement. For example, this system will execute a SQL query for any user inputโ€“ even โ€œhelloโ€. Importantly, as weโ€™ll see below, some questions require more than one query to answer. We will address these scenarios in the Agents section. ### Application state[โ€‹](#application-state "Direct link to Application state") The LangGraph [state](https://langchain-ai.github.io/langgraphjs/concepts/low_level/#state) of our application controls what data is input to the application, transferred between steps, and output by the application. For this application, we can just keep track of the input question, generated query, query result, and generated answer: import { Annotation } from "@langchain/langgraph";const InputStateAnnotation = Annotation.Root({ question: Annotation,});const StateAnnotation = Annotation.Root({ question: Annotation, query: Annotation, result: Annotation, answer: Annotation,}); Now we just need functions that operate on this state and populate its contents. ### Convert question to SQL query[โ€‹](#convert-question-to-sql-query "Direct link to Convert question to SQL query") The first step is to take the user input and convert it to a SQL query. To reliably obtain SQL queries (absent markdown formatting and explanations or clarifications), we will make use of LangChainโ€™s [structured output](/docs/concepts/structured_outputs/) abstraction. Letโ€™s select a chat model for our application: ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); We will pull a prompt from the [Prompt Hub](https://smith.langchain.com/hub) to instruct the model. import { pull } from "langchain/hub";import { ChatPromptTemplate } from "@langchain/core/prompts";const queryPromptTemplate = await pull( "langchain-ai/sql-query-system-prompt");console.log(queryPromptTemplate.promptMessages[0].lc_kwargs.prompt.template); Given an input question, create a syntactically correct {dialect} query to run to help find the answer. Unless the user specifies in his question a specific number of examples they wish to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.Never query for all the columns from a specific table, only ask for a the few relevant columns given the question.Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.Only use the following tables:{table_info}Question: {input} The prompt includes several parameters we will need to populate, such as the SQL dialect and table schemas. LangChainโ€™s [SqlDatabase](https://api.js.langchain.com/classes/langchain.sql_db.SqlDatabase.html) object includes methods to help with this. Our `writeQuery` step will just populate these parameters and prompt a model to generate the SQL query: import { z } from "zod";const queryOutput = z.object({ query: z.string().describe("Syntactically valid SQL query."),});const structuredLlm = llm.withStructuredOutput(queryOutput);const writeQuery = async (state: typeof InputStateAnnotation.State) => { const promptValue = await queryPromptTemplate.invoke({ dialect: db.appDataSourceOptions.type, top_k: 10, table_info: await db.getTableInfo(), input: state.question, }); const result = await structuredLlm.invoke(promptValue); return { query: result.query };}; Letโ€™s test it out: await writeQuery({ question: "How many Employees are there?" }); { query: 'SELECT COUNT(*) AS EmployeeCount FROM Employee;' } ### Execute query[โ€‹](#execute-query "Direct link to Execute query") **This is the most dangerous part of creating a SQL chain.** Consider carefully if it is OK to run automated queries over your data. Minimize the database connection permissions as much as possible. Consider adding a human approval step to you chains before query execution (see below). To execute the query, we will load a tool from [langchain-community](/docs/concepts/architecture#langchaincommunity) . Our `executeQuery` node will just wrap this tool: import { QuerySqlTool } from "langchain/tools/sql";const executeQuery = async (state: typeof StateAnnotation.State) => { const executeQueryTool = new QuerySqlTool(db); return { result: await executeQueryTool.invoke(state.query) };}; Testing this step: await executeQuery({ question: "", query: "SELECT COUNT(*) AS EmployeeCount FROM Employee;", result: "", answer: "",}); { result: '[{"EmployeeCount":8}]' } ### Generate answer[โ€‹](#generate-answer "Direct link to Generate answer") Finally, our last step generates an answer to the question given the information pulled from the database: const generateAnswer = async (state: typeof StateAnnotation.State) => { const promptValue = "Given the following user question, corresponding SQL query, " + "and SQL result, answer the user question.\n\n" + `Question: ${state.question}\n` + `SQL Query: ${state.query}\n` + `SQL Result: ${state.result}\n`; const response = await llm.invoke(promptValue); return { answer: response.content };}; ### Orchestrating with LangGraph[โ€‹](#orchestrating-with-langgraph "Direct link to Orchestrating with LangGraph") Finally, we compile our application into a single `graph` object. In this case, we are just connecting the three steps into a single sequence. import { StateGraph } from "@langchain/langgraph";const graphBuilder = new StateGraph({ stateSchema: StateAnnotation,}) .addNode("writeQuery", writeQuery) .addNode("executeQuery", executeQuery) .addNode("generateAnswer", generateAnswer) .addEdge("__start__", "writeQuery") .addEdge("writeQuery", "executeQuery") .addEdge("executeQuery", "generateAnswer") .addEdge("generateAnswer", "__end__"); const graph = graphBuilder.compile(); LangGraph also comes with built-in utilities for visualizing the control flow of your application: // Note: tslab only works inside a jupyter notebook. Don't worry about running this code yourself!import * as tslab from "tslab";const image = await graph.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_sql_qa](/assets/images/graph_img_sql_qa-e15c213cd7e0db2348b8f1c99066c701.png) Letโ€™s test our application! Note that we can stream the results of individual steps: let inputs = { question: "How many employees are there?" };console.log(inputs);console.log("\n====\n");for await (const step of await graph.stream(inputs, { streamMode: "updates",})) { console.log(step); console.log("\n====\n");} { question: 'How many employees are there?' }===={ writeQuery: { query: 'SELECT COUNT(*) AS EmployeeCount FROM Employee;' }}===={ executeQuery: { result: '[{"EmployeeCount":8}]' } }===={ generateAnswer: { answer: 'There are 8 employees.' } }==== Check out the [LangSmith trace](https://smith.langchain.com/public/4cb42037-55cf-4da9-8b3a-8410482dbd32/r) . ### Human-in-the-loop[โ€‹](#human-in-the-loop "Direct link to Human-in-the-loop") LangGraph supports a number of features that can be useful for this workflow. One of them is [human-in-the-loop](https://langchain-ai.github.io/langgraphjs/concepts/human_in_the_loop/) : we can interrupt our application before sensitive steps (such as the execution of a SQL query) for human review. This is enabled by LangGraphโ€™s [persistence](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) layer, which saves run progress to your storage of choice. Below, we specify storage in-memory: import { MemorySaver } from "@langchain/langgraph";const checkpointer = new MemorySaver();const graphWithInterrupt = graphBuilder.compile({ checkpointer: checkpointer, interruptBefore: ["executeQuery"],});// Now that we're using persistence, we need to specify a thread ID// so that we can continue the run after review.const threadConfig = { configurable: { thread_id: "1" }, streamMode: "updates" as const,}; const image = await graphWithInterrupt.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_sql_qa_interrupt](/assets/images/graph_img_sql_qa_interrupt-733903891c7e3815866a331bcb327aa5.png) Letโ€™s repeat the same run, adding in a simple yes/no approval step: console.log(inputs);console.log("\n====\n");for await (const step of await graphWithInterrupt.stream( inputs, threadConfig)) { console.log(step); console.log("\n====\n");}// Will log when the graph is interrupted, after `executeQuery`.console.log("---GRAPH INTERRUPTED---"); { question: 'How many employees are there?' }===={ writeQuery: { query: 'SELECT COUNT(*) AS EmployeeCount FROM Employee;' }}====---GRAPH INTERRUPTED--- The run interrupts before the query is executed. At this point, our application can handle an interaction with a user, who reviews the query. If approved, running the same thread with a `null` input will continue from where we left off. This is enabled by LangGraphโ€™s [persistence](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) layer. for await (const step of await graphWithInterrupt.stream(null, threadConfig)) { console.log(step); console.log("\n====\n");} { executeQuery: { result: '[{"EmployeeCount":8}]' } }===={ generateAnswer: { answer: 'There are 8 employees.' } }==== See [this](https://langchain-ai.github.io/langgraphjs/concepts/human_in_the_loop/) LangGraph guide for more detail and examples. ### Next steps[โ€‹](#next-steps "Direct link to Next steps") For more complex query-generation, we may want to create few-shot prompts or add query-checking steps. For advanced techniques like this and more check out: * [Prompting strategies](/docs/how_to/sql_prompting) : Advanced prompt engineering techniques. * [Query checking](/docs/how_to/sql_query_checking) : Add query validation and error handling. * [Large databases](/docs/how_to/sql_large_db) : Techniques for working with large databases. Agents[โ€‹](#agents "Direct link to Agents") ------------------------------------------- [Agents](/docs/concepts/agents) leverage the reasoning capabilities of LLMs to make decisions during execution. Using agents allows you to offload additional discretion over the query generation and execution process. Although their behavior is less predictable than the above โ€œchainโ€, they feature some advantages: * They can query the database as many times as needed to answer the user question. * They can recover from errors by running a generated query, catching the traceback and regenerating it correctly. * They can answer questions based on the databasesโ€™ schema as well as on the databasesโ€™ content (like describing a specific table). Below we assemble a minimal SQL agent. We will equip it with a set of tools using LangChainโ€™s [SqlToolkit](https://api.js.langchain.com/classes/langchain.agents_toolkits_sql.SqlToolkit.html) . Using LangGraphโ€™s [pre-built ReAct agent constructor](https://langchain-ai.github.io/langgraphjs/how-tos/create-react-agent/) , we can do this in one line. The `SqlToolkit` includes tools that can: * Create and execute queries * Check query syntax * Retrieve table descriptions * โ€ฆ and more import { SqlToolkit } from "langchain/agents/toolkits/sql";const toolkit = new SqlToolkit(db, llm);const tools = toolkit.getTools();console.log( tools.map((tool) => ({ name: tool.name, description: tool.description, }))); [ { name: 'query-sql', description: 'Input to this tool is a detailed and correct SQL query, output is a result from the database.\n' + ' If the query is not correct, an error message will be returned.\n' + ' If an error is returned, rewrite the query, check the query, and try again.' }, { name: 'info-sql', description: 'Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables.\n' + ' Be sure that the tables actually exist by calling list-tables-sql first!\n' + '\n' + ' Example Input: "table1, table2, table3.' }, { name: 'list-tables-sql', description: 'Input is an empty string, output is a comma-separated list of tables in the database.' }, { name: 'query-checker', description: 'Use this tool to double check if your query is correct before executing it.\n' + ' Always use this tool before executing a query with query-sql!' }] ### System Prompt[โ€‹](#system-prompt "Direct link to System Prompt") We will also want to load a system prompt for our agent. This will consist of instructions for how to behave. import { pull } from "langchain/hub";import { ChatPromptTemplate } from "@langchain/core/prompts";const systemPromptTemplate = await pull( "langchain-ai/sql-agent-system-prompt");console.log(systemPromptTemplate.promptMessages[0].lc_kwargs.prompt.template); You are an agent designed to interact with a SQL database.Given an input question, create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.Unless the user specifies a specific number of examples they wish to obtain, always limit your query to at most {top_k} results.You can order the results by a relevant column to return the most interesting examples in the database.Never query for all the columns from a specific table, only ask for the relevant columns given the question.You have access to tools for interacting with the database.Only use the below tools. Only use the information returned by the below tools to construct your final answer.You MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.To start you should ALWAYS look at the tables in the database to see what you can query.Do NOT skip this step.Then you should query the schema of the most relevant tables. Letโ€™s populate the parameters highlighted in the prompt: const systemMessage = await systemPromptTemplate.format({ dialect: "SQLite", top_k: 5,}); ### Initializing agent[โ€‹](#initializing-agent "Direct link to Initializing agent") We will use a prebuilt [LangGraph](/docs/concepts/architecture/#langgraph) agent to build our agent import { createReactAgent } from "@langchain/langgraph/prebuilt";const agent = createReactAgent({ llm: llm, tools: tools, stateModifier: systemMessage,}); Consider how the agent responds to the below question: Expand for \`prettyPrint\` code. import { AIMessage, BaseMessage, isAIMessage } from "@langchain/core/messages";const prettyPrint = (message: BaseMessage) => { let txt = `[${message._getType()}]: ${message.content}`; if ((isAIMessage(message) && message.tool_calls?.length) || 0 > 0) { const tool_calls = (message as AIMessage)?.tool_calls ?.map((tc) => `- ${tc.name}(${JSON.stringify(tc.args)})`) .join("\n"); txt += ` \nTools: \n${tool_calls}`; } console.log(txt);}; let inputs2 = { messages: [ { role: "user", content: "Which country's customers spent the most?" }, ],};for await (const step of await agent.stream(inputs2, { streamMode: "values",})) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: Which country's customers spent the most?-----[ai]:Tools:- list-tables-sql({"input":""})-----[tool]: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track-----[ai]:Tools:- info-sql({"input":"Customer, Invoice, InvoiceLine"})- info-sql({"input":"Invoice"})-----[tool]: CREATE TABLE Invoice (InvoiceId INTEGER NOT NULL, CustomerId INTEGER NOT NULL, InvoiceDate DATETIME NOT NULL, BillingAddress NVARCHAR(70) , BillingCity NVARCHAR(40) , BillingState NVARCHAR(40) , BillingCountry NVARCHAR(40) , BillingPostalCode NVARCHAR(10) , Total NUMERIC(10,2) NOT NULL)SELECT * FROM "Invoice" LIMIT 3; InvoiceId CustomerId InvoiceDate BillingAddress BillingCity BillingState BillingCountry BillingPostalCode Total 1 2 2021-01-01 00:00:00 Theodor-Heuss-StraรŸe 34 Stuttgart null Germany 70174 1.98 2 4 2021-01-02 00:00:00 Ullevรฅlsveien 14 Oslo null Norway 0171 3.96 3 8 2021-01-03 00:00:00 Grรฉtrystraat 63 Brussels null Belgium 1000 5.94-----[ai]:Tools:- query-checker({"input":"SELECT c.Country, SUM(i.Total) AS TotalSpent \nFROM Customer c \nJOIN Invoice i ON c.CustomerId = i.CustomerId \nGROUP BY c.Country \nORDER BY TotalSpent DESC \nLIMIT 5;"})-----[tool]: The SQL query you provided appears to be correct and does not contain any of the common mistakes listed. It properly joins the `Customer` and `Invoice` tables, groups the results by country, and orders the total spending in descending order while limiting the results to the top 5 countries.Here is the original query reproduced:```sqlSELECT c.Country, SUM(i.Total) AS TotalSpentFROM Customer cJOIN Invoice i ON c.CustomerId = i.CustomerIdGROUP BY c.CountryORDER BY TotalSpent DESCLIMIT 5;```No changes are necessary.-----[ai]:Tools:- query-sql({"input":"SELECT c.Country, SUM(i.Total) AS TotalSpent \nFROM Customer c \nJOIN Invoice i ON c.CustomerId = i.CustomerId \nGROUP BY c.Country \nORDER BY TotalSpent DESC \nLIMIT 5;"})-----[tool]: [{"Country":"USA","TotalSpent":523.0600000000003},{"Country":"Canada","TotalSpent":303.9599999999999},{"Country":"France","TotalSpent":195.09999999999994},{"Country":"Brazil","TotalSpent":190.09999999999997},{"Country":"Germany","TotalSpent":156.48}]-----[ai]: The countries whose customers spent the most are:1. **USA** - $523.062. **Canada** - $303.963. **France** - $195.104. **Brazil** - $190.105. **Germany** - $156.48----- You can also use the [LangSmith trace](https://smith.langchain.com/public/f4313ba4-a93e-418b-b863-1c2626c330d1/r) to visualize these steps and associated metadata. Note that the agent executes multiple queries until it has the information it needs: 1. List available tables; 2. Retrieves the schema for three tables; 3. Queries multiple of the tables via a join operation. The agent is then able to use the result of the final query to generate an answer to the original question. The agent can similarly handle qualitative questions: let inputs3 = { messages: [{ role: "user", content: "Describe the playlisttrack table" }],};for await (const step of await agent.stream(inputs3, { streamMode: "values",})) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: Describe the playlisttrack table-----[ai]:Tools:- list-tables-sql({"input":""})-----[tool]: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track-----[ai]:Tools:- info-sql({"input":"PlaylistTrack"})-----[tool]: CREATE TABLE PlaylistTrack (PlaylistId INTEGER NOT NULL, TrackId INTEGER NOT NULL)SELECT * FROM "PlaylistTrack" LIMIT 3; PlaylistId TrackId 1 3402 1 3389 1 3390-----[ai]: The `PlaylistTrack` table has the following schema:- **PlaylistId**: INTEGER (NOT NULL)- **TrackId**: INTEGER (NOT NULL)This table is used to associate tracks with playlists. Here are some sample rows from the table:| PlaylistId | TrackId ||------------|---------|| 1 | 3402 || 1 | 3389 || 1 | 3390 |----- ### Dealing with high-cardinality columns[โ€‹](#dealing-with-high-cardinality-columns "Direct link to Dealing with high-cardinality columns") In order to filter columns that contain proper nouns such as addresses, song names or artists, we first need to double-check the spelling in order to filter the data correctly. We can achieve this by creating a vector store with all the distinct proper nouns that exist in the database. We can then have the agent query that vector store each time the user includes a proper noun in their question, to find the correct spelling for that word. In this way, the agent can make sure it understands which entity the user is referring to before building the target query. First we need the unique values for each entity we want, for which we define a function that parses the result into a list of elements: async function queryAsList(database: any, query: string): Promise { const res: Array<{ [key: string]: string }> = JSON.parse( await database.run(query) ) .flat() .filter((el: any) => el != null); const justValues: Array = res.map((item) => Object.values(item)[0] .replace(/\b\d+\b/g, "") .trim() ); return justValues;}// Gather entities into a listlet artists: string[] = await queryAsList(db, "SELECT Name FROM Artist");let albums: string[] = await queryAsList(db, "SELECT Title FROM Album");let properNouns = artists.concat(albums);console.log(`Total: ${properNouns.length}\n`);console.log(`Sample: ${properNouns.slice(0, 5)}...`); Total: 622Sample: AC/DC,Accept,Aerosmith,Alanis Morissette,Alice In Chains... Using this function, we can create a **retriever tool** that the agent can execute at its discretion. Letโ€™s select an [embeddings model](/docs/integrations/text_embedding/) and [vector store](/docs/integrations/vectorstores/) for this step: ### Pick your embedding model: * OpenAI * Azure * AWS * VertexAI * MistralAI * Cohere #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai OPENAI_API_KEY=your-api-key import { OpenAIEmbeddings } from "@langchain/openai";const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-large"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai AZURE_OPENAI_API_INSTANCE_NAME=AZURE_OPENAI_API_KEY=AZURE_OPENAI_API_VERSION="2024-02-01" import { AzureOpenAIEmbeddings } from "@langchain/openai";const embeddings = new AzureOpenAIEmbeddings({ azureOpenAIApiEmbeddingsDeploymentName: "text-embedding-ada-002"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/aws yarn add @langchain/aws pnpm add @langchain/aws BEDROCK_AWS_REGION=your-region import { BedrockEmbeddings } from "@langchain/aws";const embeddings = new BedrockEmbeddings({ model: "amazon.titan-embed-text-v1"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai GOOGLE_APPLICATION_CREDENTIALS=credentials.json import { VertexAIEmbeddings } from "@langchain/google-vertexai";const embeddings = new VertexAIEmbeddings({ model: "text-embedding-004"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai MISTRAL_API_KEY=your-api-key import { MistralAIEmbeddings } from "@langchain/mistralai";const embeddings = new MistralAIEmbeddings({ model: "mistral-embed"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/cohere yarn add @langchain/cohere pnpm add @langchain/cohere COHERE_API_KEY=your-api-key import { CohereEmbeddings } from "@langchain/cohere";const embeddings = new CohereEmbeddings({ model: "embed-english-v3.0"}); ### Pick your vector store: * Memory * Chroma * FAISS * MongoDB * PGVector * Pinecone * Qdrant #### Install dependencies * npm * yarn * pnpm npm i langchain yarn add langchain pnpm add langchain import { MemoryVectorStore } from "langchain/vectorstores/memory";const vectorStore = new MemoryVectorStore(embeddings); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { Chroma } from "@langchain/community/vectorstores/chroma";const vectorStore = new Chroma(embeddings, { collectionName: "a-test-collection",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { FaissStore } from "@langchain/community/vectorstores/faiss";const vectorStore = new FaissStore(embeddings, {}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mongodb yarn add @langchain/mongodb pnpm add @langchain/mongodb import { MongoDBAtlasVectorSearch } from "@langchain/mongodb"import { MongoClient } from "mongodb";const client = new MongoClient(process.env.MONGODB_ATLAS_URI || "");const collection = client .db(process.env.MONGODB_ATLAS_DB_NAME) .collection(process.env.MONGODB_ATLAS_COLLECTION_NAME);const vectorStore = new MongoDBAtlasVectorSearch(embeddings, { collection: collection, indexName: "vector_index", textKey: "text", embeddingKey: "embedding",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import PGVectorStore from "@langchain/community/vectorstores/pgvector";const vectorStore = await PGVectorStore.initialize(embeddings, {}) #### Install dependencies * npm * yarn * pnpm npm i @langchain/pinecone yarn add @langchain/pinecone pnpm add @langchain/pinecone import { PineconeStore } from "@langchain/pinecone";import { Pinecone as PineconeClient } from "@pinecone-database/pinecone";const pinecone = new PineconeClient();const vectorStore = new PineconeStore(embeddings, { pineconeIndex, maxConcurrency: 5,}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/qdrant yarn add @langchain/qdrant pnpm add @langchain/qdrant import { QdrantVectorStore } from "@langchain/qdrant";const vectorStore = await QdrantVectorStore.fromExistingCollection(embeddings, { url: process.env.QDRANT_URL, collectionName: "langchainjs-testing",}); We can now construct a retrieval tool that can search over relevant proper nouns in the database: import { createRetrieverTool } from "langchain/tools/retriever";import { Document } from "@langchain/core/documents";const documents = properNouns.map( (text) => new Document({ pageContent: text }));await vectorStore.addDocuments(documents);const retriever = vectorStore.asRetriever(5);const retrieverTool = createRetrieverTool(retriever, { name: "searchProperNouns", description: "Use to look up values to filter on. Input is an approximate spelling " + "of the proper noun, output is valid proper nouns. Use the noun most " + "similar to the search.",}); Letโ€™s try it out: console.log(await retrieverTool.invoke({ query: "Alice Chains" })); Alice In ChainsAlanis MorissetteJagged Little PillAngel DustAmy Winehouse This way, if the agent determines it needs to write a filter based on an artist along the lines of โ€œAlice Chainsโ€, it can first use the retriever tool to observe relevant values of a column. Putting this together: // Add to system messagelet suffix = "If you need to filter on a proper noun like a Name, you must ALWAYS first look up " + "the filter value using the 'search_proper_nouns' tool! Do not try to " + "guess at the proper name - use this function to find similar ones.";const system = systemMessage + suffix;const updatedTools = tools.concat(retrieverTool);const agent2 = createReactAgent({ llm: llm, tools: updatedTools, stateModifier: system,}); let inputs4 = { messages: [ { role: "user", content: "How many albums does alis in chain have?" }, ],};for await (const step of await agent2.stream(inputs4, { streamMode: "values",})) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: How many albums does alis in chain have?-----[ai]:Tools:- searchProperNouns({"query":"alis in chain"})-----[tool]: Alice In ChainsAlanis MorissetteUp An' AtomA-SidesJagged Little Pill-----[ai]:Tools:- query-sql({"input":"SELECT COUNT(*) FROM albums WHERE artist_name = 'Alice In Chains'"})-----[tool]: QueryFailedError: SQLITE_ERROR: no such table: albums-----[ai]:Tools:- list-tables-sql({"input":""})-----[tool]: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track-----[ai]:Tools:- info-sql({"input":"Album"})- info-sql({"input":"Artist"})-----[tool]: CREATE TABLE Artist (ArtistId INTEGER NOT NULL, Name NVARCHAR(120) )SELECT * FROM "Artist" LIMIT 3; ArtistId Name 1 AC/DC 2 Accept 3 Aerosmith-----[ai]:Tools:- query-sql({"input":"SELECT COUNT(*) FROM Album WHERE ArtistId = (SELECT ArtistId FROM Artist WHERE Name = 'Alice In Chains')"})-----[tool]: [{"COUNT(*)":1}]-----[ai]: Alice In Chains has released 1 album.----- As we can see, both in the streamed steps and in the [LangSmith trace](https://smith.langchain.com/public/8b14a4a4-c08b-4b85-8086-c050931ae03d/r) , the agent used the `searchProperNouns` tool in order to check how to correctly query the database for this specific artist. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [โš ๏ธ Security note โš ๏ธ](#security-note) * [Architecture](#architecture) * [Setup](#setup) * [Sample data](#sample-data) * [Chains](#chains) * [Application state](#application-state) * [Convert question to SQL query](#convert-question-to-sql-query) * [Execute query](#execute-query) * [Generate answer](#generate-answer) * [Orchestrating with LangGraph](#orchestrating-with-langgraph) * [Human-in-the-loop](#human-in-the-loop) * [Next steps](#next-steps) * [Agents](#agents) * [System Prompt](#system-prompt) * [Initializing agent](#initializing-agent) * [Dealing with high-cardinality columns](#dealing-with-high-cardinality-columns) --- # Build a Chatbot | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Prerequisites This guide assumes familiarity with the following concepts: * [Chat Models](/docs/concepts/chat_models) * [Prompt Templates](/docs/concepts/prompt_templates) * [Chat History](/docs/concepts/chat_history) This guide requires `langgraph >= 0.2.28`. note This tutorial previously built a chatbot using [RunnableWithMessageHistory](https://api.js.langchain.com/classes/_langchain_core.runnables.RunnableWithMessageHistory.html) . You can access this version of the tutorial in the [v0.2 docs](https://js.langchain.com/v0.2/docs/tutorials/chatbot/) . The LangGraph implementation offers a number of advantages over `RunnableWithMessageHistory`, including the ability to persist arbitrary components of an application's state (instead of only messages). Overview[โ€‹](#overview "Direct link to Overview") ------------------------------------------------- Weโ€™ll go over an example of how to design and implement an LLM-powered chatbot. This chatbot will be able to have a conversation and remember previous interactions. Note that this chatbot that we build will only use the language model to have a conversation. There are several other related concepts that you may be looking for: * [Conversational RAG](/docs/tutorials/qa_chat_history) : Enable a chatbot experience over an external source of data * [Agents](https://langchain-ai.github.io/langgraphjs/tutorials/multi_agent/agent_supervisor/) : Build a chatbot that can take actions This tutorial will cover the basics which will be helpful for those two more advanced topics, but feel free to skip directly to there should you choose. Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Jupyter Notebook[โ€‹](#jupyter-notebook "Direct link to Jupyter Notebook") This guide (and most of the other guides in the documentation) uses [Jupyter notebooks](https://jupyter.org/) and assumes the reader is as well. Jupyter notebooks are perfect for learning how to work with LLM systems because oftentimes things can go wrong (unexpected output, API down, etc) and going through guides in an interactive environment is a great way to better understand them. This and other tutorials are perhaps most conveniently run in a Jupyter notebook. See [here](https://jupyter.org/install) for instructions on how to install. ### Installation[โ€‹](#installation "Direct link to Installation") For this tutorial we will need `@langchain/core` and `langgraph`: * npm * yarn * pnpm npm i @langchain/core @langchain/langgraph uuid yarn add @langchain/core @langchain/langgraph uuid pnpm add @langchain/core @langchain/langgraph uuid For more details, see our [Installation guide](/docs/how_to/installation) . ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com) . After you sign up at the link above, make sure to set your environment variables to start logging traces: process.env.LANGSMITH_TRACING = "true";process.env.LANGSMITH_API_KEY = "..."; Quickstart[โ€‹](#quickstart "Direct link to Quickstart") ------------------------------------------------------- First up, letโ€™s learn how to use a language model by itself. LangChain supports many different language models that you can use interchangeably - select the one you want to use below! ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); Letโ€™s first use the model directly. `ChatModel`s are instances of LangChain โ€œRunnablesโ€, which means they expose a standard interface for interacting with them. To just simply call the model, we can pass in a list of messages to the `.invoke` method. await llm.invoke([{ role: "user", content: "Hi im bob" }]); AIMessage { "id": "chatcmpl-AekDrrCyaBauLYHuVv3dkacxW2G1J", "content": "Hi Bob! How can I help you today?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 10, "completionTokens": 10, "totalTokens": 20 }, "finish_reason": "stop", "usage": { "prompt_tokens": 10, "completion_tokens": 10, "total_tokens": 20, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 10, "input_tokens": 10, "total_tokens": 20, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} The model on its own does not have any concept of state. For example, if you ask a followup question: await llm.invoke([{ role: "user", content: "Whats my name" }]); AIMessage { "id": "chatcmpl-AekDuOk1LjOdBVLtuCvuHjAs5aoad", "content": "I'm sorry, but I don't have access to personal information about users unless you've shared it with me in this conversation. How can I assist you today?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 10, "completionTokens": 30, "totalTokens": 40 }, "finish_reason": "stop", "usage": { "prompt_tokens": 10, "completion_tokens": 30, "total_tokens": 40, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 30, "input_tokens": 10, "total_tokens": 40, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} Letโ€™s take a look at the example [LangSmith trace](https://smith.langchain.com/public/3b768e44-a319-453a-bd6e-30f9df75f16a/r) We can see that it doesnโ€™t take the previous conversation turn into context, and cannot answer the question. This makes for a terrible chatbot experience! To get around this, we need to pass the entire conversation history into the model. Letโ€™s see what happens when we do that: await llm.invoke([ { role: "user", content: "Hi! I'm Bob" }, { role: "assistant", content: "Hello Bob! How can I assist you today?" }, { role: "user", content: "What's my name?" },]); AIMessage { "id": "chatcmpl-AekDyJdj6y9IREyNIf3tkKGRKhN1Z", "content": "Your name is Bob! How can I help you today, Bob?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 33, "completionTokens": 14, "totalTokens": 47 }, "finish_reason": "stop", "usage": { "prompt_tokens": 33, "completion_tokens": 14, "total_tokens": 47, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 14, "input_tokens": 33, "total_tokens": 47, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} And now we can see that we get a good response! This is the basic idea underpinning a chatbotโ€™s ability to interact conversationally. So how do we best implement this? Message persistence[โ€‹](#message-persistence "Direct link to Message persistence") ---------------------------------------------------------------------------------- [LangGraph](https://langchain-ai.github.io/langgraphjs/) implements a built-in persistence layer, making it ideal for chat applications that support multiple conversational turns. Wrapping our chat model in a minimal LangGraph application allows us to automatically persist the message history, simplifying the development of multi-turn applications. LangGraph comes with a simple in-memory checkpointer, which we use below. import { START, END, MessagesAnnotation, StateGraph, MemorySaver,} from "@langchain/langgraph";// Define the function that calls the modelconst callModel = async (state: typeof MessagesAnnotation.State) => { const response = await llm.invoke(state.messages); return { messages: response };};// Define a new graphconst workflow = new StateGraph(MessagesAnnotation) // Define the node and edge .addNode("model", callModel) .addEdge(START, "model") .addEdge("model", END);// Add memoryconst memory = new MemorySaver();const app = workflow.compile({ checkpointer: memory }); We now need to create a `config` that we pass into the runnable every time. This config contains information that is not part of the input directly, but is still useful. In this case, we want to include a `thread_id`. This should look like: import { v4 as uuidv4 } from "uuid";const config = { configurable: { thread_id: uuidv4() } }; This enables us to support multiple conversation threads with a single application, a common requirement when your application has multiple users. We can then invoke the application: const input = [ { role: "user", content: "Hi! I'm Bob.", },];const output = await app.invoke({ messages: input }, config);// The output contains all messages in the state.// This will long the last message in the conversation.console.log(output.messages[output.messages.length - 1]); AIMessage { "id": "chatcmpl-AekEFPclmrO7YfAe7J0zUAanS4ifx", "content": "Hi Bob! How can I assist you today?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 12, "completionTokens": 10, "totalTokens": 22 }, "finish_reason": "stop", "usage": { "prompt_tokens": 12, "completion_tokens": 10, "total_tokens": 22, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 10, "input_tokens": 12, "total_tokens": 22, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} const input2 = [ { role: "user", content: "What's my name?", },];const output2 = await app.invoke({ messages: input2 }, config);console.log(output2.messages[output2.messages.length - 1]); AIMessage { "id": "chatcmpl-AekEJgCfLodGCcuLgLQdJevH7CpCJ", "content": "Your name is Bob! How can I help you today, Bob?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 34, "completionTokens": 14, "totalTokens": 48 }, "finish_reason": "stop", "usage": { "prompt_tokens": 34, "completion_tokens": 14, "total_tokens": 48, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 14, "input_tokens": 34, "total_tokens": 48, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} Great! Our chatbot now remembers things about us. If we change the config to reference a different `thread_id`, we can see that it starts the conversation fresh. const config2 = { configurable: { thread_id: uuidv4() } };const input3 = [ { role: "user", content: "What's my name?", },];const output3 = await app.invoke({ messages: input3 }, config2);console.log(output3.messages[output3.messages.length - 1]); AIMessage { "id": "chatcmpl-AekELvPXLtjOKgLN63mQzZwvyo12J", "content": "I'm sorry, but I don't have access to personal information about individuals unless you share it with me. How can I assist you today?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 11, "completionTokens": 27, "totalTokens": 38 }, "finish_reason": "stop", "usage": { "prompt_tokens": 11, "completion_tokens": 27, "total_tokens": 38, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_39a40c96a0" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 27, "input_tokens": 11, "total_tokens": 38, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} However, we can always go back to the original conversation (since we are persisting it in a database) const output4 = await app.invoke({ messages: input2 }, config);console.log(output4.messages[output4.messages.length - 1]); AIMessage { "id": "chatcmpl-AekEQ8Z5JmYquSfzPsCWv1BDTKZSh", "content": "Your name is Bob. Is there something specific you would like to talk about?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 60, "completionTokens": 16, "totalTokens": 76 }, "finish_reason": "stop", "usage": { "prompt_tokens": 60, "completion_tokens": 16, "total_tokens": 76, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_39a40c96a0" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 16, "input_tokens": 60, "total_tokens": 76, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} This is how we can support a chatbot having conversations with many users! Right now, all weโ€™ve done is add a simple persistence layer around the model. We can start to make the more complicated and personalized by adding in a prompt template. Prompt templates[โ€‹](#prompt-templates "Direct link to Prompt templates") ------------------------------------------------------------------------- Prompt Templates help to turn raw user information into a format that the LLM can work with. In this case, the raw user input is just a message, which we are passing to the LLM. Letโ€™s now make that a bit more complicated. First, letโ€™s add in a system message with some custom instructions (but still taking messages as input). Next, weโ€™ll add in more input besides just the messages. To add in a system message, we will create a `ChatPromptTemplate`. We will utilize `MessagesPlaceholder` to pass all the messages in. import { ChatPromptTemplate } from "@langchain/core/prompts";const promptTemplate = ChatPromptTemplate.fromMessages([ [ "system", "You talk like a pirate. Answer all questions to the best of your ability.", ], ["placeholder", "{messages}"],]); We can now update our application to incorporate this template: import { START, END, MessagesAnnotation, StateGraph, MemorySaver,} from "@langchain/langgraph";// Define the function that calls the modelconst callModel2 = async (state: typeof MessagesAnnotation.State) => { const prompt = await promptTemplate.invoke(state); const response = await llm.invoke(prompt); // Update message history with response: return { messages: [response] };};// Define a new graphconst workflow2 = new StateGraph(MessagesAnnotation) // Define the (single) node in the graph .addNode("model", callModel2) .addEdge(START, "model") .addEdge("model", END);// Add memoryconst app2 = workflow2.compile({ checkpointer: new MemorySaver() }); We invoke the application in the same way: const config3 = { configurable: { thread_id: uuidv4() } };const input4 = [ { role: "user", content: "Hi! I'm Jim.", },];const output5 = await app2.invoke({ messages: input4 }, config3);console.log(output5.messages[output5.messages.length - 1]); AIMessage { "id": "chatcmpl-AekEYAQVqh9OFZRGdzGiPz33WPf1v", "content": "Ahoy, Jim! A pleasure to meet ye, matey! What be on yer mind this fine day?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 32, "completionTokens": 23, "totalTokens": 55 }, "finish_reason": "stop", "usage": { "prompt_tokens": 32, "completion_tokens": 23, "total_tokens": 55, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_39a40c96a0" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 23, "input_tokens": 32, "total_tokens": 55, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} const input5 = [ { role: "user", content: "What is my name?", },];const output6 = await app2.invoke({ messages: input5 }, config3);console.log(output6.messages[output6.messages.length - 1]); AIMessage { "id": "chatcmpl-AekEbrpFI3K8BxemHZ5fG4xF2tT8x", "content": "Ye be callin' yerself Jim, if I heard ye right, savvy? What else can I do fer ye, me hearty?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 68, "completionTokens": 29, "totalTokens": 97 }, "finish_reason": "stop", "usage": { "prompt_tokens": 68, "completion_tokens": 29, "total_tokens": 97, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 29, "input_tokens": 68, "total_tokens": 97, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} Awesome! Letโ€™s now make our prompt a little bit more complicated. Letโ€™s assume that the prompt template now looks something like this: const promptTemplate2 = ChatPromptTemplate.fromMessages([ [ "system", "You are a helpful assistant. Answer all questions to the best of your ability in {language}.", ], ["placeholder", "{messages}"],]); Note that we have added a new `language` input to the prompt. Our application now has two parametersโ€“ the input `messages` and `language`. We should update our applicationโ€™s state to reflect this: import { START, END, StateGraph, MemorySaver, MessagesAnnotation, Annotation,} from "@langchain/langgraph";// Define the Stateconst GraphAnnotation = Annotation.Root({ ...MessagesAnnotation.spec, language: Annotation(),});// Define the function that calls the modelconst callModel3 = async (state: typeof GraphAnnotation.State) => { const prompt = await promptTemplate2.invoke(state); const response = await llm.invoke(prompt); return { messages: [response] };};const workflow3 = new StateGraph(GraphAnnotation) .addNode("model", callModel3) .addEdge(START, "model") .addEdge("model", END);const app3 = workflow3.compile({ checkpointer: new MemorySaver() }); const config4 = { configurable: { thread_id: uuidv4() } };const input6 = { messages: [ { role: "user", content: "Hi im bob", }, ], language: "Spanish",};const output7 = await app3.invoke(input6, config4);console.log(output7.messages[output7.messages.length - 1]); AIMessage { "id": "chatcmpl-AekF4R7ioefFo6PmOYo3YuCbGpROq", "content": "ยกHola, Bob! ยฟCรณmo puedo ayudarte hoy?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 32, "completionTokens": 11, "totalTokens": 43 }, "finish_reason": "stop", "usage": { "prompt_tokens": 32, "completion_tokens": 11, "total_tokens": 43, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_39a40c96a0" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 11, "input_tokens": 32, "total_tokens": 43, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} Note that the entire state is persisted, so we can omit parameters like `language` if no changes are desired: const input7 = { messages: [ { role: "user", content: "What is my name?", }, ],};const output8 = await app3.invoke(input7, config4);console.log(output8.messages[output8.messages.length - 1]); AIMessage { "id": "chatcmpl-AekF8yN7H81ITccWlBzSahmduP69T", "content": "Tu nombre es Bob. ยฟEn quรฉ puedo ayudarte, Bob?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 56, "completionTokens": 13, "totalTokens": 69 }, "finish_reason": "stop", "usage": { "prompt_tokens": 56, "completion_tokens": 13, "total_tokens": 69, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 13, "input_tokens": 56, "total_tokens": 69, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} To help you understand whatโ€™s happening internally, check out [this LangSmith trace](https://smith.langchain.com/public/d61630b7-6a52-4dc9-974c-8452008c498a/r) . Managing Conversation History[โ€‹](#managing-conversation-history "Direct link to Managing Conversation History") ---------------------------------------------------------------------------------------------------------------- One important concept to understand when building chatbots is how to manage conversation history. If left unmanaged, the list of messages will grow unbounded and potentially overflow the context window of the LLM. Therefore, it is important to add a step that limits the size of the messages you are passing in. **Importantly, you will want to do this BEFORE the prompt template but AFTER you load previous messages from Message History.** We can do this by adding a simple step in front of the prompt that modifies the `messages` key appropriately, and then wrap that new chain in the Message History class. LangChain comes with a few built-in helpers for [managing a list of messages](/docs/how_to/#messages) . In this case weโ€™ll use the [trimMessages](/docs/how_to/trim_messages/) helper to reduce how many messages weโ€™re sending to the model. The trimmer allows us to specify how many tokens we want to keep, along with other parameters like if we want to always keep the system message and whether to allow partial messages: import { SystemMessage, HumanMessage, AIMessage, trimMessages,} from "@langchain/core/messages";const trimmer = trimMessages({ maxTokens: 10, strategy: "last", tokenCounter: (msgs) => msgs.length, includeSystem: true, allowPartial: false, startOn: "human",});const messages = [ new SystemMessage("you're a good assistant"), new HumanMessage("hi! I'm bob"), new AIMessage("hi!"), new HumanMessage("I like vanilla ice cream"), new AIMessage("nice"), new HumanMessage("whats 2 + 2"), new AIMessage("4"), new HumanMessage("thanks"), new AIMessage("no problem!"), new HumanMessage("having fun?"), new AIMessage("yes!"),];await trimmer.invoke(messages); [ SystemMessage { "content": "you're a good assistant", "additional_kwargs": {}, "response_metadata": {} }, HumanMessage { "content": "I like vanilla ice cream", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "content": "nice", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "content": "whats 2 + 2", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "content": "4", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "content": "thanks", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "content": "no problem!", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "content": "having fun?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "content": "yes!", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }] To use it in our chain, we just need to run the trimmer before we pass the `messages` input to our prompt. const callModel4 = async (state: typeof GraphAnnotation.State) => { const trimmedMessage = await trimmer.invoke(state.messages); const prompt = await promptTemplate2.invoke({ messages: trimmedMessage, language: state.language, }); const response = await llm.invoke(prompt); return { messages: [response] };};const workflow4 = new StateGraph(GraphAnnotation) .addNode("model", callModel4) .addEdge(START, "model") .addEdge("model", END);const app4 = workflow4.compile({ checkpointer: new MemorySaver() }); Now if we try asking the model our name, it wonโ€™t know it since we trimmed that part of the chat history: const config5 = { configurable: { thread_id: uuidv4() } };const input8 = { messages: [...messages, new HumanMessage("What is my name?")], language: "English",};const output9 = await app4.invoke(input8, config5);console.log(output9.messages[output9.messages.length - 1]); AIMessage { "id": "chatcmpl-AekHyVN7f0Pnuyc2RHVL8CxKmFfMQ", "content": "I don't know your name. You haven't shared it yet!", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 97, "completionTokens": 12, "totalTokens": 109 }, "finish_reason": "stop", "usage": { "prompt_tokens": 97, "completion_tokens": 12, "total_tokens": 109, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 12, "input_tokens": 97, "total_tokens": 109, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} But if we ask about information that is within the last few messages, it remembers: const config6 = { configurable: { thread_id: uuidv4() } };const input9 = { messages: [...messages, new HumanMessage("What math problem did I ask?")], language: "English",};const output10 = await app4.invoke(input9, config6);console.log(output10.messages[output10.messages.length - 1]); AIMessage { "id": "chatcmpl-AekI1jwlErzHuZ3BhAxr97Ct818Pp", "content": "You asked what 2 + 2 equals.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "promptTokens": 99, "completionTokens": 10, "totalTokens": 109 }, "finish_reason": "stop", "usage": { "prompt_tokens": 99, "completion_tokens": 10, "total_tokens": 109, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_6fc10e10eb" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "output_tokens": 10, "input_tokens": 99, "total_tokens": 109, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } }} If you take a look at LangSmith, you can see exactly what is happening under the hood in the [LangSmith trace](https://smith.langchain.com/public/ac63745d-8429-4ae5-8c11-9ec79d9632f2/r) . Next Steps[โ€‹](#next-steps "Direct link to Next Steps") ------------------------------------------------------- Now that you understand the basics of how to create a chatbot in LangChain, some more advanced tutorials you may be interested in are: * [Conversational RAG](/docs/tutorials/qa_chat_history) : Enable a chatbot experience over an external source of data * [Agents](https://langchain-ai.github.io/langgraphjs/tutorials/multi_agent/agent_supervisor/) : Build a chatbot that can take actions If you want to dive deeper on specifics, some things worth checking out are: * [Streaming](/docs/how_to/streaming) : streaming is _crucial_ for chat applications * [How to add message history](/docs/how_to/message_history) : for a deeper dive into all things related to message history * [How to manage large message history](/docs/how_to/trim_messages/) : more techniques for managing a large chat history * [LangGraph main docs](https://langchain-ai.github.io/langgraphjs/) : for more detail on building with LangGraph * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Overview](#overview) * [Setup](#setup) * [Jupyter Notebook](#jupyter-notebook) * [Installation](#installation) * [LangSmith](#langsmith) * [Quickstart](#quickstart) * [Message persistence](#message-persistence) * [Prompt templates](#prompt-templates) * [Managing Conversation History](#managing-conversation-history) * [Next Steps](#next-steps) --- # Tagging | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Classify Text into Labels ========================= Tagging means labeling a document with classes such as: * sentiment * language * style (formal, informal etc.) * covered topics * political tendency ![Image description](/assets/images/tagging-93990e95451d92b715c2b47066384224.png) Overview[โ€‹](#overview "Direct link to Overview") ------------------------------------------------- Tagging has a few components: * `function`: Like [extraction](/docs/tutorials/extraction) , tagging uses [functions](https://openai.com/blog/function-calling-and-other-api-updates) to specify how the model should tag a document * `schema`: defines how we want to tag the document Quickstart[โ€‹](#quickstart "Direct link to Quickstart") ------------------------------------------------------- Letโ€™s see a very straightforward example of how we can use tool calling for tagging in LangChain. Weโ€™ll use the `.withStructuredOutput()`, supported on [selected chat models](/docs/integrations/chat/) . ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); Letโ€™s specify a [Zod](https://zod.dev) schema with a few properties and their expected type in our schema. import { ChatPromptTemplate } from "@langchain/core/prompts";import { z } from "zod";const taggingPrompt = ChatPromptTemplate.fromTemplate( `Extract the desired information from the following passage.Only extract the properties mentioned in the 'Classification' function.Passage:{input}`);const classificationSchema = z.object({ sentiment: z.string().describe("The sentiment of the text"), aggressiveness: z .number() .int() .min(1) .max(10) .describe("How aggressive the text is on a scale from 1 to 10"), language: z.string().describe("The language the text is written in"),});// Name is optional, but gives the models more clues as to what your schema representsconst llmWihStructuredOutput = llm.withStructuredOutput(classificationSchema, { name: "extractor",}); const prompt1 = await taggingPrompt.invoke({ input: "Estoy increiblemente contento de haberte conocido! Creo que seremos muy buenos amigos!",});await llmWihStructuredOutput.invoke(prompt1); { sentiment: 'contento', aggressiveness: 1, language: 'es' } As we can see in the example, it correctly interprets what we want. The results vary so that we may get, for example, sentiments in different languages (โ€˜positiveโ€™, โ€˜enojadoโ€™ etc.). We will see how to control these results in the next section. Finer control[โ€‹](#finer-control "Direct link to Finer control") ---------------------------------------------------------------- Careful schema definition gives us more control over the modelโ€™s output. Specifically, we can define: * possible values for each property * description to make sure that the model understands the property * required properties to be returned Letโ€™s redeclare our Zod schema to control for each of the previously mentioned aspects using enums: import { z } from "zod";const classificationSchema2 = z.object({ sentiment: z .enum(["happy", "neutral", "sad"]) .describe("The sentiment of the text"), aggressiveness: z .number() .int() .min(1) .max(5) .describe( "describes how aggressive the statement is, the higher the number the more aggressive" ), language: z .enum(["spanish", "english", "french", "german", "italian"]) .describe("The language the text is written in"),}); const taggingPrompt2 = ChatPromptTemplate.fromTemplate( `Extract the desired information from the following passage.Only extract the properties mentioned in the 'Classification' function.Passage:{input}`);const llmWihStructuredOutput2 = llm.withStructuredOutput( classificationSchema2, { name: "extractor" }); Now the answers will be restricted in a way we expect! const prompt2 = await taggingPrompt2.invoke({ input: "Estoy increiblemente contento de haberte conocido! Creo que seremos muy buenos amigos!",});await llmWihStructuredOutput2.invoke(prompt2); { sentiment: 'happy', aggressiveness: 1, language: 'spanish' } const prompt3 = await taggingPrompt2.invoke({ input: "Estoy muy enojado con vos! Te voy a dar tu merecido!",});await llmWihStructuredOutput2.invoke(prompt3); { sentiment: 'sad', aggressiveness: 5, language: 'spanish' } const prompt4 = await taggingPrompt2.invoke({ input: "Weather is ok here, I can go outside without much more than a coat",});await llmWihStructuredOutput2.invoke(prompt4); { sentiment: 'neutral', aggressiveness: 1, language: 'english' } The [LangSmith trace](https://smith.langchain.com/public/455f5404-8784-49ce-8851-0619b99e936f/r) lets us peek under the hood: ![](/assets/images/classification_ls_trace-7b269b067c3751c6d06289c560505656.png) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Overview](#overview) * [Quickstart](#quickstart) * [Finer control](#finer-control) --- # How to use example selectors | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Prerequisites This guide assumes familiarity with the following concepts: * [Prompt templates](/docs/concepts/prompt_templates) * [Few-shot examples](/docs/how_to/few_shot_examples) If you have a large number of examples, you may need to select which ones to include in the prompt. The Example Selector is the class responsible for doing so. The base interface is defined as below: class BaseExampleSelector { addExample(example: Example): Promise; selectExamples(input_variables: Example): Promise;} The only method it needs to define is a `selectExamples` method. This takes in the input variables and then returns a list of examples. It is up to each specific implementation as to how those examples are selected. LangChain has a few different types of example selectors. For an overview of all these types, see the below table. In this guide, we will walk through creating a custom example selector. Examples[โ€‹](#examples "Direct link to Examples") ------------------------------------------------- In order to use an example selector, we need to create a list of examples. These should generally be example inputs and outputs. For this demo purpose, letโ€™s imagine we are selecting examples of how to translate English to Italian. const examples = [ { input: "hi", output: "ciao" }, { input: "bye", output: "arrivaderci" }, { input: "soccer", output: "calcio" },]; Custom Example Selector[โ€‹](#custom-example-selector "Direct link to Custom Example Selector") ---------------------------------------------------------------------------------------------- Letโ€™s write an example selector that chooses what example to pick based on the length of the word. import { BaseExampleSelector } from "@langchain/core/example_selectors";import { Example } from "@langchain/core/prompts";class CustomExampleSelector extends BaseExampleSelector { private examples: Example[]; constructor(examples: Example[]) { super(); this.examples = examples; } async addExample(example: Example): Promise { this.examples.push(example); return; } async selectExamples(inputVariables: Example): Promise { // This assumes knowledge that part of the input will be a 'text' key const newWord = inputVariables.input; const newWordLength = newWord.length; // Initialize variables to store the best match and its length difference let bestMatch: Example | null = null; let smallestDiff = Infinity; // Iterate through each example for (const example of this.examples) { // Calculate the length difference with the first word of the example const currentDiff = Math.abs(example.input.length - newWordLength); // Update the best match if the current one is closer in length if (currentDiff < smallestDiff) { smallestDiff = currentDiff; bestMatch = example; } } return bestMatch ? [bestMatch] : []; }} const exampleSelector = new CustomExampleSelector(examples); await exampleSelector.selectExamples({ input: "okay" }); [ { input: "bye", output: "arrivaderci" } ] await exampleSelector.addExample({ input: "hand", output: "mano" }); await exampleSelector.selectExamples({ input: "okay" }); [ { input: "hand", output: "mano" } ] Use in a Prompt[โ€‹](#use-in-a-prompt "Direct link to Use in a Prompt") ---------------------------------------------------------------------- We can now use this example selector in a prompt import { PromptTemplate, FewShotPromptTemplate } from "@langchain/core/prompts";const examplePrompt = PromptTemplate.fromTemplate( "Input: {input} -> Output: {output}"); const prompt = new FewShotPromptTemplate({ exampleSelector, examplePrompt, suffix: "Input: {input} -> Output:", prefix: "Translate the following words from English to Italian:", inputVariables: ["input"],});console.log(await prompt.format({ input: "word" })); Translate the following words from English to Italian:Input: hand -> Output: manoInput: word -> Output: Example Selector Types[โ€‹](#example-selector-types "Direct link to Example Selector Types") ------------------------------------------------------------------------------------------- | Name | Description | | --- | --- | | Similarity | Uses semantic similarity between inputs and examples to decide which examples to choose. | | Length | Selects examples based on how many can fit within a certain length | Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Youโ€™ve now learned a bit about using example selectors to few shot LLMs. Next, check out some guides on some other techniques for selecting examples: * [How to select examples by length](/docs/how_to/example_selectors_length_based) * [How to select examples by similarity](/docs/how_to/example_selectors_similarity) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Examples](#examples) * [Custom Example Selector](#custom-example-selector) * [Use in a Prompt](#use-in-a-prompt) * [Example Selector Types](#example-selector-types) * [Next steps](#next-steps) --- # Build a Retrieval Augmented Generation (RAG) App: Part 2 | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page In many Q&A applications we want to allow the user to have a back-and-forth conversation, meaning the application needs some sort of โ€œmemoryโ€ of past questions and answers, and some logic for incorporating those into its current thinking. This is a the second part of a multi-part tutorial: * [Part 1](/docs/tutorials/rag) introduces RAG and walks through a minimal implementation. * [Part 2](/docs/tutorials/qa_chat_history) (this guide) extends the implementation to accommodate conversation-style interactions and multi-step retrieval processes. Here we focus on **adding logic for incorporating historical messages.** This involves the management of a [chat history](/docs/concepts/chat_history) . We will cover two approaches: 1. [Chains](/docs/tutorials/qa_chat_history/#chains) , in which we execute at most one retrieval step; 2. [Agents](/docs/tutorials/qa_chat_history/#agents) , in which we give an LLM discretion to execute multiple retrieval steps. note The methods presented here leverage [tool-calling](/docs/concepts/tool_calling/) capabilities in modern [chat models](/docs/concepts/chat_models) . See [this page](/docs/integrations/chat/) for a table of models supporting tool calling features. For the external knowledge source, we will use the same [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng from the [Part 1](/docs/tutorials/rag) of the RAG tutorial. Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Components[โ€‹](#components "Direct link to Components") We will need to select three components from LangChainโ€™s suite of integrations. A [chat model](/docs/integrations/chat/) : ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); An [embedding model](/docs/integrations/text_embedding/) : ### Pick your embedding model: * OpenAI * Azure * AWS * VertexAI * MistralAI * Cohere #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai OPENAI_API_KEY=your-api-key import { OpenAIEmbeddings } from "@langchain/openai";const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-large"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai AZURE_OPENAI_API_INSTANCE_NAME=AZURE_OPENAI_API_KEY=AZURE_OPENAI_API_VERSION="2024-02-01" import { AzureOpenAIEmbeddings } from "@langchain/openai";const embeddings = new AzureOpenAIEmbeddings({ azureOpenAIApiEmbeddingsDeploymentName: "text-embedding-ada-002"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/aws yarn add @langchain/aws pnpm add @langchain/aws BEDROCK_AWS_REGION=your-region import { BedrockEmbeddings } from "@langchain/aws";const embeddings = new BedrockEmbeddings({ model: "amazon.titan-embed-text-v1"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai GOOGLE_APPLICATION_CREDENTIALS=credentials.json import { VertexAIEmbeddings } from "@langchain/google-vertexai";const embeddings = new VertexAIEmbeddings({ model: "text-embedding-004"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai MISTRAL_API_KEY=your-api-key import { MistralAIEmbeddings } from "@langchain/mistralai";const embeddings = new MistralAIEmbeddings({ model: "mistral-embed"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/cohere yarn add @langchain/cohere pnpm add @langchain/cohere COHERE_API_KEY=your-api-key import { CohereEmbeddings } from "@langchain/cohere";const embeddings = new CohereEmbeddings({ model: "embed-english-v3.0"}); And a [vector store](/docs/integrations/vectorstores/) : ### Pick your vector store: * Memory * Chroma * FAISS * MongoDB * PGVector * Pinecone * Qdrant #### Install dependencies * npm * yarn * pnpm npm i langchain yarn add langchain pnpm add langchain import { MemoryVectorStore } from "langchain/vectorstores/memory";const vectorStore = new MemoryVectorStore(embeddings); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { Chroma } from "@langchain/community/vectorstores/chroma";const vectorStore = new Chroma(embeddings, { collectionName: "a-test-collection",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { FaissStore } from "@langchain/community/vectorstores/faiss";const vectorStore = new FaissStore(embeddings, {}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mongodb yarn add @langchain/mongodb pnpm add @langchain/mongodb import { MongoDBAtlasVectorSearch } from "@langchain/mongodb"import { MongoClient } from "mongodb";const client = new MongoClient(process.env.MONGODB_ATLAS_URI || "");const collection = client .db(process.env.MONGODB_ATLAS_DB_NAME) .collection(process.env.MONGODB_ATLAS_COLLECTION_NAME);const vectorStore = new MongoDBAtlasVectorSearch(embeddings, { collection: collection, indexName: "vector_index", textKey: "text", embeddingKey: "embedding",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import PGVectorStore from "@langchain/community/vectorstores/pgvector";const vectorStore = await PGVectorStore.initialize(embeddings, {}) #### Install dependencies * npm * yarn * pnpm npm i @langchain/pinecone yarn add @langchain/pinecone pnpm add @langchain/pinecone import { PineconeStore } from "@langchain/pinecone";import { Pinecone as PineconeClient } from "@pinecone-database/pinecone";const pinecone = new PineconeClient();const vectorStore = new PineconeStore(embeddings, { pineconeIndex, maxConcurrency: 5,}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/qdrant yarn add @langchain/qdrant pnpm add @langchain/qdrant import { QdrantVectorStore } from "@langchain/qdrant";const vectorStore = await QdrantVectorStore.fromExistingCollection(embeddings, { url: process.env.QDRANT_URL, collectionName: "langchainjs-testing",}); ### Dependencies[โ€‹](#dependencies "Direct link to Dependencies") In addition, weโ€™ll use the following packages: * npm * yarn * pnpm npm i langchain @langchain/community @langchain/langgraph cheerio yarn add langchain @langchain/community @langchain/langgraph cheerio pnpm add langchain @langchain/community @langchain/langgraph cheerio ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://docs.smith.langchain.com) . Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces: export LANGSMITH_TRACING=trueexport LANGSMITH_API_KEY=YOUR_KEY# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true Chains[โ€‹](#chains "Direct link to Chains") ------------------------------------------- Letโ€™s first revisit the vector store we built in [Part 1](/docs/tutorials/rag) , which indexes an [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng. import "cheerio";import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";// Load and chunk contents of the blogconst pTagSelector = "p";const cheerioLoader = new CheerioWebBaseLoader( "https://lilianweng.github.io/posts/2023-06-23-agent/", { selector: pTagSelector, });const docs = await cheerioLoader.load();const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200,});const allSplits = await splitter.splitDocuments(docs); // Index chunksawait vectorStore.addDocuments(allSplits); In the [Part 1](/docs/tutorials/rag) of the RAG tutorial, we represented the user input, retrieved context, and generated answer as separate keys in the state. Conversational experiences can be naturally represented using a sequence of [messages](/docs/concepts/messages/) . In addition to messages from the user and assistant, retrieved documents and other artifacts can be incorporated into a message sequence via [tool messages](/docs/concepts/messages/#toolmessage) . This motivates us to represent the state of our RAG application using a sequence of messages. Specifically, we will have 1. User input as a `HumanMessage`; 2. Vector store query as an `AIMessage` with tool calls; 3. Retrieved documents as a `ToolMessage`; 4. Final response as a `AIMessage`. This model for state is so versatile that LangGraph offers a built-in version for convenience: import { MessagesAnnotation, StateGraph } from "@langchain/langgraph";const graph = new StateGraph(MessagesAnnotation); Leveraging [tool-calling](/docs/concepts/tool_calling/) to interact with a retrieval step has another benefit, which is that the query for the retrieval is generated by our model. This is especially important in a conversational setting, where user queries may require contextualization based on the chat history. For instance, consider the following exchange: > Human: โ€œWhat is Task Decomposition?โ€ > > AI: โ€œTask decomposition involves breaking down complex tasks into smaller and simpler steps to make them more manageable for an agent or model.โ€ > > Human: โ€œWhat are common ways of doing it?โ€ In this scenario, a model could generate a query such as `"common approaches to task decomposition"`. Tool-calling facilitates this naturally. As in the [query analysis](/docs/tutorials/rag#query-analysis) section of the RAG tutorial, this allows a model to rewrite user queries into more effective search queries. It also provides support for direct responses that do not involve a retrieval step (e.g., in response to a generic greeting from the user). Letโ€™s turn our retrieval step into a [tool](/docs/concepts/tools) : import { z } from "zod";import { tool } from "@langchain/core/tools";const retrieveSchema = z.object({ query: z.string() });const retrieve = tool( async ({ query }) => { const retrievedDocs = await vectorStore.similaritySearch(query, 2); const serialized = retrievedDocs .map( (doc) => `Source: ${doc.metadata.source}\nContent: ${doc.pageContent}` ) .join("\n"); return [serialized, retrievedDocs]; }, { name: "retrieve", description: "Retrieve information related to a query.", schema: retrieveSchema, responseFormat: "content_and_artifact", }); See [this guide](/docs/how_to/custom_tools/) for more detail on creating tools. Our graph will consist of three nodes: 1. A node that fields the user input, either generating a query for the retriever or responding directly; 2. A node for the retriever tool that executes the retrieval step; 3. A node that generates the final response using the retrieved context. We build them below. Note that we leverage another pre-built LangGraph component, [ToolNode](https://langchain-ai.github.io/langgraph/reference/prebuilt/#langgraph.prebuilt.tool_node.ToolNode) , that executes the tool and adds the result as a `ToolMessage` to the state. import { AIMessage, HumanMessage, SystemMessage, ToolMessage,} from "@langchain/core/messages";import { MessagesAnnotation } from "@langchain/langgraph";import { ToolNode } from "@langchain/langgraph/prebuilt";// Step 1: Generate an AIMessage that may include a tool-call to be sent.async function queryOrRespond(state: typeof MessagesAnnotation.State) { const llmWithTools = llm.bindTools([retrieve]); const response = await llmWithTools.invoke(state.messages); // MessagesState appends messages to state instead of overwriting return { messages: [response] };}// Step 2: Execute the retrieval.const tools = new ToolNode([retrieve]);// Step 3: Generate a response using the retrieved content.async function generate(state: typeof MessagesAnnotation.State) { // Get generated ToolMessages let recentToolMessages = []; for (let i = state["messages"].length - 1; i >= 0; i--) { let message = state["messages"][i]; if (message instanceof ToolMessage) { recentToolMessages.push(message); } else { break; } } let toolMessages = recentToolMessages.reverse(); // Format into prompt const docsContent = toolMessages.map((doc) => doc.content).join("\n"); const systemMessageContent = "You are an assistant for question-answering tasks. " + "Use the following pieces of retrieved context to answer " + "the question. If you don't know the answer, say that you " + "don't know. Use three sentences maximum and keep the " + "answer concise." + "\n\n" + `${docsContent}`; const conversationMessages = state.messages.filter( (message) => message instanceof HumanMessage || message instanceof SystemMessage || (message instanceof AIMessage && message.tool_calls.length == 0) ); const prompt = [ new SystemMessage(systemMessageContent), ...conversationMessages, ]; // Run const response = await llm.invoke(prompt); return { messages: [response] };} Finally, we compile our application into a single `graph` object. In this case, we are just connecting the steps into a sequence. We also allow the first `query_or_respond` step to โ€œshort-circuitโ€ and respond directly to the user if it does not generate a tool call. This allows our application to support conversational experiencesโ€“ e.g., responding to generic greetings that may not require a retrieval step import { StateGraph } from "@langchain/langgraph";import { toolsCondition } from "@langchain/langgraph/prebuilt";const graphBuilder = new StateGraph(MessagesAnnotation) .addNode("queryOrRespond", queryOrRespond) .addNode("tools", tools) .addNode("generate", generate) .addEdge("__start__", "queryOrRespond") .addConditionalEdges("queryOrRespond", toolsCondition, { __end__: "__end__", tools: "tools", }) .addEdge("tools", "generate") .addEdge("generate", "__end__");const graph = graphBuilder.compile(); // Note: tslab only works inside a jupyter notebook. Don't worry about running this code yourself!import * as tslab from "tslab";const image = await graph.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_rag_part_2](/assets/images/graph_img_rag_part_2-5fa94e3b895cf0874ec635d926d80f0c.png) Letโ€™s test our application. Expand for \`prettyPrint\` code. import { BaseMessage, isAIMessage } from "@langchain/core/messages";const prettyPrint = (message: BaseMessage) => { let txt = `[${message._getType()}]: ${message.content}`; if ((isAIMessage(message) && message.tool_calls?.length) || 0 > 0) { const tool_calls = (message as AIMessage)?.tool_calls ?.map((tc) => `- ${tc.name}(${JSON.stringify(tc.args)})`) .join("\n"); txt += ` \nTools: \n${tool_calls}`; } console.log(txt);}; Note that it responds appropriately to messages that do not require an additional retrieval step: let inputs1 = { messages: [{ role: "user", content: "Hello" }] };for await (const step of await graph.stream(inputs1, { streamMode: "values",})) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: Hello-----[ai]: Hello! How can I assist you today?----- And when executing a search, we can stream the steps to observe the query generation, retrieval, and answer generation: let inputs2 = { messages: [{ role: "user", content: "What is Task Decomposition?" }],};for await (const step of await graph.stream(inputs2, { streamMode: "values",})) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: What is Task Decomposition?-----[ai]:Tools:- retrieve({"query":"Task Decomposition"})-----[tool]: Source: https://lilianweng.github.io/posts/2023-06-23-agent/Content: hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning DomainSource: https://lilianweng.github.io/posts/2023-06-23-agent/Content: System message:Think step by step and reason yourself to the right decisions to make sure we get it right.You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose.Then you will output the content of each file including ALL code.Each file must strictly follow a markdown code block format, where the following tokens must be replaced such thatFILENAME is the lowercase file name including the file extension,LANG is the markup code block language for the codeโ€™s language, and CODE is the code:FILENAMEYou will start with the โ€œentrypointโ€ file, then go to the ones that are imported by that file, and so on.Please note that the code should be fully functional. No placeholders.Follow a language and framework appropriate best practice file naming convention.Make sure that files contain all imports, types etc. Make sure that code in different files are compatible with each other.-----[ai]: Task decomposition is the process of breaking down a complex task into smaller, more manageable steps or subgoals. This can be achieved through various methods, such as using prompts for large language models (LLMs), task-specific instructions, or human inputs. It helps in simplifying the problem-solving process and enhances understanding of the task at hand.----- Check out the LangSmith trace [here](https://smith.langchain.com/public/c6ed4e16-b9ed-46cc-912e-6a580d3c47ed/r) . ### Stateful management of chat history[โ€‹](#stateful-management-of-chat-history "Direct link to Stateful management of chat history") note This section of the tutorial previously used the [RunnableWithMessageHistory](https://api.js.langchain.com/classes/_langchain_core.runnables.RunnableWithMessageHistory.html) abstraction. You can access that version of the documentation in the [v0.2 docs](https://js.langchain.com/v0.2/docs/tutorials/qa_chat_history) . As of the v0.3 release of LangChain, we recommend that LangChain users take advantage of [LangGraph persistence](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) to incorporate `memory` into new LangChain applications. If your code is already relying on `RunnableWithMessageHistory` or `BaseChatMessageHistory`, you do **not** need to make any changes. We do not plan on deprecating this functionality in the near future as it works for simple chat applications and any code that uses `RunnableWithMessageHistory` will continue to work as expected. Please see [How to migrate to LangGraph Memory](/docs/versions/migrating_memory/) for more details. In production, the Q&A application will usually persist the chat history into a database, and be able to read and update it appropriately. [LangGraph](https://langchain-ai.github.io/langgraphjs/) implements a built-in [persistence layer](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) , making it ideal for chat applications that support multiple conversational turns. To manage multiple conversational turns and threads, all we have to do is specify a [checkpointer](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) when compiling our application. Because the nodes in our graph are appending messages to the state, we will retain a consistent chat history across invocations. LangGraph comes with a simple in-memory checkpointer, which we use below. See its [documentation](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) for more detail, including how to use different persistence backends (e.g., SQLite or Postgres). For a detailed walkthrough of how to manage message history, head to the [How to add message history (memory)](/docs/how_to/message_history) guide. import { MemorySaver } from "@langchain/langgraph";const checkpointer = new MemorySaver();const graphWithMemory = graphBuilder.compile({ checkpointer });// Specify an ID for the threadconst threadConfig = { configurable: { thread_id: "abc123" }, streamMode: "values" as const,}; We can now invoke similar to before: let inputs3 = { messages: [{ role: "user", content: "What is Task Decomposition?" }],};for await (const step of await graphWithMemory.stream(inputs3, threadConfig)) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: What is Task Decomposition?-----[ai]:Tools:- retrieve({"query":"Task Decomposition"})-----[tool]: Source: https://lilianweng.github.io/posts/2023-06-23-agent/Content: hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning DomainSource: https://lilianweng.github.io/posts/2023-06-23-agent/Content: System message:Think step by step and reason yourself to the right decisions to make sure we get it right.You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose.Then you will output the content of each file including ALL code.Each file must strictly follow a markdown code block format, where the following tokens must be replaced such thatFILENAME is the lowercase file name including the file extension,LANG is the markup code block language for the codeโ€™s language, and CODE is the code:FILENAMEYou will start with the โ€œentrypointโ€ file, then go to the ones that are imported by that file, and so on.Please note that the code should be fully functional. No placeholders.Follow a language and framework appropriate best practice file naming convention.Make sure that files contain all imports, types etc. Make sure that code in different files are compatible with each other.-----[ai]: Task decomposition is the process of breaking down a complex task into smaller, more manageable steps or subgoals. This can be achieved through various methods, such as using prompts for large language models (LLMs), task-specific instructions, or human inputs. It helps in simplifying the problem-solving process and enhances understanding of the task at hand.----- let inputs4 = { messages: [ { role: "user", content: "Can you look up some common ways of doing it?" }, ],};for await (const step of await graphWithMemory.stream(inputs4, threadConfig)) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: Can you look up some common ways of doing it?-----[ai]:Tools:- retrieve({"query":"common methods of task decomposition"})-----[tool]: Source: https://lilianweng.github.io/posts/2023-06-23-agent/Content: hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning DomainSource: https://lilianweng.github.io/posts/2023-06-23-agent/Content: be provided by other developers (as in Plugins) or self-defined (as in function calls).HuggingGPT (Shen et al. 2023) is a framework to use ChatGPT as the task planner to select models available in HuggingFace platform according to the model descriptions and summarize the response based on the execution results.The system comprises of 4 stages:(1) Task planning: LLM works as the brain and parses the user requests into multiple tasks. There are four attributes associated with each task: task type, ID, dependencies, and arguments. They use few-shot examples to guide LLM to do task parsing and planning.Instruction:(2) Model selection: LLM distributes the tasks to expert models, where the request is framed as a multiple-choice question. LLM is presented with a list of models to choose from. Due to the limited context length, task type based filtration is needed.Instruction:(3) Task execution: Expert models execute on the specific tasks and log results.Instruction:(4) Response generation:-----[ai]: Common ways of task decomposition include using large language models (LLMs) with simple prompts like "Steps for XYZ" or "What are the subgoals for achieving XYZ?", employing task-specific instructions (e.g., "Write a story outline"), and incorporating human inputs. Additionally, methods like the Tree of Thoughts approach explore multiple reasoning possibilities at each step, creating a structured tree of thoughts. These techniques facilitate breaking down tasks into manageable components for better execution.----- Note that the query generated by the model in the second question incorporates the conversational context. The [LangSmith](https://smith.langchain.com/public/c8b2c1ba-8c8b-47ab-b298-3502e0688711/r) trace is particularly informative here, as we can see exactly what messages are visible to our chat model at each step. Agents[โ€‹](#agents "Direct link to Agents") ------------------------------------------- [Agents](/docs/concepts/agents) leverage the reasoning capabilities of LLMs to make decisions during execution. Using agents allows you to offload additional discretion over the retrieval process. Although their behavior is less predictable than the above โ€œchainโ€, they are able to execute multiple retrieval steps in service of a query, or iterate on a single search. Below we assemble a minimal RAG agent. Using LangGraphโ€™s [pre-built ReAct agent constructor](https://langchain-ai.github.io/langgraph/how-tos/#langgraph.prebuilt.chat_agent_executor.create_react_agent) , we can do this in one line. tip Check out LangGraph's [Agentic RAG](https://langchain-ai.github.io/langgraphjs/tutorials/rag/langgraph_agentic_rag/) tutorial for more advanced formulations. import { createReactAgent } from "@langchain/langgraph/prebuilt";const agent = createReactAgent({ llm: llm, tools: [retrieve] }); Letโ€™s inspect the graph: // Note: tslab only works inside a jupyter notebook. Don't worry about running this code yourself!import * as tslab from "tslab";const image = await agent.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_react](data:image/png;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAERAPYDASIAAhEBAxEB/8QAHQABAAIDAQEBAQAAAAAAAAAAAAYHAwQFCAIBCf/EAFIQAAEEAQIDAgYLDAgEBgMAAAEAAgMEBQYRBxIhEzEUFRYiQZQIFzJRVFVWYdHS0zQ2N0JTcXJ1kpOxsxgjc4GRlaSyJCVDUigzR2KCg6Gjw//EABsBAQEAAwEBAQAAAAAAAAAAAAABAgMEBQcG/8QAMhEBAAECAQkGBQUBAAAAAAAAAAECEQMEEhMhMVFxkdEUM0FhobEFIlOB8CNSYpLB4f/aAAwDAQACEQMRAD8A/qmiIgIiICIiAiIgIiICwWr1aiwPs2Iq7D3OleGj/wDK4dq3d1JbnpYyw/H0K7zFZyLWAySPHfHBvuN2no55BAILQC7mLPupoHT1R5kOKr2rJ2LrV1vhE7iO4mSTdx7z6fSVvzKKe8nXuj/fyVtvbflVhPjih60z6U8qsJ8cUPWmfSv3yWwvxRQ9WZ9CeS2F+KKHqzPoV/R8/RdT88qsJ8cUPWmfSnlVhPjih60z6V++S2F+KKHqzPoTyWwvxRQ9WZ9Cfo+foan55VYT44oetM+lPKrCfHFD1pn0r98lsL8UUPVmfQnkthfiih6sz6E/R8/Q1PzyqwnxxQ9aZ9K3ql+tfYXVrEVho73RPDgP8FpeS2F+KKHqzPoWnc0Bp648SeKa1ayNy21Tb4PO0nvLZI+Vw7h3H0BLYM+MxynompIEUbrWrumbUFTJWJMhjp3tir5CRoEkTz0Ec22wO52DXgDckNd12LpItVdGb5wTAiIsEEREBERAREQEREBERAREQEREBERAXE1rlp8JpbI2qpa24IxFXLxu0TPIZGSPe5nN3XbUa4jRPfo2/MxrnmoYrpaxvM5whlZMQB6Tsw7BbsCInFoidl491ja7GGxMGCxVTH1QRBWjbG3mO7jsO8k9ST3knqSSSt1fMcjZY2vY4PY4BzXA7gg+lRbUvFnQ+jMl4u1BrLT+CyHIJPBMllIK8vId9ncj3g7HY9dvQtUzMzMztRK1AeJPGLHcN8vg8O7DZrUWbzLZ5KmMwdZk0xihDTLIed7Ghredn425J6Ar5/pC8LB/6l6P/wA+q/aKv+NVihxm09Qm0Ngq3Emam6wK2oNMalrVbODucjOzcyYSD3Qdu4B3c1u7Hg9IOpc47ZyHj7idGwaNzFrC3dPx5N0scEDJ4ZJJ42dpJ2k7S2KNri17Q0v5t9g4Bd3J8fMdgta1cBltManxVS3kWYmtqC3j2tx01l55Y2NeHl+z3ea1xYGkkdVDINK8SNI8QNB6us4ZmuMiNIt07nnUrsNZ8VvtYpXWR2pYHsLmvBDevcQ30Kt9T8DNcZPMT3bWgWah1RT1hDnW6vs5iAusY+O62WOtVje7miLYg1hjcI2eY4hziQCF7w+yApZLV+pNN4bSOp89f09aNS/JRr1xCx3YtlYRJJOwO5w7lAHnBw84NaQ46/sZeL2a4z8Msdns5p63hrksQkNl7ImVLfM943rhs0j+VoaAe0DTuRtv1W9wh0bl9L6s4pXcnT8Gr5vUnh9CTtGP7aDwOtHz7NJLfPjeNnbHpvtsQohwMytzgRwzx+lOJEGO0dj8K59GpqHIZmqypkyZJHs7MF4e1xYObleAeh79kF+Iq/HshOFhBI4laQIHU/8APqvT/wDYutpnixojWmROP09rHT+eviMymrjMpBYlDAQC7lY4nYbjrt6Qg7+WxdfN4y1QtsL61mN0TwDsdiNtwfQR3gjuK5uiMpPmNLULFtwkuta6Cy9o2Dpo3GOQgegF7HHZdxzgxpc4hrQNySdgAo1w4jcNH053Nczw2Se+GubykNnmfMAR6DtIOi6I14M8Y9pv7QvgkyIi50EREBERAREQEREBERAREQEREBERAX4QHAgjcHoQV+ogimPtR6GEeLvObBhgeTH3XEhkTfxYJXHo0j3LCehADfdDzpLLTrzu5pII5Hd272AlfcsTJ4nxSsbJG8FrmPG4cD3gj0hRscPsfV6Yy1kMLHuNoKFtzIW7dwbEd2NHzNaB/guiZoxNdc2nnE9PVlql3vFlP4JB+7H0LLDBHXaWxRsjaTuQxoAUa8iJ/lTnv38X2SeRE/ypz37+L7JNHh/v9JLRvSlFFvIif5U579/F9kom7HZQcV2ad8qcz4uOEdfJ7WLtO1E4Z39n7nlPvd6aPD/f6SWjetVY5oI7DQ2WNkjQd9ntBCjXkRP8qc9+/i+yTyIn+VOe/fxfZJo8P9/pJaN6QeLafwWD92PoX3FTggdzRwRxu7t2MAKjnkRP8qc9+/i+yX0eH2PtdMlbyOZj3JMF6490LvmdE3Zjh8zgQmZhxtr5R1slo3seRuM1wJcVjpGzYkkx5C9G4ljm/jQROHRzj7lxB2aCR7rulTWtY0NaA1oGwAGwAXzDDHXhZFExsUTGhrGMGzWgdwAHcF9rCuuJiKadUQCIi1IIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAq9ft/SBi79/Jd/o6fdbVYSr14/8AEFEdjt5LvG+3T7rb6UFhIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgKvH7f0gou7fyXf7+/3W3+5WGq9eD/SAiO3TyYf16/C2/3ILCREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQERQ+5q7KX7MzcDQqT1IXuidcvWHxtke07OEbWscXNBBHMSOoOwI6rbh4VWJPyra6YIoR481h8BwfrU32aePNYfAcH61N9mt/Za98c4LJuv5/S+z3y7PZFDFHhVYOpWMOmfFQzLdzYNkHfn7Du3G3d3dV7K8eaw+A4P1qb7NVA/2P80nsiWcXjQw3jptLwfwTt5eyM+3ILG/J7rs/N229496dlr3xzgs9LIoR481h8BwfrU32aePNYfAcH61N9mnZa98c4LJuihHjzWHwHB+tTfZrPV1flMfPEM9QqQU5XtiFyjYdIInuOze0a5jSGkkDmBOxPUAbuEnJcSNlp+8FkwREXIgiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgKuuHx30hjye8h5P5+0crFVdcPfvPx35n/73L0Mn7qvjHtUvgkSIizQREQERaJzmPbm2Yc3YPGr67rYpdoO1MIcGmTl7+XmcBv3blBvKO8QztorLH0iHcfn3CkSjvET7ycv/AGB/iFuwO9p4x7rG2FioiLxkEREBERAREQEREBERAREQEREBERAREQEREBV1w9+8/Hfmf/vcrFVdcPfvPx35n/73L0Mn7qvjHtUvgkS8h6l1nqEa1oa20vc1G3TbtaV8HPYymoC6paY62K08UOOEZaIw4vDZC5rwW77FevFWuR9jhw6y2Su37WnA+xbteHP5blhjGWecPM8TGyBsUpcNzJGGuO53J3O6qJnYig85mdQ6l1vlaPlFqtuva2uYa0WnaVixDjxhmzxua9zY9mCM1x2jpSQ4u6b9djtVjxX4v5LW+Z09fko3sZnruJxzvKqWnXoeDycsbZqDaj45twA93aPJcH9CwbbSjVXsddX5bX2SyeEsYjTMFvKC+3NY7NZVluNvO1z96XaeDPe4Agk7NPMSWq1M3wD0FqHVcmpLuAa7LzSRyzyw2p4Y7D49uR0sTHiOVw2Gxe0noFhmzIpjU9fUWfyvHi7Z1dqDF3NLU6tvG1cTk5Iataz4qjmeQwbc7DI3qx+7Tu48oc4lbeAwsevPZH6Oz1/IZavduaAr5d8dHKT143SizCSzkY8AxHm3dGRyuPUglXxLw507PJquR+P5n6pjbFmD28n/ABTRD2AHuvM/qxy+Zy+/39VzM1wV0bqAae8NxDi/T8IrY2aC3PBLBEA0dmXxva57dmN3a8kHbqss2RN1HeIn3k5f+wP8QpEo7xE+8nL/ANgf4hdWB3tPGPdY2wsVEReMgiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgKuuHv3n478z/8Ae5WKq0pS5PDXrmOxWEs5zERctivdryRxtIlLn9k0yuaJAwEHnaSNnNHeCu7J5iaaqL2mZjbq2X6so2WShFxPG2f+RuT9ap/bp42z/wAjcn61T+3XTo/5R/anqWdtFxPG2f8Akbk/Wqf26iruM9VnEdmgjhbo1a+j4xbjfCavMYObbfm7bl36b8u/Nt1226po/wCUf2p6lliouJ42z/yNyfrVP7dPG2f+RuT9ap/bpo/5R/anqWdtR3iJ95OX/sD/ABCz+Ns/8jcn61T+3WCrFk9ciSjZxM+DoRzBts3Xt7Z4Y7fkY1hI5Xbe732LSdt99xnRbDqiuqqLRO+J9pIi03WKiIvFYiIiAiIgIiICIiAiIgIiICIiAiIgIiIC1MtlqWCx8t7IWoqdSLbnmmdytG5AA/OSQAO8kgDqV85LM08S6oy1YjimuTeD1YXOAfYl5HP5I297ncjHuIHc1jnHYNJGhisbbvOjyOY5o7MteIOxTZRLWqyNcX7tPKC9+5aC89P6tvKG7u5gxyUshqKy9t1kmNxUcliu+gTHI7IRloY173Dfs2dZHBrTzH+rLi3zo126tWGjWhrVoY69eFgjjhiaGsY0DYNaB0AAGwAWVEBERAX87bHsZOOMnsvhrjyg0oNRukOdbH4faMIqCQQirv4PzbdmQzbbbl9O6/okq8j3l9kHPsOlfS8e5/tLb9v5RQWGiIgLk5rARZNwuQdlUzcFeaClkzCJJK3aBvMOu3MwuZG5zCdnGNhPVrSOsiDl43LyTWJql6s6lbjk7Nhe5vZ2hyNcZIdiSW9S0h2zgWu6bcrndRaeRxNTK+Dm1AyV9aUT15CPPhkALQ9h72nlc5u47w5wPQkHTwlq7BI3F5IS2bVevG45Ps2sitk8wcQAfNeOXdzdthzt2J6hodhERAREQEREBERAREQFp+Nqfwhi3FWWo9R4vSOEuZjNXoMbi6jO0ntWH8rGN7up+ckADvJIA6oLB8bU/hDE8bU/hDFSmM45aIyunsvnI814Ni8S1j7s9+pPU7IP35Dyysa53MRs3lB3PQblfFLjzoS/p/MZpmeENDDmIZA2qs9eWqJCBG58UjGyNa4no7l2OxO+wKC7vG1P4QxPG1P4QxUdFx20hkMVqC3jb016fC0jfnpmlYimfDseV8bHRh0jHFpAewOb86iX9JOjlOBlXWtV7MJkLEdWPbL4u++nBZla15aXRwh0ke3MBKwchPL53UBB6e8bU/hDFr5LUMFKhYnhY69PHG50dWBzGyTOA3DGl5a0E93nED3yFTGpOPehNI5jJ4rK5w172LdG2/GylYlFQPjbIx8rmRlrGFr2nncQ3fcb7ggSTHavw+V1Jcwla62fKVKsN2SBsTgGQTFwjcH7cp5jG/oDuNuoHTcJziZIYJrFy5e7a5ZLXGIvD46o5GgxRHkaSzdpcS4czi4noOVrel42p/l2qkrXHjQ1TTWEzz82X47Ntc/HCCnPLPZa33TmwNjMuzfSeXYbjfbcKU6W1ViNa4StmMHfiyWNsb9nYhPQkEhwIPUEEEEEAggghBaCIiAiIgKvdJ/8y4ycQMgGtLadXGYbnB3IdGya04Hr0O12M+g7EfMp/LKyGN8kj2xxsBc57jsGgd5JUC4JQvtaNm1DNH2djU1+xmyPT2MrtqwPzisyuD84Pd3ALAREQEREBaWVw9LOVW171aOzE2RkzA8blkjHBzHtP4rmuAIcOoIBC3UQcbEZKxFbOKycgmyTYzOLEFV8UE0fOQNiS5vOBsHN5ifxtgHADsrRy+JizNVkMktiAxyxzxy1ZnRPY9jg4dR3jpsWndrgS1wIJBw6fy02UpEXYIaOUhPZ26UNptgQv9HnAAlrm7PbzNaS1w3a07gB1EREBERAREQEREBecvZFaayuodFYuxicdJmpMNnKGYsYmIjnvQQTB8kTQ4gF23nAHvLQF6NUR8S3fyB/aH0oPPHEzO3+LOjal3DaQ1K0aczuMzM2OyuNdTlyMUU3NLFCyQgvc1o5tiACQ0AuUD4t4jP8VjxA1RiNKZ2ljnacpYWvWv46SG5kJxfE7nMrkdpyxsO25A35nbbgbr2H4lu/kD+0PpTxLd/IH9ofSgpLVWmMnlePctivSn8CsaGu4/w4xO7ATutRFkbpNtubbmIbvvtuVXlhuZ1B7D2bR7NKahqaiwmLxmOnpWcZK0zyxSxNeYCARM0CIu5mbjYgr1h4lu/kD+0PpTxLd/IH9ofSg86ZjTGVsXvZGgYm89mXpQsoObWcRcIxLWFsXT+s8/zNm79enetLRj8xw01pSyuQ0xnslBltGYilF4uoPmdHbr9r2kE35Fx7VvnScre/dw2K9IHHWvGLa/g0/aGIyBwYez23A2L/AHPN82++252Wx4lu/kD+0PpQeHtFcPszpbG8NM7qLAa0OIZpQ4ezW01Jcr5DH2hafKDLDA5kpY9rgD0OxY0kDoV6f4QacxOndHMdh8Xl8PDkLMt+atnZpJbvavd5z5TI97uZ2wdsXb9eux3VheJbv5A/tD6V+jC3d/8AyD+0PpQS1ERARFr5HIVcRj7V67YjqUqsTp57Ezg1kUbQS5ziegAAJJ+ZBB+MD5s5iqWiqZkba1RI6nYliJBr0Gje5LuPcnsz2TSOoknjPvqewwx1oWRRMbFFG0NYxg2a0DoAAO4KDcOcfYzd2/rjKV5at7MRsio1LDeV9LHsLnQsc0+4kkLjLIO8FzWHfsmqeICIiAiIgIiICjOppYNMXYtSPmx+Opjkr5azPX3klgBcIf61vUCOSUnzt2hskh83fdSZfE0YmifGege0tPQHv/P0QfaKP6GyZyWnIGS5J+Xu0Xvx9y7LV8GdNYhcYpXmMdG8zmlw5fNIcC3oQpAgIiICIiAiIgIiICIiAiIgj2Zi8G1fp6+2DJzmRtnHO8FfvVhbI1s3azs+Y1gxr+9plI7nlSFcnVGFZnsNLWcJTIx8dmHsZzC/tYntkj2eN9hzMbv0II3BBBIOrj9c4izkMbiLl+ljNTXaYujT892F11rOvP5jXnmDXNc0ubu3dp2JQSBERAREQFXeW24oapdhoyXaVwdlj8pIPcZC23zmVAfxo4jyvl9BdyR7naZg6Wss7eyF8aT09OYM1agEtvIR8pOJqu5mifZwIMji17YmuBBc1ziC2NwMiwGBoaXw1TFYyuK1Gqzkjj5i4++S5ziS5xJJLnElxJJJJJQdBERAREQEREBERAREQRzB3iNX6lxsl+1bkYa11kE0AbHVikjMYZG8e7BfXled+oLz6CFI1HGXuTiLNSN+07tMUyZtAwjsGcszgZRJ/wB552tLfeaCpGgIiICIiAiIgIiICIiAuHltcadwNp1bI5zH0bLQC6Geyxj27jcbtJ3G/oWXV+Tlwmk81kYDtNUpT2GEjfZzI3OHT09QuHgsbBisXBBC38UPfIerpXnq57iernOJJJO5JJXXhYVNVOfXs2al85cjiBqfSWutG5TBQ8Qo9OzXIw2PKYfKNgtV3Bwc1zHtcCOoAI3G4JHpXiH2L3DC/wAFvZg38lqjUdLOY2ahcnj1R4aJY7ckrm7ukeXEtlJLtw87k7nqDuf6Cot+iwd0846LqYfbU0d8qMT65H9Ke2po75UYn1yP6VmRNFg7p5x0NTD7amjvlRifXI/pXC1hxt09h8WwYfL4vKZe3KK1SI22iFjyCe0neD5kTA1znHvOwa3me5jTI0TRYO6ecdDU5PDi7pmjXfSx+pKOdzd6R1u9cbPGZ70/K0OkLWnuDWsaGjoxjWNHRoU5UNzuNhyuLngmb+KXskHR0bx1a9pHVrmkAgggggLt6Qycua0nhcjOeaa3SgsPO227nxtceno6laMXCpppz6NmzWnnDroiLkQREQEREBFpZbN4/A1TZyd6tj64O3a2pWxt397ckdVF5OM+jIzt47jf123jhleP8Q0hb8PAxcWL4dEzwiZW0ymqKD+3Voz45/0s31E9urRnxz/pZvqLb2PKfpVcpLSi7/ZH8MBryKD21tOCDxe/esMtU8F7TtWjmMvP0l23AZ6RufQrgX83cj7HTR9r2ZrNTi1EeG0snjyVvg8nKLW+5rcnJvymTz+7blO2+4Xur26tGfHP+lm+onY8p+lVyktKcIoP7dWjPjn/AEs31F9M40aMedvHbGfPJBKwf4loCdjyn6VXKS0psi0cRncbqCt4RjL9bIQb7GSrK2RoPvEg9D8y3lyTE0zaYtKCIigIiICIiCOcSPwd6p/VVr+S5Yqv3LD+gP4LLxI/B3qn9VWv5Lliq/csP6A/gvRwu5jjPtC+DKiw3XzxU531YWWLLY3GKKSTs2vft5rS7Y8oJ2G+x294rzvwq4+aox/Abyx1xiY78r7LquOdj7rZbOUsyXZYGQdl2UbItncjAeZ27QXEDbZJmIR6ORUpc9khJoxuoq+vtLS6ayuKxTMzDVoXm5Bl6B0ogDY5Axm0nbOjYWuA6yNIJB3Xxk+MmoDWzmndT6Wk0VnbWnbuUxM1XKNuMlELNpG9o1jDHNGXxu2G467hx2TOgXci8/YXjhmNO6N4P4OlhTqrU2pNNQXjNk8s2m2Yx14TJ/XSNeZZnGTfl7z1JI71flWSSWtE+aLsJXMDnxcwdyOI6t3HQ7d26RNwtfcs36B/gsvDf8Helv1VV/ktWK19yzfoH+Cy8N/wd6W/VVX+S1MXuZ4x7Sy8EjREXnMRERAUV4ha4ZorFxuijZZydollWu92wJG3M93p5W7jfbv3A6b7qVLz5xNyT8pxFyrHkmPHMhpxj0DeNsriPzmUA/oD3l6vw3JacqyjNr2RF5/OMqjt2WfLZB1/Izvv33f9ec78o95g7mN/9rdh/f1X4iL6FERTERGxhM3EXzNMyvC+WRwZGxpc5zjsAB1JKpfD+ycxWWyuMaK1BuJyduOnWlizMEt4GR3LG+WoPOY0kjfziWg7kDrtqxMbDwpiK5tcXUiqeHjffNY5SfSxg07FmXYaxf8AGDXSMeLJrtlbFyeczm5d93Agk7BwG553FXinmrOl9ewaXws81PDVZ6trPMvis6vYEXM7sW7czzHzAk7t67gbkLVVlWHFM1RPpP59xdKLRwMj5sFjpJHOfI+tG5znHcklo3JK3l1RN4uj9qPmxt9l+hPJQvs9zYgOzj8zh3Pb0HmuBCvfh1rtutMbK2xGyvlqnK21BGfMIO/LIzfryu5XbA9QWuG523NDrv8ADnIvxXELDFjtmXe1pSjbvaY3SN/wdG39oryfieSUZRgVV2+amLxPDwZxN9T0OiIvnwIiICIiCOcSPwd6p/VVr+S5Yqv3LD+gP4LLxI/B3qn9VWv5Lliq/csP6A/gvRwu5jjPtC+DKvO9DgFrKLhta0LNkcHFQxOR8baby8RmfYFhl024hZhLQ0NBJYSxxJB3HVeiESYujz1qT2P2q+LdjUeV1vksPistZwjMNioMCZbEFQtssteESOlawvcZoYfNAADWEbknddiHhNrPXurG5ziBcwdQ0cJdxGPq6edNK3ntBjZ7D3StaQeWNobGAQOvnFXaimbA87Z3gzxBzHBXT2gbdHQuZbQxpxklnIG0DCY2NjrWoCGEtlaxvM4dPOPmvA7700piLGn9LYfF3L8mUt0qcNaa9N7uy9jA10jup6uIJP511UViLDFa+5Zv0D/BZeG/4O9Lfqqr/JasVr7lm/QP8Fl4b/g70t+qqv8AJamL3M8Y9pZeCRoiLzmIiIgLz5xOxj8XxFyr3AiPIsiuRu9BIjbC4D83ZtJ/THvr0Govr/REWtMWxjJG1sjWJfVsubzBpO3M13p5XAAH8wPeAvV+G5VTkuPnV7Ji0/nGFedsrlIMLj5rtkTGCEbuFeCSd/ft0ZG1znd/oBUYHFvT5/6Wc/v07kB//BTTIQz4XInH5OB+PvDuhm6CQf8AdG7ue3527+8dj0Xyv3051URVRMWn7/6wtbaho4i6e1AfFfY5k+G/8NtLgr0TPP8AN6vdCGtHXvJAHpK4fDzResNFRYrAzv0/e07jd4o75ZKL0kDQezaWbcjXDzQXcx3De7c7qzkWOimqqKqp1xu1dUVPPwmy8nDLJ6dFml4bZz5yjJC9/ZiI5BtnYnk35uQbbbbc3p26rQ1Jws1kzH66wmAtYSXBaofYs82SdMyxVmnYGyNHI1zXNJG4J2Ldz0dt1udFrnJsOYt5W+2vqqFQ8RMNp6CLF2mZd1mkxteV1fBXpYy5oAPK9sJa4bjoQdisjuLWn2HYxZzuB6aeyB7/AP6FMUW3NxI1RMcv+o0sNl6+exsN6oJ2wS78os1pK8nQkHeORrXDqD3gbjqOhClvDjGvyvEPDBo3jo9rdlO/cBG6No/vdID/APE+8uBSiny19tDHV5Mhed/0IBvy/O89zG/O4gf3q+eHmhWaLxshmkZYytvldasRjZvTfljbv15W8ztt+pJcem+w834lldOT4E0TPzVRa3HbLOItrSxERfPwREQEREHJ1djJM3pTNY6EbzW6U1dgJ2858bmjr6OpXCwWTgyuNhlhds5rQyWJ3R8Tx0cxzT1a4EEEEehTNcXLaK09n7BsZPBY3ITnYGW1Ujkcdug6kE9F14WLTTTmV7NupfKWsiw+1Zoz5JYT/L4vqp7VmjPklhP8vi+qt2lwd88o6rqZkWH2rNGfJLCf5fF9VPas0Z8ksJ/l8X1U0uDvnlHU1MyLD7VmjPklhP8AL4vqp7VmjPklhP8AL4vqppcHfPKOpqaudycGKxs0szvOc0sjib1fK89Gsa0dXOJIAAHpXd0jjJcJpTC46YbTVKUFd4B385kbWnr6eoWPE6K09gLAsYzBY3HzjcCWrUjjcN+h6gA9V2lpxcWmqnMo2bdaeUCIi5EEREBERBqZTEUc3UdVyNKvfqu6mG1E2Rh/ucCFGZOD+jZDucBWb6dmFzB/gCApii3UY+LhRaiqY4TMLeYQv2mtGfEUP7yT6ye01oz4ih/eSfWU0Rbe15T9SrnJed6F+01oz4ih/eSfWT2mtGfEUP7yT6ymiJ2vKfqVc5LzvQv2mtGfEUP7yT6y+mcHtGxnfxBXd8z3PcP8CdlMkTteUfUq5yXne0sVhsfgqorY2jXoVwd+yrRNjbv7+wAW6iLlmZqm8oIiKAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIg//9k=) The key difference from our earlier implementation is that instead of a final generation step that ends the run, here the tool invocation loops back to the original LLM call. The model can then either answer the question using the retrieved context, or generate another tool call to obtain more information. Letโ€™s test this out. We construct a question that would typically require an iterative sequence of retrieval steps to answer: let inputMessage = `What is the standard method for Task Decomposition?Once you get the answer, look up common extensions of that method.`;let inputs5 = { messages: [{ role: "user", content: inputMessage }] };for await (const step of await agent.stream(inputs5, { streamMode: "values",})) { const lastMessage = step.messages[step.messages.length - 1]; prettyPrint(lastMessage); console.log("-----\n");} [human]: What is the standard method for Task Decomposition?Once you get the answer, look up common extensions of that method.-----[ai]:Tools:- retrieve({"query":"standard method for Task Decomposition"})-----[tool]: Source: https://lilianweng.github.io/posts/2023-06-23-agent/Content: hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning DomainSource: https://lilianweng.github.io/posts/2023-06-23-agent/Content: System message:Think step by step and reason yourself to the right decisions to make sure we get it right.You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose.Then you will output the content of each file including ALL code.Each file must strictly follow a markdown code block format, where the following tokens must be replaced such thatFILENAME is the lowercase file name including the file extension,LANG is the markup code block language for the codeโ€™s language, and CODE is the code:FILENAMEYou will start with the โ€œentrypointโ€ file, then go to the ones that are imported by that file, and so on.Please note that the code should be fully functional. No placeholders.Follow a language and framework appropriate best practice file naming convention.Make sure that files contain all imports, types etc. Make sure that code in different files are compatible with each other.-----[ai]:Tools:- retrieve({"query":"common extensions of Task Decomposition method"})-----[tool]: Source: https://lilianweng.github.io/posts/2023-06-23-agent/Content: hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning DomainSource: https://lilianweng.github.io/posts/2023-06-23-agent/Content: be provided by other developers (as in Plugins) or self-defined (as in function calls).HuggingGPT (Shen et al. 2023) is a framework to use ChatGPT as the task planner to select models available in HuggingFace platform according to the model descriptions and summarize the response based on the execution results.The system comprises of 4 stages:(1) Task planning: LLM works as the brain and parses the user requests into multiple tasks. There are four attributes associated with each task: task type, ID, dependencies, and arguments. They use few-shot examples to guide LLM to do task parsing and planning.Instruction:(2) Model selection: LLM distributes the tasks to expert models, where the request is framed as a multiple-choice question. LLM is presented with a list of models to choose from. Due to the limited context length, task type based filtration is needed.Instruction:(3) Task execution: Expert models execute on the specific tasks and log results.Instruction:(4) Response generation:-----[ai]: ### Standard Method for Task DecompositionThe standard method for task decomposition involves breaking down hard tasks into smaller, more manageable steps. This can be achieved through various approaches:1. **Chain of Thought (CoT)**: This method transforms large tasks into multiple manageable tasks, providing insight into the model's reasoning process.2. **Prompting**: Using simple prompts like "Steps for XYZ" or "What are the subgoals for achieving XYZ?" to guide the decomposition.3. **Task-Specific Instructions**: Providing specific instructions tailored to the task, such as "Write a story outline" for writing a novel.4. **Human Inputs**: Involving human input to assist in the decomposition process.### Common Extensions of Task DecompositionSeveral extensions have been developed to enhance the task decomposition process:1. **Tree of Thoughts (ToT)**: This method extends CoT by exploring multiple reasoning possibilities at each step. It decomposes the problem into multiple thought steps and generates various thoughts per step, creating a tree structure. The search process can utilize either breadth-first search (BFS) or depth-first search (DFS), with each state evaluated by a classifier or through majority voting.2. **LLM+P**: This approach involves using an external classical planner for long-horizon planning, integrating planning domains to enhance the decomposition process.3. **HuggingGPT**: This framework utilizes ChatGPT as a task planner to select models from the HuggingFace platform based on model descriptions. It consists of four stages: - **Task Planning**: Parsing user requests into multiple tasks with attributes like task type, ID, dependencies, and arguments. - **Model Selection**: Distributing tasks to expert models based on a multiple-choice question format. - **Task Execution**: Expert models execute specific tasks and log results. - **Response Generation**: Compiling the results into a coherent response.These extensions aim to improve the efficiency and effectiveness of task decomposition, making it easier to manage complex tasks.----- Note that the agent: 1. Generates a query to search for a standard method for task decomposition; 2. Receiving the answer, generates a second query to search for common extensions of it; 3. Having received all necessary context, answers the question. We can see the full sequence of steps, along with latency and other metadata, in the [LangSmith trace](https://smith.langchain.com/public/67b7642b-78d0-482a-bb49-fe08674bf972/r) . Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Weโ€™ve covered the steps to build a basic conversational Q&A application: * We used chains to build a predictable application that generates at most one query per user input; * We used agents to build an application that can iterate on a sequence of queries. To explore different types of retrievers and retrieval strategies, visit the [retrievers](/docs/how_to/#retrievers) section of the how-to guides. For a detailed walkthrough of LangChainโ€™s conversation memory abstractions, visit the [How to add message history (memory)](/docs/how_to/message_history) guide. To learn more about agents, check out the [conceptual guide](/docs/concepts/agents) and LangGraph [agent architectures](https://langchain-ai.github.io/langgraphjs/concepts/agentic_concepts/) page. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Setup](#setup) * [Components](#components) * [Dependencies](#dependencies) * [LangSmith](#langsmith) * [Chains](#chains) * [Stateful management of chat history](#stateful-management-of-chat-history) * [Agents](#agents) * [Next steps](#next-steps) --- # How to stream responses from an LLM | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page How to stream responses from an LLM =================================== All [`LLM`s](https://api.js.langchain.com/classes/langchain_core.language_models_llms.BaseLLM.html) implement the [Runnable interface](https://api.js.langchain.com/classes/langchain_core.runnables.Runnable.html) , which comes with **default** implementations of standard runnable methods (i.e. `ainvoke`, `batch`, `abatch`, `stream`, `astream`, `astream_events`). The **default** streaming implementations provide an `AsyncGenerator` that yields a single value: the final output from the underlying chat model provider. The ability to stream the output token-by-token depends on whether the provider has implemented proper streaming support. See which [integrations support token-by-token streaming here](/docs/integrations/llms/) . :::{.callout-note} The **default** implementation does **not** provide support for token-by-token streaming, but it ensures that the model can be swapped in for any other model as it supports the same standard interface. ::: Using `.stream()`[โ€‹](#using-stream "Direct link to using-stream") ------------------------------------------------------------------ The easiest way to stream is to use the `.stream()` method. This returns an readable stream that you can also iterate over: tip See [this section for general instructions on installing integration packages](/docs/how_to/installation#installing-integration-packages) . * npm * Yarn * pnpm npm install @langchain/openai @langchain/core yarn add @langchain/openai @langchain/core pnpm add @langchain/openai @langchain/core import { OpenAI } from "@langchain/openai";const model = new OpenAI({ maxTokens: 25,});const stream = await model.stream("Tell me a joke.");for await (const chunk of stream) { console.log(chunk);}/*Q: What did the fish say when it hit the wall?A: Dam!*/ #### API Reference: * OpenAI from `@langchain/openai` For models that do not support streaming, the entire response will be returned as a single chunk. Using a callback handler[โ€‹](#using-a-callback-handler "Direct link to Using a callback handler") ------------------------------------------------------------------------------------------------- You can also use a [`CallbackHandler`](https://api.js.langchain.com/classes/langchain_core.callbacks_base.BaseCallbackHandler.html) like so: import { OpenAI } from "@langchain/openai";// To enable streaming, we pass in `streaming: true` to the LLM constructor.// Additionally, we pass in a handler for the `handleLLMNewToken` event.const model = new OpenAI({ maxTokens: 25, streaming: true,});const response = await model.invoke("Tell me a joke.", { callbacks: [ { handleLLMNewToken(token: string) { console.log({ token }); }, }, ],});console.log(response);/*{ token: '\n' }{ token: '\n' }{ token: 'Q' }{ token: ':' }{ token: ' Why' }{ token: ' did' }{ token: ' the' }{ token: ' chicken' }{ token: ' cross' }{ token: ' the' }{ token: ' playground' }{ token: '?' }{ token: '\n' }{ token: 'A' }{ token: ':' }{ token: ' To' }{ token: ' get' }{ token: ' to' }{ token: ' the' }{ token: ' other' }{ token: ' slide' }{ token: '.' }Q: Why did the chicken cross the playground?A: To get to the other slide.*/ #### API Reference: * OpenAI from `@langchain/openai` We still have access to the end `LLMResult` if using `generate`. However, `tokenUsage` may not be currently supported for all model providers when streaming. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Using `.stream()`](#using-stream) * [Using a callback handler](#using-a-callback-handler) --- # How to add memory to chatbots | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page A key feature of chatbots is their ability to use content of previous conversation turns as context. This state management can take several forms, including: * Simply stuffing previous messages into a chat model prompt. * The above, but trimming old messages to reduce the amount of distracting information the model has to deal with. * More complex modifications like synthesizing summaries for long running conversations. Weโ€™ll go into more detail on a few techniques below! This how-to guide previously built a chatbot using [RunnableWithMessageHistory](https://v03.api.js.langchain.com/classes/_langchain_core.runnables.RunnableWithMessageHistory.html) . You can access this version of the tutorial in the [v0.2 docs](https://js.langchain.com/v0.2/docs/how_to/chatbots_memory/) . The LangGraph implementation offers a number of advantages over `RunnableWithMessageHistory`, including the ability to persist arbitrary components of an applicationโ€™s state (instead of only messages). Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- Youโ€™ll need to install a few packages, select your chat model, and set its enviroment variable. * npm * yarn * pnpm npm i @langchain/core @langchain/langgraph yarn add @langchain/core @langchain/langgraph pnpm add @langchain/core @langchain/langgraph Letโ€™s set up a chat model that weโ€™ll use for the below examples. ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); Message passing[โ€‹](#message-passing "Direct link to Message passing") ---------------------------------------------------------------------- The simplest form of memory is simply passing chat history messages into a chain. Hereโ€™s an example: import { HumanMessage, AIMessage } from "@langchain/core/messages";import { ChatPromptTemplate, MessagesPlaceholder,} from "@langchain/core/prompts";const prompt = ChatPromptTemplate.fromMessages([ [ "system", "You are a helpful assistant. Answer all questions to the best of your ability.", ], new MessagesPlaceholder("messages"),]);const chain = prompt.pipe(llm);await chain.invoke({ messages: [ new HumanMessage( "Translate this sentence from English to French: I love programming." ), new AIMessage("J'adore la programmation."), new HumanMessage("What did you just say?"), ],}); AIMessage { "id": "chatcmpl-ABSxUXVIBitFRBh9MpasB5jeEHfCA", "content": "I said \"J'adore la programmation,\" which means \"I love programming\" in French.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 18, "promptTokens": 58, "totalTokens": 76 }, "finish_reason": "stop", "system_fingerprint": "fp_e375328146" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 58, "output_tokens": 18, "total_tokens": 76 }} We can see that by passing the previous conversation into a chain, it can use it as context to answer questions. This is the basic concept underpinning chatbot memory - the rest of the guide will demonstrate convenient techniques for passing or reformatting messages. Automatic history management[โ€‹](#automatic-history-management "Direct link to Automatic history management") ------------------------------------------------------------------------------------------------------------- The previous examples pass messages to the chain (and model) explicitly. This is a completely acceptable approach, but it does require external management of new messages. LangChain also provides a way to build applications that have memory using LangGraphโ€™s persistence. You can enable persistence in LangGraph applications by providing a `checkpointer` when compiling the graph. import { START, END, MessagesAnnotation, StateGraph, MemorySaver,} from "@langchain/langgraph";// Define the function that calls the modelconst callModel = async (state: typeof MessagesAnnotation.State) => { const systemPrompt = "You are a helpful assistant. " + "Answer all questions to the best of your ability."; const messages = [ { role: "system", content: systemPrompt }, ...state.messages, ]; const response = await llm.invoke(messages); return { messages: response };};const workflow = new StateGraph(MessagesAnnotation) // Define the node and edge .addNode("model", callModel) .addEdge(START, "model") .addEdge("model", END);// Add simple in-memory checkpointerconst memory = new MemorySaver();const app = workflow.compile({ checkpointer: memory }); Weโ€™ll pass the latest input to the conversation here and let the LangGraph keep track of the conversation history using the checkpointer: await app.invoke( { messages: [ { role: "user", content: "Translate to French: I love programming.", }, ], }, { configurable: { thread_id: "1" }, }); { messages: [ HumanMessage { "id": "227b82a9-4084-46a5-ac79-ab9a3faa140e", "content": "Translate to French: I love programming.", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "chatcmpl-ABSxVrvztgnasTeMSFbpZQmyYqjJZ", "content": "J'adore la programmation.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 5, "promptTokens": 35, "totalTokens": 40 }, "finish_reason": "stop", "system_fingerprint": "fp_52a7f40b0b" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 35, "output_tokens": 5, "total_tokens": 40 } } ]} await app.invoke( { messages: [ { role: "user", content: "What did I just ask you?", }, ], }, { configurable: { thread_id: "1" }, }); { messages: [ HumanMessage { "id": "1a0560a4-9dcb-47a1-b441-80717e229706", "content": "Translate to French: I love programming.", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "chatcmpl-ABSxVrvztgnasTeMSFbpZQmyYqjJZ", "content": "J'adore la programmation.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 5, "promptTokens": 35, "totalTokens": 40 }, "finish_reason": "stop", "system_fingerprint": "fp_52a7f40b0b" }, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "id": "4f233a7d-4b08-4f53-bb60-cf0141a59721", "content": "What did I just ask you?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "chatcmpl-ABSxVs5QnlPfbihTOmJrCVg1Dh7Ol", "content": "You asked me to translate \"I love programming\" into French.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 13, "promptTokens": 55, "totalTokens": 68 }, "finish_reason": "stop", "system_fingerprint": "fp_9f2bfdaa89" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 55, "output_tokens": 13, "total_tokens": 68 } } ]} Modifying chat history[โ€‹](#modifying-chat-history "Direct link to Modifying chat history") ------------------------------------------------------------------------------------------- Modifying stored chat messages can help your chatbot handle a variety of situations. Here are some examples: ### Trimming messages[โ€‹](#trimming-messages "Direct link to Trimming messages") LLMs and chat models have limited context windows, and even if youโ€™re not directly hitting limits, you may want to limit the amount of distraction the model has to deal with. One solution is trim the history messages before passing them to the model. Letโ€™s use an example history with the `app` we declared above: const demoEphemeralChatHistory = [ { role: "user", content: "Hey there! I'm Nemo." }, { role: "assistant", content: "Hello!" }, { role: "user", content: "How are you today?" }, { role: "assistant", content: "Fine thanks!" },];await app.invoke( { messages: [ ...demoEphemeralChatHistory, { role: "user", content: "What's my name?" }, ], }, { configurable: { thread_id: "2" }, }); { messages: [ HumanMessage { "id": "63057c3d-f980-4640-97d6-497a9f83ddee", "content": "Hey there! I'm Nemo.", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "c9f0c20a-8f55-4909-b281-88f2a45c4f05", "content": "Hello!", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "id": "fd7fb3a0-7bc7-4e84-99a9-731b30637b55", "content": "How are you today?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "09b0debb-1d4a-4856-8821-b037f5d96ecf", "content": "Fine thanks!", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "id": "edc13b69-25a0-40ac-81b3-175e65dc1a9a", "content": "What's my name?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "chatcmpl-ABSxWKCTdRuh2ZifXsvFHSo5z5I0J", "content": "Your name is Nemo! How can I assist you today, Nemo?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 14, "promptTokens": 63, "totalTokens": 77 }, "finish_reason": "stop", "system_fingerprint": "fp_a5d11b2ef2" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 63, "output_tokens": 14, "total_tokens": 77 } } ]} We can see the app remembers the preloaded name. But letโ€™s say we have a very small context window, and we want to trim the number of messages passed to the model to only the 2 most recent ones. We can use the built in [trimMessages](/docs/how_to/trim_messages/) util to trim messages based on their token count before they reach our prompt. In this case weโ€™ll count each message as 1 โ€œtokenโ€ and keep only the last two messages: import { START, END, MessagesAnnotation, StateGraph, MemorySaver,} from "@langchain/langgraph";import { trimMessages } from "@langchain/core/messages";// Define trimmer// count each message as 1 "token" (tokenCounter: (msgs) => msgs.length) and keep only the last two messagesconst trimmer = trimMessages({ strategy: "last", maxTokens: 2, tokenCounter: (msgs) => msgs.length,});// Define the function that calls the modelconst callModel2 = async (state: typeof MessagesAnnotation.State) => { const trimmedMessages = await trimmer.invoke(state.messages); const systemPrompt = "You are a helpful assistant. " + "Answer all questions to the best of your ability."; const messages = [ { role: "system", content: systemPrompt }, ...trimmedMessages, ]; const response = await llm.invoke(messages); return { messages: response };};const workflow2 = new StateGraph(MessagesAnnotation) // Define the node and edge .addNode("model", callModel2) .addEdge(START, "model") .addEdge("model", END);// Add simple in-memory checkpointerconst app2 = workflow2.compile({ checkpointer: new MemorySaver() }); Letโ€™s call this new app and check the response await app2.invoke( { messages: [ ...demoEphemeralChatHistory, { role: "user", content: "What is my name?" }, ], }, { configurable: { thread_id: "3" }, }); { messages: [ HumanMessage { "id": "0d9330a0-d9d1-4aaf-8171-ca1ac6344f7c", "content": "What is my name?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "3a24e88b-7525-4797-9fcd-d751a378d22c", "content": "Fine thanks!", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "id": "276039c8-eba8-4c68-b015-81ec7704140d", "content": "How are you today?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "2ad4f461-20e1-4982-ba3b-235cb6b02abd", "content": "Hello!", "additional_kwargs": {}, "response_metadata": {}, "tool_calls": [], "invalid_tool_calls": [] }, HumanMessage { "id": "52213cae-953a-463d-a4a0-a7368c9ee4db", "content": "Hey there! I'm Nemo.", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "chatcmpl-ABSxWe9BRDl1pmzkNIDawWwU3hvKm", "content": "I'm sorry, but I don't have access to personal information about you unless you've shared it with me during our conversation. How can I assist you today?", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 30, "promptTokens": 39, "totalTokens": 69 }, "finish_reason": "stop", "system_fingerprint": "fp_3537616b13" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 39, "output_tokens": 30, "total_tokens": 69 } } ]} We can see that `trimMessages` was called and only the two most recent messages will be passed to the model. In this case, this means that the model forgot the name we gave it. Check out our [how to guide on trimming messages](/docs/how_to/trim_messages/) for more. ### Summary memory[โ€‹](#summary-memory "Direct link to Summary memory") We can use this same pattern in other ways too. For example, we could use an additional LLM call to generate a summary of the conversation before calling our app. Letโ€™s recreate our chat history: const demoEphemeralChatHistory2 = [ { role: "user", content: "Hey there! I'm Nemo." }, { role: "assistant", content: "Hello!" }, { role: "user", content: "How are you today?" }, { role: "assistant", content: "Fine thanks!" },]; And now, letโ€™s update the model-calling function to distill previous interactions into a summary: import { START, END, MessagesAnnotation, StateGraph, MemorySaver,} from "@langchain/langgraph";import { RemoveMessage } from "@langchain/core/messages";// Define the function that calls the modelconst callModel3 = async (state: typeof MessagesAnnotation.State) => { const systemPrompt = "You are a helpful assistant. " + "Answer all questions to the best of your ability. " + "The provided chat history includes a summary of the earlier conversation."; const systemMessage = { role: "system", content: systemPrompt }; const messageHistory = state.messages.slice(0, -1); // exclude the most recent user input // Summarize the messages if the chat history reaches a certain size if (messageHistory.length >= 4) { const lastHumanMessage = state.messages[state.messages.length - 1]; // Invoke the model to generate conversation summary const summaryPrompt = "Distill the above chat messages into a single summary message. " + "Include as many specific details as you can."; const summaryMessage = await llm.invoke([ ...messageHistory, { role: "user", content: summaryPrompt }, ]); // Delete messages that we no longer want to show up const deleteMessages = state.messages.map( (m) => new RemoveMessage({ id: m.id }) ); // Re-add user message const humanMessage = { role: "user", content: lastHumanMessage.content }; // Call the model with summary & response const response = await llm.invoke([ systemMessage, summaryMessage, humanMessage, ]); return { messages: [summaryMessage, humanMessage, response, ...deleteMessages], }; } else { const response = await llm.invoke([systemMessage, ...state.messages]); return { messages: response }; }};const workflow3 = new StateGraph(MessagesAnnotation) // Define the node and edge .addNode("model", callModel3) .addEdge(START, "model") .addEdge("model", END);// Add simple in-memory checkpointerconst app3 = workflow3.compile({ checkpointer: new MemorySaver() }); Letโ€™s see if it remembers the name we gave it: await app3.invoke( { messages: [ ...demoEphemeralChatHistory2, { role: "user", content: "What did I say my name was?" }, ], }, { configurable: { thread_id: "4" }, }); { messages: [ AIMessage { "id": "chatcmpl-ABSxXjFDj6WRo7VLSneBtlAxUumPE", "content": "Nemo greeted the assistant and asked how it was doing, to which the assistant responded that it was fine.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 22, "promptTokens": 60, "totalTokens": 82 }, "finish_reason": "stop", "system_fingerprint": "fp_e375328146" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 60, "output_tokens": 22, "total_tokens": 82 } }, HumanMessage { "id": "8b1309b7-c09e-47fb-9ab3-34047f6973e3", "content": "What did I say my name was?", "additional_kwargs": {}, "response_metadata": {} }, AIMessage { "id": "chatcmpl-ABSxYAQKiBsQ6oVypO4CLFDsi1HRH", "content": "You mentioned that your name is Nemo.", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 8, "promptTokens": 73, "totalTokens": 81 }, "finish_reason": "stop", "system_fingerprint": "fp_52a7f40b0b" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 73, "output_tokens": 8, "total_tokens": 81 } } ]} Note that invoking the app again will keep accumulating the history until it reaches the specified number of messages (four in our case). At that point we will generate another summary generated from the initial summary plus new messages and so on. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Setup](#setup) * [Message passing](#message-passing) * [Automatic history management](#automatic-history-management) * [Modifying chat history](#modifying-chat-history) * [Trimming messages](#trimming-messages) * [Summary memory](#summary-memory) --- # Build a semantic search engine | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page This tutorial will familiarize you with LangChainโ€™s [document loader](/docs/concepts/document_loaders) , [embedding](/docs/concepts/embedding_models) , and [vector store](/docs/concepts/vectorstores) abstractions. These abstractions are designed to support retrieval of dataโ€“ from (vector) databases and other sourcesโ€“ for integration with LLM workflows. They are important for applications that fetch data to be reasoned over as part of model inference, as in the case of retrieval-augmented generation, or [RAG](/docs/concepts/rag) (see our RAG tutorial [here](/docs/tutorials/rag) ). Here we will build a search engine over a PDF document. This will allow us to retrieve passages in the PDF that are similar to an input query. Concepts[โ€‹](#concepts "Direct link to Concepts") ------------------------------------------------- This guide focuses on retrieval of text data. We will cover the following concepts: * Documents and document loaders; * Text splitters; * Embeddings; * Vector stores and retrievers. Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Jupyter Notebook[โ€‹](#jupyter-notebook "Direct link to Jupyter Notebook") This and other tutorials are perhaps most conveniently run in a Jupyter notebook. See [here](https://jupyter.org/install) for instructions on how to install. ### Installation[โ€‹](#installation "Direct link to Installation") This guide requires `@langchain/community` and `pdf-parse`: * npm * yarn * pnpm npm i @langchain/community pdf-parse yarn add @langchain/community pdf-parse pnpm add @langchain/community pdf-parse For more details, see our [Installation guide](/docs/how_to/installation/) . ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com) . After you sign up at the link above, make sure to set your environment variables to start logging traces: export LANGSMITH_TRACING="true"export LANGSMITH_API_KEY="..."# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true Documents and Document Loaders[โ€‹](#documents-and-document-loaders "Direct link to Documents and Document Loaders") ------------------------------------------------------------------------------------------------------------------- LangChain implements a [Document](https://api.js.langchain.com/classes/_langchain_core.documents.Document.html) abstraction, which is intended to represent a unit of text and associated metadata. It has three attributes: * `pageContent`: a string representing the content; * `metadata`: records of arbitrary metadata; * `id`: (optional) a string identifier for the document. The `metadata` attribute can capture information about the source of the document, its relationship to other documents, and other information. Note that an individual `Document` object often represents a chunk of a larger document. We can generate sample documents when desired: import { Document } from "@langchain/core/documents";const documents = [ new Document({ pageContent: "Dogs are great companions, known for their loyalty and friendliness.", metadata: { source: "mammal-pets-doc" }, }), new Document({ pageContent: "Cats are independent pets that often enjoy their own space.", metadata: { source: "mammal-pets-doc" }, }),]; However, the LangChain ecosystem implements [document loaders](/docs/concepts/document_loaders) that [integrate with hundreds of common sources](/docs/integrations/document_loaders/) . This makes it easy to incorporate data from these sources into your AI application. ### Loading documents[โ€‹](#loading-documents "Direct link to Loading documents") Letโ€™s load a PDF into a sequence of `Document` objects. There is a sample PDF in the LangChain repo [here](https://github.com/langchain-ai/langchainjs/blob/main/docs/core_docs/data/nke-10k-2023.pdf) โ€“ a 10-k filing for Nike from 2023. LangChain implements a [PDFLoader](/docs/integrations/document_loaders/file_loaders/pdf/) that we can use to parse the PDF: import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";const loader = new PDFLoader("../../data/nke-10k-2023.pdf");const docs = await loader.load();console.log(docs.length); 107 tip See [this guide](/docs/how_to/document_loader_pdf/) for more detail on PDF document loaders. `PDFLoader` loads one `Document` object per PDF page. For each, we can easily access: * The string content of the page; * Metadata containing the file name and page number. docs[0].pageContent.slice(0, 200); Table of ContentsUNITED STATESSECURITIES AND EXCHANGE COMMISSIONWashington, D.C. 20549FORM 10-K(Mark One)โ˜‘ ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(D) OF THE SECURITIES EXCHANGE ACT OF 1934FO docs[0].metadata; { source: '../../data/nke-10k-2023.pdf', pdf: { version: '1.10.100', info: { PDFFormatVersion: '1.4', IsAcroFormPresent: false, IsXFAPresent: false, Title: '0000320187-23-000039', Author: 'EDGAR Online, a division of Donnelley Financial Solutions', Subject: 'Form 10-K filed on 2023-07-20 for the period ending 2023-05-31', Keywords: '0000320187-23-000039; ; 10-K', Creator: 'EDGAR Filing HTML Converter', Producer: 'EDGRpdf Service w/ EO.Pdf 22.0.40.0', CreationDate: "D:20230720162200-04'00'", ModDate: "D:20230720162208-04'00'" }, metadata: null, totalPages: 107 }, loc: { pageNumber: 1 }} ### Splitting[โ€‹](#splitting "Direct link to Splitting") For both information retrieval and downstream question-answering purposes, a page may be too coarse a representation. Our goal in the end will be to retrieve `Document` objects that answer an input query, and further splitting our PDF will help ensure that the meanings of relevant portions of the document are not โ€œwashed outโ€ by surrounding text. We can use [text splitters](/docs/concepts/text_splitters) for this purpose. Here we will use a simple text splitter that partitions based on characters. We will split our documents into chunks of 1000 characters with 200 characters of overlap between chunks. The overlap helps mitigate the possibility of separating a statement from important context related to it. We use the [RecursiveCharacterTextSplitter](/docs/how_to/recursive_text_splitter) , which will recursively split the document using common separators like new lines until each chunk is the appropriate size. This is the recommended text splitter for generic text use cases. We set `add_start_index=True` so that the character index where each split Document starts within the initial Document is preserved as metadata attribute โ€œstart\_indexโ€. See [this guide](/docs/how_to/document_loader_pdf/) for more detail about working with PDFs. import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200,});const allSplits = await textSplitter.splitDocuments(docs);allSplits.length; 513 Embeddings[โ€‹](#embeddings "Direct link to Embeddings") ------------------------------------------------------- Vector search is a common way to store and search over unstructured data (such as unstructured text). The idea is to store numeric vectors that are associated with the text. Given a query, we can [embed](/docs/concepts/embedding_models) it as a vector of the same dimension and use vector similarity metrics (such as cosine similarity) to identify related text. LangChain supports embeddings from [dozens of providers](/docs/integrations/text_embedding/) . These models specify how text should be converted into a numeric vector. Letโ€™s select a model. ### Pick your embedding model: * OpenAI * Azure * AWS * VertexAI * MistralAI * Cohere #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai OPENAI_API_KEY=your-api-key import { OpenAIEmbeddings } from "@langchain/openai";const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-large"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai AZURE_OPENAI_API_INSTANCE_NAME=AZURE_OPENAI_API_KEY=AZURE_OPENAI_API_VERSION="2024-02-01" import { AzureOpenAIEmbeddings } from "@langchain/openai";const embeddings = new AzureOpenAIEmbeddings({ azureOpenAIApiEmbeddingsDeploymentName: "text-embedding-ada-002"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/aws yarn add @langchain/aws pnpm add @langchain/aws BEDROCK_AWS_REGION=your-region import { BedrockEmbeddings } from "@langchain/aws";const embeddings = new BedrockEmbeddings({ model: "amazon.titan-embed-text-v1"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai GOOGLE_APPLICATION_CREDENTIALS=credentials.json import { VertexAIEmbeddings } from "@langchain/google-vertexai";const embeddings = new VertexAIEmbeddings({ model: "text-embedding-004"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai MISTRAL_API_KEY=your-api-key import { MistralAIEmbeddings } from "@langchain/mistralai";const embeddings = new MistralAIEmbeddings({ model: "mistral-embed"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/cohere yarn add @langchain/cohere pnpm add @langchain/cohere COHERE_API_KEY=your-api-key import { CohereEmbeddings } from "@langchain/cohere";const embeddings = new CohereEmbeddings({ model: "embed-english-v3.0"}); const vector1 = await embeddings.embedQuery(allSplits[0].pageContent);const vector2 = await embeddings.embedQuery(allSplits[1].pageContent);console.assert(vector1.length === vector2.length);console.log(`Generated vectors of length ${vector1.length}\n`);console.log(vector1.slice(0, 10)); Generated vectors of length 3072[ 0.014310152, -0.01681044, -0.0011537228, 0.010546423, 0.022808468, -0.028327717, -0.00058849837, 0.0419197, -0.0012900416, 0.0661778] Armed with a model for generating text embeddings, we can next store them in a special data structure that supports efficient similarity search. Vector stores[โ€‹](#vector-stores "Direct link to Vector stores") ---------------------------------------------------------------- LangChain [VectorStore](https://api.js.langchain.com/classes/_langchain_core.vectorstores.VectorStore.html) objects contain methods for adding text and `Document` objects to the store, and querying them using various similarity metrics. They are often initialized with [embedding](/docs/how_to/embed_text) models, which determine how text data is translated to numeric vectors. LangChain includes a suite of [integrations](/docs/integrations/vectorstores) with different vector store technologies. Some vector stores are hosted by a provider (e.g., various cloud providers) and require specific credentials to use; some (such as [Postgres](/docs/integrations/vectorstores/pgvector) ) run in separate infrastructure that can be run locally or via a third-party; others can run in-memory for lightweight workloads. ### Pick your vector store: * Memory * Chroma * FAISS * MongoDB * PGVector * Pinecone * Qdrant #### Install dependencies * npm * yarn * pnpm npm i langchain yarn add langchain pnpm add langchain import { MemoryVectorStore } from "langchain/vectorstores/memory";const vectorStore = new MemoryVectorStore(embeddings); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { Chroma } from "@langchain/community/vectorstores/chroma";const vectorStore = new Chroma(embeddings, { collectionName: "a-test-collection",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { FaissStore } from "@langchain/community/vectorstores/faiss";const vectorStore = new FaissStore(embeddings, {}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mongodb yarn add @langchain/mongodb pnpm add @langchain/mongodb import { MongoDBAtlasVectorSearch } from "@langchain/mongodb"import { MongoClient } from "mongodb";const client = new MongoClient(process.env.MONGODB_ATLAS_URI || "");const collection = client .db(process.env.MONGODB_ATLAS_DB_NAME) .collection(process.env.MONGODB_ATLAS_COLLECTION_NAME);const vectorStore = new MongoDBAtlasVectorSearch(embeddings, { collection: collection, indexName: "vector_index", textKey: "text", embeddingKey: "embedding",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import PGVectorStore from "@langchain/community/vectorstores/pgvector";const vectorStore = await PGVectorStore.initialize(embeddings, {}) #### Install dependencies * npm * yarn * pnpm npm i @langchain/pinecone yarn add @langchain/pinecone pnpm add @langchain/pinecone import { PineconeStore } from "@langchain/pinecone";import { Pinecone as PineconeClient } from "@pinecone-database/pinecone";const pinecone = new PineconeClient();const vectorStore = new PineconeStore(embeddings, { pineconeIndex, maxConcurrency: 5,}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/qdrant yarn add @langchain/qdrant pnpm add @langchain/qdrant import { QdrantVectorStore } from "@langchain/qdrant";const vectorStore = await QdrantVectorStore.fromExistingCollection(embeddings, { url: process.env.QDRANT_URL, collectionName: "langchainjs-testing",}); Having instantiated our vector store, we can now index the documents. await vectorStore.addDocuments(allSplits); Note that most vector store implementations will allow you to connect to an existing vector storeโ€“ e.g., by providing a client, index name, or other information. See the documentation for a specific [integration](/docs/integrations/vectorstores) for more detail. Once weโ€™ve instantiated a `VectorStore` that contains documents, we can query it. [VectorStore](https://api.js.langchain.com/classes/_langchain_core.vectorstores.VectorStore.html) includes methods for querying: - Synchronously and asynchronously; - By string query and by vector; - With and without returning similarity scores; - By similarity and [maximum marginal relevance](https://api.js.langchain.com/classes/_langchain_core.vectorstores.VectorStore.html#maxMarginalRelevanceSearch) (to balance similarity with query to diversity in retrieved results). The methods will generally include a list of [Document](https://api.js.langchain.com/classes/_langchain_core.documents.Document.html) objects in their outputs. ### Usage[โ€‹](#usage "Direct link to Usage") Embeddings typically represent text as a โ€œdenseโ€ vector such that texts with similar meanings are gemoetrically close. This lets us retrieve relevant information just by passing in a question, without knowledge of any specific key-terms used in the document. Return documents based on similarity to a string query: const results1 = await vectorStore.similaritySearch( "When was Nike incorporated?");results1[0]; Document { pageContent: 'Table of Contents\n' + 'PART I\n' + 'ITEM 1. BUSINESS\n' + 'GENERAL\n' + 'NIKE, Inc. was incorporated in 1967 under the laws of the State of Oregon. As used in this Annual Report on Form 10-K (this "Annual Report"), the terms "we," "us," "our,"\n' + '"NIKE" and the "Company" refer to NIKE, Inc. and its predecessors, subsidiaries and affiliates, collectively, unless the context indicates otherwise.\n' + 'Our principal business activity is the design, development and worldwide marketing and selling of athletic footwear, apparel, equipment, accessories and services. NIKE is\n' + 'the largest seller of athletic footwear and apparel in the world. We sell our products through NIKE Direct operations, which are comprised of both NIKE-owned retail stores\n' + 'and sales through our digital platforms (also referred to as "NIKE Brand Digital"), to retail accounts and to a mix of independent distributors, licensees and sales', metadata: { source: '../../data/nke-10k-2023.pdf', pdf: { version: '1.10.100', info: [Object], metadata: null, totalPages: 107 }, loc: { pageNumber: 4, lines: [Object] } }, id: undefined} Return scores: const results2 = await vectorStore.similaritySearchWithScore( "What was Nike's revenue in 2023?");results2[0]; [ Document { pageContent: 'Table of Contents\n' + 'FISCAL 2023 NIKE BRAND REVENUE HIGHLIGHTS\n' + 'The following tables present NIKE Brand revenues disaggregated by reportable operating segment, distribution channel and major product line:\n' + 'FISCAL 2023 COMPARED TO FISCAL 2022\n' + 'โ€ขNIKE, Inc. Revenues were $51.2 billion in fiscal 2023, which increased 10% and 16% compared to fiscal 2022 on a reported and currency-neutral basis, respectively.\n' + 'The increase was due to higher revenues in North America, Europe, Middle East & Africa ("EMEA"), APLA and Greater China, which contributed approximately 7, 6,\n' + '2 and 1 percentage points to NIKE, Inc. Revenues, respectively.\n' + 'โ€ขNIKE Brand revenues, which represented over 90% of NIKE, Inc. Revenues, increased 10% and 16% on a reported and currency-neutral basis, respectively. This\n' + "increase was primarily due to higher revenues in Men's, the Jordan Brand, Women's and Kids' which grew 17%, 35%,11% and 10%, respectively, on a wholesale\n" + 'equivalent basis.', metadata: { source: '../../data/nke-10k-2023.pdf', pdf: [Object], loc: [Object] }, id: undefined }, 0.6992287611800424] Return documents based on similarity to an embedded query: const embedding = await embeddings.embedQuery( "How were Nike's margins impacted in 2023?");const results3 = await vectorStore.similaritySearchVectorWithScore( embedding, 1);results3[0]; [ Document { pageContent: 'Table of Contents\n' + 'GROSS MARGIN\n' + 'FISCAL 2023 COMPARED TO FISCAL 2022\n' + 'For fiscal 2023, our consolidated gross profit increased 4% to $22,292 million compared to $21,479 million for fiscal 2022. Gross margin decreased 250 basis points to\n' + '43.5% for fiscal 2023 compared to 46.0% for fiscal 2022 due to the following:\n' + '*Wholesale equivalent\n' + 'The decrease in gross margin for fiscal 2023 was primarily due to:\n' + 'โ€ขHigher NIKE Brand product costs, on a wholesale equivalent basis, primarily due to higher input costs and elevated inbound freight and logistics costs as well as\n' + 'product mix;\n' + 'โ€ขLower margin in our NIKE Direct business, driven by higher promotional activity to liquidate inventory in the current period compared to lower promotional activity in\n' + 'the prior period resulting from lower available inventory supply;\n' + 'โ€ขUnfavorable changes in net foreign currency exchange rates, including hedges; and\n' + 'โ€ขLower off-price margin, on a wholesale equivalent basis.\n' + 'This was partially offset by:', metadata: { source: '../../data/nke-10k-2023.pdf', pdf: [Object], loc: [Object] }, id: undefined }, 0.7368815472158006] Learn more: * [API reference](https://api.js.langchain.com/classes/_langchain_core.vectorstores.VectorStore.html) * [How-to guide](/docs/how_to/vectorstores) * [Integration-specific docs](/docs/integrations/vectorstores) Retrievers[โ€‹](#retrievers "Direct link to Retrievers") ------------------------------------------------------- LangChain `VectorStore` objects do not subclass [Runnable](https://api.js.langchain.com/classes/_langchain_core.runnables.Runnable.html) . LangChain [Retrievers](https://api.js.langchain.com/classes/_langchain_core.retrievers.BaseRetriever.html) are Runnables, so they implement a standard set of methods (e.g., synchronous and asynchronous `invoke` and `batch` operations). Although we can construct retrievers from vector stores, retrievers can interface with non-vector store sources of data, as well (such as external APIs). Vectorstores implement an [as retriever](https://api.js.langchain.com/classes/_langchain_core.vectorstores.VectorStore.html#asRetriever) method that will generate a Retriever, specifically a [VectorStoreRetriever](https://api.js.langchain.com/classes/_langchain_core.vectorstores.VectorStoreRetriever.html) . These retrievers include specific `search_type` and `search_kwargs` attributes that identify what methods of the underlying vector store to call, and how to parameterize them. const retriever = vectorStore.asRetriever({ searchType: "mmr", searchKwargs: { fetchK: 1, },});await retriever.batch([ "When was Nike incorporated?", "What was Nike's revenue in 2023?",]); [ [ Document { pageContent: 'Table of Contents\n' + 'PART I\n' + 'ITEM 1. BUSINESS\n' + 'GENERAL\n' + 'NIKE, Inc. was incorporated in 1967 under the laws of the State of Oregon. As used in this Annual Report on Form 10-K (this "Annual Report"), the terms "we," "us," "our,"\n' + '"NIKE" and the "Company" refer to NIKE, Inc. and its predecessors, subsidiaries and affiliates, collectively, unless the context indicates otherwise.\n' + 'Our principal business activity is the design, development and worldwide marketing and selling of athletic footwear, apparel, equipment, accessories and services. NIKE is\n' + 'the largest seller of athletic footwear and apparel in the world. We sell our products through NIKE Direct operations, which are comprised of both NIKE-owned retail stores\n' + 'and sales through our digital platforms (also referred to as "NIKE Brand Digital"), to retail accounts and to a mix of independent distributors, licensees and sales', metadata: [Object], id: undefined } ], [ Document { pageContent: 'Table of Contents\n' + 'FISCAL 2023 NIKE BRAND REVENUE HIGHLIGHTS\n' + 'The following tables present NIKE Brand revenues disaggregated by reportable operating segment, distribution channel and major product line:\n' + 'FISCAL 2023 COMPARED TO FISCAL 2022\n' + 'โ€ขNIKE, Inc. Revenues were $51.2 billion in fiscal 2023, which increased 10% and 16% compared to fiscal 2022 on a reported and currency-neutral basis, respectively.\n' + 'The increase was due to higher revenues in North America, Europe, Middle East & Africa ("EMEA"), APLA and Greater China, which contributed approximately 7, 6,\n' + '2 and 1 percentage points to NIKE, Inc. Revenues, respectively.\n' + 'โ€ขNIKE Brand revenues, which represented over 90% of NIKE, Inc. Revenues, increased 10% and 16% on a reported and currency-neutral basis, respectively. This\n' + "increase was primarily due to higher revenues in Men's, the Jordan Brand, Women's and Kids' which grew 17%, 35%,11% and 10%, respectively, on a wholesale\n" + 'equivalent basis.', metadata: [Object], id: undefined } ]] `VectorStoreRetriever` supports search types of `"similarity"` (default) and `"mmr"` (maximum marginal relevance, described above). Retrievers can easily be incorporated into more complex applications, such as [retrieval-augmented generation (RAG)](/docs/concepts/rag) applications that combine a given question with retrieved context into a prompt for a LLM. To learn more about building such an application, check out the [RAG tutorial](/docs/tutorials/rag) tutorial. ### Learn more:[โ€‹](#learn-more "Direct link to Learn more:") Retrieval strategies can be rich and complex. For example: * We can [infer hard rules and filters](/docs/how_to/self_query/) from a query (e.g., โ€œusing documents published after 2020โ€); * We can [return documents that are linked](/docs/how_to/parent_document_retriever/) to the retrieved context in some way (e.g., via some document taxonomy); * We can generate [multiple embeddings](/docs/how_to/multi_vector) for each unit of context; * We can [ensemble results](/docs/how_to/ensemble_retriever) from multiple retrievers; * We can assign weights to documents, e.g., to weigh [recent documents](/docs/how_to/time_weighted_vectorstore/) higher. The [retrievers](/docs/how_to#retrievers) section of the how-to guides covers these and other built-in retrieval strategies. It is also straightforward to extend the [BaseRetriever](https://api.js.langchain.com/classes/_langchain_core.retrievers.BaseRetriever.html) class in order to implement custom retrievers. See our how-to guide [here](/docs/how_to/custom_retriever) . Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Youโ€™ve now seen how to build a semantic search engine over a PDF document. For more on document loaders: * [Conceptual guide](/docs/concepts/document_loaders) * [How-to guides](/docs/how_to/#document-loaders) * [Available integrations](/docs/integrations/document_loaders/) For more on embeddings: * [Conceptual guide](/docs/concepts/embedding_models/) * [How-to guides](/docs/how_to/#embedding-models) * [Available integrations](/docs/integrations/text_embedding/) For more on vector stores: * [Conceptual guide](/docs/concepts/vectorstores/) * [How-to guides](/docs/how_to/#vector-stores) * [Available integrations](/docs/integrations/vectorstores/) For more on RAG, see: * [Build a Retrieval Augmented Generation (RAG) App](/docs/tutorials/rag/) * [Related how-to guides](/docs/how_to/#qa-with-rag) * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Concepts](#concepts) * [Setup](#setup) * [Jupyter Notebook](#jupyter-notebook) * [Installation](#installation) * [LangSmith](#langsmith) * [Documents and Document Loaders](#documents-and-document-loaders) * [Loading documents](#loading-documents) * [Splitting](#splitting) * [Embeddings](#embeddings) * [Vector stores](#vector-stores) * [Usage](#usage) * [Retrievers](#retrievers) * [Learn more:](#learn-more) * [Next steps](#next-steps) --- # Installation | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Installation ============ Supported Environments[โ€‹](#supported-environments "Direct link to Supported Environments") ------------------------------------------------------------------------------------------- LangChain is written in TypeScript and can be used in: * Node.js (ESM and CommonJS) - 18.x, 19.x, 20.x * Cloudflare Workers * Vercel / Next.js (Browser, Serverless and Edge functions) * Supabase Edge Functions * Browser * Deno * Bun However, note that individual integrations may not be supported in all environments. Installation[โ€‹](#installation-1 "Direct link to Installation") --------------------------------------------------------------- To install the main `langchain` package, run: * npm * Yarn * pnpm npm install langchain @langchain/core yarn add langchain @langchain/core pnpm add langchain @langchain/core While this package acts as a sane starting point to using LangChain, much of the value of LangChain comes when integrating it with various model providers, datastores, etc. By default, the dependencies needed to do that are NOT installed. You will need to install the dependencies for specific integrations separately. We'll show how to do that in the next sections of this guide. Please also see the section on [installing integration packages](/docs/how_to/installation/#installing-integration-packages) for some special considerations when installing LangChain packages. Ecosystem packages[โ€‹](#ecosystem-packages "Direct link to Ecosystem packages") ------------------------------------------------------------------------------- With the exception of the `langsmith` SDK, all packages in the LangChain ecosystem depend on `@langchain/core`, which contains base classes and abstractions that other packages use. The dependency graph below shows how the difference packages are related. A directed arrow indicates that the source package depends on the target package: ![](/assets/images/ecosystem_packages-32943b32657e7a187770c9b585f22a64.png) **Note:** It is important that your app only uses one version of `@langchain/core`. Common package managers may introduce additional versions when resolving direct dependencies, even if you don't intend this. See [this section on installing integration packages](/docs/how_to/installation/#installing-integration-packages) for more information and ways to remedy this. ### @langchain/community[โ€‹](#langchaincommunity "Direct link to @langchain/community") The [@langchain/community](https://www.npmjs.com/package/@langchain/community) package contains a range of third-party integrations. Install with: * npm * Yarn * pnpm npm install @langchain/community @langchain/core yarn add @langchain/community @langchain/core pnpm add @langchain/community @langchain/core There are also more granular packages containing LangChain integrations for individual providers. ### @langchain/core[โ€‹](#langchaincore "Direct link to @langchain/core") The [@langchain/core](https://www.npmjs.com/package/@langchain/core) package contains base abstractions that the rest of the LangChain ecosystem uses, along with the LangChain Expression Language. It should be installed separately: * npm * Yarn * pnpm npm install @langchain/core yarn add @langchain/core pnpm add @langchain/core ### LangGraph[โ€‹](#langgraph "Direct link to LangGraph") [LangGraph.js](https://langchain-ai.github.io/langgraphjs/) is a library for building stateful, multi-actor applications with LLMs. It integrates smoothly with LangChain, but can be used without it. Install with: * npm * Yarn * pnpm npm install @langchain/langgraph @langchain/core yarn add @langchain/langgraph @langchain/core pnpm add @langchain/langgraph @langchain/core ### LangSmith SDK[โ€‹](#langsmith-sdk "Direct link to LangSmith SDK") The LangSmith SDK is automatically installed by LangChain. If you're not using it with LangChain, install with: * npm * Yarn * pnpm npm install langsmith yarn add langsmith pnpm add langsmith tip See [this section for general instructions on installing integration packages](/docs/how_to/installation#installing-integration-packages) . Installing integration packages[โ€‹](#installing-integration-packages "Direct link to Installing integration packages") ---------------------------------------------------------------------------------------------------------------------- LangChain supports packages that contain module integrations with individual third-party providers. They can be as specific as [`@langchain/anthropic`](/docs/integrations/platforms/anthropic/) , which contains integrations just for Anthropic models, or as broad as [`@langchain/community`](https://www.npmjs.com/package/@langchain/community) , which contains broader variety of community contributed integrations. These packages, as well as the main LangChain package, all have [`@langchain/core`](https://www.npmjs.com/package/@langchain/core) as a peer dependency to avoid package managers installing multiple versions of the same package. It contains the base abstractions that these integration packages extend. To ensure that all integrations and their types interact with each other properly, it is important that they all use the same version of `@langchain/core`. If you encounter type errors around base classes, you may need to guarantee that your package manager is resolving a single version of `@langchain/core`. To do so, you can add a `"resolutions"` or `"overrides"` field like the following in your project's `package.json`. The name will depend on your package manager: tip The `resolutions` or `pnpm.overrides` fields for `yarn` or `pnpm` must be set in the root `package.json` file. If you are using `yarn`: yarn package.json { "name": "your-project", "version": "0.0.0", "private": true, "engines": { "node": ">=18" }, "dependencies": { "@langchain/anthropic": "^0.0.2", "@langchain/core": "^0.3.0", "langchain": "0.0.207" }, "resolutions": { "@langchain/core": "0.3.0" }} You can also try running the [`yarn dedupe`](https://yarnpkg.com/cli/dedupe) command if you are on `yarn` version 2 or higher. Or for `npm`: npm package.json { "name": "your-project", "version": "0.0.0", "private": true, "engines": { "node": ">=18" }, "dependencies": { "@langchain/anthropic": "^0.0.2", "@langchain/core": "^0.3.0", "langchain": "0.0.207" }, "overrides": { "@langchain/core": "0.3.0" }} You can also try the [`npm dedupe`](https://docs.npmjs.com/cli/commands/npm-dedupe) command. Or for `pnpm`: pnpm package.json { "name": "your-project", "version": "0.0.0", "private": true, "engines": { "node": ">=18" }, "dependencies": { "@langchain/anthropic": "^0.0.2", "@langchain/core": "^0.3.0", "langchain": "0.0.207" }, "pnpm": { "overrides": { "@langchain/core": "0.3.0" } }} You can also try the [`pnpm dedupe`](https://pnpm.io/cli/dedupe) command. Loading the library[โ€‹](#loading-the-library "Direct link to Loading the library") ---------------------------------------------------------------------------------- ### TypeScript[โ€‹](#typescript "Direct link to TypeScript") LangChain is written in TypeScript and provides type definitions for all of its public APIs. ### ESM[โ€‹](#esm "Direct link to ESM") LangChain provides an ESM build targeting Node.js environments. You can import it using the following syntax: * npm * Yarn * pnpm npm install @langchain/openai @langchain/core yarn add @langchain/openai @langchain/core pnpm add @langchain/openai @langchain/core import { ChatOpenAI } from "@langchain/openai"; If you are using TypeScript in an ESM project we suggest updating your `tsconfig.json` to include the following: tsconfig.json { "compilerOptions": { ... "target": "ES2020", // or higher "module": "nodenext", }} ### CommonJS[โ€‹](#commonjs "Direct link to CommonJS") LangChain provides a CommonJS build targeting Node.js environments. You can import it using the following syntax: const { ChatOpenAI } = require("@langchain/openai"); ### Cloudflare Workers[โ€‹](#cloudflare-workers "Direct link to Cloudflare Workers") LangChain can be used in Cloudflare Workers. You can import it using the following syntax: import { ChatOpenAI } from "@langchain/openai"; ### Vercel / Next.js[โ€‹](#vercel--nextjs "Direct link to Vercel / Next.js") LangChain can be used in Vercel / Next.js. We support using LangChain in frontend components, in Serverless functions and in Edge functions. You can import it using the following syntax: import { ChatOpenAI } from "@langchain/openai"; ### Deno / Supabase Edge Functions[โ€‹](#deno--supabase-edge-functions "Direct link to Deno / Supabase Edge Functions") LangChain can be used in Deno / Supabase Edge Functions. You can import it using the following syntax: import { ChatOpenAI } from "https://esm.sh/@langchain/openai"; or import { ChatOpenAI } from "npm:@langchain/openai"; ### Browser[โ€‹](#browser "Direct link to Browser") LangChain can be used in the browser. In our CI we test bundling LangChain with Webpack and Vite, but other bundlers should work too. You can import it using the following syntax: import { ChatOpenAI } from "@langchain/openai"; Unsupported: Node.js 16[โ€‹](#unsupported-nodejs-16 "Direct link to Unsupported: Node.js 16") -------------------------------------------------------------------------------------------- We do not support Node.js 16, but if you still want to run LangChain on Node.js 16, you will need to follow the instructions in this section. We do not guarantee that these instructions will continue to work in the future. You will have to make `fetch` available globally, either: * run your application with `NODE_OPTIONS='--experimental-fetch' node ...`, or * install `node-fetch` and follow the instructions [here](https://github.com/node-fetch/node-fetch#providing-global-access) You'll also need to [polyfill `ReadableStream`](https://www.npmjs.com/package/web-streams-polyfill) by installing: * npm * Yarn * pnpm npm i web-streams-polyfill@4 yarn add web-streams-polyfill@4 pnpm add web-streams-polyfill@4 And then adding it to the global namespace in your main entrypoint: import "web-streams-polyfill/polyfill"; Additionally you'll have to polyfill `structuredClone`, eg. by installing `core-js` and following the instructions [here](https://github.com/zloirock/core-js) . If you are running Node.js 18+, you do not need to do anything. * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Supported Environments](#supported-environments) * [Installation](#installation-1) * [Ecosystem packages](#ecosystem-packages) * [@langchain/community](#langchaincommunity) * [@langchain/core](#langchaincore) * [LangGraph](#langgraph) * [LangSmith SDK](#langsmith-sdk) * [Installing integration packages](#installing-integration-packages) * [Loading the library](#loading-the-library) * [TypeScript](#typescript) * [ESM](#esm) * [CommonJS](#commonjs) * [Cloudflare Workers](#cloudflare-workers) * [Vercel / Next.js](#vercel--nextjs) * [Deno / Supabase Edge Functions](#deno--supabase-edge-functions) * [Browser](#browser) * [Unsupported: Node.js 16](#unsupported-nodejs-16) --- # How to cache chat model responses | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page How to cache chat model responses ================================= Prerequisites This guide assumes familiarity with the following concepts: * [Chat models](/docs/concepts/chat_models) * [LLMs](/docs/concepts/text_llms) LangChain provides an optional caching layer for chat models. This is useful for two reasons: It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider. import { ChatOpenAI } from "@langchain/openai";// To make the caching really obvious, lets use a slower model.const model = new ChatOpenAI({ model: "gpt-4", cache: true,}); In Memory Cache[โ€‹](#in-memory-cache "Direct link to In Memory Cache") ---------------------------------------------------------------------- The default cache is stored in-memory. This means that if you restart your application, the cache will be cleared. console.time();// The first time, it is not yet in cache, so it should take longerconst res = await model.invoke("Tell me a joke!");console.log(res);console.timeEnd();/* AIMessage { lc_serializable: true, lc_kwargs: { content: "Why don't scientists trust atoms?\n\nBecause they make up everything!", additional_kwargs: { function_call: undefined, tool_calls: undefined } }, lc_namespace: [ 'langchain_core', 'messages' ], content: "Why don't scientists trust atoms?\n\nBecause they make up everything!", name: undefined, additional_kwargs: { function_call: undefined, tool_calls: undefined } } default: 2.224s*/ console.time();// The second time it is, so it goes fasterconst res2 = await model.invoke("Tell me a joke!");console.log(res2);console.timeEnd();/* AIMessage { lc_serializable: true, lc_kwargs: { content: "Why don't scientists trust atoms?\n\nBecause they make up everything!", additional_kwargs: { function_call: undefined, tool_calls: undefined } }, lc_namespace: [ 'langchain_core', 'messages' ], content: "Why don't scientists trust atoms?\n\nBecause they make up everything!", name: undefined, additional_kwargs: { function_call: undefined, tool_calls: undefined } } default: 181.98ms*/ Caching with Redis[โ€‹](#caching-with-redis "Direct link to Caching with Redis") ------------------------------------------------------------------------------- LangChain also provides a Redis-based cache. This is useful if you want to share the cache across multiple processes or servers. To use it, you'll need to install the `redis` package: * npm * Yarn * pnpm npm install ioredis @langchain/community @langchain/core yarn add ioredis @langchain/community @langchain/core pnpm add ioredis @langchain/community @langchain/core Then, you can pass a `cache` option when you instantiate the LLM. For example: import { ChatOpenAI } from "@langchain/openai";import { Redis } from "ioredis";import { RedisCache } from "@langchain/community/caches/ioredis";const client = new Redis("redis://localhost:6379");const cache = new RedisCache(client, { ttl: 60, // Optional key expiration value});const model = new ChatOpenAI({ cache });const response1 = await model.invoke("Do something random!");console.log(response1);/* AIMessage { content: "Sure! I'll generate a random number for you: 37", additional_kwargs: {} }*/const response2 = await model.invoke("Do something random!");console.log(response2);/* AIMessage { content: "Sure! I'll generate a random number for you: 37", additional_kwargs: {} }*/await client.disconnect(); #### API Reference: * ChatOpenAI from `@langchain/openai` * RedisCache from `@langchain/community/caches/ioredis` Caching with Upstash Redis[โ€‹](#caching-with-upstash-redis "Direct link to Caching with Upstash Redis") ------------------------------------------------------------------------------------------------------- LangChain provides an Upstash Redis-based cache. Like the Redis-based cache, this cache is useful if you want to share the cache across multiple processes or servers. The Upstash Redis client uses HTTP and supports edge environments. To use it, you'll need to install the `@upstash/redis` package: * npm * Yarn * pnpm npm install @upstash/redis yarn add @upstash/redis pnpm add @upstash/redis You'll also need an [Upstash account](https://docs.upstash.com/redis#create-account) and a [Redis database](https://docs.upstash.com/redis#create-a-database) to connect to. Once you've done that, retrieve your REST URL and REST token. Then, you can pass a `cache` option when you instantiate the LLM. For example: import { ChatOpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// See https://docs.upstash.com/redis/howto/connectwithupstashredis#quick-start for connection optionsconst cache = new UpstashRedisCache({ config: { url: "UPSTASH_REDIS_REST_URL", token: "UPSTASH_REDIS_REST_TOKEN", }, ttl: 3600,});const model = new ChatOpenAI({ cache }); #### API Reference: * ChatOpenAI from `@langchain/openai` * UpstashRedisCache from `@langchain/community/caches/upstash_redis` You can also directly pass in a previously created [@upstash/redis](https://docs.upstash.com/redis/sdks/javascriptsdk/overview) client instance: import { Redis } from "@upstash/redis";import https from "https";import { ChatOpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// const client = new Redis({// url: process.env.UPSTASH_REDIS_REST_URL!,// token: process.env.UPSTASH_REDIS_REST_TOKEN!,// agent: new https.Agent({ keepAlive: true }),// });// Or simply call Redis.fromEnv() to automatically load the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN environment variables.const client = Redis.fromEnv({ agent: new https.Agent({ keepAlive: true }),});const cache = new UpstashRedisCache({ client });const model = new ChatOpenAI({ cache }); #### API Reference: * ChatOpenAI from `@langchain/openai` * UpstashRedisCache from `@langchain/community/caches/upstash_redis` Caching with Vercel KV[โ€‹](#caching-with-vercel-kv "Direct link to Caching with Vercel KV") ------------------------------------------------------------------------------------------- LangChain provides an Vercel KV-based cache. Like the Redis-based cache, this cache is useful if you want to share the cache across multiple processes or servers. The Vercel KV client uses HTTP and supports edge environments. To use it, you'll need to install the `@vercel/kv` package: * npm * Yarn * pnpm npm install @vercel/kv yarn add @vercel/kv pnpm add @vercel/kv You'll also need an Vercel account and a [KV database](https://vercel.com/docs/storage/vercel-kv/kv-reference) to connect to. Once you've done that, retrieve your REST URL and REST token. Then, you can pass a `cache` option when you instantiate the LLM. For example: import { ChatOpenAI } from "@langchain/openai";import { VercelKVCache } from "@langchain/community/caches/vercel_kv";import { createClient } from "@vercel/kv";// See https://vercel.com/docs/storage/vercel-kv/kv-reference#createclient-example for connection optionsconst cache = new VercelKVCache({ client: createClient({ url: "VERCEL_KV_API_URL", token: "VERCEL_KV_API_TOKEN", }), ttl: 3600,});const model = new ChatOpenAI({ model: "gpt-4o-mini", cache,}); #### API Reference: * ChatOpenAI from `@langchain/openai` * VercelKVCache from `@langchain/community/caches/vercel_kv` Caching with Cloudflare KV[โ€‹](#caching-with-cloudflare-kv "Direct link to Caching with Cloudflare KV") ------------------------------------------------------------------------------------------------------- info This integration is only supported in Cloudflare Workers. If you're deploying your project as a Cloudflare Worker, you can use LangChain's Cloudflare KV-powered LLM cache. For information on how to set up KV in Cloudflare, see [the official documentation](https://developers.cloudflare.com/kv/) . **Note:** If you are using TypeScript, you may need to install types if they aren't already present: * npm * Yarn * pnpm npm install -S @cloudflare/workers-types yarn add @cloudflare/workers-types pnpm add @cloudflare/workers-types import type { KVNamespace } from "@cloudflare/workers-types";import { ChatOpenAI } from "@langchain/openai";import { CloudflareKVCache } from "@langchain/cloudflare";export interface Env { KV_NAMESPACE: KVNamespace; OPENAI_API_KEY: string;}export default { async fetch(_request: Request, env: Env) { try { const cache = new CloudflareKVCache(env.KV_NAMESPACE); const model = new ChatOpenAI({ cache, model: "gpt-3.5-turbo", apiKey: env.OPENAI_API_KEY, }); const response = await model.invoke("How are you today?"); return new Response(JSON.stringify(response), { headers: { "content-type": "application/json" }, }); } catch (err: any) { console.log(err.message); return new Response(err.message, { status: 500 }); } },}; #### API Reference: * ChatOpenAI from `@langchain/openai` * CloudflareKVCache from `@langchain/cloudflare` Caching on the File System[โ€‹](#caching-on-the-file-system "Direct link to Caching on the File System") ------------------------------------------------------------------------------------------------------- danger This cache is not recommended for production use. It is only intended for local development. LangChain provides a simple file system cache. By default the cache is stored a temporary directory, but you can specify a custom directory if you want. const cache = await LocalFileCache.create(); Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- You've now learned how to cache model responses to save time and money. Next, check out the other how-to guides on chat models, like [how to get a model to return structured output](/docs/how_to/structured_output) or [how to create your own custom chat model](/docs/how_to/custom_chat) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [In Memory Cache](#in-memory-cache) * [Caching with Redis](#caching-with-redis) * [Caching with Upstash Redis](#caching-with-upstash-redis) * [Caching with Vercel KV](#caching-with-vercel-kv) * [Caching with Cloudflare KV](#caching-with-cloudflare-kv) * [Caching on the File System](#caching-on-the-file-system) * [Next steps](#next-steps) --- # How to stream chat model responses | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page All [chat models](https://api.js.langchain.com/classes/langchain_core.language_models_chat_models.BaseChatModel.html) implement the [Runnable interface](https://.api.js.langchain.com/classes/langchain_core.runnables.Runnable.html) , which comes with **default** implementations of standard runnable methods (i.e.ย `invoke`, `batch`, `stream`, `streamEvents`). This guide covers how to use these methods to stream output from chat models. tip The **default** implementation does **not** provide support for token-by-token streaming, and will instead return an [`AsyncGenerator`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncGenerator) that will yield all model output in a single chunk. It exists to ensures that the the model can be swapped in for any other model as it supports the same standard interface. The ability to stream the output token-by-token depends on whether the provider has implemented token-by-token streaming support. You can see which [integrations support token-by-token streaming here](/docs/integrations/chat/) . Streaming[โ€‹](#streaming "Direct link to Streaming") ---------------------------------------------------- Below, we use a `---` to help visualize the delimiter between tokens. ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); const stream = await model.stream( "Write me a 1 verse song about goldfish on the moon");for await (const chunk of stream) { console.log(`${chunk.content}\n---`);} ---Here's--- a one-------verse song about goldfish on--- the moon:Verse---:Swimming--- through the stars---,--- in--- a cosmic--- lag---oon---Little--- golden--- scales---,--- reflecting the moon---No--- gravity to--- hold them,--- they--- float with--- gleeGoldfish--- astron---auts, on a lunar--- sp---ree---Bub---bles rise--- like--- com---ets, in the--- star---ry night---Their fins like--- tiny--- rockets, a--- w---ondrous sightWho--- knew--- these--- small--- creatures---,--- could con---quer space?---Goldfish on the moon,--- with--- such--- fis---hy grace--------- Stream events[โ€‹](#stream-events "Direct link to Stream events") ---------------------------------------------------------------- Chat models also support the standard [`streamEvents()`](https://api.js.langchain.com/classes/langchain_core.runnables.Runnable.html#streamEvents) method to stream more granular events from within chains. This method is useful if youโ€™re streaming output from a larger LLM application that contains multiple steps (e.g., a chain composed of a prompt, chat model and parser): const eventStream = await model.streamEvents( "Write me a 1 verse song about goldfish on the moon", { version: "v2", });const events = [];for await (const event of eventStream) { events.push(event);}events.slice(0, 3); [ { event: "on_chat_model_start", data: { input: "Write me a 1 verse song about goldfish on the moon" }, name: "ChatAnthropic", tags: [], run_id: "d60a87d6-acf0-4ae1-bf27-e570aa101960", metadata: { ls_provider: "openai", ls_model_name: "claude-3-5-sonnet-20240620", ls_model_type: "chat", ls_temperature: 1, ls_max_tokens: 2048, ls_stop: undefined } }, { event: "on_chat_model_stream", run_id: "d60a87d6-acf0-4ae1-bf27-e570aa101960", name: "ChatAnthropic", tags: [], metadata: { ls_provider: "openai", ls_model_name: "claude-3-5-sonnet-20240620", ls_model_type: "chat", ls_temperature: 1, ls_max_tokens: 2048, ls_stop: undefined }, data: { chunk: AIMessageChunk { lc_serializable: true, lc_kwargs: { content: "", additional_kwargs: [Object], tool_calls: [], invalid_tool_calls: [], tool_call_chunks: [], response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "", name: undefined, additional_kwargs: { id: "msg_01JaaH9ZUXg7bUnxzktypRak", type: "message", role: "assistant", model: "claude-3-5-sonnet-20240620" }, response_metadata: {}, id: undefined, tool_calls: [], invalid_tool_calls: [], tool_call_chunks: [], usage_metadata: undefined } } }, { event: "on_chat_model_stream", run_id: "d60a87d6-acf0-4ae1-bf27-e570aa101960", name: "ChatAnthropic", tags: [], metadata: { ls_provider: "openai", ls_model_name: "claude-3-5-sonnet-20240620", ls_model_type: "chat", ls_temperature: 1, ls_max_tokens: 2048, ls_stop: undefined }, data: { chunk: AIMessageChunk { lc_serializable: true, lc_kwargs: { content: "Here's", additional_kwargs: {}, tool_calls: [], invalid_tool_calls: [], tool_call_chunks: [], response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "Here's", name: undefined, additional_kwargs: {}, response_metadata: {}, id: undefined, tool_calls: [], invalid_tool_calls: [], tool_call_chunks: [], usage_metadata: undefined } } }] Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Youโ€™ve now seen a few ways you can stream chat model responses. Next, check out this guide for more on [streaming with other LangChain modules](/docs/how_to/streaming) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Streaming](#streaming) * [Stream events](#stream-events) * [Next steps](#next-steps) --- # How to embed text data | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page How to embed text data ====================== info Head to [Integrations](/docs/integrations/text_embedding) for documentation on built-in integrations with text embedding providers. Prerequisites This guide assumes familiarity with the following concepts: * [Embeddings](/docs/concepts/embedding_models) Embeddings create a vector representation of a piece of text. This is useful because it means we can think about text in the vector space, and do things like semantic search where we look for pieces of text that are most similar in the vector space. The base Embeddings class in LangChain exposes two methods: one for embedding documents and one for embedding a query. The former takes as input multiple texts, while the latter takes a single text. The reason for having these as two separate methods is that some embedding providers have different embedding methods for documents (to be searched over) vs queries (the search query itself). Get started[โ€‹](#get-started "Direct link to Get started") ---------------------------------------------------------- Below is an example of how to use the OpenAI embeddings. Embeddings occasionally have different embedding methods for queries versus documents, so the embedding class exposes a `embedQuery` and `embedDocuments` method. tip See [this section for general instructions on installing integration packages](/docs/how_to/installation#installing-integration-packages) . * npm * Yarn * pnpm npm install @langchain/openai @langchain/core yarn add @langchain/openai @langchain/core pnpm add @langchain/openai @langchain/core Get started[โ€‹](#get-started-1 "Direct link to Get started") ------------------------------------------------------------ import { OpenAIEmbeddings } from "@langchain/openai";const embeddings = new OpenAIEmbeddings(); Embed queries[โ€‹](#embed-queries "Direct link to Embed queries") ---------------------------------------------------------------- const res = await embeddings.embedQuery("Hello world");/*[ -0.004845875, 0.004899438, -0.016358767, -0.024475135, -0.017341806, 0.012571548, -0.019156644, 0.009036391, -0.010227379, -0.026945334, 0.022861943, 0.010321903, -0.023479493, -0.0066544134, 0.007977734, 0.0026371893, 0.025206111, -0.012048521, 0.012943339, 0.013094575, -0.010580265, -0.003509951, 0.004070787, 0.008639394, -0.020631202, ... 1511 more items]*/ Embed documents[โ€‹](#embed-documents "Direct link to Embed documents") ---------------------------------------------------------------------- const documentRes = await embeddings.embedDocuments(["Hello world", "Bye bye"]);/*[ [ -0.004845875, 0.004899438, -0.016358767, -0.024475135, -0.017341806, 0.012571548, -0.019156644, 0.009036391, -0.010227379, -0.026945334, 0.022861943, 0.010321903, -0.023479493, -0.0066544134, 0.007977734, 0.0026371893, 0.025206111, -0.012048521, 0.012943339, 0.013094575, -0.010580265, -0.003509951, 0.004070787, 0.008639394, -0.020631202, ... 1511 more items ] [ -0.009446913, -0.013253193, 0.013174579, 0.0057552797, -0.038993083, 0.0077763423, -0.0260478, -0.0114384955, -0.0022683728, -0.016509168, 0.041797023, 0.01787183, 0.00552271, -0.0049789557, 0.018146982, -0.01542166, 0.033752076, 0.006112323, 0.023872782, -0.016535373, -0.006623321, 0.016116094, -0.0061090477, -0.0044155475, -0.016627092, ... 1511 more items ]]*/ Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- You've now learned how to use embeddings models with queries and text. Next, check out how to [avoid excessively recomputing embeddings with caching](/docs/how_to/caching_embeddings) , or the [full tutorial on retrieval-augmented generation](/docs/tutorials/rag) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Get started](#get-started) * [Get started](#get-started-1) * [Embed queries](#embed-queries) * [Embed documents](#embed-documents) * [Next steps](#next-steps) --- # How to use few shot examples in chat models | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page This guide covers how to prompt a chat model with example inputs and outputs. Providing the model with a few such examples is called few-shotting, and is a simple yet powerful way to guide generation and in some cases drastically improve model performance. There does not appear to be solid consensus on how best to do few-shot prompting, and the optimal prompt compilation will likely vary by model. Because of this, we provide few-shot prompt templates like the [FewShotChatMessagePromptTemplate](https://api.js.langchain.com/classes/langchain_core.prompts.FewShotChatMessagePromptTemplate.html) as a flexible starting point, and you can modify or replace them as you see fit. The goal of few-shot prompt templates are to dynamically select examples based on an input, and then format the examples in a final prompt to provide for the model. **Note:** The following code examples are for chat models only, since `FewShotChatMessagePromptTemplates` are designed to output formatted [chat messages](/docs/concepts/messages) rather than pure strings. For similar few-shot prompt examples for pure string templates compatible with completion models (LLMs), see the [few-shot prompt templates](/docs/how_to/few_shot_examples/) guide. Prerequisites This guide assumes familiarity with the following concepts: * [Prompt templates](/docs/concepts/prompt_templates) * [Example selectors](/docs/concepts/example_selectors) * [Chat models](/docs/concepts/chat_models) * [Vectorstores](/docs/concepts/#vectorstores) Fixed Examples[โ€‹](#fixed-examples "Direct link to Fixed Examples") ------------------------------------------------------------------- The most basic (and common) few-shot prompting technique is to use fixed prompt examples. This way you can select a chain, evaluate it, and avoid worrying about additional moving parts in production. The basic components of the template are: - `examples`: An array of object examples to include in the final prompt. - `examplePrompt`: converts each example into 1 or more messages through its [`formatMessages`](https://api.js.langchain.com/classes/langchain_core.prompts.FewShotChatMessagePromptTemplate.html#formatMessages) method. A common example would be to convert each example into one human message and one AI message response, or a human message followed by a function call message. Below is a simple demonstration. First, define the examples youโ€™d like to include: import { ChatPromptTemplate, FewShotChatMessagePromptTemplate,} from "@langchain/core/prompts";const examples = [ { input: "2+2", output: "4" }, { input: "2+3", output: "5" },]; Next, assemble them into the few-shot prompt template. // This is a prompt template used to format each individual example.const examplePrompt = ChatPromptTemplate.fromMessages([ ["human", "{input}"], ["ai", "{output}"],]);const fewShotPrompt = new FewShotChatMessagePromptTemplate({ examplePrompt, examples, inputVariables: [], // no input variables});const result = await fewShotPrompt.invoke({});console.log(result.toChatMessages()); [ HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+2", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+2", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "4", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "4", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] }, HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+3", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+3", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "5", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "5", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] }] Finally, we assemble the final prompt as shown below, passing `fewShotPrompt` directly into the `fromMessages` factory method, and use it with a model: const finalPrompt = ChatPromptTemplate.fromMessages([ ["system", "You are a wondrous wizard of math."], fewShotPrompt, ["human", "{input}"],]); ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); const chain = finalPrompt.pipe(model);await chain.invoke({ input: "What's the square of a triangle?" }); AIMessage { lc_serializable: true, lc_kwargs: { content: "A triangle does not have a square. The square of a number is the result of multiplying the number by"... 8 more characters, tool_calls: [], invalid_tool_calls: [], additional_kwargs: { function_call: undefined, tool_calls: undefined }, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "A triangle does not have a square. The square of a number is the result of multiplying the number by"... 8 more characters, name: undefined, additional_kwargs: { function_call: undefined, tool_calls: undefined }, response_metadata: { tokenUsage: { completionTokens: 23, promptTokens: 52, totalTokens: 75 }, finish_reason: "stop" }, tool_calls: [], invalid_tool_calls: []} Dynamic few-shot prompting[โ€‹](#dynamic-few-shot-prompting "Direct link to Dynamic few-shot prompting") ------------------------------------------------------------------------------------------------------- Sometimes you may want to select only a few examples from your overall set to show based on the input. For this, you can replace the `examples` passed into `FewShotChatMessagePromptTemplate` with an `exampleSelector`. The other components remain the same as above! Our dynamic few-shot prompt template would look like: * `exampleSelector`: responsible for selecting few-shot examples (and the order in which they are returned) for a given input. These implement the [BaseExampleSelector](https://api.js.langchain.com/classes/langchain_core.example_selectors.BaseExampleSelector.html) interface. A common example is the vectorstore-backed [SemanticSimilarityExampleSelector](https://api.js.langchain.com/classes/langchain_core.example_selectors.SemanticSimilarityExampleSelector.html) * `examplePrompt`: convert each example into 1 or more messages through its [`formatMessages`](https://api.js.langchain.com/classes/langchain_core.prompts.FewShotChatMessagePromptTemplate.html#formatMessages) method. A common example would be to convert each example into one human message and one AI message response, or a human message followed by a function call message. These once again can be composed with other messages and chat templates to assemble your final prompt. Letโ€™s walk through an example with the `SemanticSimilarityExampleSelector`. Since this implementation uses a vectorstore to select examples based on semantic similarity, we will want to first populate the store. Since the basic idea here is that we want to search for and return examples most similar to the text input, we embed the `values` of our prompt examples rather than considering the keys: import { SemanticSimilarityExampleSelector } from "@langchain/core/example_selectors";import { MemoryVectorStore } from "langchain/vectorstores/memory";import { OpenAIEmbeddings } from "@langchain/openai";const examples = [ { input: "2+2", output: "4" }, { input: "2+3", output: "5" }, { input: "2+4", output: "6" }, { input: "What did the cow say to the moon?", output: "nothing at all" }, { input: "Write me a poem about the moon", output: "One for the moon, and one for me, who are we to talk about the moon?", },];const toVectorize = examples.map( (example) => `${example.input} ${example.output}`);const embeddings = new OpenAIEmbeddings();const vectorStore = await MemoryVectorStore.fromTexts( toVectorize, examples, embeddings); ### Create the `exampleSelector`[โ€‹](#create-the-exampleselector "Direct link to create-the-exampleselector") With a vectorstore created, we can create the `exampleSelector`. Here we will call it in isolation, and set `k` on it to only fetch the two example closest to the input. const exampleSelector = new SemanticSimilarityExampleSelector({ vectorStore, k: 2,});// The prompt template will load examples by passing the input do the `select_examples` methodawait exampleSelector.selectExamples({ input: "horse" }); [ { input: "What did the cow say to the moon?", output: "nothing at all" }, { input: "2+4", output: "6" }] ### Create prompt template[โ€‹](#create-prompt-template "Direct link to Create prompt template") We now assemble the prompt template, using the `exampleSelector` created above. import { ChatPromptTemplate, FewShotChatMessagePromptTemplate,} from "@langchain/core/prompts";// Define the few-shot prompt.const fewShotPrompt = new FewShotChatMessagePromptTemplate({ // The input variables select the values to pass to the example_selector inputVariables: ["input"], exampleSelector, // Define how ech example will be formatted. // In this case, each example will become 2 messages: // 1 human, and 1 AI examplePrompt: ChatPromptTemplate.fromMessages([ ["human", "{input}"], ["ai", "{output}"], ]),});const results = await fewShotPrompt.invoke({ input: "What's 3+3?" });console.log(results.toChatMessages()); [ HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+3", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+3", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "5", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "5", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] }, HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+2", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+2", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "4", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "4", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] }] And we can pass this few-shot chat message prompt template into another chat prompt template: const finalPrompt = ChatPromptTemplate.fromMessages([ ["system", "You are a wondrous wizard of math."], fewShotPrompt, ["human", "{input}"],]);const result = await fewShotPrompt.invoke({ input: "What's 3+3?" });console.log(result); ChatPromptValue { lc_serializable: true, lc_kwargs: { messages: [ HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+3", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+3", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "5", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "5", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] }, HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+2", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+2", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "4", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "4", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] } ] }, lc_namespace: [ "langchain_core", "prompt_values" ], messages: [ HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+3", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+3", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "5", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "5", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] }, HumanMessage { lc_serializable: true, lc_kwargs: { content: "2+2", additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "2+2", name: undefined, additional_kwargs: {}, response_metadata: {} }, AIMessage { lc_serializable: true, lc_kwargs: { content: "4", tool_calls: [], invalid_tool_calls: [], additional_kwargs: {}, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "4", name: undefined, additional_kwargs: {}, response_metadata: {}, tool_calls: [], invalid_tool_calls: [] } ]} ### Use with an chat model[โ€‹](#use-with-an-chat-model "Direct link to Use with an chat model") Finally, you can connect your model to the few-shot prompt. ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); const chain = finalPrompt.pipe(model);await chain.invoke({ input: "What's 3+3?" }); AIMessage { lc_serializable: true, lc_kwargs: { content: "6", tool_calls: [], invalid_tool_calls: [], additional_kwargs: { function_call: undefined, tool_calls: undefined }, response_metadata: {} }, lc_namespace: [ "langchain_core", "messages" ], content: "6", name: undefined, additional_kwargs: { function_call: undefined, tool_calls: undefined }, response_metadata: { tokenUsage: { completionTokens: 1, promptTokens: 51, totalTokens: 52 }, finish_reason: "stop" }, tool_calls: [], invalid_tool_calls: []} Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Youโ€™ve now learned how to add few-shot examples to your chat prompts. Next, check out the other how-to guides on prompt templates in this section, the related how-to guide on [few shotting with text completion models](/docs/how_to/few_shot_examples) , or the other [example selector how-to guides](/docs/how_to/example_selectors/) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Fixed Examples](#fixed-examples) * [Dynamic few-shot prompting](#dynamic-few-shot-prompting) * [Create the `exampleSelector`](#create-the-exampleselector) * [Create prompt template](#create-prompt-template) * [Use with an chat model](#use-with-an-chat-model) * [Next steps](#next-steps) --- # How to cache model responses | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page How to cache model responses ============================ LangChain provides an optional caching layer for LLMs. This is useful for two reasons: It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider. tip See [this section for general instructions on installing integration packages](/docs/how_to/installation#installing-integration-packages) . * npm * Yarn * pnpm npm install @langchain/openai @langchain/core yarn add @langchain/openai @langchain/core pnpm add @langchain/openai @langchain/core import { OpenAI } from "@langchain/openai";const model = new OpenAI({ model: "gpt-3.5-turbo-instruct", cache: true,}); In Memory Cache[โ€‹](#in-memory-cache "Direct link to In Memory Cache") ---------------------------------------------------------------------- The default cache is stored in-memory. This means that if you restart your application, the cache will be cleared. console.time();// The first time, it is not yet in cache, so it should take longerconst res = await model.invoke("Tell me a long joke");console.log(res);console.timeEnd();/* A man walks into a bar and sees a jar filled with money on the counter. Curious, he asks the bartender about it. The bartender explains, "We have a challenge for our customers. If you can complete three tasks, you win all the money in the jar." Intrigued, the man asks what the tasks are. The bartender replies, "First, you have to drink a whole bottle of tequila without making a face. Second, there's a pitbull out back with a sore tooth. You have to pull it out. And third, there's an old lady upstairs who has never had an orgasm. You have to give her one." The man thinks for a moment and then confidently says, "I'll do it." He grabs the bottle of tequila and downs it in one gulp, without flinching. He then heads to the back and after a few minutes of struggling, emerges with the pitbull's tooth in hand. The bar erupts in cheers and the bartender leads the man upstairs to the old lady's room. After a few minutes, the man walks out with a big smile on his face and the old lady is giggling with delight. The bartender hands the man the jar of money and asks, "How default: 4.187s*/ console.time();// The second time it is, so it goes fasterconst res2 = await model.invoke("Tell me a joke");console.log(res2);console.timeEnd();/* A man walks into a bar and sees a jar filled with money on the counter. Curious, he asks the bartender about it. The bartender explains, "We have a challenge for our customers. If you can complete three tasks, you win all the money in the jar." Intrigued, the man asks what the tasks are. The bartender replies, "First, you have to drink a whole bottle of tequila without making a face. Second, there's a pitbull out back with a sore tooth. You have to pull it out. And third, there's an old lady upstairs who has never had an orgasm. You have to give her one." The man thinks for a moment and then confidently says, "I'll do it." He grabs the bottle of tequila and downs it in one gulp, without flinching. He then heads to the back and after a few minutes of struggling, emerges with the pitbull's tooth in hand. The bar erupts in cheers and the bartender leads the man upstairs to the old lady's room. After a few minutes, the man walks out with a big smile on his face and the old lady is giggling with delight. The bartender hands the man the jar of money and asks, "How default: 175.74ms*/ Caching with Momento[โ€‹](#caching-with-momento "Direct link to Caching with Momento") ------------------------------------------------------------------------------------- LangChain also provides a Momento-based cache. [Momento](https://gomomento.com) is a distributed, serverless cache that requires zero setup or infrastructure maintenance. Given Momento's compatibility with Node.js, browser, and edge environments, ensure you install the relevant package. To install for **Node.js**: * npm * Yarn * pnpm npm install @gomomento/sdk yarn add @gomomento/sdk pnpm add @gomomento/sdk To install for **browser/edge workers**: * npm * Yarn * pnpm npm install @gomomento/sdk-web yarn add @gomomento/sdk-web pnpm add @gomomento/sdk-web Next you'll need to sign up and create an API key. Once you've done that, pass a `cache` option when you instantiate the LLM like this: import { OpenAI } from "@langchain/openai";import { CacheClient, Configurations, CredentialProvider,} from "@gomomento/sdk";import { MomentoCache } from "@langchain/community/caches/momento";// See https://github.com/momentohq/client-sdk-javascript for connection optionsconst client = new CacheClient({ configuration: Configurations.Laptop.v1(), credentialProvider: CredentialProvider.fromEnvironmentVariable({ environmentVariableName: "MOMENTO_API_KEY", }), defaultTtlSeconds: 60 * 60 * 24,});const cache = await MomentoCache.fromProps({ client, cacheName: "langchain",});const model = new OpenAI({ cache }); #### API Reference: * OpenAI from `@langchain/openai` * MomentoCache from `@langchain/community/caches/momento` Caching with Redis[โ€‹](#caching-with-redis "Direct link to Caching with Redis") ------------------------------------------------------------------------------- LangChain also provides a Redis-based cache. This is useful if you want to share the cache across multiple processes or servers. To use it, you'll need to install the `redis` package: * npm * Yarn * pnpm npm install ioredis yarn add ioredis pnpm add ioredis Then, you can pass a `cache` option when you instantiate the LLM. For example: import { OpenAI } from "@langchain/openai";import { RedisCache } from "@langchain/community/caches/ioredis";import { Redis } from "ioredis";// See https://github.com/redis/ioredis for connection optionsconst client = new Redis({});const cache = new RedisCache(client);const model = new OpenAI({ cache }); Caching with Upstash Redis[โ€‹](#caching-with-upstash-redis "Direct link to Caching with Upstash Redis") ------------------------------------------------------------------------------------------------------- LangChain provides an Upstash Redis-based cache. Like the Redis-based cache, this cache is useful if you want to share the cache across multiple processes or servers. The Upstash Redis client uses HTTP and supports edge environments. To use it, you'll need to install the `@upstash/redis` package: * npm * Yarn * pnpm npm install @upstash/redis yarn add @upstash/redis pnpm add @upstash/redis You'll also need an [Upstash account](https://docs.upstash.com/redis#create-account) and a [Redis database](https://docs.upstash.com/redis#create-a-database) to connect to. Once you've done that, retrieve your REST URL and REST token. Then, you can pass a `cache` option when you instantiate the LLM. For example: import { OpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// See https://docs.upstash.com/redis/howto/connectwithupstashredis#quick-start for connection optionsconst cache = new UpstashRedisCache({ config: { url: "UPSTASH_REDIS_REST_URL", token: "UPSTASH_REDIS_REST_TOKEN", }, ttl: 3600,});const model = new OpenAI({ cache }); #### API Reference: * OpenAI from `@langchain/openai` * UpstashRedisCache from `@langchain/community/caches/upstash_redis` You can also directly pass in a previously created [@upstash/redis](https://docs.upstash.com/redis/sdks/javascriptsdk/overview) client instance: import { Redis } from "@upstash/redis";import https from "https";import { OpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// const client = new Redis({// url: process.env.UPSTASH_REDIS_REST_URL!,// token: process.env.UPSTASH_REDIS_REST_TOKEN!,// agent: new https.Agent({ keepAlive: true }),// });// Or simply call Redis.fromEnv() to automatically load the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN environment variables.const client = Redis.fromEnv({ agent: new https.Agent({ keepAlive: true }),});const cache = new UpstashRedisCache({ client });const model = new OpenAI({ cache }); #### API Reference: * OpenAI from `@langchain/openai` * UpstashRedisCache from `@langchain/community/caches/upstash_redis` Caching with Vercel KV[โ€‹](#caching-with-vercel-kv "Direct link to Caching with Vercel KV") ------------------------------------------------------------------------------------------- LangChain provides an Vercel KV-based cache. Like the Redis-based cache, this cache is useful if you want to share the cache across multiple processes or servers. The Vercel KV client uses HTTP and supports edge environments. To use it, you'll need to install the `@vercel/kv` package: * npm * Yarn * pnpm npm install @vercel/kv yarn add @vercel/kv pnpm add @vercel/kv You'll also need an Vercel account and a [KV database](https://vercel.com/docs/storage/vercel-kv/kv-reference) to connect to. Once you've done that, retrieve your REST URL and REST token. Then, you can pass a `cache` option when you instantiate the LLM. For example: import { OpenAI } from "@langchain/openai";import { VercelKVCache } from "@langchain/community/caches/vercel_kv";import { createClient } from "@vercel/kv";// See https://vercel.com/docs/storage/vercel-kv/kv-reference#createclient-example for connection optionsconst cache = new VercelKVCache({ client: createClient({ url: "VERCEL_KV_API_URL", token: "VERCEL_KV_API_TOKEN", }), ttl: 3600,});const model = new OpenAI({ cache }); #### API Reference: * OpenAI from `@langchain/openai` * VercelKVCache from `@langchain/community/caches/vercel_kv` Caching with Cloudflare KV[โ€‹](#caching-with-cloudflare-kv "Direct link to Caching with Cloudflare KV") ------------------------------------------------------------------------------------------------------- info This integration is only supported in Cloudflare Workers. If you're deploying your project as a Cloudflare Worker, you can use LangChain's Cloudflare KV-powered LLM cache. For information on how to set up KV in Cloudflare, see [the official documentation](https://developers.cloudflare.com/kv/) . **Note:** If you are using TypeScript, you may need to install types if they aren't already present: * npm * Yarn * pnpm npm install -S @cloudflare/workers-types yarn add @cloudflare/workers-types pnpm add @cloudflare/workers-types import type { KVNamespace } from "@cloudflare/workers-types";import { OpenAI } from "@langchain/openai";import { CloudflareKVCache } from "@langchain/cloudflare";export interface Env { KV_NAMESPACE: KVNamespace; OPENAI_API_KEY: string;}export default { async fetch(_request: Request, env: Env) { try { const cache = new CloudflareKVCache(env.KV_NAMESPACE); const model = new OpenAI({ cache, model: "gpt-3.5-turbo-instruct", apiKey: env.OPENAI_API_KEY, }); const response = await model.invoke("How are you today?"); return new Response(JSON.stringify(response), { headers: { "content-type": "application/json" }, }); } catch (err: any) { console.log(err.message); return new Response(err.message, { status: 500 }); } },}; #### API Reference: * OpenAI from `@langchain/openai` * CloudflareKVCache from `@langchain/cloudflare` Caching on the File System[โ€‹](#caching-on-the-file-system "Direct link to Caching on the File System") ------------------------------------------------------------------------------------------------------- danger This cache is not recommended for production use. It is only intended for local development. LangChain provides a simple file system cache. By default the cache is stored a temporary directory, but you can specify a custom directory if you want. const cache = await LocalFileCache.create(); Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- You've now learned how to cache model responses to save time and money. Next, check out the other how-to guides on LLMs, like [how to create your own custom LLM class](/docs/how_to/custom_llm) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [In Memory Cache](#in-memory-cache) * [Caching with Momento](#caching-with-momento) * [Caching with Redis](#caching-with-redis) * [Caching with Upstash Redis](#caching-with-upstash-redis) * [Caching with Vercel KV](#caching-with-vercel-kv) * [Caching with Cloudflare KV](#caching-with-cloudflare-kv) * [Caching on the File System](#caching-on-the-file-system) * [Next steps](#next-steps) --- # Richer outputs | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) How to create a custom LLM class ================================ Prerequisites This guide assumes familiarity with the following concepts: * [LLMs](/docs/concepts/text_llms) This notebook goes over how to create a custom LLM wrapper, in case you want to use your own LLM or a different wrapper than one that is directly supported in LangChain. There are a few required things that a custom LLM needs to implement after extending the [`LLM` class](https://api.js.langchain.com/classes/langchain_core.language_models_llms.LLM.html) : * A `_call` method that takes in a string and call options (which includes things like `stop` sequences), and returns a string. * A `_llmType` method that returns a string. Used for logging purposes only. You can also implement the following optional method: * A `_streamResponseChunks` method that returns an `AsyncIterator` and yields [`GenerationChunks`](https://api.js.langchain.com/classes/langchain_core.outputs.GenerationChunk.html) . This allows the LLM to support streaming outputs. Letโ€™s implement a very simple custom LLM that just echoes back the first `n` characters of the input. import { LLM, type BaseLLMParams } from "@langchain/core/language_models/llms";import type { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";import { GenerationChunk } from "@langchain/core/outputs";interface CustomLLMInput extends BaseLLMParams { n: number;}class CustomLLM extends LLM { n: number; constructor(fields: CustomLLMInput) { super(fields); this.n = fields.n; } _llmType() { return "custom"; } async _call( prompt: string, options: this["ParsedCallOptions"], runManager: CallbackManagerForLLMRun ): Promise { // Pass `runManager?.getChild()` when invoking internal runnables to enable tracing // await subRunnable.invoke(params, runManager?.getChild()); return prompt.slice(0, this.n); } async *_streamResponseChunks( prompt: string, options: this["ParsedCallOptions"], runManager?: CallbackManagerForLLMRun ): AsyncGenerator { // Pass `runManager?.getChild()` when invoking internal runnables to enable tracing // await subRunnable.invoke(params, runManager?.getChild()); for (const letter of prompt.slice(0, this.n)) { yield new GenerationChunk({ text: letter, }); // Trigger the appropriate callback await runManager?.handleLLMNewToken(letter); } }} We can now use this as any other LLM: const llm = new CustomLLM({ n: 4 });await llm.invoke("I am an LLM"); I am And support streaming: const stream = await llm.stream("I am an LLM");for await (const chunk of stream) { console.log(chunk);} Iam If you want to take advantage of LangChainโ€™s callback system for functionality like token tracking, you can extend the [`BaseLLM`](https://api.js.langchain.com/classes/langchain_core.language_models_llms.BaseLLM.html) class and implement the lower level `_generate` method. Rather than taking a single string as input and a single string output, it can take multiple input strings and map each to multiple string outputs. Additionally, it returns a `Generation` output with fields for additional metadata rather than just a string. import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";import { LLMResult } from "@langchain/core/outputs";import { BaseLLM, BaseLLMCallOptions, BaseLLMParams,} from "@langchain/core/language_models/llms";interface AdvancedCustomLLMCallOptions extends BaseLLMCallOptions {}interface AdvancedCustomLLMParams extends BaseLLMParams { n: number;}class AdvancedCustomLLM extends BaseLLM { n: number; constructor(fields: AdvancedCustomLLMParams) { super(fields); this.n = fields.n; } _llmType() { return "advanced_custom_llm"; } async _generate( inputs: string[], options: this["ParsedCallOptions"], runManager?: CallbackManagerForLLMRun ): Promise { const outputs = inputs.map((input) => input.slice(0, this.n)); // Pass `runManager?.getChild()` when invoking internal runnables to enable tracing // await subRunnable.invoke(params, runManager?.getChild()); // One input could generate multiple outputs. const generations = outputs.map((output) => [ { text: output, // Optional additional metadata for the generation generationInfo: { outputCount: 1 }, }, ]); const tokenUsage = { usedTokens: this.n, }; return { generations, llmOutput: { tokenUsage }, }; }} This will pass the additional returned information in callback events and in the \`streamEvents method: const llm = new AdvancedCustomLLM({ n: 4 });const eventStream = await llm.streamEvents("I am an LLM", { version: "v2",});for await (const event of eventStream) { if (event.event === "on_llm_end") { console.log(JSON.stringify(event, null, 2)); }} { "event": "on_llm_end", "data": { "output": { "generations": [ [ { "text": "I am", "generationInfo": { "outputCount": 1 } } ] ], "llmOutput": { "tokenUsage": { "usedTokens": 4 } } } }, "run_id": "a9ce50e4-f85b-41eb-bcbe-793efc52f9d8", "name": "AdvancedCustomLLM", "tags": [], "metadata": {}} * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . --- # How to use few shot examples | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page In this guide, weโ€™ll learn how to create a simple prompt template that provides the model with example inputs and outputs when generating. Providing the LLM with a few such examples is called few-shotting, and is a simple yet powerful way to guide generation and in some cases drastically improve model performance. A few-shot prompt template can be constructed from either a set of examples, or from an [Example Selector](https://api.js.langchain.com/classes/langchain_core.example_selectors.BaseExampleSelector.html) class responsible for choosing a subset of examples from the defined set. This guide will cover few-shotting with string prompt templates. For a guide on few-shotting with chat messages for chat models, see [here](/docs/how_to/few_shot_examples_chat/) . Prerequisites This guide assumes familiarity with the following concepts: * [Prompt templates](/docs/concepts/prompt_templates) * [Example selectors](/docs/concepts/example_selectors) * [LLMs](/docs/concepts/text_llms) * [Vectorstores](/docs/concepts/#vectorstores) Create a formatter for the few-shot examples[โ€‹](#create-a-formatter-for-the-few-shot-examples "Direct link to Create a formatter for the few-shot examples") ------------------------------------------------------------------------------------------------------------------------------------------------------------- Configure a formatter that will format the few-shot examples into a string. This formatter should be a `PromptTemplate` object. import { PromptTemplate } from "@langchain/core/prompts";const examplePrompt = PromptTemplate.fromTemplate( "Question: {question}\n{answer}"); Creating the example set[โ€‹](#creating-the-example-set "Direct link to Creating the example set") ------------------------------------------------------------------------------------------------- Next, weโ€™ll create a list of few-shot examples. Each example should be a dictionary representing an example input to the formatter prompt we defined above. const examples = [ { question: "Who lived longer, Muhammad Ali or Alan Turing?", answer: ` Are follow up questions needed here: Yes. Follow up: How old was Muhammad Ali when he died? Intermediate answer: Muhammad Ali was 74 years old when he died. Follow up: How old was Alan Turing when he died? Intermediate answer: Alan Turing was 41 years old when he died. So the final answer is: Muhammad Ali `, }, { question: "When was the founder of craigslist born?", answer: ` Are follow up questions needed here: Yes. Follow up: Who was the founder of craigslist? Intermediate answer: Craigslist was founded by Craig Newmark. Follow up: When was Craig Newmark born? Intermediate answer: Craig Newmark was born on December 6, 1952. So the final answer is: December 6, 1952 `, }, { question: "Who was the maternal grandfather of George Washington?", answer: ` Are follow up questions needed here: Yes. Follow up: Who was the mother of George Washington? Intermediate answer: The mother of George Washington was Mary Ball Washington. Follow up: Who was the father of Mary Ball Washington? Intermediate answer: The father of Mary Ball Washington was Joseph Ball. So the final answer is: Joseph Ball `, }, { question: "Are both the directors of Jaws and Casino Royale from the same country?", answer: ` Are follow up questions needed here: Yes. Follow up: Who is the director of Jaws? Intermediate Answer: The director of Jaws is Steven Spielberg. Follow up: Where is Steven Spielberg from? Intermediate Answer: The United States. Follow up: Who is the director of Casino Royale? Intermediate Answer: The director of Casino Royale is Martin Campbell. Follow up: Where is Martin Campbell from? Intermediate Answer: New Zealand. So the final answer is: No `, },]; ### Pass the examples and formatter to `FewShotPromptTemplate`[โ€‹](#pass-the-examples-and-formatter-to-fewshotprompttemplate "Direct link to pass-the-examples-and-formatter-to-fewshotprompttemplate") Finally, create a [`FewShotPromptTemplate`](https://api.js.langchain.com/classes/langchain_core.prompts.FewShotPromptTemplate.html) object. This object takes in the few-shot examples and the formatter for the few-shot examples. When this `FewShotPromptTemplate` is formatted, it formats the passed examples using the `examplePrompt`, then and adds them to the final prompt before `suffix`: import { FewShotPromptTemplate } from "@langchain/core/prompts";const prompt = new FewShotPromptTemplate({ examples, examplePrompt, suffix: "Question: {input}", inputVariables: ["input"],});const formatted = await prompt.format({ input: "Who was the father of Mary Ball Washington?",});console.log(formatted.toString()); Question: Who lived longer, Muhammad Ali or Alan Turing? Are follow up questions needed here: Yes. Follow up: How old was Muhammad Ali when he died? Intermediate answer: Muhammad Ali was 74 years old when he died. Follow up: How old was Alan Turing when he died? Intermediate answer: Alan Turing was 41 years old when he died. So the final answer is: Muhammad AliQuestion: When was the founder of craigslist born? Are follow up questions needed here: Yes. Follow up: Who was the founder of craigslist? Intermediate answer: Craigslist was founded by Craig Newmark. Follow up: When was Craig Newmark born? Intermediate answer: Craig Newmark was born on December 6, 1952. So the final answer is: December 6, 1952Question: Who was the maternal grandfather of George Washington? Are follow up questions needed here: Yes. Follow up: Who was the mother of George Washington? Intermediate answer: The mother of George Washington was Mary Ball Washington. Follow up: Who was the father of Mary Ball Washington? Intermediate answer: The father of Mary Ball Washington was Joseph Ball. So the final answer is: Joseph BallQuestion: Are both the directors of Jaws and Casino Royale from the same country? Are follow up questions needed here: Yes. Follow up: Who is the director of Jaws? Intermediate Answer: The director of Jaws is Steven Spielberg. Follow up: Where is Steven Spielberg from? Intermediate Answer: The United States. Follow up: Who is the director of Casino Royale? Intermediate Answer: The director of Casino Royale is Martin Campbell. Follow up: Where is Martin Campbell from? Intermediate Answer: New Zealand. So the final answer is: NoQuestion: Who was the father of Mary Ball Washington? By providing the model with examples like this, we can guide the model to a better response. Using an example selector[โ€‹](#using-an-example-selector "Direct link to Using an example selector") ---------------------------------------------------------------------------------------------------- We will reuse the example set and the formatter from the previous section. However, instead of feeding the examples directly into the `FewShotPromptTemplate` object, we will feed them into an implementation of `ExampleSelector` called [`SemanticSimilarityExampleSelector`](https://api.js.langchain.com/classes/langchain_core.example_selectors.SemanticSimilarityExampleSelector.html) instance. This class selects few-shot examples from the initial set based on their similarity to the input. It uses an embedding model to compute the similarity between the input and the few-shot examples, as well as a vector store to perform the nearest neighbor search. To show what it looks like, letโ€™s initialize an instance and call it in isolation: Set your OpenAI API key for the embeddings model export OPENAI_API_KEY="..." import { SemanticSimilarityExampleSelector } from "@langchain/core/example_selectors";import { MemoryVectorStore } from "langchain/vectorstores/memory";import { OpenAIEmbeddings } from "@langchain/openai";const exampleSelector = await SemanticSimilarityExampleSelector.fromExamples( // This is the list of examples available to select from. examples, // This is the embedding class used to produce embeddings which are used to measure semantic similarity. new OpenAIEmbeddings(), // This is the VectorStore class that is used to store the embeddings and do a similarity search over. MemoryVectorStore, { // This is the number of examples to produce. k: 1, });// Select the most similar example to the input.const question = "Who was the father of Mary Ball Washington?";const selectedExamples = await exampleSelector.selectExamples({ question });console.log(`Examples most similar to the input: ${question}`);for (const example of selectedExamples) { console.log("\n"); console.log( Object.entries(example) .map(([k, v]) => `${k}: ${v}`) .join("\n") );} Examples most similar to the input: Who was the father of Mary Ball Washington?question: Who was the maternal grandfather of George Washington?answer: Are follow up questions needed here: Yes. Follow up: Who was the mother of George Washington? Intermediate answer: The mother of George Washington was Mary Ball Washington. Follow up: Who was the father of Mary Ball Washington? Intermediate answer: The father of Mary Ball Washington was Joseph Ball. So the final answer is: Joseph Ball Now, letโ€™s create a `FewShotPromptTemplate` object. This object takes in the example selector and the formatter prompt for the few-shot examples. const prompt = new FewShotPromptTemplate({ exampleSelector, examplePrompt, suffix: "Question: {input}", inputVariables: ["input"],});const formatted = await prompt.invoke({ input: "Who was the father of Mary Ball Washington?",});console.log(formatted.toString()); Question: Who was the maternal grandfather of George Washington? Are follow up questions needed here: Yes. Follow up: Who was the mother of George Washington? Intermediate answer: The mother of George Washington was Mary Ball Washington. Follow up: Who was the father of Mary Ball Washington? Intermediate answer: The father of Mary Ball Washington was Joseph Ball. So the final answer is: Joseph BallQuestion: Who was the father of Mary Ball Washington? Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Youโ€™ve now learned how to add few-shot examples to your prompts. Next, check out the other how-to guides on prompt templates in this section, the related how-to guide on [few shotting with chat models](/docs/how_to/few_shot_examples_chat) , or the other [example selector how-to guides](/docs/how_to/example_selectors/) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Create a formatter for the few-shot examples](#create-a-formatter-for-the-few-shot-examples) * [Creating the example set](#creating-the-example-set) * [Pass the examples and formatter to `FewShotPromptTemplate`](#pass-the-examples-and-formatter-to-fewshotprompttemplate) * [Using an example selector](#using-an-example-selector) * [Next steps](#next-steps) --- # Build a Retrieval Augmented Generation (RAG) App: Part 1 | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page One of the most powerful applications enabled by LLMs is sophisticated question-answering (Q&A) chatbots. These are applications that can answer questions about specific source information. These applications use a technique known as Retrieval Augmented Generation, or [RAG](/docs/concepts/rag/) . This is a multi-part tutorial: * [Part 1](/docs/tutorials/rag) (this guide) introduces RAG and walks through a minimal implementation. * [Part 2](/docs/tutorials/qa_chat_history) extends the implementation to accommodate conversation-style interactions and multi-step retrieval processes. This tutorial will show how to build a simple Q&A application over a text data source. Along the way weโ€™ll go over a typical Q&A architecture and highlight additional resources for more advanced Q&A techniques. Weโ€™ll also see how LangSmith can help us trace and understand our application. LangSmith will become increasingly helpful as our application grows in complexity. If youโ€™re already familiar with basic retrieval, you might also be interested in this [high-level overview of different retrieval techinques](/docs/concepts/retrieval) . **Note**: Here we focus on Q&A for unstructured data. If you are interested for RAG over structured data, check out our tutorial on doing [question/answering over SQL data](/docs/tutorials/sql_qa) . Overview[โ€‹](#overview "Direct link to Overview") ------------------------------------------------- A typical RAG application has two main components: **Indexing**: a pipeline for ingesting data from a source and indexing it. _This usually happens offline._ **Retrieval and generation**: the actual RAG chain, which takes the user query at run time and retrieves the relevant data from the index, then passes that to the model. Note: the indexing portion of this tutorial will largely follow the [semantic search tutorial](/docs/tutorials/retrievers) . The most common full sequence from raw data to answer looks like: ### Indexing[โ€‹](#indexing "Direct link to Indexing") 1. **Load**: First we need to load our data. This is done with [Document Loaders](/docs/concepts/document_loaders) . 2. **Split**: [Text splitters](/docs/concepts/text_splitters) break large `Documents` into smaller chunks. This is useful both for indexing data and passing it into a model, as large chunks are harder to search over and wonโ€™t fit in a modelโ€™s finite context window. 3. **Store**: We need somewhere to store and index our splits, so that they can be searched over later. This is often done using a [VectorStore](/docs/concepts/vectorstores) and [Embeddings](/docs/concepts/embedding_models) model. ![index_diagram](/assets/images/rag_indexing-8160f90a90a33253d0154659cf7d453f.png) ### Retrieval and generation[โ€‹](#retrieval-and-generation "Direct link to Retrieval and generation") 1. **Retrieve**: Given a user input, relevant splits are retrieved from storage using a [Retriever](/docs/concepts/retrievers) . 2. **Generate**: A [ChatModel](/docs/concepts/chat_models) / [LLM](/docs/concepts/text_llms) produces an answer using a prompt that includes both the question with the retrieved data ![retrieval_diagram](/assets/images/rag_retrieval_generation-1046a4668d6bb08786ef73c56d4f228a.png) Once weโ€™ve indexed our data, we will use [LangGraph](https://langchain-ai.github.io/langgraphjs/) as our orchestration framework to implement the retrieval and generation steps. Setup[โ€‹](#setup "Direct link to Setup") ---------------------------------------- ### Jupyter Notebook[โ€‹](#jupyter-notebook "Direct link to Jupyter Notebook") This and other tutorials are perhaps most conveniently run in a [Jupyter notebooks](https://jupyter.org/) . Going through guides in an interactive environment is a great way to better understand them. See [here](https://jupyter.org/install) for instructions on how to install. ### Installation[โ€‹](#installation "Direct link to Installation") This guide requires the following dependencies: * npm * yarn * pnpm npm i langchain @langchain/core @langchain/langgraph yarn add langchain @langchain/core @langchain/langgraph pnpm add langchain @langchain/core @langchain/langgraph For more details, see our [Installation guide](/docs/how_to/installation) . ### LangSmith[โ€‹](#langsmith "Direct link to LangSmith") Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com) . After you sign up at the link above, make sure to set your environment variables to start logging traces: export LANGSMITH_TRACING="true"export LANGSMITH_API_KEY="..."# Reduce tracing latency if you are not in a serverless environment# export LANGCHAIN_CALLBACKS_BACKGROUND=true Components[โ€‹](#components "Direct link to Components") ------------------------------------------------------- We will need to select three components from LangChainโ€™s suite of integrations. A [chat model](/docs/integrations/chat/) : ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); An [embedding model](/docs/integrations/text_embedding/) : ### Pick your embedding model: * OpenAI * Azure * AWS * VertexAI * MistralAI * Cohere #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai OPENAI_API_KEY=your-api-key import { OpenAIEmbeddings } from "@langchain/openai";const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-large"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai AZURE_OPENAI_API_INSTANCE_NAME=AZURE_OPENAI_API_KEY=AZURE_OPENAI_API_VERSION="2024-02-01" import { AzureOpenAIEmbeddings } from "@langchain/openai";const embeddings = new AzureOpenAIEmbeddings({ azureOpenAIApiEmbeddingsDeploymentName: "text-embedding-ada-002"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/aws yarn add @langchain/aws pnpm add @langchain/aws BEDROCK_AWS_REGION=your-region import { BedrockEmbeddings } from "@langchain/aws";const embeddings = new BedrockEmbeddings({ model: "amazon.titan-embed-text-v1"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai GOOGLE_APPLICATION_CREDENTIALS=credentials.json import { VertexAIEmbeddings } from "@langchain/google-vertexai";const embeddings = new VertexAIEmbeddings({ model: "text-embedding-004"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai MISTRAL_API_KEY=your-api-key import { MistralAIEmbeddings } from "@langchain/mistralai";const embeddings = new MistralAIEmbeddings({ model: "mistral-embed"}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/cohere yarn add @langchain/cohere pnpm add @langchain/cohere COHERE_API_KEY=your-api-key import { CohereEmbeddings } from "@langchain/cohere";const embeddings = new CohereEmbeddings({ model: "embed-english-v3.0"}); And a [vector store](/docs/integrations/vectorstores/) : ### Pick your vector store: * Memory * Chroma * FAISS * MongoDB * PGVector * Pinecone * Qdrant #### Install dependencies * npm * yarn * pnpm npm i langchain yarn add langchain pnpm add langchain import { MemoryVectorStore } from "langchain/vectorstores/memory";const vectorStore = new MemoryVectorStore(embeddings); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { Chroma } from "@langchain/community/vectorstores/chroma";const vectorStore = new Chroma(embeddings, { collectionName: "a-test-collection",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import { FaissStore } from "@langchain/community/vectorstores/faiss";const vectorStore = new FaissStore(embeddings, {}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/mongodb yarn add @langchain/mongodb pnpm add @langchain/mongodb import { MongoDBAtlasVectorSearch } from "@langchain/mongodb"import { MongoClient } from "mongodb";const client = new MongoClient(process.env.MONGODB_ATLAS_URI || "");const collection = client .db(process.env.MONGODB_ATLAS_DB_NAME) .collection(process.env.MONGODB_ATLAS_COLLECTION_NAME);const vectorStore = new MongoDBAtlasVectorSearch(embeddings, { collection: collection, indexName: "vector_index", textKey: "text", embeddingKey: "embedding",}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community import PGVectorStore from "@langchain/community/vectorstores/pgvector";const vectorStore = await PGVectorStore.initialize(embeddings, {}) #### Install dependencies * npm * yarn * pnpm npm i @langchain/pinecone yarn add @langchain/pinecone pnpm add @langchain/pinecone import { PineconeStore } from "@langchain/pinecone";import { Pinecone as PineconeClient } from "@pinecone-database/pinecone";const pinecone = new PineconeClient();const vectorStore = new PineconeStore(embeddings, { pineconeIndex, maxConcurrency: 5,}); #### Install dependencies * npm * yarn * pnpm npm i @langchain/qdrant yarn add @langchain/qdrant pnpm add @langchain/qdrant import { QdrantVectorStore } from "@langchain/qdrant";const vectorStore = await QdrantVectorStore.fromExistingCollection(embeddings, { url: process.env.QDRANT_URL, collectionName: "langchainjs-testing",}); Preview[โ€‹](#preview "Direct link to Preview") ---------------------------------------------- In this guide weโ€™ll build an app that answers questions about the websiteโ€™s content. The specific website we will use is the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng, which allows us to ask questions about the contents of the post. We can create a simple indexing pipeline and RAG chain to do this in ~50 lines of code. import "cheerio";import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";import { Document } from "@langchain/core/documents";import { ChatPromptTemplate } from "@langchain/core/prompts";import { pull } from "langchain/hub";import { Annotation, StateGraph } from "@langchain/langgraph";import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";// Load and chunk contents of blogconst pTagSelector = "p";const cheerioLoader = new CheerioWebBaseLoader( "https://lilianweng.github.io/posts/2023-06-23-agent/", { selector: pTagSelector });const docs = await cheerioLoader.load();const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200});const allSplits = await splitter.splitDocuments(docs);// Index chunksawait vectorStore.addDocuments(allSplits)// Define prompt for question-answeringconst promptTemplate = await pull("rlm/rag-prompt");// Define state for applicationconst InputStateAnnotation = Annotation.Root({ question: Annotation,});const StateAnnotation = Annotation.Root({ question: Annotation, context: Annotation, answer: Annotation,});// Define application stepsconst retrieve = async (state: typeof InputStateAnnotation.State) => { const retrievedDocs = await vectorStore.similaritySearch(state.question) return { context: retrievedDocs };};const generate = async (state: typeof StateAnnotation.State) => { const docsContent = state.context.map(doc => doc.pageContent).join("\n"); const messages = await promptTemplate.invoke({ question: state.question, context: docsContent }); const response = await llm.invoke(messages); return { answer: response.content };};// Compile application and testconst graph = new StateGraph(StateAnnotation) .addNode("retrieve", retrieve) .addNode("generate", generate) .addEdge("__start__", "retrieve") .addEdge("retrieve", "generate") .addEdge("generate", "__end__") .compile(); let inputs = { question: "What is Task Decomposition?" };const result = await graph.invoke(inputs);console.log(result.answer); Task decomposition is the process of breaking down complex tasks into smaller, more manageable steps. This can be achieved through various methods, including prompting large language models (LLMs) or using task-specific instructions. Techniques like Chain of Thought (CoT) and Tree of Thoughts further enhance this process by structuring reasoning and exploring multiple possibilities at each step. Check out the [LangSmith trace](https://smith.langchain.com/public/84a36239-b466-41bd-ac84-befc33ab50df/r) . Detailed walkthrough[โ€‹](#detailed-walkthrough "Direct link to Detailed walkthrough") ------------------------------------------------------------------------------------- Letโ€™s go through the above code step-by-step to really understand whatโ€™s going on. 1\. Indexing[โ€‹](#indexing "Direct link to 1. Indexing") -------------------------------------------------------- note This section is an abbreviated version of the content in the [semantic search tutorial](/docs/tutorials/retrievers) . If you're comfortable with [document loaders](/docs/concepts/document_loaders) , [embeddings](/docs/concepts/embedding_models) , and [vector stores](/docs/concepts/vectorstores) , feel free to skip to the next section on [retrieval and generation](/docs/tutorials/rag/#orchestration) . ### Loading documents[โ€‹](#loading-documents "Direct link to Loading documents") We need to first load the blog post contents. We can use [DocumentLoaders](/docs/concepts/document_loaders) for this, which are objects that load in data from a source and return a list of [Documents](https://api.js.langchain.com/classes/langchain_core.documents.Document.html) . A Document is an object with some pageContent (`string`) and metadata (`Record`). In this case weโ€™ll use the [CheerioWebBaseLoader](https://api.js.langchain.com/classes/langchain.document_loaders_web_cheerio.CheerioWebBaseLoader.html) , which uses cheerio to load HTML form web URLs and parse it to text. We can pass custom selectors to the constructor to only parse specific elements: import "cheerio";import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";const pTagSelector = "p";const cheerioLoader = new CheerioWebBaseLoader( "https://lilianweng.github.io/posts/2023-06-23-agent/", { selector: pTagSelector, });const docs = await cheerioLoader.load();console.assert(docs.length === 1);console.log(`Total characters: ${docs[0].pageContent.length}`); Total characters: 22360 console.log(docs[0].pageContent.slice(0, 500)); Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.In a LLM-powered autonomous agent system, LLM functions as the agentโ€™s brain, complemented by several key components:A complicated task usually involv #### Go deeper[โ€‹](#go-deeper "Direct link to Go deeper") `DocumentLoader`: Class that loads data from a source as list of Documents. * [Docs](/docs/concepts/document_loaders) : Detailed documentation on how to use * [Integrations](/docs/integrations/document_loaders/) * [Interface](https:/api.js.langchain.com/classes/langchain.document_loaders_base.BaseDocumentLoader.html) : API reference for the base interface. ### Splitting documents[โ€‹](#splitting-documents "Direct link to Splitting documents") Our loaded document is over 42k characters which is too long to fit into the context window of many models. Even for those models that could fit the full post in their context window, models can struggle to find information in very long inputs. To handle this weโ€™ll split the `Document` into chunks for embedding and vector storage. This should help us retrieve only the most relevant parts of the blog post at run time. As in the [semantic search tutorial](/docs/tutorials/retrievers) , we use a [RecursiveCharacterTextSplitter](/docs/how_to/recursive_text_splitter) , which will recursively split the document using common separators like new lines until each chunk is the appropriate size. This is the recommended text splitter for generic text use cases. import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200,});const allSplits = await splitter.splitDocuments(docs);console.log(`Split blog post into ${allSplits.length} sub-documents.`); Split blog post into 29 sub-documents. #### Go deeper[โ€‹](#go-deeper-1 "Direct link to Go deeper") `TextSplitter`: Object that splits a list of `Document`s into smaller chunks. Subclass of `DocumentTransformers`. - Explore `Context-aware splitters`, which keep the location (โ€œcontextโ€) of each split in the original `Document`: - [Markdown files](/docs/how_to/code_splitter/#markdown) - [Code](/docs/how_to/code_splitter/) (15+ langs) - [Interface](https://api.js.langchain.com/classes/langchain_textsplitters.TextSplitter.html) : API reference for the base interface. `DocumentTransformer`: Object that performs a transformation on a list of `Document`s. - Docs: Detailed documentation on how to use `DocumentTransformer`s - [Integrations](/docs/integrations/document_transformers) - [Interface](https://api.js.langchain.com/classes/langchain_core.documents.BaseDocumentTransformer.html) : API reference for the base interface. ### Storing documents[โ€‹](#storing-documents "Direct link to Storing documents") Now we need to index our 66 text chunks so that we can search over them at runtime. Following the [semantic search tutorial](/docs/tutorials/retrievers) , our approach is to [embed](/docs/concepts/embedding_models/) the contents of each document split and insert these embeddings into a [vector store](/docs/concepts/vectorstores/) . Given an input query, we can then use vector search to retrieve relevant documents. We can embed and store all of our document splits in a single command using the vector store and embeddings model selected at the [start of the tutorial](/docs/tutorials/rag/#components) . await vectorStore.addDocuments(allSplits); #### Go deeper[โ€‹](#go-deeper-2 "Direct link to Go deeper") `Embeddings`: Wrapper around a text embedding model, used for converting text to embeddings. - [Docs](/docs/concepts/embedding_models) : Detailed documentation on how to use embeddings. - [Integrations](/docs/integrations/text_embedding) : 30+ integrations to choose from. - [Interface](https://api.js.langchain.com/classes/langchain_core.embeddings.Embeddings.html) : API reference for the base interface. `VectorStore`: Wrapper around a vector database, used for storing and querying embeddings. - [Docs](/docs/concepts/vectorstores) : Detailed documentation on how to use vector stores. - [Integrations](/docs/integrations/vectorstores) : 40+ integrations to choose from. - [Interface](https://api.js.langchain.com/classes/langchain_core.vectorstores.VectorStore.html) : API reference for the base interface. This completes the **Indexing** portion of the pipeline. At this point we have a query-able vector store containing the chunked contents of our blog post. Given a user question, we should ideally be able to return the snippets of the blog post that answer the question. 2\. Retrieval and Generation[โ€‹](#orchestration "Direct link to 2. Retrieval and Generation") --------------------------------------------------------------------------------------------- Now letโ€™s write the actual application logic. We want to create a simple application that takes a user question, searches for documents relevant to that question, passes the retrieved documents and initial question to a model, and returns an answer. For generation, we will use the chat model selected at the [start of the tutorial](/docs/tutorials/rag/#components) . Weโ€™ll use a prompt for RAG that is checked into the LangChain prompt hub ([here](https://smith.langchain.com/hub/rlm/rag-prompt) ). import { pull } from "langchain/hub";import { ChatPromptTemplate } from "@langchain/core/prompts";const promptTemplate = await pull("rlm/rag-prompt");// Example:const example_prompt = await promptTemplate.invoke({ context: "(context goes here)", question: "(question goes here)",});const example_messages = example_prompt.messages;console.assert(example_messages.length === 1);example_messages[0].content; You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.Question: (question goes here)Context: (context goes here)Answer: Weโ€™ll use [LangGraph](https://langchain-ai.github.io/langgraphjs/) to tie together the retrieval and generation steps into a single application. This will bring a number of benefits: * We can define our application logic once and automatically support multiple invocation modes, including streaming, async, and batched calls. * We get streamlined deployments via [LangGraph Platform](https://langchain-ai.github.io/langgraphjs/concepts/langgraph_platform/) . * LangSmith will automatically trace the steps of our application together. * We can easily add key features to our application, including [persistence](https://langchain-ai.github.io/langgraphjs/concepts/persistence/) and [human-in-the-loop approval](https://langchain-ai.github.io/langgraphjs/concepts/human_in_the_loop/) , with minimal code changes. To use LangGraph, we need to define three things: 1. The state of our application; 2. The nodes of our application (i.e., application steps); 3. The โ€œcontrol flowโ€ of our application (e.g., the ordering of the steps). #### State:[โ€‹](#state "Direct link to State:") The [state](https://langchain-ai.github.io/langgraphjs/concepts/low_level/#state) of our application controls what data is input to the application, transferred between steps, and output by the application. For a simple RAG application, we can just keep track of the input question, retrieved context, and generated answer. Read more about defining graph states [here](https://langchain-ai.github.io/langgraphjs/how-tos/define-state/) . import { Document } from "@langchain/core/documents";import { Annotation } from "@langchain/langgraph";const InputStateAnnotation = Annotation.Root({ question: Annotation,});const StateAnnotation = Annotation.Root({ question: Annotation, context: Annotation, answer: Annotation,}); #### Nodes (application steps)[โ€‹](#nodes-application-steps "Direct link to Nodes (application steps)") Letโ€™s start with a simple sequence of two steps: retrieval and generation. import { concat } from "@langchain/core/utils/stream";const retrieve = async (state: typeof InputStateAnnotation.State) => { const retrievedDocs = await vectorStore.similaritySearch(state.question); return { context: retrievedDocs };};const generate = async (state: typeof StateAnnotation.State) => { const docsContent = state.context.map((doc) => doc.pageContent).join("\n"); const messages = await promptTemplate.invoke({ question: state.question, context: docsContent, }); const response = await llm.invoke(messages); return { answer: response.content };}; Our retrieval step simply runs a similarity search using the input question, and the generation step formats the retrieved context and original question into a prompt for the chat model. #### Control flow[โ€‹](#control-flow "Direct link to Control flow") Finally, we compile our application into a single `graph` object. In this case, we are just connecting the retrieval and generation steps into a single sequence. import { StateGraph } from "@langchain/langgraph";const graph = new StateGraph(StateAnnotation) .addNode("retrieve", retrieve) .addNode("generate", generate) .addEdge("__start__", "retrieve") .addEdge("retrieve", "generate") .addEdge("generate", "__end__") .compile(); LangGraph also comes with built-in utilities for visualizing the control flow of your application: // Note: tslab only works inside a jupyter notebook. Don't worry about running this code yourself!import * as tslab from "tslab";const image = await graph.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_rag](data:image/png;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAFNAG4DASIAAhEBAxEB/8QAHQABAAMBAQEBAQEAAAAAAAAAAAUGBwQIAgMBCf/EAFQQAAEDBAADAggHCwcJCQEAAAECAwQABQYRBxIhEzEUFyJBUVaU0wgVFlR0stEyNDZCVWFxdYGV0iM1R5GTobQJJURScpKjsfAYJDM3Q1djosHU/8QAGwEBAAMBAQEBAAAAAAAAAAAAAAECAwQFBgf/xAA1EQACAAMFBAYKAwEAAAAAAAAAAQIDEQQSITFRFFKR0WFxkqGxwQUTFSMyM0FigeFCU/Ai/9oADAMBAAIRAxEAPwD/AFTpSoq+XldvLEWIx4Xc5XMGGd6SAPunHD+KhOxs9+yAASQDaGFxOiGZKKUEJKlEJSBsknoBUavJrO0opXdYKFDzKkoB/wCdRaMDhz1pkX9ZyGWCFalj/uzZ/wDjY2UJAPcTzK7tqOt1IoxOxtoCU2a3pSOgSIqAB/dW1JKzbfUv94ItgfXyqsv5Yge0o+2nyqsv5Yge0o+2nyVsv5HgezI+ynyVsv5HgezI+ynuenuGA+VVl/LED2lH20+VVl/LED2lH20+Stl/I8D2ZH2U+Stl/I8D2ZH2U9z09wwHyqsv5Yge0o+2nyqsv5Yge0o+2nyVsv5HgezI+ynyVsv5HgezI+ynuenuGB+0a+22a4ER7jEfWToJafSon9gNd1Q0jDMfltlt+xW15BBHKuI2R16HzVwfJyVjP8vYHXXIydFyzyHSttafP2KlHbS/QN8h7iE75wuy4sIXR9PP9EYFopXJarmxeLezMjFRadB6LSUrSQdKSpJ6pUCCCD1BBB7q66waadGQKq+Kaul5v94XpSjKVb456+Qyx5Kh+ku9sdjvHKD9yKtFVjBR4Ki+W9Ww5Fu0pRBGth5fhCSPSNPAb9II81bwfLjazw4V50JWTLPSlK5yDgv19t+L2Wdd7tMagWyCyuRJlPq5UNNpG1KJ/MBWQ518K3FMf4VXHNLEJd+aizYsHwdVvlx1c7y0gKUFM8wSEKKwop5VEJSDtad6NxPt1tu/DvI4V4s0vIbW/Bdbk2qAjnkSkFJ222Np8s+bRHXXUV5lm23P8x4FcSrCzbcnvVjt0i1v4z8pYHgt4lNMvNPyWFIISpzk7LSFqSFLJI2rQNAegL3x6wvHMbtN8uc+fDg3UuJiIcs03wlzszpe44Z7VIHpUgDRB7iDX8ufH/h/aLLjd3kZGyq3ZGF/FL8dh18SyhPMpCA2hR5+mgggKKvJAKulZtxNza+ZhccMmM2viDasBktzPjJix2yTFuypaS2I6HkoAfaZILp5k8oJCeYgaqjcGMEv9ul8Eolwxe9wU2DJMnXLTcoy1mIh5uSthbjvlJUFdqgBwKKVL2ASRQGvw/hN2Gdxjh4Q3BuoZmWiLcY89VonJUpx9zlQ2tBYHZICeVRdWQkFRSSkoUK2WsPymRcML+E/DyR7H71dLHd8XasqJlogrlpjyUTVuEPBAJbSUOg86vJ8k9elbhQClKUBV7b/AJpzu5wEaTGuMZNxQgeZ1KuzeP5gR2J0PPzHvJq0VWAPDeJRWjZTb7UW1nXTmfdBA36QI+yPNzD01Z66J2cLedF/uFCWKr14gyLZdRfbeyZCy0GZsVH3T7SSSlSPS4gqVoHvCiO/WrDSsoInA6grd0s+LcVMeEa5wLdktnU4FGNMZS82lxPmUhQ8ladkEEAg7B1VcT8G7hSgKCeHGLpCxpQFpY6jYOj5PpA/qq13XDLVdppmqaciXAgAzYLy47ygO4KUgjnA9Ctj81cZwh4ABGT35CR3Dt2lf3lsmtbsqLKKnWuXJDAj8f4GcO8TvEa7WXB8ftNzjElmZDtrTTrZIKTyqSkEbBI/QTV4qr/ImR61X7+2Z91T5EyPWq/f2zPuqerl7/cxRalopWV8VLfdcOwS43e35TeTLYUwEB9xko8t5CDsdmPMo+fvq2fImR61X7+2Z91T1cvf7mKLUsrzKJDK2nUJcaWkpUhQ2FA9CCKzn/s18J//AG2xX90Mfw1YfkTI9ar9/bM+6p8iZHrVfv7Zn3VPVy9/uYotSvr+DbwocUVK4b4spROyTaWCSf8Adq4Xa/RrKWYTCBJuTqQI1vZOlqHdzH/UbHnWRod3UkA8AwYrHK/kV9kI67T4YGtj9LaUn+o1K2XHbbjzTiLfERHLpCnXOqnHSBoFaztSzrptRJpSVDi3Xw4/oYH547ZlWiK8uQtD1xmOmRLeQCErcIA0nfUJSlKUpB8yRvrupalKxiicTqyMxSlKqBSlKAUpSgM94+68U953v7uL3DZ++Wq0Ks94/JKuE95ABJ7SL3J2fvlrzVoVAKUpQClKUApSlAKUpQClKUApSlAZ5x/14p71vl12kX7revvlr0VodZ7x9BPCi8gDZ54vTr85a9FaFQClKUApSlAKUpQClKgMgyV63y0W+3RET7mtvtih10tNMt9QFLWEqI2QQAASdHuAJF4IIpjuwjMn6VSTfcw2dQLIR5ty3vd1/Pj3MPmFj9re93XTsseq4omhd6VSPj3MPmFj9re93T49zD5hY/a3vd02WPVcUKGB/Dr+ErK4LW6Djj2HOXW2X5hDzV4TODSUPNPpWtnsy2rZCUtnex933dOuy/B04xTePHDOLmMrGl4uxMfcREjOSxJLzKdJ7Xm5Ea2rnGtfi7316Uj4QvCC6/CKwE4zeY1nhdnJblRpzEh1TjC0nR0C31CklSSN+cHzVfcbbyLEsfttktdpsUa3W+O3FjsplveQ2hISkf8Ah9eg76bLHquKFDSKVSPj3MPmFj9re93T49zD5hY/a3vd02WPVcUKF3pVI+Pcw+YWP2t73dfSL9lyVArt1lWkfipmvJJ/b2R1/VTZY9VxQoXWlRtgvjV/gl9Da47ray0/Hd+7ZcGtpOuh7wQR0III6GpKuWKFwtwvMgVRSd8Rr/vzQII3+bmkf9ftq9VRP6Rsg+gwfrSK67N/Pq80WWTJilKVsVFKVwovlvdvT1oRNYXdGWEyXIaXAXUNKUpKVqT3hJKVAE9/KfRQHdSvxmS2rfDflPr5GGG1OuK0TpIGydDqeg81cuPX+BlVit95tb/hNtuDCJUZ4oUjnbWkKSrlUAobBHQgGgJClKUApSlAcOBn/PWYDuHxk0enp8Dj9f8Al/VVxqnYH/PeY/rFr/Bx6uNc1p+Z+F4IliqJ/SNkH0GD9aRV7qif0jZB9Bg/WkVezfz6vNErJkxWGZ1a7hmPwk7VjaskvlosJxKROkQ7RcXYnbuiW0hJ5kEFJHPvmSQrprfKVA7nUOrEbSvL28oMTd9bgqtqZXaL6R1OJcUjk3y/doSd6301vVaNVKnl4p4ocWsi4gysenyIT1ivcmyWtScsegNQuwCQ2t2GIriZHPsOEurPMF6HKBurbi2Frm/Cuu8673C5NXhnFbTNfZg3WQ3FW/2shDiezCgFs7RsIUOXalHW1Hel5LwCwLLsmdyC6WBLt1f5BIdZlPsJk8n3HbNtrSh3WgBzhXQAVKZJwpxfLMpteSXK2qcvlsCURprEp5hYQlYcCF9mtIcQFjm5V8yd76dTVbrBiPDTG7jlXCLL8mu+YZVJuIlXxiIGr3IZRFaalPJQlIQsbILfRStkA8oISAKqtnybPOJD/DrFYUybJbb4f2u/SSMmetEmfIeHIt5chth1x0JKRtO0jmcJVzdAPU1kwOxY7jcqwW+D4PaZS5Lj0ftnFcyn1rW8eZSiocynFnoem+mgBVevfATBMhsuPWubYtxsfjJh2txiY+xIispQEBtL7a0uFPKlIIKjvXXZpdYMjnX/ADXgNBw3MOIN7dmWqMZ1nvLbE5cpoMLKnYL69obSt5Km0sKd5ElXaioK4S+I7z/DnC5E6cq75JBuGSXRC8getbq3i4haYTUlLTq20MJd12bYTsIB2ACFelV8OMacwxjEl2hhzHGUtpRb18ykANrC0b2dnSkg9T1I618Z5wzxnibCixcktabgmI728Z1DrjD0detczbrakrQSOh5VDfnpdYIPgjY8zx3GJ8LNJaJb6bg4q3f5wVPdaiFKOVt2QppouLC+18op3y8uySN1odQ2I4faMEsTNmscTwK3MqWtLRcW4SpSipSlLWSpRKiSSSSSamausEDhwP8AnvMf1i1/g49XGqdgf895j+sWv8HHq41z2r5n4XgiWKon9I2QfQYP1pFXuqpkVlnxrwq9WphM5x1hEeVCU6G1LSgqU2ttR8nmBWoEK1sEeUOQBU2eJJxJvNU70/II6KVCqut+CiBht0IHnEmH1/49fz42v3qZdfaoXv67Ln3LtLmTQm6VCfG1+9TLr7VC9/T42v3qZdfaoXv6XPuXaXMUJulVPIc3n4raH7pdMUusaCyUBx3t4i9FSghPRLxPVSgO7z1I/G1+9TLr7VC9/S59y7S5ihN0qE+Nr96mXX2qF7+nxtfvUy6+1Qvf0ufcu0uYoTdKhPja/epl19qhe/r6Rcr+4eUYhcGz5lPSogT+0peUf7jS59y7S5kUOvA/57zH9Ytf4OPVxqExSxPWaJKcmOIcuE58yZJaJLaVcqUBCN9eVKUJG+myCrQ3oTdcE+JRzG10LgqBilKVzkClKUApSlAZ9x7G+FN56b8uL5t/6S1+Y1oNZ5x//wDKa9dAr+Ui9Dv5y16K0OgFKUoBSlKAUpSgFKUoBSlKAUpSgM84/wCvFNet612kXv3r75a9FaHWe8flcnCe9HZHlxeoOj98tVoVAKUpQClKUApSlAKUqFvGbY9j8oRrnfLdb5JHN2MmUhC9enlJ3qrwwRRukKqyaVJqlVbxpYd602j21v7aeNLDvWm0e2t/bWuzztx8GTdehaaVVvGlh3rTaPbW/tp40sO9abR7a39tNnnbj4MXXoUz4SGdY3YcAudquWQ2q3XR4RnmoUuc20+tvwlHlpQpQUU+QvqOnkn0GtHxzK7JmEJyZYbzb73DbcLK5FulIkNpWACUFSCQFaUk679EemvFH+UV4f4/xfxCz5Ri11ttzymzOiK5FiyULdkxHFdwAOyW1nm0O4LWfNW6/Btt2C8CeDthxRrJ7KZrTXhFxeRNb/lZa9FxW99QDpIP+qhNNnnbj4MXXobvSqt40sO9abR7a39tPGlh3rTaPbW/tps87cfBi69C00qreNLDvWm0e2t/bTxpYd602j21v7abPO3HwYuvQtNKj7PkFsyFhT9ruMW4spPKpcR5LoSfQSknR/NUhWLhcLpEqMqcV6mKt9nnSkAFbDDjqQfSlJI/5VUcSiNxrBCcA5n5LSH33ldVvOKSCpaiepJJ/Z3dwqz5V+DF4+hvfUNV7GvwctX0Rr6grukYSn1k/QkqUpVyBSlKAUpSgFKUoBSlKAhLqoWzIrBPYAbkPzEwnlJ6dq0pC/JV6QFAKG96I6a2av1Z/k33/jX63Z+qutArK05QPo8yXkReVfgxePob31DVexr8HLV9Ea+oKsOVfgxePob31DVexr8HLV9Ea+oKvJ+S+vyH0O2VJahRnpDyw2y0guLWfxUgbJ/qrDca+EpdL9fMIMvDE2bF8v8ACXbbeJV1SpwsNR3H+ZxlLf8AJqUhAUE85GidqBAB3V1KVtLStIUgghSVDYI8+xXhngbMt7Gd43jjybfmjSlTLbHj2q63BxWPMvNrLrgiSIyAw3oBvy3FLSFABSuu4idGiDWsf+GjYr7fbKkRLUmxXme1Ahvs5FFeuSVOr5GnHoCfLbQVFO/KUpIUCpI66mI/wlLoqGq9ycIMbEmMhXjsq6fGqFOtuCYYqHksdn5TZXyc21JUCogJUAFK7eEnDniDw3ZseLyncUueI2fbDN0Lbwub0ZKSGUKb5Q2lafIBWFkEJ+52d1wyeBF/e4M3nEUzLaLlNylV8bdLrnYhg3ZMzlJ5N8/ZpI1ojm6b11qqvAj86+GJZ8SyLIYcSJaJ8HHnlxrguXkkWFNccQAXUxYjnlPcu+XqUcygUp3qrVa+OF1yziJPxrF8UbusGFFts928Sbn4M0I0tJUDydkpXOEpJCO5WlbUjQBjIHCzPMDyrJziTmK3DHcgurl5Ub+h8SoD72i+lAbSUuoKgVJBUggqI2aumJ4FPsHFjPMnedim3X2PbGYrTSldq2Y6Hkr5wUgAHtE60T3HeqlXvqCnr+Ea7aOK9vw2/WG325FyuCrdEejZBHlzAvlUppb0RIC2kOBHRW1aKkhQBNcHCji7lizxQumbQ4MbGMdu9wCrg1cO1citsNtK8HSyGEc6AgqV2hVzEnXL56r9l+Dpm9mYxe2odxRUHHsmF/Ny/l/D7vt1wqL6uTTbnI8rqC5zKSgbSKtzHBbITJ4k43LkWqRgWaSJkxySlx1NyiuSI6W1oSjkLakhSAoKKgevUVH/AECMwL4XFszDLsfs8mDaYrOQOKagLt2SRbjKaX2anEplR2vKZKkpI2CsBWkkjdegKyrhXjHEXG12y2ZP8kpVotsXwYXG2ofE2YUpCW3FIUkIaOhtQCl7J6aFarV4a0xBBZN9/wCNfrdn6q60Cs/yb7/xr9bs/VXWgVS0/DB+fEl5EXlX4MXj6G99Q1Xsa/By1fRGvqCrTeYarjaJ0RBAW+w40CfMVJI//aqGJTG5Fhhsg8kmMyhiQwrotlxKQFIUD1BB/rGiOhFWkYymukfQmKUpVyBSlKAUpSgFKUoBSlKAgsm+/wDGv1uz9VdaBVBuYTdcisMCOoOyI8xMx9KTvsWkoX5SvRtRCQDrezrfKdX6srTlAujzJeQqFvGFY/kMgSLpY7bcXwOUOyojbiwPRtQJ1U1SuOGOKB1hdGRkVbxV4Z6p2T93tfw08VeGeqdk/d7X8NWmlbbRO33xZNXqVbxV4Z6p2T93tfw08VeGeqdk/d7X8NWmlNonb74sVepjvG7h3i9q4ZXaVBx61QZSFxuR9iG02tO5DYOlADWwSO/uNXnxV4Z6p2T93tfw1DcflFPCe8kK5DzxevX5y16K0Km0Tt98WKvUq3irwz1Tsn7va/hp4q8M9U7J+72v4atNKbRO33xYq9SreKvDPVOyfu9r+Gnirwz1Tsn7va/hq00ptE7ffFir1OC0WK24/HUxa7fFtzCjzFuIylpJPpISB1rvpSsG3E6t4kClKVAFKUoBSlKAzz4QCuXhLeiCR/KRe46/0lqtDrPePw3wnvX+3F/GCf8ASWvPWhUApSlAKUpQClKUApSlAKUpQClKUBnvH0A8J7yCCRzxeg+ktfmNaFXkD/KFcZM/4RYxafiO2WabiN3UmPLkTYzzj8eU24HUAKS4lIStKegKSfIX16jW4fBuzTM+I3CGy5RnMK22273ZJlsxLYw60huMrXZcwccWSpQ8vYOtLT02DQGn0pSgFKUoBSlKAUpVV4l5U5iOJyJcYpE95SYsTmGwHVnQVrz8o5l684Sa1lS4p0alwZvAZkXnfFZjGJS7bbowud2SB2iVL5GY+xsc6tHatdeRPXWtlIIJzWTxIzKY6XDfkQgf/ShQmggfo7QLP99V5loMo5eZSySVKW4oqUtRO1KUT1KiSSSe8kmvuvv7P6Ns8iGjhUT1ar3PIV0Jf5dZl62TPZInuafLrMvWyZ7JE9zURSuvZrP/AFQ9lciLzOLiBFn8U8Yfx7KbzIu9nfWhxcZyNGR5SFBSSFJaCgQR5iOmx3E1YGMzy2Kw2yzlEplltIQhtuFDSlKQNAABjoAPNUZSmzWf+qHsrkLzJf5dZl62TPZInuaJzvMQoE5XMIHmMSJ1/wCDVOx3LYeTTL5GitvtuWicYD5eSAFOBtDm0aJ2nTie/R2D0qaqFZ7NEqqXD2VyF5lntfFbLrU4FPy4t7Z3tTUtgMrI9CXGwAn9JQqtdwvOIGbQXHYwXHlMEJkQ39BxonuPTopJ0dKHQ6I6EEDz3X6Qr1Jxe4x73D2ZEPalNpOu2Z2C40f9oDpvuUEnzV59s9FyZ8DcqFQxfSmC6qZEp1wZ6ipX4xJTU6KzJYWHGHkJcbWnuUkjYP8AUa/avgmqYMCst4+FSbXjp2ey+MyDo/jeDva3/wDb9pFalVc4gYscwxWXb21JRLHK9FcX3JeQeZG/zEjR/MTXZYpsMm0QRxZJko8+Ur+DtELcaeaXHkNKLbzDg0ptY70n/rr0I6GqpNw++Spsh5nO7zDaccUtEdqLBKGkk7CElUcqIHcNknp1Jr9IiioqpV6qFC2V574gW1zMOMt5tN5n2KLCh22M9bI2QsOutLQrn7Z1oIfaAWFAAqOyAE61o71NWE5Ao7HEG+J6AaEO3+jv+9qkXcItt2tcOLkcePlbsYqUmVd4bDi9k73ypQEpIGh5KR3Dz9a5psDnpQtUpjjk+jB/n8Ax+yYLEuWeYbY8gntZjDbxaW8JKiosSk+FM9kSCtXOEoWACoq3oK79GoGzPRrnb+H+N5JLUMQVc71FcTJfUlt9cd9SYjDqyRtITzaST5RQkddV6Tbs0BmWzLbgxkSmGDGafSykLbaJBLaVa2E7Sk8o6dB6K5JOI2KZal2x+y25+2rcU8qG5EbUypxSipSygjRUVEknWySTWLsmlOfw4Ph3gzzgDBtlsf4gxbMGhbGsjcSwlhznQkeDR+iTs9AdjXm1rzVrNViXgrceMhjHJysPb5y48mzQoqQ+rlSkFYcaWNgJABGjrp5hrk+RGQa14wr5+nwO3/8A81dEtRSoVBdr1Up4guVfxZCUkqICQNkn0VA4/jt0tExx6dlNyvjSmygR5jEVtKTsHmBaZQrfQjqddT07qtVnxuRmV1as0cKCHtGW8np2DG/LVvzEjaU+knfcCRs5iggcceCQSqzdeGIWOG+Ldpzc/wAVxuivugOyToH8+u+rNXwy0iO0hptIQ2hISlKRoADuFfdfl8yO/HFHq6l3ixSlKzIKfm3DO35isSkurtt0SkJExhIPOB3JcSeiwP2EeYjZrN5PB7LozqktG0zmh3OiQ4yo/pQW1Af7xreKV6ln9JWmzQ3IXVaMmupgHiozL5jbfb1e7p4qMy+Y2329Xu63+ldftq06Lg+Yw0MA8VGZfMbb7er3dPFRmXzG2+3q93W/0p7atOi4PmMNDAPFRmXzG2+3q93QcKMx2NwbaB9PV7ut/pT21adFw/Yw0MQtnBTIprw+Mp1vtcf8bwMrkukegFSUJSfz6V+itXxfE7dh9t8DtzSkhR5nXnVczry9a5lq85/uA6AAACpilcFpt8+1K7MeGiyFRSlK88g//9k=) Do I need to use LangGraph? LangGraph is not required to build a RAG application. Indeed, we can implement the same application logic through invocations of the individual components: let question = "...";const retrievedDocs = await vectorStore.similaritySearch(question);const docsContent = retrievedDocs.map((doc) => doc.pageContent).join("\n");const messages = await promptTemplate.invoke({ question: question, context: docsContent,});const answer = await llm.invoke(messages); The benefits of LangGraph include: * Support for multiple invocation modes: this logic would need to be rewritten if we wanted to stream output tokens, or stream the results of individual steps; * Automatic support for tracing via [LangSmith](https://docs.smith.langchain.com/) and deployments via [LangGraph Platform](https://langchain-ai.github.io/langgraphjs/concepts/langgraph_platform/) ; * Support for persistence, human-in-the-loop, and other features. Many use-cases demand RAG in a conversational experience, such that a user can receive context-informed answers via a stateful conversation. As we will see in [Part 2](/docs/tutorials/qa_chat_history) of the tutorial, LangGraphโ€™s management and persistence of state simplifies these applications enormously. #### Usage[โ€‹](#usage "Direct link to Usage") Letโ€™s test our application! LangGraph supports multiple invocation modes, including sync, async, and streaming. Invoke: let inputs = { question: "What is Task Decomposition?" };const result = await graph.invoke(inputs);console.log(result.context.slice(0, 2));console.log(`\nAnswer: ${result["answer"]}`); [ Document { pageContent: 'hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning Domain', metadata: { source: 'https://lilianweng.github.io/posts/2023-06-23-agent/', loc: [Object] }, id: undefined }, Document { pageContent: 'Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.In a LLM-powered autonomous agent system, LLM functions as the agentโ€™s brain, complemented by several key components:A complicated task usually involves many steps. An agent needs to know what they are and plan ahead.Chain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to โ€œthink step by stepโ€ to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the modelโ€™s thinking process.Tree of Thoughts (Yao et al.', metadata: { source: 'https://lilianweng.github.io/posts/2023-06-23-agent/', loc: [Object] }, id: undefined }]Answer: Task decomposition is the process of breaking down complex tasks into smaller, more manageable steps. This can be achieved through various methods, including prompting large language models (LLMs) to outline steps or using task-specific instructions. Techniques like Chain of Thought (CoT) and Tree of Thoughts further enhance this process by structuring reasoning and exploring multiple possibilities at each step. Stream steps: console.log(inputs);console.log("\n====\n");for await (const chunk of await graph.stream(inputs, { streamMode: "updates",})) { console.log(chunk); console.log("\n====\n");} { question: 'What is Task Decomposition?' }===={ retrieve: { context: [ [Document], [Document], [Document], [Document] ] }}===={ generate: { answer: 'Task decomposition is the process of breaking down complex tasks into smaller, more manageable steps. This can be achieved through various methods, including prompting large language models (LLMs) or using task-specific instructions. Techniques like Chain of Thought (CoT) and Tree of Thoughts further enhance this process by structuring reasoning and exploring multiple possibilities at each step.' }}==== Stream [tokens](/docs/concepts/tokens/) (requires `@langchain/core` \>\= 0.3.24 and `@langchain/langgraph` \>\= 0.2.34 with above implementation): const stream = await graph.stream(inputs, { streamMode: "messages" });for await (const [message, _metadata] of stream) { process.stdout.write(message.content + "|");} |Task| decomposition| is| the| process| of| breaking| down| complex| tasks| into| smaller|,| more| manageable| steps|.| This| can| be| achieved| through| various| methods|,| including| prompting| large| language| models| (|LL|Ms|)| to| outline| steps| or| using| task|-specific| instructions|.| Techniques| like| Chain| of| Thought| (|Co|T|)| and| Tree| of| Thoughts| further| enhance| this| process| by| struct|uring| reasoning| and| exploring| multiple| possibilities| at| each| step|.|| note Streaming tokens with the current implementation, using `.invoke` in the `generate` step, requires `@langchain/core` >= 0.3.24 and `@langchain/langgraph` >= 0.2.34. See details [here](https://langchain-ai.github.io/langgraphjs/how-tos/stream-tokens/) . #### Returning sources[โ€‹](#returning-sources "Direct link to Returning sources") Note that by storing the retrieved context in the state of the graph, we recover sources for the modelโ€™s generated answer in the `"context"` field of the state. See [this guide](/docs/how_to/qa_sources/) on returning sources for more detail. #### Go deeper[โ€‹](#go-deeper-3 "Direct link to Go deeper") [Chat models](/docs/concepts/chat_models) take in a sequence of messages and return a message. * [Docs](/docs/how_to#chat-models) * [Integrations](/docs/integrations/chat/) : 25+ integrations to choose from. **Customizing the prompt** As shown above, we can load prompts (e.g., [this RAG prompt](https://smith.langchain.com/hub/rlm/rag-prompt) ) from the prompt hub. The prompt can also be easily customized. For example: const template = `Use the following pieces of context to answer the question at the end.If you don't know the answer, just say that you don't know, don't try to make up an answer.Use three sentences maximum and keep the answer as concise as possible.Always say "thanks for asking!" at the end of the answer.{context}Question: {question}Helpful Answer:`;const promptTemplateCustom = ChatPromptTemplate.fromMessages([ ["user", template],]); Query analysis[โ€‹](#query-analysis "Direct link to Query analysis") ------------------------------------------------------------------- So far, we are executing the retrieval using the raw input query. However, there are some advantages to allowing a model to generate the query for retrieval purposes. For example: * In addition to semantic search, we can build in structured filters (e.g., โ€œFind documents since the year 2020.โ€); * The model can rewrite user queries, which may be multifaceted or include irrelevant language, into more effective search queries. [Query analysis](/docs/concepts/retrieval/#query-analysis) employs models to transform or construct optimized search queries from raw user input. We can easily incorporate a query analysis step into our application. For illustrative purposes, letโ€™s add some metadata to the documents in our vector store. We will add some (contrived) sections to the document which we can filter on later. const totalDocuments = allSplits.length;const third = Math.floor(totalDocuments / 3);allSplits.forEach((document, i) => { if (i < third) { document.metadata["section"] = "beginning"; } else if (i < 2 * third) { document.metadata["section"] = "middle"; } else { document.metadata["section"] = "end"; }});allSplits[0].metadata; { source: 'https://lilianweng.github.io/posts/2023-06-23-agent/', loc: { lines: { from: 1, to: 1 } }, section: 'beginning'} We will need to update the documents in our vector store. We will use a simple [MemoryVectorStore](https://api.js.langchain.com/classes/langchain.vectorstores_memory.MemoryVectorStore.html) for this, as we will use some of its specific features (i.e., metadata filtering). Refer to the vector store [integration documentation](/docs/integrations/vectorstores/) for relevant features of your chosen vector store. import { MemoryVectorStore } from "langchain/vectorstores/memory";const vectorStoreQA = new MemoryVectorStore(embeddings);await vectorStoreQA.addDocuments(allSplits); Letโ€™s next define a schema for our search query. We will use [structured output](/docs/concepts/structured_outputs/) for this purpose. Here we define a query as containing a string query and a document section (either โ€œbeginningโ€, โ€œmiddleโ€, or โ€œendโ€), but this can be defined however you like. import { z } from "zod";const searchSchema = z.object({ query: z.string().describe("Search query to run."), section: z.enum(["beginning", "middle", "end"]).describe("Section to query."),});const structuredLlm = llm.withStructuredOutput(searchSchema); Finally, we add a step to our LangGraph application to generate a query from the userโ€™s raw input: const StateAnnotationQA = Annotation.Root({ question: Annotation, search: Annotation>, context: Annotation, answer: Annotation,});const analyzeQuery = async (state: typeof InputStateAnnotation.State) => { const result = await structuredLlm.invoke(state.question); return { search: result };};const retrieveQA = async (state: typeof StateAnnotationQA.State) => { const filter = (doc) => doc.metadata.section === state.search.section; const retrievedDocs = await vectorStore.similaritySearch( state.search.query, 2, filter ); return { context: retrievedDocs };};const generateQA = async (state: typeof StateAnnotationQA.State) => { const docsContent = state.context.map((doc) => doc.pageContent).join("\n"); const messages = await promptTemplate.invoke({ question: state.question, context: docsContent, }); const response = await llm.invoke(messages); return { answer: response.content };};const graphQA = new StateGraph(StateAnnotationQA) .addNode("analyzeQuery", analyzeQuery) .addNode("retrieveQA", retrieveQA) .addNode("generateQA", generateQA) .addEdge("__start__", "analyzeQuery") .addEdge("analyzeQuery", "retrieveQA") .addEdge("retrieveQA", "generateQA") .addEdge("generateQA", "__end__") .compile(); // Note: tslab only works inside a jupyter notebook. Don't worry about running this code yourself!import * as tslab from "tslab";const image = await graphQA.getGraph().drawMermaidPng();const arrayBuffer = await image.arrayBuffer();await tslab.display.png(new Uint8Array(arrayBuffer)); ![graph_img_rag_qa](/assets/images/graph_img_rag_qa-974636bca4635fc8cd3d168abf5c6966.png) We can test our implementation by specifically asking for context from the end of the post. Note that the model includes different information in its answer. let inputsQA = { question: "What does the end of the post say about Task Decomposition?",};console.log(inputsQA);console.log("\n====\n");for await (const chunk of await graphQA.stream(inputsQA, { streamMode: "updates",})) { console.log(chunk); console.log("\n====\n");} { question: 'What does the end of the post say about Task Decomposition?'}===={ analyzeQuery: { search: { query: 'Task Decomposition', section: 'end' } }}===={ retrieveQA: { context: [ [Document], [Document] ] } }===={ generateQA: { answer: 'The end of the post emphasizes the importance of task decomposition by outlining a structured approach to organizing code into separate files and functions. It highlights the need for clarity and compatibility among different components, ensuring that each part of the architecture is well-defined and functional. This methodical breakdown aids in maintaining best practices and enhances code readability and manageability.' }}==== In both the streamed steps and the [LangSmith trace](https://smith.langchain.com/public/8ff4742c-a5d4-41b2-adf9-22915a876a30/r) , we can now observe the structured query that was fed into the retrieval step. Query Analysis is a rich problem with a wide range of approaches. Refer to the [how-to guides](/docs/how_to/#query-analysis) for more examples. Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Weโ€™ve covered the steps to build a basic Q&A app over data: * Loading data with a [Document Loader](/docs/concepts/document_loaders) * Chunking the indexed data with a [Text Splitter](/docs/concepts/text_splitters) to make it more easily usable by a model * [Embedding the data](/docs/concepts/embedding_models) and storing the data in a [vectorstore](/docs/how_to/vectorstores) * [Retrieving](/docs/concepts/retrievers) the previously stored chunks in response to incoming questions * Generating an answer using the retrieved chunks as context. In [Part 2](/docs/tutorials/qa_chat_history) of the tutorial, we will extend the implementation here to accommodate conversation-style interactions and multi-step retrieval processes. Further reading: * [Return sources](/docs/how_to/qa_sources) : Learn how to return source documents * [Streaming](/docs/how_to/streaming) : Learn how to stream outputs and intermediate steps * [Add chat history](/docs/how_to/message_history) : Learn how to add chat history to your app * [Retrieval conceptual guide](/docs/concepts/retrieval) : A high-level overview of specific retrieval techniques * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Overview](#overview) * [Indexing](#indexing) * [Retrieval and generation](#retrieval-and-generation) * [Setup](#setup) * [Jupyter Notebook](#jupyter-notebook) * [Installation](#installation) * [LangSmith](#langsmith) * [Components](#components) * [Preview](#preview) * [Detailed walkthrough](#detailed-walkthrough) * [1\. Indexing](#indexing) * [Loading documents](#loading-documents) * [Splitting documents](#splitting-documents) * [Storing documents](#storing-documents) * [2\. Retrieval and Generation](#orchestration) * [Query analysis](#query-analysis) * [Next steps](#next-steps) --- # How to use output parsers to parse an LLM response into structured format | ๐Ÿฆœ๏ธ๐Ÿ”— Langchain [Skip to main content](#__docusaurus_skipToContent_fallback) On this page Prerequisites This guide assumes familiarity with the following concepts: * [Output parsers](/docs/concepts/output_parsers) * [Chat models](/docs/concepts/chat_models) Language models output text. But there are times where you want to get more structured information than just text back. While some model providers support [built-in ways to return structured output](/docs/how_to/structured_output) , not all do. For these providers, you must use prompting to encourage the model to return structured data in the desired format. LangChain has [output parsers](/docs/concepts/output_parsers) which can help parse model outputs into usable objects. Weโ€™ll go over a few examples below. Get started[โ€‹](#get-started "Direct link to Get started") ---------------------------------------------------------- The primary type of output parser for working with structured data in model responses is the [`StructuredOutputParser`](https://api.js.langchain.com/classes/langchain_core.output_parsers.StructuredOutputParser.html) . In the below example, we define a schema for the type of output we expect from the model using [`zod`](https://zod.dev) . First, letโ€™s see the default formatting instructions weโ€™ll plug into the prompt: ### Pick your chat model: * OpenAI * Anthropic * FireworksAI * MistralAI * Groq * VertexAI #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/openai yarn add @langchain/openai pnpm add @langchain/openai #### Add environment variables OPENAI_API_KEY=your-api-key #### Instantiate the model import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/anthropic yarn add @langchain/anthropic pnpm add @langchain/anthropic #### Add environment variables ANTHROPIC_API_KEY=your-api-key #### Instantiate the model import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({ model: "claude-3-5-sonnet-20240620", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/community yarn add @langchain/community pnpm add @langchain/community #### Add environment variables FIREWORKS_API_KEY=your-api-key #### Instantiate the model import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({ model: "accounts/fireworks/models/llama-v3p1-70b-instruct", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/mistralai yarn add @langchain/mistralai pnpm add @langchain/mistralai #### Add environment variables MISTRAL_API_KEY=your-api-key #### Instantiate the model import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({ model: "mistral-large-latest", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/groq yarn add @langchain/groq pnpm add @langchain/groq #### Add environment variables GROQ_API_KEY=your-api-key #### Instantiate the model import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({ model: "mixtral-8x7b-32768", temperature: 0}); #### Install dependencies tip See [this section for general instructions on installing integration packages](/docs/how_to/installation/#installing-integration-packages) . * npm * yarn * pnpm npm i @langchain/google-vertexai yarn add @langchain/google-vertexai pnpm add @langchain/google-vertexai #### Add environment variables GOOGLE_APPLICATION_CREDENTIALS=credentials.json #### Instantiate the model import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({ model: "gemini-1.5-flash", temperature: 0}); import { z } from "zod";import { RunnableSequence } from "@langchain/core/runnables";import { StructuredOutputParser } from "@langchain/core/output_parsers";import { ChatPromptTemplate } from "@langchain/core/prompts";const zodSchema = z.object({ answer: z.string().describe("answer to the user's question"), source: z .string() .describe( "source used to answer the user's question, should be a website." ),});const parser = StructuredOutputParser.fromZodSchema(zodSchema);const chain = RunnableSequence.from([ ChatPromptTemplate.fromTemplate( "Answer the users question as best as possible.\n{format_instructions}\n{question}" ), model, parser,]);console.log(parser.getFormatInstructions()); You must format your output as a JSON value that adheres to a given "JSON Schema" instance."JSON Schema" is a declarative language that allows you to annotate and validate JSON documents.For example, the example "JSON Schema" instance {{"properties": {{"foo": {{"description": "a list of test words", "type": "array", "items": {{"type": "string"}}}}}}, "required": ["foo"]}}}}would match an object with one required property, "foo". The "type" property specifies "foo" must be an "array", and the "description" property semantically describes it as "a list of test words". The items within "foo" must be strings.Thus, the object {{"foo": ["bar", "baz"]}} is a well-formatted instance of this example "JSON Schema". The object {{"properties": {{"foo": ["bar", "baz"]}}}} is not well-formatted.Your output will be parsed and type-checked according to the provided schema instance, so make sure all fields in your output match the schema exactly and there are no trailing commas!Here is the JSON Schema instance your output must adhere to. Include the enclosing markdown codeblock:```json{"type":"object","properties":{"answer":{"type":"string","description":"answer to the user's question"},"source":{"type":"string","description":"source used to answer the user's question, should be a website."}},"required":["answer","source"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}``` Next, letโ€™s invoke the chain: const response = await chain.invoke({ question: "What is the capital of France?", format_instructions: parser.getFormatInstructions(),});console.log(response); { answer: "The capital of France is Paris.", source: "https://en.wikipedia.org/wiki/Paris"} Output parsers implement the [Runnable interface](/docs/how_to/#langchain-expression-language-lcel) , the basic building block of [LangChain Expression Language (LCEL)](/docs/how_to/#langchain-expression-language-lcel) . This means they support `invoke`, `stream`, `batch`, `streamLog` calls. Validation[โ€‹](#validation "Direct link to Validation") ------------------------------------------------------- One feature of the `StructuredOutputParser` is that it supports stricter Zod validations. For example, if you pass a simulated model output that does not conform to the schema, we get a detailed type error: import { AIMessage } from "@langchain/core/messages";await parser.invoke(new AIMessage(`{"badfield": "foo"}`)); Error: Failed to parse. Text: "{"badfield": "foo"}". Error: [ { "code": "invalid_type", "expected": "string", "received": "undefined", "path": [ "answer" ], "message": "Required" }, { "code": "invalid_type", "expected": "string", "received": "undefined", "path": [ "source" ], "message": "Required" }] Compared to: await parser.invoke( new AIMessage(`{"answer": "Paris", "source": "I made it up"}`)); { answer: "Paris", source: "I made it up" } More advanced Zod validations are supported as well. To learn more, check out the [Zod documentation](https://zod.dev) . Streaming[โ€‹](#streaming "Direct link to Streaming") ---------------------------------------------------- While all parsers are runnables and support the streaming interface, only certain parsers can stream through partially parsed objects, since this is highly dependent on the output type. The `StructuredOutputParser` does not support partial streaming because it validates the output at each step. If you try to stream using a chain with this output parser, the chain will simply yield the fully parsed output: const stream = await chain.stream({ question: "What is the capital of France?", format_instructions: parser.getFormatInstructions(),});for await (const s of stream) { console.log(s);} { answer: "The capital of France is Paris.", source: "https://en.wikipedia.org/wiki/Paris"} The simpler [`JsonOutputParser`](https://api.js.langchain.com/classes/langchain_core.output_parsers.JsonOutputParser.html) , however, supports streaming through partial outputs: import { JsonOutputParser } from "@langchain/core/output_parsers";const template = `Return a JSON object with a single key named "answer" that answers the following question: {question}.Do not wrap the JSON output in markdown blocks.`;const jsonPrompt = ChatPromptTemplate.fromTemplate(template);const jsonParser = new JsonOutputParser();const jsonChain = jsonPrompt.pipe(model).pipe(jsonParser);const stream = await jsonChain.stream({ question: "Who invented the microscope?",});for await (const s of stream) { console.log(s);} {}{ answer: "" }{ answer: "The" }{ answer: "The invention" }{ answer: "The invention of" }{ answer: "The invention of the" }{ answer: "The invention of the microscope" }{ answer: "The invention of the microscope is" }{ answer: "The invention of the microscope is attributed" }{ answer: "The invention of the microscope is attributed to" }{ answer: "The invention of the microscope is attributed to Hans" }{ answer: "The invention of the microscope is attributed to Hans L" }{ answer: "The invention of the microscope is attributed to Hans Lippers"}{ answer: "The invention of the microscope is attributed to Hans Lippershey"}{ answer: "The invention of the microscope is attributed to Hans Lippershey,"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zach"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Jans"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen,"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Anton"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 4 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 8 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 12 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 13 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 18 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 20 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 26 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 29 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 33 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 38 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 43 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 48 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 51 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 52 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 57 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 63 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 73 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 80 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 81 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 85 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 94 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 99 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 108 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 112 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 118 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 127 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 138 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 145 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 149 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 150 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 151 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 157 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 159 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 163 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 167 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 171 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 175 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 176 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 181 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 186 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 190 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 202 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 203 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 209 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 214 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 226 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 239 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 242 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 246 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 253 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 257 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 262 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 265 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 268 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 273 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 288 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 300 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 303 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 311 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 316 more characters}{ answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 317 more characters} Next steps[โ€‹](#next-steps "Direct link to Next steps") ------------------------------------------------------- Youโ€™ve learned about using output parsers to parse structured outputs from prompted model outputs. Next, check out the [guide on tool calling](/docs/how_to/tool_calling) , a more built-in way of obtaining structured output that some model providers support, or read more about output parsers for other types of structured data like [XML](/docs/how_to/output_parser_xml) . * * * #### Was this page helpful? #### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E) . * [Get started](#get-started) * [Validation](#validation) * [Streaming](#streaming) * [Next steps](#next-steps) ---