[2](#__codelineno-1-2)
[3](#__codelineno-1-3)
[4](#__codelineno-1-4)
[5](#__codelineno-1-5) | `AZURE_OPENAI_ENDPOINT= AZURE_OPENAI_API_KEY= OPENAI_API_VERSION=2024-02-15-preview # could be different for you AZURE_OPENAI_CHAT_DEPLOYMENT=gpt-35-turbo # change to your deployment name AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT=text-embedding-ada-002 # change to your deployment name` | ### Local models[¶](#local-models "Permanent link") Pros: * Privacy. Your documents will be stored and process locally. * Choices. There are a wide range of LLMs in terms of size, domain, language to choose from. * Cost. It's free. Cons: * Quality. Local models are much smaller and thus have lower generative quality than paid APIs. * Speed. Local models are deployed using your machine so the processing speed is limited by your hardware. #### Find and download a LLM[¶](#find-and-download-a-llm "Permanent link") You can search and download a LLM to be ran locally from the [Hugging Face Hub](https://huggingface.co/models) . Currently, these model formats are supported: * GGUF You should choose a model whose size is less than your device's memory and should leave about 2 GB. For example, if you have 16 GB of RAM in total, of which 12 GB is available, then you should choose a model that take up at most 10 GB of RAM. Bigger models tend to give better generation but also take more processing time. Here are some recommendations and their size in memory: * [Qwen1.5-1.8B-Chat-GGUF](https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat-GGUF/resolve/main/qwen1_5-1_8b-chat-q8_0.gguf?download=true) : around 2 GB #### Enable local models[¶](#enable-local-models "Permanent link") To add a local model to the model pool, set the `LOCAL_MODEL` variable in the `.env` file to the path of the model file. | | | | --- | --- | | [1](#__codelineno-2-1) | `LOCAL_MODEL=
[2](#__codelineno-0-2)
[3](#__codelineno-0-3)
[4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12) | ``+----------------------------------------------------------------------------+ \| End users: Those who use apps built with `kotaemon`. \| \| (You use an app like the one in the demo above) \| \| +----------------------------------------------------------------+ \| \| \| Developers: Those who built with `kotaemon`. \| \| \| \| (You have `import kotaemon` somewhere in your project) \| \| \| \| +----------------------------------------------------+ \| \| \| \| \| Contributors: Those who make `kotaemon` better. \| \| \| \| \| \| (You make PR to this repo) \| \| \| \| \| +----------------------------------------------------+ \| \| \| +----------------------------------------------------------------+ \| +----------------------------------------------------------------------------+`` | ### For end users[¶](#for-end-users "Permanent link") * **Clean & Minimalistic UI**: A user-friendly interface for RAG-based QA. * **Support for Various LLMs**: Compatible with LLM API providers (OpenAI, AzureOpenAI, Cohere, etc.) and local LLMs (via `ollama` and `llama-cpp-python`). * **Easy Installation**: Simple scripts to get you started quickly. ### For developers[¶](#for-developers "Permanent link") * **Framework for RAG Pipelines**: Tools to build your own RAG-based document QA pipeline. * **Customizable UI**: See your RAG pipeline in action with the provided UI, built with [Gradio ](https://github.com/gradio-app/gradio) . * **Gradio Theme**: If you use Gradio for development, check out our theme here: [kotaemon-gradio-theme](https://github.com/lone17/kotaemon-gradio-theme) . Key Features[¶](#key-features "Permanent link") ------------------------------------------------ * **Host your own document QA (RAG) web-UI**: Support multi-user login, organize your files in private/public collections, collaborate and share your favorite chat with others. * **Organize your LLM & Embedding models**: Support both local LLMs & popular API providers (OpenAI, Azure, Ollama, Groq). * **Hybrid RAG pipeline**: Sane default RAG pipeline with hybrid (full-text & vector) retriever and re-ranking to ensure best retrieval quality. * **Multi-modal QA support**: Perform Question Answering on multiple documents with figures and tables support. Support multi-modal document parsing (selectable options on UI). * **Advanced citations with document preview**: By default the system will provide detailed citations to ensure the correctness of LLM answers. View your citations (incl. relevant score) directly in the _in-browser PDF viewer_ with highlights. Warning when retrieval pipeline return low relevant articles. * **Support complex reasoning methods**: Use question decomposition to answer your complex/multi-hop question. Support agent-based reasoning with `ReAct`, `ReWOO` and other agents. * **Configurable settings UI**: You can adjust most important aspects of retrieval & generation process on the UI (incl. prompts). * **Extensible**: Being built on Gradio, you are free to customize or add any UI elements as you like. Also, we aim to support multiple strategies for document indexing & retrieval. `GraphRAG` indexing pipeline is provided as an example.  Installation[¶](#installation "Permanent link") ------------------------------------------------ > If you are not a developer and just want to use the app, please check out our easy-to-follow [User Guide](https://cinnamon.github.io/kotaemon/) > . Download the `.zip` file from the [latest release](https://github.com/Cinnamon/kotaemon/releases/latest) > to get all the newest features and bug fixes. ### System requirements[¶](#system-requirements "Permanent link") 1. [Python](https://www.python.org/downloads/) >= 3.10 2. [Docker](https://www.docker.com/) : optional, if you [install with Docker](../..#with-docker-recommended) 3. [Unstructured](https://docs.unstructured.io/open-source/installation/full-installation#full-installation) if you want to process files other than `.pdf`, `.html`, `.mhtml`, and `.xlsx` documents. Installation steps differ depending on your operating system. Please visit the link and follow the specific instructions provided there. ### With Docker (recommended)[¶](#with-docker-recommended "Permanent link") 1. We support both `lite` & `full` version of Docker images. With `full`, the extra packages of `unstructured` will be installed as well, it can support additional file types (`.doc`, `.docx`, ...) but the cost is larger docker image size. For most users, the `lite` image should work well in most cases. * To use the `lite` version. | | | | --- | --- | | [1](#__codelineno-1-1)
[2](#__codelineno-1-2)
[3](#__codelineno-1-3)
[4](#__codelineno-1-4)
[5](#__codelineno-1-5)
[6](#__codelineno-1-6) | `docker run \ -e GRADIO_SERVER_NAME=0.0.0.0 \ -e GRADIO_SERVER_PORT=7860 \ -v ./ktem_app_data:/app/ktem_app_data \ -p 7860:7860 -it --rm \ ghcr.io/cinnamon/kotaemon:main-lite` | * To use the `full` version. | | | | --- | --- | | [1](#__codelineno-2-1)
[2](#__codelineno-2-2)
[3](#__codelineno-2-3)
[4](#__codelineno-2-4)
[5](#__codelineno-2-5)
[6](#__codelineno-2-6) | `docker run \ -e GRADIO_SERVER_NAME=0.0.0.0 \ -e GRADIO_SERVER_PORT=7860 \ -v ./ktem_app_data:/app/ktem_app_data \ -p 7860:7860 -it --rm \ ghcr.io/cinnamon/kotaemon:main-full` | 2. We currently support and test two platforms: `linux/amd64` and `linux/arm64` (for newer Mac). You can specify the platform by passing `--platform` in the `docker run` command. For example: | | | | --- | --- | | [1](#__codelineno-3-1)
[2](#__codelineno-3-2)
[3](#__codelineno-3-3)
[4](#__codelineno-3-4)
[5](#__codelineno-3-5)
[6](#__codelineno-3-6)
[7](#__codelineno-3-7)
[8](#__codelineno-3-8) | `# To run docker with platform linux/arm64 docker run \ -e GRADIO_SERVER_NAME=0.0.0.0 \ -e GRADIO_SERVER_PORT=7860 \ -v ./ktem_app_data:/app/ktem_app_data \ -p 7860:7860 -it --rm \ --platform linux/arm64 \ ghcr.io/cinnamon/kotaemon:main-lite` | 3. Once everything is set up correctly, you can go to `http://localhost:7860/` to access the WebUI. 4. We use [GHCR](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) to store docker images, all images can be found [here.](https://github.com/Cinnamon/kotaemon/pkgs/container/kotaemon) ### Without Docker[¶](#without-docker "Permanent link") 1. Clone and install required packages on a fresh python environment. | | | | --- | --- | | [1](#__codelineno-4-1)
[2](#__codelineno-4-2)
[3](#__codelineno-4-3)
[4](#__codelineno-4-4)
[5](#__codelineno-4-5)
[6](#__codelineno-4-6)
[7](#__codelineno-4-7)
[8](#__codelineno-4-8)
[9](#__codelineno-4-9)
[10](#__codelineno-4-10) | `# optional (setup env) conda create -n kotaemon python=3.10 conda activate kotaemon # clone this repo git clone https://github.com/Cinnamon/kotaemon cd kotaemon pip install -e "libs/kotaemon[all]" pip install -e "libs/ktem"` | 2. Create a `.env` file in the root of this project. Use `.env.example` as a template The `.env` file is there to serve use cases where users want to pre-config the models before starting up the app (e.g. deploy the app on HF hub). The file will only be used to populate the db once upon the first run, it will no longer be used in consequent runs. 3. (Optional) To enable in-browser `PDF_JS` viewer, download [PDF\_JS\_DIST](https://github.com/mozilla/pdf.js/releases/download/v4.0.379/pdfjs-4.0.379-dist.zip) then extract it to `libs/ktem/ktem/assets/prebuilt`  1. Start the web server: | | | | --- | --- | | [1](#__codelineno-5-1) | `python app.py` | * The app will be automatically launched in your browser. * Default username and password are both `admin`. You can set up additional users directly through the UI.  2. Check the `Resources` tab and `LLMs and Embeddings` and ensure that your `api_key` value is set correctly from your `.env` file. If it is not set, you can set it there. ### Setup GraphRAG[¶](#setup-graphrag "Permanent link") > \[!NOTE\] Official MS GraphRAG indexing only works with OpenAI or Ollama API. We recommend most users to use NanoGraphRAG implementation for straightforward integration with Kotaemon. Setup Nano GRAPHRAG - Install nano-GraphRAG: \`pip install nano-graphrag\` - \`nano-graphrag\` install might introduce version conflicts, see \[this issue\](https://github.com/Cinnamon/kotaemon/issues/440) - To quickly fix: \`pip uninstall hnswlib chroma-hnswlib && pip install chroma-hnswlib\` - Launch Kotaemon with \`USE\_NANO\_GRAPHRAG=true\` environment variable. - Set your default LLM & Embedding models in Resources setting and it will be recognized automatically from NanoGraphRAG. Setup LIGHTRAG - Install LightRAG: \`pip install git+https://github.com/HKUDS/LightRAG.git\` - \`LightRAG\` install might introduce version conflicts, see \[this issue\](https://github.com/Cinnamon/kotaemon/issues/440) - To quickly fix: \`pip uninstall hnswlib chroma-hnswlib && pip install chroma-hnswlib\` - Launch Kotaemon with \`USE\_LIGHTRAG=true\` environment variable. - Set your default LLM & Embedding models in Resources setting and it will be recognized automatically from LightRAG. Setup MS GRAPHRAG - \*\*Non-Docker Installation\*\*: If you are not using Docker, install GraphRAG with the following command: | | | | --- | --- | | [1](#__codelineno-6-1) | `pip install "graphrag<=0.3.6" future` | \- \*\*Setting Up API KEY\*\*: To use the GraphRAG retriever feature, ensure you set the \`GRAPHRAG\_API\_KEY\` environment variable. You can do this directly in your environment or by adding it to a \`.env\` file. - \*\*Using Local Models and Custom Settings\*\*: If you want to use GraphRAG with local models (like \`Ollama\`) or customize the default LLM and other configurations, set the \`USE\_CUSTOMIZED\_GRAPHRAG\_SETTING\` environment variable to true. Then, adjust your settings in the \`settings.yaml.example\` file. ### Setup Local Models (for local/private RAG)[¶](#setup-local-models-for-localprivate-rag "Permanent link") See [Local model setup](../local_model/) . ### Setup multimodal document parsing (OCR, table parsing, figure extraction)[¶](#setup-multimodal-document-parsing-ocr-table-parsing-figure-extraction "Permanent link") These options are available: * [Azure Document Intelligence (API)](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence) * [Adobe PDF Extract (API)](https://developer.adobe.com/document-services/docs/overview/pdf-extract-api/) * [Docling (local, open-source)](https://github.com/DS4SD/docling) * To use Docling, first install required dependencies: `pip install docling` Select corresponding loaders in `Settings -> Retrieval Settings -> File loader` ### Customize your application[¶](#customize-your-application "Permanent link") * By default, all application data is stored in the `./ktem_app_data` folder. You can back up or copy this folder to transfer your installation to a new machine. * For advanced users or specific use cases, you can customize these files: * `flowsettings.py` * `.env` #### `flowsettings.py`[¶](#flowsettingspy "Permanent link") This file contains the configuration of your application. You can use the example [here](../../flowsettings.py) as the starting point. Notable settings | | | | --- | --- | | [1](#__codelineno-7-1)
[2](#__codelineno-7-2)
[3](#__codelineno-7-3)
[4](#__codelineno-7-4)
[5](#__codelineno-7-5)
[6](#__codelineno-7-6)
[7](#__codelineno-7-7)
[8](#__codelineno-7-8)
[9](#__codelineno-7-9)
[10](#__codelineno-7-10)
[11](#__codelineno-7-11)
[12](#__codelineno-7-12)
[13](#__codelineno-7-13)
[14](#__codelineno-7-14)
[15](#__codelineno-7-15)
[16](#__codelineno-7-16) | `# setup your preferred document store (with full-text search capabilities) KH_DOCSTORE=(Elasticsearch \| LanceDB \| SimpleFileDocumentStore) # setup your preferred vectorstore (for vector-based search) KH_VECTORSTORE=(ChromaDB \| LanceDB \| InMemory \| Qdrant) # Enable / disable multimodal QA KH_REASONINGS_USE_MULTIMODAL=True # Setup your new reasoning pipeline or modify existing one. KH_REASONINGS = [ "ktem.reasoning.simple.FullQAPipeline", "ktem.reasoning.simple.FullDecomposeQAPipeline", "ktem.reasoning.react.ReactAgentPipeline", "ktem.reasoning.rewoo.RewooAgentPipeline", ]` | #### `.env`[¶](#env "Permanent link") This file provides another way to configure your models and credentials. Configure model via the .env file - Alternatively, you can configure the models via the \`.env\` file with the information needed to connect to the LLMs. This file is located in the folder of the application. If you don't see it, you can create one. - Currently, the following providers are supported: - \*\*OpenAI\*\* In the \`.env\` file, set the \`OPENAI\_API\_KEY\` variable with your OpenAI API key in order to enable access to OpenAI's models. There are other variables that can be modified, please feel free to edit them to fit your case. Otherwise, the default parameter should work for most people. | | | | --- | --- | | [1](#__codelineno-8-1)
[2](#__codelineno-8-2)
[3](#__codelineno-8-3)
[4](#__codelineno-8-4) | `OPENAI_API_BASE=https://api.openai.com/v1 OPENAI_API_KEY=
[2](#__codelineno-9-2)
[3](#__codelineno-9-3)
[4](#__codelineno-9-4)
[5](#__codelineno-9-5) | `AZURE_OPENAI_ENDPOINT= AZURE_OPENAI_API_KEY= OPENAI_API_VERSION=2024-02-15-preview AZURE_OPENAI_CHAT_DEPLOYMENT=gpt-35-turbo AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT=text-embedding-ada-002` | \- \*\*Local Models\*\* - Using \`ollama\` OpenAI compatible server: - Install \[ollama\](https://github.com/ollama/ollama) and start the application. - Pull your model, for example: | | | | --- | --- | | [1](#__codelineno-10-1)
[2](#__codelineno-10-2) | `ollama pull llama3.1:8b ollama pull nomic-embed-text` | \- Set the model names on web UI and make it as default: !\[Models\](https://raw.githubusercontent.com/Cinnamon/kotaemon/main/docs/images/models.png) - Using \`GGUF\` with \`llama-cpp-python\` You can search and download a LLM to be ran locally from the \[Hugging Face Hub\](https://huggingface.co/models). Currently, these model formats are supported: - GGUF You should choose a model whose size is less than your device's memory and should leave about 2 GB. For example, if you have 16 GB of RAM in total, of which 12 GB is available, then you should choose a model that takes up at most 10 GB of RAM. Bigger models tend to give better generation but also take more processing time. Here are some recommendations and their size in memory: - \[Qwen1.5-1.8B-Chat-GGUF\](https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat-GGUF/resolve/main/qwen1\_5-1\_8b-chat-q8\_0.gguf?download=true): around 2 GB Add a new LlamaCpp model with the provided model name on the web UI. ### Adding your own RAG pipeline[¶](#adding-your-own-rag-pipeline "Permanent link") #### Custom Reasoning Pipeline[¶](#custom-reasoning-pipeline "Permanent link") 1. Check the default pipeline implementation in [here](../../libs/ktem/ktem/reasoning/simple.py) . You can make quick adjustment to how the default QA pipeline work. 2. Add new `.py` implementation in `libs/ktem/ktem/reasoning/` and later include it in `flowssettings` to enable it on the UI. #### Custom Indexing Pipeline[¶](#custom-indexing-pipeline "Permanent link") * Check sample implementation in `libs/ktem/ktem/index/file/graph` > (more instruction WIP). Back to top --- # Customize flow logic - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/pages/app/customize-flows.md "Edit this page") Add new indexing and reasoning pipeline to the application[¶](#add-new-indexing-and-reasoning-pipeline-to-the-application "Permanent link") ============================================================================================================================================ @trducng At high level, to add new indexing and reasoning pipeline: 1. You define your indexing or reasoning pipeline as a class from `BaseComponent`. 2. You declare that class in the setting files `flowsettings.py`. Then when `python app.py`, the application will dynamically load those pipelines. The below sections talk in more detail about how the pipelines should be constructed. Define a pipeline as a class[¶](#define-a-pipeline-as-a-class "Permanent link") -------------------------------------------------------------------------------- In essence, a pipeline will subclass from `kotaemon.base.BaseComponent`. Each pipeline has 2 main parts: * All declared arguments and sub-pipelines. * The logic inside the pipeline. An example pipeline: | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3)
[4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9) | `from kotaemon.base import BaseComponent class SoSimple(BaseComponent): arg1: int arg2: str def run(self, arg3: str): return self.arg1 * self.arg2 + arg3` | This pipeline is simple for demonstration purpose, but we can imagine pipelines with much more arguments, that can take other pipelines as arguments, and have more complicated logic in the `run` method. **_An indexing or reasoning pipeline is just a class subclass from `BaseComponent` like above._** For more detail on this topic, please refer to [Creating a Component](/create-a-component/) Run signatures[¶](#run-signatures "Permanent link") ---------------------------------------------------- **Note**: this section is tentative at the moment. We will finalize `def run` function signature by latest early April. The indexing pipeline: | | | | --- | --- | | [1](#__codelineno-1-1)
[2](#__codelineno-1-2)
[3](#__codelineno-1-3)
[4](#__codelineno-1-4)
[5](#__codelineno-1-5)
[6](#__codelineno-1-6)
[7](#__codelineno-1-7)
[8](#__codelineno-1-8)
[9](#__codelineno-1-9)
[10](#__codelineno-1-10)
[11](#__codelineno-1-11)
[12](#__codelineno-1-12)
[13](#__codelineno-1-13) | ``def run( self, file_paths: str \| Path \| list[str \| Path], reindex: bool = False, **kwargs, ): """Index files to intermediate representation (e.g. vector, database...) Args: file_paths: the list of paths to files reindex: if True, files in `file_paths` that already exists in database should be reindex. """`` | The reasoning pipeline: | | | | --- | --- | | [1](#__codelineno-2-1)
[2](#__codelineno-2-2)
[3](#__codelineno-2-3)
[4](#__codelineno-2-4)
[5](#__codelineno-2-5)
[6](#__codelineno-2-6)
[7](#__codelineno-2-7)
[8](#__codelineno-2-8)
[9](#__codelineno-2-9)
[10](#__codelineno-2-10) | `def run(self, question: str, history: list, **kwargs) -> Document: """Answer the question Args: question: the user input history: the chat history [(user_msg1, bot_msg1), (user_msg2, bot_msg2)...] Returns: kotaemon.base.Document: the final answer """` | Register your pipeline to ktem[¶](#register-your-pipeline-to-ktem "Permanent link") ------------------------------------------------------------------------------------ To register your pipelines to ktem, you declare it in the `flowsettings.py` file. This file locates at the current working directory where you start the ktem. In most use cases, it is this [one](https://github.com/Cinnamon/kotaemon/blob/main/flowsettings.py) . | | | | --- | --- | | [1](#__codelineno-3-1)
[2](#__codelineno-3-2)
[3](#__codelineno-3-3) | `KH_REASONING = ["
[2](#__codelineno-4-2)
[3](#__codelineno-4-3)
[4](#__codelineno-4-4)
[5](#__codelineno-4-5)
[6](#__codelineno-4-6)
[7](#__codelineno-4-7)
[8](#__codelineno-4-8)
[9](#__codelineno-4-9)
[10](#__codelineno-4-10)
[11](#__codelineno-4-11)
[12](#__codelineno-4-12)
[13](#__codelineno-4-13)
[14](#__codelineno-4-14)
[15](#__codelineno-4-15)
[16](#__codelineno-4-16)
[17](#__codelineno-4-17)
[18](#__codelineno-4-18)
[19](#__codelineno-4-19)
[20](#__codelineno-4-20)
[21](#__codelineno-4-21)
[22](#__codelineno-4-22)
[23](#__codelineno-4-23)
[24](#__codelineno-4-24)
[25](#__codelineno-4-25)
[26](#__codelineno-4-26)
[27](#__codelineno-4-27) | `class SoSimple(BaseComponent): ... # as above @classmethod def get_user_settings(cls) -> dict: """The settings to the user""" return { "setting_1": { "name": "Human-friendly name", "value": "Default value", "choices": [("Human-friendly Choice 1", "choice1-id"), ("HFC 2", "choice2-id")], # optional "component": "Which Gradio UI component to render, can be: text, number, checkbox, dropdown, radio, checkboxgroup" }, "setting_2": { # follow the same rule as above } } @classmethod def get_info(cls) -> dict: """Pipeline information for bookkeeping purpose""" return { "id": "a unique id to differentiate this pipeline from other pipeline", "name": "Human-friendly name of the pipeline", "description": "Can be a short description of this pipeline" }` | Once adding these methods to your pipeline class, `ktem` will automatically extract and add them to the settings. Construct to pipeline object[¶](#construct-to-pipeline-object "Permanent link") -------------------------------------------------------------------------------- Once `ktem` runs your pipeline, it will call your classmethod `get_pipeline` with the full user settings and expect to obtain the pipeline object. Within this `get_pipeline` method, you implement all the necessary logics to initiate the pipeline object. Example: | | | | --- | --- | | [1](#__codelineno-5-1)
[2](#__codelineno-5-2)
[3](#__codelineno-5-3)
[4](#__codelineno-5-4)
[5](#__codelineno-5-5)
[6](#__codelineno-5-6)
[7](#__codelineno-5-7) | `class SoSimple(BaseComponent): ... # as above @classmethod def get_pipeline(self, setting): obj = cls(arg1=setting["reasoning.id.setting1"]) return obj` | Reasoning: Stream output to UI[¶](#reasoning-stream-output-to-ui "Permanent link") ----------------------------------------------------------------------------------- For fast user experience, you can stream the output directly to UI. This way, user can start observing the output as soon as the LLM model generates the 1st token, rather than having to wait the pipeline finishes to read the whole message. To stream the output, you need to; 1. Turn the `run` function to async. 2. Pass in the output to a special queue with `self.report_output`. | | | | --- | --- | | [1](#__codelineno-6-1)
[2](#__codelineno-6-2)
[3](#__codelineno-6-3) | `async def run(self, question: str, history: list, **kwargs) -> Document: for char in "This is a long messages": self.report_output({"output": text.text})` | The argument to `self.report_output` is a dictionary, that contains either or all of these 2 keys: "output", "evidence". The "output" string will be streamed to the chat message, and the "evidence" string will be streamed to the information panel. Access application LLMs, Embeddings[¶](#access-application-llms-embeddings "Permanent link") --------------------------------------------------------------------------------------------- You can access users' collections of LLMs and embedding models with: | | | | --- | --- | | [1](#__codelineno-7-1)
[2](#__codelineno-7-2)
[3](#__codelineno-7-3)
[4](#__codelineno-7-4)
[5](#__codelineno-7-5)
[6](#__codelineno-7-6) | `from ktem.embeddings.manager import embeddings from ktem.llms.manager import llms llm = llms.get_default() embedding_model = embeddings.get_default()` | You can also allow the users to specifically select which llms or embedding models they want to use through the settings. | | | | --- | --- | | [1](#__codelineno-8-1)
[2](#__codelineno-8-2)
[3](#__codelineno-8-3)
[4](#__codelineno-8-4)
[5](#__codelineno-8-5)
[6](#__codelineno-8-6)
[7](#__codelineno-8-7)
[8](#__codelineno-8-8)
[9](#__codelineno-8-9)
[10](#__codelineno-8-10)
[11](#__codelineno-8-11)
[12](#__codelineno-8-12)
[13](#__codelineno-8-13) | `@classmethod def get_user_settings(cls) -> dict: from ktem.llms.manager import llms return { "citation_llm": { "name": "LLM for citation", "value": llms.get_default(), "component: "dropdown", "choices": list(llms.options().keys()), }, ... }` | Optional: Access application data[¶](#optional-access-application-data "Permanent link") ----------------------------------------------------------------------------------------- You can access the user's application database, vector store as follow: | | | | --- | --- | | [1](#__codelineno-9-1)
[2](#__codelineno-9-2)
[3](#__codelineno-9-3)
[4](#__codelineno-9-4) | `# get the database that contains the source files from ktem.db.models import Source, Index, Conversation, User # get the vector store` | Back to top --- # Creating a Component - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/development/create-a-component.md "Edit this page") Creating a component[¶](#creating-a-component "Permanent link") ================================================================ A fundamental concept in kotaemon is "component". Anything that isn't data or data structure is a "component". A component can be thought of as a step within a pipeline. It takes in some input, processes it, and returns an output, just the same as a Python function! The output will then become an input for the next component in a pipeline. In fact, a pipeline is just a component. More appropriately, a nested component: a component that makes use of one or more other components in the processing step. So in reality, there isn't a difference between a pipeline and a component! Because of that, in kotaemon, we will consider them the same as "component". To define a component, you will: 1. Create a class that subclasses from `kotaemon.base.BaseComponent` 2. Declare init params with type annotation 3. Declare nodes (nodes are just other components!) with type annotation 4. Implement the processing logic in `run`. The syntax of a component is as follow: | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3)
[4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19) | `from kotaemon.base import BaseComponent from kotaemon.llms import LCAzureChatOpenAI from kotaemon.parsers import RegexExtractor class FancyPipeline(BaseComponent): param1: str = "This is param1" param2: int = 10 param3: float node1: BaseComponent # this is a node because of BaseComponent type annotation node2: LCAzureChatOpenAI # this is also a node because LCAzureChatOpenAI subclasses BaseComponent node3: RegexExtractor # this is also a node bceause RegexExtractor subclasses BaseComponent def run(self, some_text: str): prompt = (self.param1 + some_text) * int(self.param2 + self.param3) llm_pred = self.node2(prompt).text matches = self.node3(llm_pred) return matches` | Then this component can be used as follow: | | | | --- | --- | | [1](#__codelineno-1-1)
[2](#__codelineno-1-2)
[3](#__codelineno-1-3)
[4](#__codelineno-1-4)
[5](#__codelineno-1-5)
[6](#__codelineno-1-6)
[7](#__codelineno-1-7)
[8](#__codelineno-1-8)
[9](#__codelineno-1-9)
[10](#__codelineno-1-10)
[11](#__codelineno-1-11) | `llm = LCAzureChatOpenAI(endpoint="some-endpont") extractor = RegexExtractor(pattern=["yes", "Yes"]) component = FancyPipeline( param1="Hello" param3=1.5 node1=llm, node2=llm, node3=extractor ) component("goodbye")` | This way, we can define each operation as a reusable component, and use them to compose larger reusable components! Benefits of component[¶](#benefits-of-component "Permanent link") ------------------------------------------------------------------ By defining a component as above, we formally encapsulate all the necessary information inside a single class. This introduces several benefits: 1. Allow tools like promptui to inspect the inner working of a component in order to automatically generate the promptui. 2. Allow visualizing a pipeline for debugging purpose. Back to top --- # Agents - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/agents/__init__.py "Edit this page") Agents ====== BaseAgent [¶](#agents.BaseAgent "Permanent link") -------------------------------------------------- Bases: `BaseComponent` Define base agent interface Source code in `libs/kotaemon/kotaemon/agents/base.py` | | | | --- | --- | | [10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57) | `class BaseAgent(BaseComponent): """Define base agent interface""" name: str = Param(help="Name of the agent.") agent_type: AgentType = Param(help="Agent type, must be one of AgentType") description: str = Param( help=( "Description used to tell the model how/when/why to use the agent. You can" " provide few-shot examples as a part of the description. This will be" " input to the prompt of LLM." ) ) llm: Optional[BaseLLM] = Node( help=( "LLM to be used for the agent (optional). LLM must implement BaseLLM" " interface." ) ) prompt_template: Optional[Union[PromptTemplate, dict[str, PromptTemplate]]] = Param( help="A prompt template or a dict to supply different prompt to the agent" ) plugins: list[BaseTool] = Param( default_callback=lambda _: [], help="List of plugins / tools to be used in the agent", ) @staticmethod def safeguard_run(run_func, *args, **kwargs): def wrapper(self, *args, **kwargs): try: return run_func(self, *args, **kwargs) except Exception as e: return AgentOutput( text="", agent_type=self.agent_type, status="failed", error=str(e), ) return wrapper def add_tools(self, tools: list[BaseTool]) -> None: """Helper method to add tools and update agent state if needed""" self.plugins.extend(tools) def run(self, *args, **kwargs) -> AgentOutput \| list[AgentOutput]: """Run the component.""" raise NotImplementedError()` | ### add\_tools [¶](#agents.BaseAgent.add_tools "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `add_tools(tools)` | Helper method to add tools and update agent state if needed Source code in `libs/kotaemon/kotaemon/agents/base.py` | | | | --- | --- | | [51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53) | `def add_tools(self, tools: list[BaseTool]) -> None: """Helper method to add tools and update agent state if needed""" self.plugins.extend(tools)` | ### run [¶](#agents.BaseAgent.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(*args, **kwargs)` | Run the component. Source code in `libs/kotaemon/kotaemon/agents/base.py` | | | | --- | --- | | [55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57) | `def run(self, *args, **kwargs) -> AgentOutput \| list[AgentOutput]: """Run the component.""" raise NotImplementedError()` | AgentFinish [¶](#agents.AgentFinish "Permanent link") ------------------------------------------------------ Bases: `NamedTuple` Agent's return value when finishing execution. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `return_values` | | The return values of the agent. | _required_ | | `log` | | The log message. | _required_ | Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238) | `class AgentFinish(NamedTuple): """Agent's return value when finishing execution. Args: return_values: The return values of the agent. log: The log message. """ return_values: dict log: str` | AgentOutput [¶](#agents.AgentOutput "Permanent link") ------------------------------------------------------ Bases: `LLMInterface` Output from an agent. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `text` | | The text output from the agent. | _required_ | | `agent_type` | | The type of agent. | _required_ | | `status` | | The status after executing the agent. | _required_ | | `error` | | The error message if any. | _required_ | Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258) | `class AgentOutput(LLMInterface): """Output from an agent. Args: text: The text output from the agent. agent_type: The type of agent. status: The status after executing the agent. error: The error message if any. """ model_config = ConfigDict(extra="allow") text: str type: str = "agent" agent_type: AgentType status: Literal["thinking", "finished", "stopped", "failed"] error: Optional[str] = None intermediate_steps: Optional[list] = None` | AgentType [¶](#agents.AgentType "Permanent link") -------------------------------------------------- Bases: `Enum` Enumerated type for agent types. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33) | `class AgentType(Enum): """ Enumerated type for agent types. """ openai = "openai" openai_multi = "openai_multi" openai_tool = "openai_tool" self_ask = "self_ask" react = "react" rewoo = "rewoo" vanilla = "vanilla"` | BaseScratchPad [¶](#agents.BaseScratchPad "Permanent link") ------------------------------------------------------------ Base class for output handlers. #### Attributes:[¶](#agents.BaseScratchPad--attributes "Permanent link") logger : logging.Logger The logger object to log messages. #### Methods:[¶](#agents.BaseScratchPad--methods "Permanent link") stop(): Stop the output. update\_status(output: str, \*\*kwargs): Update the status of the output. thinking(name: str): Log that a process is thinking. done(\_all=False): Log that the process is done. stream\_print(item: str): Not implemented. json\_print(item: Dict\[str, Any\]): Log a JSON object. panel\_print(item: Any, title: str = "Output", stream: bool = False): Log a panel output. clear(): Not implemented. print(content: str, \*\*kwargs): Log arbitrary content. format\_json(json\_obj: str): Format a JSON object. debug(content: str, \*\*kwargs): Log a debug message. info(content: str, \*\*kwargs): Log an informational message. warning(content: str, \*\*kwargs): Log a warning message. error(content: str, \*\*kwargs): Log an error message. critical(content: str, \*\*kwargs): Log a critical message. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211) | `class BaseScratchPad: """ Base class for output handlers. Attributes: ----------- logger : logging.Logger The logger object to log messages. Methods: -------- stop(): Stop the output. update_status(output: str, **kwargs): Update the status of the output. thinking(name: str): Log that a process is thinking. done(_all=False): Log that the process is done. stream_print(item: str): Not implemented. json_print(item: Dict[str, Any]): Log a JSON object. panel_print(item: Any, title: str = "Output", stream: bool = False): Log a panel output. clear(): Not implemented. print(content: str, **kwargs): Log arbitrary content. format_json(json_obj: str): Format a JSON object. debug(content: str, **kwargs): Log a debug message. info(content: str, **kwargs): Log an informational message. warning(content: str, **kwargs): Log a warning message. error(content: str, **kwargs): Log an error message. critical(content: str, **kwargs): Log a critical message. """ def __init__(self): """ Initialize the BaseOutput object. """ self.logger = logging self.log = [] def stop(self): """ Stop the output. """ def update_status(self, output: str, **kwargs): """ Update the status of the output. """ if check_log(): self.logger.info(output) def thinking(self, name: str): """ Log that a process is thinking. """ if check_log(): self.logger.info(f"{name} is thinking...") def done(self, _all=False): """ Log that the process is done. """ if check_log(): self.logger.info("Done") def stream_print(self, item: str): """ Stream print. """ def json_print(self, item: Dict[str, Any]): """ Log a JSON object. """ if check_log(): self.logger.info(json.dumps(item, indent=2)) def panel_print(self, item: Any, title: str = "Output", stream: bool = False): """ Log a panel output. Args: item : Any The item to log. title : str, optional The title of the panel, defaults to "Output". stream : bool, optional """ if not stream: self.log.append(item) if check_log(): self.logger.info("-" * 20) self.logger.info(item) self.logger.info("-" * 20) def clear(self): """ Not implemented. """ def print(self, content: str, **kwargs): """ Log arbitrary content. """ self.log.append(content) if check_log(): self.logger.info(content) def format_json(self, json_obj: str): """ Format a JSON object. """ formatted_json = json.dumps(json_obj, indent=2) return formatted_json def debug(self, content: str, **kwargs): """ Log a debug message. """ if check_log(): self.logger.debug(content, **kwargs) def info(self, content: str, **kwargs): """ Log an informational message. """ if check_log(): self.logger.info(content, **kwargs) def warning(self, content: str, **kwargs): """ Log a warning message. """ if check_log(): self.logger.warning(content, **kwargs) def error(self, content: str, **kwargs): """ Log an error message. """ if check_log(): self.logger.error(content, **kwargs) def critical(self, content: str, **kwargs): """ Log a critical message. """ if check_log(): self.logger.critical(content, **kwargs)` | ### stop [¶](#agents.BaseScratchPad.stop "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `stop()` | Stop the output. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104) | `def stop(self): """ Stop the output. """` | ### update\_status [¶](#agents.BaseScratchPad.update_status "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `update_status(output, **kwargs)` | Update the status of the output. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111) | `def update_status(self, output: str, **kwargs): """ Update the status of the output. """ if check_log(): self.logger.info(output)` | ### thinking [¶](#agents.BaseScratchPad.thinking "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `thinking(name)` | Log that a process is thinking. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118) | `def thinking(self, name: str): """ Log that a process is thinking. """ if check_log(): self.logger.info(f"{name} is thinking...")` | ### done [¶](#agents.BaseScratchPad.done "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `done(_all=False)` | Log that the process is done. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126) | `def done(self, _all=False): """ Log that the process is done. """ if check_log(): self.logger.info("Done")` | ### stream\_print [¶](#agents.BaseScratchPad.stream_print "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `stream_print(item)` | Stream print. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131) | `def stream_print(self, item: str): """ Stream print. """` | ### json\_print [¶](#agents.BaseScratchPad.json_print "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `json_print(item)` | Log a JSON object. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138) | `def json_print(self, item: Dict[str, Any]): """ Log a JSON object. """ if check_log(): self.logger.info(json.dumps(item, indent=2))` | ### panel\_print [¶](#agents.BaseScratchPad.panel_print "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `panel_print(item, title='Output', stream=False)` | Log a panel output. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `item` | | Any The item to log. | _required_ | | `title` | | str, optional The title of the panel, defaults to "Output". | `'Output'` | | `stream` | | bool, optional | `False` | Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156) | `def panel_print(self, item: Any, title: str = "Output", stream: bool = False): """ Log a panel output. Args: item : Any The item to log. title : str, optional The title of the panel, defaults to "Output". stream : bool, optional """ if not stream: self.log.append(item) if check_log(): self.logger.info("-" * 20) self.logger.info(item) self.logger.info("-" * 20)` | ### clear [¶](#agents.BaseScratchPad.clear "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `clear()` | Not implemented. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161) | `def clear(self): """ Not implemented. """` | ### print [¶](#agents.BaseScratchPad.print "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `print(content, **kwargs)` | Log arbitrary content. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169) | `def print(self, content: str, **kwargs): """ Log arbitrary content. """ self.log.append(content) if check_log(): self.logger.info(content)` | ### format\_json [¶](#agents.BaseScratchPad.format_json "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `format_json(json_obj)` | Format a JSON object. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176) | `def format_json(self, json_obj: str): """ Format a JSON object. """ formatted_json = json.dumps(json_obj, indent=2) return formatted_json` | ### debug [¶](#agents.BaseScratchPad.debug "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `debug(content, **kwargs)` | Log a debug message. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183) | `def debug(self, content: str, **kwargs): """ Log a debug message. """ if check_log(): self.logger.debug(content, **kwargs)` | ### info [¶](#agents.BaseScratchPad.info "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `info(content, **kwargs)` | Log an informational message. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190) | `def info(self, content: str, **kwargs): """ Log an informational message. """ if check_log(): self.logger.info(content, **kwargs)` | ### warning [¶](#agents.BaseScratchPad.warning "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `warning(content, **kwargs)` | Log a warning message. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197) | `def warning(self, content: str, **kwargs): """ Log a warning message. """ if check_log(): self.logger.warning(content, **kwargs)` | ### error [¶](#agents.BaseScratchPad.error "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `error(content, **kwargs)` | Log an error message. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204) | `def error(self, content: str, **kwargs): """ Log an error message. """ if check_log(): self.logger.error(content, **kwargs)` | ### critical [¶](#agents.BaseScratchPad.critical "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `critical(content, **kwargs)` | Log a critical message. Source code in `libs/kotaemon/kotaemon/agents/io/base.py` | | | | --- | --- | | [206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211) | `def critical(self, content: str, **kwargs): """ Log a critical message. """ if check_log(): self.logger.critical(content, **kwargs)` | LangchainAgent [¶](#agents.LangchainAgent "Permanent link") ------------------------------------------------------------ Bases: `BaseAgent` Wrapper for Langchain Agent Source code in `libs/kotaemon/kotaemon/agents/langchain_based.py` | | | | --- | --- | | [14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79) | `class LangchainAgent(BaseAgent): """Wrapper for Langchain Agent""" name: str = "LangchainAgent" agent_type: AgentType description: str = "LangchainAgent for answering multi-step reasoning questions" AGENT_TYPE_MAP = { AgentType.openai: LCAgentType.OPENAI_FUNCTIONS, AgentType.openai_multi: LCAgentType.OPENAI_MULTI_FUNCTIONS, AgentType.react: LCAgentType.ZERO_SHOT_REACT_DESCRIPTION, AgentType.self_ask: LCAgentType.SELF_ASK_WITH_SEARCH, } agent: Optional[LCAgentExecutor] = None def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) if self.agent_type not in self.AGENT_TYPE_MAP: raise NotImplementedError( f"AgentType {self.agent_type } not supported by Langchain wrapper" ) self.update_agent_tools() def update_agent_tools(self): assert isinstance(self.llm, (ChatLLM, LLM)) langchain_plugins = [tool.to_langchain_format() for tool in self.plugins] # a fix for search_doc tool name: # use "Intermediate Answer" for self-ask agent found_search_tool = False if self.agent_type == AgentType.self_ask: for plugin in langchain_plugins: if plugin.name == "search_doc": plugin.name = "Intermediate Answer" langchain_plugins = [plugin] found_search_tool = True break if self.agent_type != AgentType.self_ask or found_search_tool: # reinit Langchain AgentExecutor self.agent = initialize_agent( langchain_plugins, self.llm.to_langchain_format(), agent=self.AGENT_TYPE_MAP[self.agent_type], handle_parsing_errors=True, verbose=True, ) def add_tools(self, tools: List[BaseTool]) -> None: super().add_tools(tools) self.update_agent_tools() return def run(self, instruction: str) -> AgentOutput: assert ( self.agent is not None ), "Lanchain AgentExecutor is not correctly initialized" # Langchain AgentExecutor call output = self.agent(instruction)["output"] return AgentOutput( text=output, agent_type=self.agent_type, status="finished", )` | ReactAgent [¶](#agents.ReactAgent "Permanent link") ---------------------------------------------------- Bases: `BaseAgent` Sequential ReactAgent class inherited from BaseAgent. Implementing ReAct agent paradigm https://arxiv.org/pdf/2210.03629.pdf Source code in `libs/kotaemon/kotaemon/agents/react/agent.py` | | | | --- | --- | | [18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310)
[311](#__codelineno-0-311)
[312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317)
[318](#__codelineno-0-318)
[319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345) | ``class ReactAgent(BaseAgent): """ Sequential ReactAgent class inherited from BaseAgent. Implementing ReAct agent paradigm https://arxiv.org/pdf/2210.03629.pdf """ name: str = "ReactAgent" agent_type: AgentType = AgentType.react description: str = "ReactAgent for answering multi-step reasoning questions" llm: BaseLLM prompt_template: Optional[PromptTemplate] = None output_lang: str = "English" plugins: list[BaseTool] = Param( default_callback=lambda _: [], help="List of tools to be used in the agent. " ) examples: dict[str, str \| list[str]] = Param( default_callback=lambda _: {}, help="Examples to be used in the agent. " ) intermediate_steps: list[tuple[AgentAction \| AgentFinish, str]] = Param( default_callback=lambda _: [], help="List of AgentAction and observation (tool) output", ) max_iterations: int = 5 strict_decode: bool = False max_context_length: int = Param( default=3000, help="Max context length for each tool output.", ) trim_func: TokenSplitter \| None = None def _compose_plugin_description(self) -> str: """ Compose the worker prompt from the workers. Example: toolname1[input]: tool1 description toolname2[input]: tool2 description """ prompt = "" try: for plugin in self.plugins: prompt += f"{plugin.name}[input]: {plugin.description}\n" except Exception: raise ValueError("Worker must have a name and description.") return prompt def _construct_scratchpad( self, intermediate_steps: list[tuple[AgentAction \| AgentFinish, str]] = [] ) -> str: """Construct the scratchpad that lets the agent continue its thought process.""" thoughts = "" for action, observation in intermediate_steps: thoughts += action.log thoughts += f"\nObservation: {observation}\nThought:" return thoughts def _parse_output(self, text: str) -> Optional[AgentAction \| AgentFinish]: """ Parse text output from LLM for the next Action or Final Answer Using Regex to parse "Action:\n Action Input:\n" for the next Action Using FINAL_ANSWER_ACTION to parse Final Answer Args: text[str]: input text to parse """ includes_answer = FINAL_ANSWER_ACTION in text regex = ( r"Action\s*\d*\s*:[\s]*(.*?)[\s]*Action\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)" ) action_match = re.search(regex, text, re.DOTALL) action_output: Optional[AgentAction \| AgentFinish] = None if action_match: if includes_answer: raise Exception( "Parsing LLM output produced both a final answer " f"and a parse-able action: {text}" ) action = action_match.group(1).strip() action_input = action_match.group(2) tool_input = action_input.strip(" ") # ensure if its a well formed SQL query we don't remove any trailing " chars if tool_input.startswith("SELECT ") is False: tool_input = tool_input.strip('"') action_output = AgentAction(action, tool_input, text) elif includes_answer: action_output = AgentFinish( {"output": text.split(FINAL_ANSWER_ACTION)[-1].strip()}, text ) else: if self.strict_decode: raise Exception(f"Could not parse LLM output: `{text}`") else: action_output = AgentFinish({"output": text}, text) return action_output def _compose_prompt(self, instruction) -> str: """ Compose the prompt from template, worker description, examples and instruction. """ agent_scratchpad = self._construct_scratchpad(self.intermediate_steps) tool_description = self._compose_plugin_description() tool_names = ", ".join([plugin.name for plugin in self.plugins]) if self.prompt_template is None: from .prompt import zero_shot_react_prompt self.prompt_template = zero_shot_react_prompt return self.prompt_template.populate( instruction=instruction, agent_scratchpad=agent_scratchpad, tool_description=tool_description, tool_names=tool_names, lang=self.output_lang, ) def _format_function_map(self) -> dict[str, BaseTool]: """Format the function map for the open AI function API. Return: Dict[str, Callable]: The function map. """ # Map the function name to the real function object. function_map = {} for plugin in self.plugins: function_map[plugin.name] = plugin return function_map def _trim(self, text: str \| Document) -> str: """ Trim the text to the maximum token length. """ evidence_trim_func = ( self.trim_func if self.trim_func else TokenSplitter( chunk_size=self.max_context_length, chunk_overlap=0, separator=" ", tokenizer=partial( tiktoken.encoding_for_model("gpt-3.5-turbo").encode, allowed_special=set(), disallowed_special="all", ), ) ) if isinstance(text, str): texts = evidence_trim_func([Document(text=text)]) elif isinstance(text, Document): texts = evidence_trim_func([text]) else: raise ValueError("Invalid text type to trim") trim_text = texts[0].text logging.info(f"len (trimmed): {len(trim_text)}") return trim_text def clear(self): """ Clear and reset the agent. """ self.intermediate_steps = [] def run(self, instruction, max_iterations=None) -> AgentOutput: """ Run the agent with the given instruction. Args: instruction: Instruction to run the agent with. max_iterations: Maximum number of iterations of reasoning steps, defaults to 10. Return: AgentOutput object. """ if not max_iterations: max_iterations = self.max_iterations assert max_iterations > 0 self.clear() logging.info(f"Running {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 status = "failed" response_text = None for step_count in range(1, max_iterations + 1): prompt = self._compose_prompt(instruction) logging.info(f"Prompt: {prompt}") response = self.llm( prompt, stop=["Observation:"] ) # could cause bugs if llm doesn't have `stop` as a parameter response_text = response.text logging.info(f"Response: {response_text}") action_step = self._parse_output(response_text) if action_step is None: raise ValueError("Invalid action") is_finished_chain = isinstance(action_step, AgentFinish) if is_finished_chain: result = "" else: assert isinstance(action_step, AgentAction) action_name = action_step.tool tool_input = action_step.tool_input logging.info(f"Action: {action_name}") logging.info(f"Tool Input: {tool_input}") result = self._format_function_map()[action_name](tool_input) # trim the worker output to 1000 tokens, as we are appending # all workers' logs and it can exceed the token limit if we # don't limit each. Fix this number regarding to the LLM capacity. result = self._trim(result) logging.info(f"Result: {result}") self.intermediate_steps.append((action_step, result)) if is_finished_chain: logging.info(f"Finished after {step_count} steps.") status = "finished" break else: status = "stopped" return AgentOutput( text=response_text, agent_type=self.agent_type, status=status, total_tokens=total_token, total_cost=total_cost, intermediate_steps=self.intermediate_steps, max_iterations=max_iterations, ) def stream(self, instruction, max_iterations=None): """ Stream the agent with the given instruction. Args: instruction: Instruction to run the agent with. max_iterations: Maximum number of iterations of reasoning steps, defaults to 10. Return: AgentOutput object. """ if not max_iterations: max_iterations = self.max_iterations assert max_iterations > 0 self.clear() logging.info(f"Running {self.name} with instruction: {instruction}") print(f"Running {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 status = "failed" response_text = None for step_count in range(1, max_iterations + 1): prompt = self._compose_prompt(instruction) logging.info(f"Prompt: {prompt}") print(f"Prompt: {prompt}") response = self.llm( prompt, stop=["Observation:"] ) # TODO: could cause bugs if llm doesn't have `stop` as a parameter response_text = response.text logging.info(f"Response: {response_text}") print(f"Response: {response_text}") action_step = self._parse_output(response_text) if action_step is None: raise ValueError("Invalid action") is_finished_chain = isinstance(action_step, AgentFinish) if is_finished_chain: result = response_text if "Final Answer:" in response_text: result = response_text.split("Final Answer:")[-1].strip() else: assert isinstance(action_step, AgentAction) action_name = action_step.tool tool_input = action_step.tool_input logging.info(f"Action: {action_name}") print(f"Action: {action_name}") logging.info(f"Tool Input: {tool_input}") print(f"Tool Input: {tool_input}") result = self._format_function_map()[action_name](tool_input) # trim the worker output to 1000 tokens, as we are appending # all workers' logs and it can exceed the token limit if we # don't limit each. Fix this number regarding to the LLM capacity. result = self._trim(result) logging.info(f"Result: {result}") print(f"Result: {result}") self.intermediate_steps.append((action_step, result)) if is_finished_chain: logging.info(f"Finished after {step_count} steps.") status = "finished" yield AgentOutput( text=result, agent_type=self.agent_type, status=status, intermediate_steps=self.intermediate_steps[-1], ) break else: yield AgentOutput( text="", agent_type=self.agent_type, status="thinking", intermediate_steps=self.intermediate_steps[-1], ) else: status = "stopped" yield AgentOutput( text="", agent_type=self.agent_type, status=status, intermediate_steps=self.intermediate_steps[-1], ) return AgentOutput( text=response_text, agent_type=self.agent_type, status=status, total_tokens=total_token, total_cost=total_cost, intermediate_steps=self.intermediate_steps, max_iterations=max_iterations, )`` | ### clear [¶](#agents.ReactAgent.clear "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `clear()` | Clear and reset the agent. Source code in `libs/kotaemon/kotaemon/agents/react/agent.py` | | | | --- | --- | | [175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179) | `def clear(self): """ Clear and reset the agent. """ self.intermediate_steps = []` | ### run [¶](#agents.ReactAgent.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(instruction, max_iterations=None)` | Run the agent with the given instruction. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `instruction` | | Instruction to run the agent with. | _required_ | | `max_iterations` | | Maximum number of iterations of reasoning steps, defaults to 10. | `None` | Return AgentOutput object. Source code in `libs/kotaemon/kotaemon/agents/react/agent.py` | | | | --- | --- | | [181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248) | ``def run(self, instruction, max_iterations=None) -> AgentOutput: """ Run the agent with the given instruction. Args: instruction: Instruction to run the agent with. max_iterations: Maximum number of iterations of reasoning steps, defaults to 10. Return: AgentOutput object. """ if not max_iterations: max_iterations = self.max_iterations assert max_iterations > 0 self.clear() logging.info(f"Running {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 status = "failed" response_text = None for step_count in range(1, max_iterations + 1): prompt = self._compose_prompt(instruction) logging.info(f"Prompt: {prompt}") response = self.llm( prompt, stop=["Observation:"] ) # could cause bugs if llm doesn't have `stop` as a parameter response_text = response.text logging.info(f"Response: {response_text}") action_step = self._parse_output(response_text) if action_step is None: raise ValueError("Invalid action") is_finished_chain = isinstance(action_step, AgentFinish) if is_finished_chain: result = "" else: assert isinstance(action_step, AgentAction) action_name = action_step.tool tool_input = action_step.tool_input logging.info(f"Action: {action_name}") logging.info(f"Tool Input: {tool_input}") result = self._format_function_map()[action_name](tool_input) # trim the worker output to 1000 tokens, as we are appending # all workers' logs and it can exceed the token limit if we # don't limit each. Fix this number regarding to the LLM capacity. result = self._trim(result) logging.info(f"Result: {result}") self.intermediate_steps.append((action_step, result)) if is_finished_chain: logging.info(f"Finished after {step_count} steps.") status = "finished" break else: status = "stopped" return AgentOutput( text=response_text, agent_type=self.agent_type, status=status, total_tokens=total_token, total_cost=total_cost, intermediate_steps=self.intermediate_steps, max_iterations=max_iterations, )`` | ### stream [¶](#agents.ReactAgent.stream "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `stream(instruction, max_iterations=None)` | Stream the agent with the given instruction. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `instruction` | | Instruction to run the agent with. | _required_ | | `max_iterations` | | Maximum number of iterations of reasoning steps, defaults to 10. | `None` | Return AgentOutput object. Source code in `libs/kotaemon/kotaemon/agents/react/agent.py` | | | | --- | --- | | [250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310)
[311](#__codelineno-0-311)
[312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317)
[318](#__codelineno-0-318)
[319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345) | ``def stream(self, instruction, max_iterations=None): """ Stream the agent with the given instruction. Args: instruction: Instruction to run the agent with. max_iterations: Maximum number of iterations of reasoning steps, defaults to 10. Return: AgentOutput object. """ if not max_iterations: max_iterations = self.max_iterations assert max_iterations > 0 self.clear() logging.info(f"Running {self.name} with instruction: {instruction}") print(f"Running {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 status = "failed" response_text = None for step_count in range(1, max_iterations + 1): prompt = self._compose_prompt(instruction) logging.info(f"Prompt: {prompt}") print(f"Prompt: {prompt}") response = self.llm( prompt, stop=["Observation:"] ) # TODO: could cause bugs if llm doesn't have `stop` as a parameter response_text = response.text logging.info(f"Response: {response_text}") print(f"Response: {response_text}") action_step = self._parse_output(response_text) if action_step is None: raise ValueError("Invalid action") is_finished_chain = isinstance(action_step, AgentFinish) if is_finished_chain: result = response_text if "Final Answer:" in response_text: result = response_text.split("Final Answer:")[-1].strip() else: assert isinstance(action_step, AgentAction) action_name = action_step.tool tool_input = action_step.tool_input logging.info(f"Action: {action_name}") print(f"Action: {action_name}") logging.info(f"Tool Input: {tool_input}") print(f"Tool Input: {tool_input}") result = self._format_function_map()[action_name](tool_input) # trim the worker output to 1000 tokens, as we are appending # all workers' logs and it can exceed the token limit if we # don't limit each. Fix this number regarding to the LLM capacity. result = self._trim(result) logging.info(f"Result: {result}") print(f"Result: {result}") self.intermediate_steps.append((action_step, result)) if is_finished_chain: logging.info(f"Finished after {step_count} steps.") status = "finished" yield AgentOutput( text=result, agent_type=self.agent_type, status=status, intermediate_steps=self.intermediate_steps[-1], ) break else: yield AgentOutput( text="", agent_type=self.agent_type, status="thinking", intermediate_steps=self.intermediate_steps[-1], ) else: status = "stopped" yield AgentOutput( text="", agent_type=self.agent_type, status=status, intermediate_steps=self.intermediate_steps[-1], ) return AgentOutput( text=response_text, agent_type=self.agent_type, status=status, total_tokens=total_token, total_cost=total_cost, intermediate_steps=self.intermediate_steps, max_iterations=max_iterations, )`` | RewooAgent [¶](#agents.RewooAgent "Permanent link") ---------------------------------------------------- Bases: `BaseAgent` Distributive RewooAgent class inherited from BaseAgent. Implementing ReWOO paradigm https://arxiv.org/pdf/2305.18323.pdf Source code in `libs/kotaemon/kotaemon/agents/rewoo/agent.py` | | | | --- | --- | | [22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310)
[311](#__codelineno-0-311)
[312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317)
[318](#__codelineno-0-318)
[319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345)
[346](#__codelineno-0-346)
[347](#__codelineno-0-347)
[348](#__codelineno-0-348)
[349](#__codelineno-0-349)
[350](#__codelineno-0-350)
[351](#__codelineno-0-351)
[352](#__codelineno-0-352)
[353](#__codelineno-0-353)
[354](#__codelineno-0-354)
[355](#__codelineno-0-355)
[356](#__codelineno-0-356)
[357](#__codelineno-0-357)
[358](#__codelineno-0-358)
[359](#__codelineno-0-359)
[360](#__codelineno-0-360)
[361](#__codelineno-0-361)
[362](#__codelineno-0-362)
[363](#__codelineno-0-363)
[364](#__codelineno-0-364)
[365](#__codelineno-0-365)
[366](#__codelineno-0-366)
[367](#__codelineno-0-367)
[368](#__codelineno-0-368)
[369](#__codelineno-0-369)
[370](#__codelineno-0-370)
[371](#__codelineno-0-371)
[372](#__codelineno-0-372)
[373](#__codelineno-0-373)
[374](#__codelineno-0-374)
[375](#__codelineno-0-375)
[376](#__codelineno-0-376)
[377](#__codelineno-0-377)
[378](#__codelineno-0-378)
[379](#__codelineno-0-379)
[380](#__codelineno-0-380)
[381](#__codelineno-0-381)
[382](#__codelineno-0-382)
[383](#__codelineno-0-383)
[384](#__codelineno-0-384) | `class RewooAgent(BaseAgent): """Distributive RewooAgent class inherited from BaseAgent. Implementing ReWOO paradigm https://arxiv.org/pdf/2305.18323.pdf""" name: str = "RewooAgent" agent_type: AgentType = AgentType.rewoo description: str = "RewooAgent for answering multi-step reasoning questions" output_lang: str = "English" planner_llm: BaseLLM solver_llm: BaseLLM prompt_template: dict[str, PromptTemplate] = Param( default_callback=lambda _: {}, help="A dict to supply different prompt to the agent.", ) plugins: list[BaseTool] = Param( default_callback=lambda _: [], help="A list of plugins to be used in the model." ) examples: dict[str, str \| list[str]] = Param( default_callback=lambda _: {}, help="Examples to be used in the agent." ) max_context_length: int = Param( default=3000, help="Max context length for each tool output.", ) trim_func: TokenSplitter \| None = None @Node.auto(depends_on=["planner_llm", "plugins", "prompt_template", "examples"]) def planner(self): return Planner( model=self.planner_llm, plugins=self.plugins, prompt_template=self.prompt_template.get("Planner", None), examples=self.examples.get("Planner", None), ) @Node.auto(depends_on=["solver_llm", "prompt_template", "examples"]) def solver(self): return Solver( model=self.solver_llm, prompt_template=self.prompt_template.get("Solver", None), examples=self.examples.get("Solver", None), output_lang=self.output_lang, ) def _parse_plan_map( self, planner_response: str ) -> tuple[dict[str, list[str]], dict[str, str]]: """ Parse planner output. It should be an n-to-n mapping from Plans to #Es. This is because sometimes LLM cannot follow the strict output format. Example: #Plan1 #E1 #E2 should result in: {"#Plan1": ["#E1", "#E2"]} Or: #Plan1 #Plan2 #E1 should result in: {"#Plan1": [], "#Plan2": ["#E1"]} This function should also return a plan map. Returns: tuple[Dict[str, List[str]], Dict[str, str]]: A list of plan map """ valid_chunk = [ line for line in planner_response.splitlines() if line.startswith("#Plan") or line.startswith("#E") ] plan_to_es: dict[str, list[str]] = dict() plans: dict[str, str] = dict() prev_key = "" for line in valid_chunk: key, description = line.split(":", 1) key = key.strip() if key.startswith("#Plan"): plans[key] = description.strip() plan_to_es[key] = [] prev_key = key elif key.startswith("#E"): plan_to_es[prev_key].append(key) return plan_to_es, plans def _parse_planner_evidences( self, planner_response: str ) -> tuple[dict[str, str], list[list[str]]]: """ Parse planner output. This should return a mapping from #E to tool call. It should also identify the level of each #E in dependency map. Example: { "#E1": "Tool1", "#E2": "Tool2", "#E3": "Tool3", "#E4": "Tool4" }, [[#E1, #E2], [#E3, #E4]] Returns: tuple[dict[str, str], List[List[str]]]: A mapping from #E to tool call and a list of levels. """ evidences: dict[str, str] = dict() dependence: dict[str, list[str]] = dict() for line in planner_response.splitlines(): if line.startswith("#E") and line[2].isdigit(): e, tool_call = line.split(":", 1) e, tool_call = e.strip(), tool_call.strip() if len(e) == 3: dependence[e] = [] evidences[e] = tool_call for var in re.findall(r"#E\d+", tool_call): if var in evidences: dependence[e].append(var) else: evidences[e] = "No evidence found" level = [] while dependence: select = [i for i in dependence if not dependence[i]] if len(select) == 0: raise ValueError("Circular dependency detected.") level.append(select) for item in select: dependence.pop(item) for item in dependence: for i in select: if i in dependence[item]: dependence[item].remove(i) return evidences, level def _run_plugin( self, e: str, planner_evidences: dict[str, str], worker_evidences: dict[str, str], output=BaseScratchPad(), ): """ Run a plugin for a given evidence. This function should also cumulate the cost and tokens. """ result = dict(e=e, plugin_cost=0, plugin_token=0, evidence="") tool_call = planner_evidences[e] if "[" not in tool_call: result["evidence"] = tool_call else: tool, tool_input = tool_call.split("[", 1) tool_input = tool_input[:-1] # find variables in input and replace with previous evidences for var in re.findall(r"#E\d+", tool_input): print("Tool input: ", tool_input) print("Var: ", var) print("Worker evidences: ", worker_evidences) if var in worker_evidences: tool_input = tool_input.replace( var, worker_evidences.get(var, "") or "" ) try: selected_plugin = self._find_plugin(tool) if selected_plugin is None: raise ValueError("Invalid plugin detected") tool_response = selected_plugin(tool_input) result["evidence"] = get_plugin_response_content(tool_response) except ValueError: result["evidence"] = "No evidence found." finally: output.panel_print( result["evidence"], f"[green] Function Response of [blue]{tool}: " ) return result def _get_worker_evidence( self, planner_evidences: dict[str, str], evidences_level: list[list[str]], output=BaseScratchPad(), ) -> Any: """ Parallel execution of plugins in DAG for speedup. This is one of core benefits of ReWOO agents. Args: planner_evidences: A mapping from #E to tool call. evidences_level: A list of levels of evidences. Calculated from DAG of plugin calls. output: Output object, defaults to BaseOutput(). Returns: A mapping from #E to tool call. """ worker_evidences: dict[str, str] = dict() plugin_cost, plugin_token = 0.0, 0.0 with ThreadPoolExecutor() as pool: for level in evidences_level: results = [] for e in level: results.append( pool.submit( self._run_plugin, e, planner_evidences, worker_evidences, output, ) ) if len(results) > 1: output.update_status(f"Running tasks {level} in parallel.") else: output.update_status(f"Running task {level[0]}.") for r in results: resp = r.result() plugin_cost += resp["plugin_cost"] plugin_token += resp["plugin_token"] worker_evidences[resp["e"]] = self._trim_evidence(resp["evidence"]) output.done() return worker_evidences, plugin_cost, plugin_token def _find_plugin(self, name: str): for p in self.plugins: if p.name == name: return p def _trim_evidence(self, evidence: str): evidence_trim_func = ( self.trim_func if self.trim_func else TokenSplitter( chunk_size=self.max_context_length, chunk_overlap=0, separator=" ", tokenizer=partial( tiktoken.encoding_for_model("gpt-3.5-turbo").encode, allowed_special=set(), disallowed_special="all", ), ) ) if evidence: texts = evidence_trim_func([Document(text=evidence)]) evidence = texts[0].text logging.info(f"len (trimmed): {len(evidence)}") return evidence @BaseAgent.safeguard_run def run(self, instruction: str, use_citation: bool = False) -> AgentOutput: """ Run the agent with a given instruction. """ logging.info(f"Running {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 # Plan planner_output = self.planner(instruction) planner_text_output = planner_output.text plan_to_es, plans = self._parse_plan_map(planner_text_output) planner_evidences, evidence_level = self._parse_planner_evidences( planner_text_output ) # Work worker_evidences, plugin_cost, plugin_token = self._get_worker_evidence( planner_evidences, evidence_level ) worker_log = "" for plan in plan_to_es: worker_log += f"{plan}: {plans[plan]}\n" for e in plan_to_es[plan]: worker_log += f"{e}: {worker_evidences[e]}\n" # Solve solver_output = self.solver(instruction, worker_log) solver_output_text = solver_output.text if use_citation: citation_pipeline = CitationPipeline(llm=self.solver_llm) citation = citation_pipeline(context=worker_log, question=instruction) else: citation = None return AgentOutput( text=solver_output_text, agent_type=self.agent_type, status="finished", total_tokens=total_token, total_cost=total_cost, citation=citation, metadata={"citation": citation, "worker_log": worker_log}, ) def stream(self, instruction: str, use_citation: bool = False): """ Stream the agent with a given instruction. """ logging.info(f"Streaming {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 # Plan planner_output = self.planner(instruction) planner_text_output = planner_output.text plan_to_es, plans = self._parse_plan_map(planner_text_output) planner_evidences, evidence_level = self._parse_planner_evidences( planner_text_output ) print("Planner output:", planner_text_output) # output planner to info panel yield AgentOutput( text="", agent_type=self.agent_type, status="thinking", intermediate_steps=[{"planner_log": planner_text_output}], ) # Work worker_evidences, plugin_cost, plugin_token = self._get_worker_evidence( planner_evidences, evidence_level ) worker_log = "" for plan in plan_to_es: worker_log += f"{plan}: {plans[plan]}\n" current_progress = f"{plan}: {plans[plan]}\n" for e in plan_to_es[plan]: worker_log += f"#Action: {planner_evidences.get(e, None)}\n" worker_log += f"{e}: {worker_evidences[e]}\n" current_progress += f"#Action: {planner_evidences.get(e, None)}\n" current_progress += f"{e}: {worker_evidences[e]}\n" yield AgentOutput( text="", agent_type=self.agent_type, status="thinking", intermediate_steps=[{"worker_log": current_progress}], ) # Solve solver_response = "" for solver_output in self.solver.stream(instruction, worker_log): solver_output_text = solver_output.text solver_response += solver_output_text yield AgentOutput( text=solver_output_text, agent_type=self.agent_type, status="thinking", ) if use_citation: citation_pipeline = CitationPipeline(llm=self.solver_llm) citation = citation_pipeline.invoke( context=worker_log, question=instruction ) else: citation = None return AgentOutput( text="", agent_type=self.agent_type, status="finished", total_tokens=total_token, total_cost=total_cost, citation=citation, metadata={"citation": citation, "worker_log": worker_log}, )` |\ \ ### run [¶](#agents.RewooAgent.run "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `run(instruction, use_citation=False)` |\ \ Run the agent with a given instruction.\ \ Source code in `libs/kotaemon/kotaemon/agents/rewoo/agent.py`\ \ | | |\ | --- | --- |\ | [266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310) | `@BaseAgent.safeguard_run def run(self, instruction: str, use_citation: bool = False) -> AgentOutput: """ Run the agent with a given instruction. """ logging.info(f"Running {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 # Plan planner_output = self.planner(instruction) planner_text_output = planner_output.text plan_to_es, plans = self._parse_plan_map(planner_text_output) planner_evidences, evidence_level = self._parse_planner_evidences( planner_text_output ) # Work worker_evidences, plugin_cost, plugin_token = self._get_worker_evidence( planner_evidences, evidence_level ) worker_log = "" for plan in plan_to_es: worker_log += f"{plan}: {plans[plan]}\n" for e in plan_to_es[plan]: worker_log += f"{e}: {worker_evidences[e]}\n" # Solve solver_output = self.solver(instruction, worker_log) solver_output_text = solver_output.text if use_citation: citation_pipeline = CitationPipeline(llm=self.solver_llm) citation = citation_pipeline(context=worker_log, question=instruction) else: citation = None return AgentOutput( text=solver_output_text, agent_type=self.agent_type, status="finished", total_tokens=total_token, total_cost=total_cost, citation=citation, metadata={"citation": citation, "worker_log": worker_log}, )` |\ \ ### stream [¶](#agents.RewooAgent.stream "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `stream(instruction, use_citation=False)` |\ \ Stream the agent with a given instruction.\ \ Source code in `libs/kotaemon/kotaemon/agents/rewoo/agent.py`\ \ | | |\ | --- | --- |\ | [312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317)
[318](#__codelineno-0-318)
[319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345)
[346](#__codelineno-0-346)
[347](#__codelineno-0-347)
[348](#__codelineno-0-348)
[349](#__codelineno-0-349)
[350](#__codelineno-0-350)
[351](#__codelineno-0-351)
[352](#__codelineno-0-352)
[353](#__codelineno-0-353)
[354](#__codelineno-0-354)
[355](#__codelineno-0-355)
[356](#__codelineno-0-356)
[357](#__codelineno-0-357)
[358](#__codelineno-0-358)
[359](#__codelineno-0-359)
[360](#__codelineno-0-360)
[361](#__codelineno-0-361)
[362](#__codelineno-0-362)
[363](#__codelineno-0-363)
[364](#__codelineno-0-364)
[365](#__codelineno-0-365)
[366](#__codelineno-0-366)
[367](#__codelineno-0-367)
[368](#__codelineno-0-368)
[369](#__codelineno-0-369)
[370](#__codelineno-0-370)
[371](#__codelineno-0-371)
[372](#__codelineno-0-372)
[373](#__codelineno-0-373)
[374](#__codelineno-0-374)
[375](#__codelineno-0-375)
[376](#__codelineno-0-376)
[377](#__codelineno-0-377)
[378](#__codelineno-0-378)
[379](#__codelineno-0-379)
[380](#__codelineno-0-380)
[381](#__codelineno-0-381)
[382](#__codelineno-0-382)
[383](#__codelineno-0-383)
[384](#__codelineno-0-384) | `def stream(self, instruction: str, use_citation: bool = False): """ Stream the agent with a given instruction. """ logging.info(f"Streaming {self.name} with instruction: {instruction}") total_cost = 0.0 total_token = 0 # Plan planner_output = self.planner(instruction) planner_text_output = planner_output.text plan_to_es, plans = self._parse_plan_map(planner_text_output) planner_evidences, evidence_level = self._parse_planner_evidences( planner_text_output ) print("Planner output:", planner_text_output) # output planner to info panel yield AgentOutput( text="", agent_type=self.agent_type, status="thinking", intermediate_steps=[{"planner_log": planner_text_output}], ) # Work worker_evidences, plugin_cost, plugin_token = self._get_worker_evidence( planner_evidences, evidence_level ) worker_log = "" for plan in plan_to_es: worker_log += f"{plan}: {plans[plan]}\n" current_progress = f"{plan}: {plans[plan]}\n" for e in plan_to_es[plan]: worker_log += f"#Action: {planner_evidences.get(e, None)}\n" worker_log += f"{e}: {worker_evidences[e]}\n" current_progress += f"#Action: {planner_evidences.get(e, None)}\n" current_progress += f"{e}: {worker_evidences[e]}\n" yield AgentOutput( text="", agent_type=self.agent_type, status="thinking", intermediate_steps=[{"worker_log": current_progress}], ) # Solve solver_response = "" for solver_output in self.solver.stream(instruction, worker_log): solver_output_text = solver_output.text solver_response += solver_output_text yield AgentOutput( text=solver_output_text, agent_type=self.agent_type, status="thinking", ) if use_citation: citation_pipeline = CitationPipeline(llm=self.solver_llm) citation = citation_pipeline.invoke( context=worker_log, question=instruction ) else: citation = None return AgentOutput( text="", agent_type=self.agent_type, status="finished", total_tokens=total_token, total_cost=total_cost, citation=citation, metadata={"citation": citation, "worker_log": worker_log}, )` |\ \ BaseTool [¶](#agents.BaseTool "Permanent link")\ \ ------------------------------------------------\ \ Bases: `BaseComponent`\ \ Source code in `libs/kotaemon/kotaemon/agents/tools/base.py`\ \ | | |\ | --- | --- |\ | [19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120) | ``class BaseTool(BaseComponent): name: str """The unique name of the tool that clearly communicates its purpose.""" description: str """Description used to tell the model how/when/why to use the tool. You can provide few-shot examples as a part of the description. This will be input to the prompt of LLM. """ args_schema: Optional[Type[BaseModel]] = None """Pydantic model class to validate and parse the tool's input arguments.""" verbose: bool = False """Whether to log the tool's progress.""" handle_tool_error: Optional[ Union[bool, str, Callable[[ToolException], str]] ] = False """Handle the content of the ToolException thrown.""" def _parse_input( self, tool_input: Union[str, Dict], ) -> Union[str, Dict[str, Any]]: """Convert tool input to pydantic model.""" args_schema = self.args_schema if isinstance(tool_input, str): if args_schema is not None: key_ = next(iter(args_schema.model_fields.keys())) args_schema.validate({key_: tool_input}) return tool_input else: if args_schema is not None: result = args_schema.parse_obj(tool_input) return {k: v for k, v in result.dict().items() if k in tool_input} return tool_input def _run_tool( self, *args: Any, **kwargs: Any, ) -> Any: """Call tool.""" raise NotImplementedError(f"_run_tool is not implemented for {self.name}") def _to_args_and_kwargs(self, tool_input: Union[str, Dict]) -> Tuple[Tuple, Dict]: # For backwards compatibility, if run_input is a string, # pass as a positional argument. if isinstance(tool_input, str): return (tool_input,), {} else: return (), tool_input def _handle_tool_error(self, e: ToolException) -> Any: """Handle the content of the ToolException thrown.""" observation = None if not self.handle_tool_error: raise e elif isinstance(self.handle_tool_error, bool): if e.args: observation = e.args[0] else: observation = "Tool execution error" elif isinstance(self.handle_tool_error, str): observation = self.handle_tool_error elif callable(self.handle_tool_error): observation = self.handle_tool_error(e) else: raise ValueError( f"Got unexpected type of `handle_tool_error`. Expected bool, str " f"or callable. Received: {self.handle_tool_error}" ) return observation def to_langchain_format(self) -> LCTool: """Convert this tool to Langchain format to use with its agent""" return LCTool(name=self.name, description=self.description, func=self.run) def run( self, tool_input: Union[str, Dict], verbose: Optional[bool] = None, **kwargs: Any, ) -> Any: """Run the tool.""" parsed_input = self._parse_input(tool_input) # TODO (verbose_): Add logging try: tool_args, tool_kwargs = self._to_args_and_kwargs(parsed_input) call_kwargs = {**kwargs, **tool_kwargs} observation = self._run_tool(*tool_args, **call_kwargs) except ToolException as e: observation = self._handle_tool_error(e) return observation else: return observation @classmethod def from_langchain_format(cls, langchain_tool: LCTool) -> "BaseTool": """Wrapper for Langchain Tool""" new_tool = BaseTool( name=langchain_tool.name, description=langchain_tool.description ) new_tool._run_tool = langchain_tool._run # type: ignore return new_tool`` |\ \ ### name `instance-attribute` [¶](#agents.BaseTool.name "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `name` |\ \ The unique name of the tool that clearly communicates its purpose.\ \ ### description `instance-attribute` [¶](#agents.BaseTool.description "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `description` |\ \ Description used to tell the model how/when/why to use the tool. You can provide few-shot examples as a part of the description. This will be input to the prompt of LLM.\ \ ### args\_schema `class-attribute` `instance-attribute` [¶](#agents.BaseTool.args_schema "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `args_schema = None` |\ \ Pydantic model class to validate and parse the tool's input arguments.\ \ ### verbose `class-attribute` `instance-attribute` [¶](#agents.BaseTool.verbose "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `verbose = False` |\ \ Whether to log the tool's progress.\ \ ### handle\_tool\_error `class-attribute` `instance-attribute` [¶](#agents.BaseTool.handle_tool_error "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `handle_tool_error = False` |\ \ Handle the content of the ToolException thrown.\ \ ### to\_langchain\_format [¶](#agents.BaseTool.to_langchain_format "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `to_langchain_format()` |\ \ Convert this tool to Langchain format to use with its agent\ \ Source code in `libs/kotaemon/kotaemon/agents/tools/base.py`\ \ | | |\ | --- | --- |\ | [90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92) | `def to_langchain_format(self) -> LCTool: """Convert this tool to Langchain format to use with its agent""" return LCTool(name=self.name, description=self.description, func=self.run)` |\ \ ### run [¶](#agents.BaseTool.run "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `run(tool_input, verbose=None, **kwargs)` |\ \ Run the tool.\ \ Source code in `libs/kotaemon/kotaemon/agents/tools/base.py`\ \ | | |\ | --- | --- |\ | [94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111) | `def run( self, tool_input: Union[str, Dict], verbose: Optional[bool] = None, **kwargs: Any, ) -> Any: """Run the tool.""" parsed_input = self._parse_input(tool_input) # TODO (verbose_): Add logging try: tool_args, tool_kwargs = self._to_args_and_kwargs(parsed_input) call_kwargs = {**kwargs, **tool_kwargs} observation = self._run_tool(*tool_args, **call_kwargs) except ToolException as e: observation = self._handle_tool_error(e) return observation else: return observation` |\ \ ### from\_langchain\_format `classmethod` [¶](#agents.BaseTool.from_langchain_format "Permanent link")\ \ | | |\ | --- | --- |\ | [1](#__codelineno-0-1) | `from_langchain_format(langchain_tool)` |\ \ Wrapper for Langchain Tool\ \ Source code in `libs/kotaemon/kotaemon/agents/tools/base.py`\ \ | | |\ | --- | --- |\ | [113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120) | `@classmethod def from_langchain_format(cls, langchain_tool: LCTool) -> "BaseTool": """Wrapper for Langchain Tool""" new_tool = BaseTool( name=langchain_tool.name, description=langchain_tool.description ) new_tool._run_tool = langchain_tool._run # type: ignore return new_tool` |\ \ ComponentTool [¶](#agents.ComponentTool "Permanent link")\ \ ----------------------------------------------------------\ \ Bases: `BaseTool`\ \ Wrapper around other BaseComponent to use it as a tool\ \ Parameters:\ \ | Name | Type | Description | Default |\ | --- | --- | --- | --- |\ | `component` | | BaseComponent-based component to wrap | _required_ |\ | `postprocessor` | | Optional postprocessor for the component output | _required_ |\ \ Source code in `libs/kotaemon/kotaemon/agents/tools/base.py`\ \ | | |\ | --- | --- |\ | [123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139) | `class ComponentTool(BaseTool): """Wrapper around other BaseComponent to use it as a tool Args: component: BaseComponent-based component to wrap postprocessor: Optional postprocessor for the component output """ component: BaseComponent postprocessor: Optional[Callable] = None def _run_tool(self, *args: Any, **kwargs: Any) -> Any: output = self.component(*args, **kwargs) if self.postprocessor: output = self.postprocessor(output) return output` |\ \ WikipediaTool [¶](#agents.WikipediaTool "Permanent link")\ \ ----------------------------------------------------------\ \ Bases: `BaseTool`\ \ Tool that adds the capability to query the Wikipedia API.\ \ Source code in `libs/kotaemon/kotaemon/agents/tools/wikipedia.py`\ \ | | |\ | --- | --- |\ | [48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66) | `class WikipediaTool(BaseTool): """Tool that adds the capability to query the Wikipedia API.""" name: str = "wikipedia" description: str = ( "Search engine from Wikipedia, retrieving relevant wiki page. " "Useful when you need to get holistic knowledge about people, " "places, companies, historical events, or other subjects. " "Input should be a search query." ) args_schema: Optional[Type[BaseModel]] = WikipediaArgs doc_store: Any = None def _run_tool(self, query: AnyStr) -> AnyStr: if not self.doc_store: self.doc_store = Wiki() tool = self.doc_store evidence = tool.search(query) return evidence` |\ \ Back to top --- # File index - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/pages/app/index/file.md "Edit this page") File index ========== The file index stores files in a local folder and index them for retrieval. This file index provides the following infrastructure to support the indexing: * SQL table Source: store the list of files that are indexed by the system * Vector store: contain the embedding of segments of the files * Document store: contain the text of segments of the files. Each text stored in this document store is associated with a vector in the vector store. * SQL table Index: store the relationship between (1) the source and the docstore, and (2) the source and the vector store. The indexing and retrieval pipelines are encouraged to use the above software infrastructure. Indexing pipeline[¶](#indexing-pipeline "Permanent link") ---------------------------------------------------------- The ktem has default indexing pipeline: `ktem.index.file.pipelines.IndexDocumentPipeline`. This default pipeline works as follow: * **Input**: list of file paths * **Output**: list of nodes that are indexed into database * **Process**: * Read files into texts. Different file types has different ways to read texts. * Split text files into smaller segments * Run each segments into embeddings. * Store the embeddings into vector store. Store the texts of each segment into docstore. Store the list of files in Source. Store the linking between Sources and docstore + vectorstore in Index table. You can customize this default pipeline if your indexing process is close to the default pipeline. You can create your own indexing pipeline if there are too much different logic. ### Customize the default pipeline[¶](#customize-the-default-pipeline "Permanent link") The default pipeline provides the contact points in `flowsettings.py`. 1. `FILE_INDEX_PIPELINE_FILE_EXTRACTORS`. Supply overriding file extractor, based on file extension. Example: `{".pdf": "path.to.PDFReader", ".xlsx": "path.to.ExcelReader"}` 2. `FILE_INDEX_PIPELINE_SPLITTER_CHUNK_SIZE`. The expected number of characters of each text segment. Example: 1024. 3. `FILE_INDEX_PIPELINE_SPLITTER_CHUNK_OVERLAP`. The expected number of characters that consecutive text segments should overlap with each other. Example: 256. ### Create your own indexing pipeline[¶](#create-your-own-indexing-pipeline "Permanent link") Your indexing pipeline will subclass `BaseFileIndexIndexing`. You should define the following methods: * `run(self, file_paths)`: run the indexing given the pipeline * `get_pipeline(cls, user_settings, index_settings)`: return the fully-initialized pipeline, ready to be used by ktem. * `user_settings`: is a dictionary contains user settings (e.g. `{"pdf_mode": True, "num_retrieval": 5}`). You can declare these settings in the `get_user_settings` classmethod. ktem will collect these settings into the app Settings page, and will supply these user settings to your `get_pipeline` method. * `index_settings`: is a dictionary. Currently it's empty for File Index. * `get_user_settings`: to declare user settings, return a dictionary. By subclassing `BaseFileIndexIndexing`, You will have access to the following resources: * `self._Source`: the source table * `self._Index`: the index table * `self._VS`: the vector store * `self._DS`: the docstore Once you have prepared your pipeline, register it in `flowsettings.py`: `FILE_INDEX_PIPELINE = "
\- name (str): the name of the file
\- path (str): the path of the file
\- size (int): the file size in bytes
\- note (dict): allow extra optional information about the file
\- date\_created (datetime): the time the file is created (auto) | This is SQLALchemy ORM class. Can consult | | SQL table Index | self.\_Index | \- id (int): id of the index entry (auto)
\- source\_id (int): the id of a file in the Source table
\- target\_id: the id of the segment in docstore or vector store
\- relation\_type (str): if the link is "document" or "vector" | This is SQLAlchemy ORM class | | Vector store | self.\_VS | \- self.\_VS.add: add the list of embeddings to the vector store (optionally associate metadata and ids)
\- self.\_VS.delete: delete vector entries based on ids
\- self.\_VS.query: get embeddings based on embeddings. | kotaemon > storages > vectorstores > BaseVectorStore | | Doc store | self.\_DS | \- self.\_DS.add: add the segments to document stores
\- self.\_DS.get: get the segments based on id
\- self.\_DS.get\_all: get all segments
\- self.\_DS.delete: delete segments based on id | kotaemon > storages > docstores > base > BaseDocumentStore | Back to top --- # User management - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/pages/app/ext/user-management.md "Edit this page") User management =============== `ktem` provides user management as an extension. To enable user management, in your `flowsettings.py`, set the following variables: * `KH_FEATURE_USER_MANAGEMENT`: True to enable. * `KH_FEATURE_USER_MANAGEMENT_ADMIN`: the admin username. This user will be created when the app 1st start. * `KH_FEATURE_USER_MANAGEMENT_PASSWORD`: the admin password. This value accompanies the admin username. Once enabled, you have access to the following features: * User login/logout (located in Settings Tab) * User changing password (located in Settings Tab) * Create / List / Edit / Delete user (located in Resources > Users Tab) Back to top --- # Settings - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/pages/app/settings/overview.md "Edit this page") Overview[¶](#overview "Permanent link") ======================================== There are 3 kinds of settings in `ktem`, geared towards different stakeholders for different use cases: * Developer settings. These settings are meant for very basic app customization, such as database URL, cloud config, logging config, which features to enable... You will be interested in the developer settings if you deploy `ktem` to your customers, or if you build extension for `ktem` for developers. These settings are declared inside `flowsettings.py`. * Admin settings. These settings show up in the Admin page, and are meant to allow admin-level user to customize low level features, such as which credentials to connect to data sources, which keys to use for LLM... * [User settings](/pages/app/settings/user-settings/) . These settings are meant for run-time users to tweak ktem to their personal needs, such as which output languages the chatbot should generate, which reasoning type to use... Back to top --- # Contributing - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/development/contributing.md "Edit this page") Contributing[¶](#contributing "Permanent link") ================================================ Setting up[¶](#setting-up "Permanent link") -------------------------------------------- * Clone the repo | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2) | `git clone git@github.com:Cinnamon/kotaemon.git cd kotaemon` | * Install the environment * Create a conda environment (python >= 3.10 is recommended) | | | | --- | --- | | [1](#__codelineno-1-1)
[2](#__codelineno-1-2)
[3](#__codelineno-1-3)
[4](#__codelineno-1-4)
[5](#__codelineno-1-5)
[6](#__codelineno-1-6) | `conda create -n kotaemon python=3.10 conda activate kotaemon # install dependencies cd libs/kotaemon pip install -e ".[all]"` | * Or run the installer (one of the `scripts/run_*` scripts depends on your OS), then you will have all the dependencies installed as a conda environment at `install_dir/env`. | | | | --- | --- | | [1](#__codelineno-2-1) | `conda activate install_dir/env` | * Pre-commit | | | | --- | --- | | [1](#__codelineno-3-1) | `pre-commit install` | * Test | | | | --- | --- | | [1](#__codelineno-4-1) | `pytest tests` | Package overview[¶](#package-overview "Permanent link") -------------------------------------------------------- `kotaemon` library focuses on the AI building blocks to implement a RAG-based QA application. It consists of base interfaces, core components and a list of utilities: * Base interfaces: `kotaemon` defines the base interface of a component in a pipeline. A pipeline is also a component. By clearly define this interface, a pipeline of steps can be easily constructed and orchestrated. * Core components: `kotaemon` implements (or wraps 3rd-party libraries like Langchain, llama-index,... when possible) commonly used components in kotaemon use cases. Some of these components are: LLM, vector store, document store, retriever... For a detailed list and description of these components, please refer to the [API Reference](../../reference/Summary/) section. * List of utilities: `kotaemon` provides utilities and tools that are usually needed in client project. For example, it provides a prompt engineering UI for AI developers in a project to quickly create a prompt engineering tool for DMs and QALs. It also provides a command to quickly spin up a project code base. For a full list and description of these utilities, please refer to the [Utilities](../utilities/) section. mindmap root((kotaemon)) Base Interfaces Document LLMInterface RetrievedDocument BaseEmbeddings BaseChat BaseCompletion ... Core Components LLMs AzureOpenAI OpenAI Embeddings AzureOpenAI OpenAI HuggingFaceEmbedding VectorStore InMemoryVectorstore ChromaVectorstore Agent Tool DocumentStore ... Utilities Scaffold project PromptUI Documentation Support Common conventions[¶](#common-conventions "Permanent link") ------------------------------------------------------------ * PR title: One-line description (example: Feat: Declare BaseComponent and decide LLM call interface). * \[Encouraged\] Provide a quick description in the PR, so that: * Reviewers can quickly understand the direction of the PR. * It will be included in the commit message when the PR is merged. Environment caching on PR[¶](#environment-caching-on-pr "Permanent link") -------------------------------------------------------------------------- * To speed up CI, environments are cached based on the version specified in `__init__.py`. * Since dependencies versions in `setup.py` are not pinned, you need to pump the version in order to use a new environment. That environment will then be cached and used by your subsequence commits within the PR, until you pump the version again * The new environment created during your PR is cached and will be available to others once the PR is merged. * If you are experimenting with new dependencies and want a fresh environment every time, add `[ignore cache]` in your commit message. The CI will create a fresh environment to run your commit and then discard it. * If your PR include updated dependencies, the recommended workflow would be: * Doing development as usual. * When you want to run the CI, push a commit with the message containing `[ignore cache]`. * Once the PR is final, pump the version in `__init__.py` and push a final commit not containing `[ignore cache]`. Merge PR guideline[¶](#merge-pr-guideline "Permanent link") ------------------------------------------------------------ * Use squash and merge option * 1st line message is the PR title. * The text area is the PR description. Back to top --- # User settings - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/docs/pages/app/settings/user-settings.md "Edit this page") User settings[¶](#user-settings "Permanent link") ================================================== `ktem` allows developers to extend the index and the reasoning pipeline. In many cases, these components can have settings that should be modified by users at run-time, (e.g. `topk`, `chunksize`...). These are the user settings. `ktem` allows developers to declare such user settings in their code. Once declared, `ktem` will render them in a Settings page. There are 2 places that `ktem` looks for declared user settings. You can refer to the respective pages. * In the index. * In the reasoning pipeline. Syntax of a settings[¶](#syntax-of-a-settings "Permanent link") ---------------------------------------------------------------- A collection of settings is a dictionary of type `dict[str, dict]`, where the key is a setting id, and the value is the description of the setting. | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3)
[4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13) | `settings = { "topk": { "name": "Top-k chunks", "value": 10, "component": "number", }, "lang": { "name": "Languages", "value": "en", "component": "dropdown", "choices": [("en", "English"), ("cn", "Chinese")], } }` | Each setting description must have: * name: the human-understandable name of the settings. * value: the default value of the settings. * component: the UI component to render such setting on the UI. Available: * "text": single-value * "number": single-value * "checkbox": single-value * "dropdown": choices * "radio": choices * "checkboxgroup": choices * choices: the list of choices, if the component type allows. Settings page structure[¶](#settings-page-structure "Permanent link") ---------------------------------------------------------------------- Back to top --- # Component - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/base/component.py "Edit this page") Component ========= BaseComponent [¶](#base.component.BaseComponent "Permanent link") ------------------------------------------------------------------ Bases: `Function` A component is a class that can be used to compose a pipeline. Benefits of component * Auto caching, logging * Allow deployment For each component, the spirit is * Tolerate multiple input types, e.g. str, Document, List\[str\], List\[Document\] * Enforce single output type. Hence, the output type of a component should be as generic as possible. Source code in `libs/kotaemon/kotaemon/base/component.py` | | | | --- | --- | | [9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63) | `class BaseComponent(Function): """A component is a class that can be used to compose a pipeline. !!! tip "Benefits of component" - Auto caching, logging - Allow deployment !!! tip "For each component, the spirit is" - Tolerate multiple input types, e.g. str, Document, List[str], List[Document] - Enforce single output type. Hence, the output type of a component should be as generic as possible. """ inflow = None def flow(self): if self.inflow is None: raise ValueError("No inflow provided.") if not isinstance(self.inflow, BaseComponent): raise ValueError( f"inflow must be a BaseComponent, found {type(self.inflow)}" ) return self.__call__(self.inflow.flow()) def set_output_queue(self, queue): self._queue = queue for name in self._ff_nodes: node = getattr(self, name) if isinstance(node, BaseComponent): node.set_output_queue(queue) def report_output(self, output: Optional[Document]): if self._queue is not None: self._queue.put_nowait(output) def invoke(self, *args, **kwargs) -> Document \| list[Document] \| None: ... async def ainvoke(self, *args, **kwargs) -> Document \| list[Document] \| None: ... def stream(self, *args, **kwargs) -> Iterator[Document] \| None: ... def astream(self, *args, **kwargs) -> AsyncGenerator[Document, None] \| None: ... @abstractmethod def run( self, *args, **kwargs ) -> Document \| list[Document] \| Iterator[Document] \| None \| Any: """Run the component.""" ...` | ### run `abstractmethod` [¶](#base.component.BaseComponent.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(*args, **kwargs)` | Run the component. Source code in `libs/kotaemon/kotaemon/base/component.py` | | | | --- | --- | | [58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63) | `@abstractmethod def run( self, *args, **kwargs ) -> Document \| list[Document] \| Iterator[Document] \| None \| Any: """Run the component.""" ...` | Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/base/__init__.py "Edit this page") Base ==== BaseComponent [¶](#base.BaseComponent "Permanent link") -------------------------------------------------------- Bases: `Function` A component is a class that can be used to compose a pipeline. Benefits of component * Auto caching, logging * Allow deployment For each component, the spirit is * Tolerate multiple input types, e.g. str, Document, List\[str\], List\[Document\] * Enforce single output type. Hence, the output type of a component should be as generic as possible. Source code in `libs/kotaemon/kotaemon/base/component.py` | | | | --- | --- | | [9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63) | `class BaseComponent(Function): """A component is a class that can be used to compose a pipeline. !!! tip "Benefits of component" - Auto caching, logging - Allow deployment !!! tip "For each component, the spirit is" - Tolerate multiple input types, e.g. str, Document, List[str], List[Document] - Enforce single output type. Hence, the output type of a component should be as generic as possible. """ inflow = None def flow(self): if self.inflow is None: raise ValueError("No inflow provided.") if not isinstance(self.inflow, BaseComponent): raise ValueError( f"inflow must be a BaseComponent, found {type(self.inflow)}" ) return self.__call__(self.inflow.flow()) def set_output_queue(self, queue): self._queue = queue for name in self._ff_nodes: node = getattr(self, name) if isinstance(node, BaseComponent): node.set_output_queue(queue) def report_output(self, output: Optional[Document]): if self._queue is not None: self._queue.put_nowait(output) def invoke(self, *args, **kwargs) -> Document \| list[Document] \| None: ... async def ainvoke(self, *args, **kwargs) -> Document \| list[Document] \| None: ... def stream(self, *args, **kwargs) -> Iterator[Document] \| None: ... def astream(self, *args, **kwargs) -> AsyncGenerator[Document, None] \| None: ... @abstractmethod def run( self, *args, **kwargs ) -> Document \| list[Document] \| Iterator[Document] \| None \| Any: """Run the component.""" ...` | ### run `abstractmethod` [¶](#base.BaseComponent.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(*args, **kwargs)` | Run the component. Source code in `libs/kotaemon/kotaemon/base/component.py` | | | | --- | --- | | [58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63) | `@abstractmethod def run( self, *args, **kwargs ) -> Document \| list[Document] \| Iterator[Document] \| None \| Any: """Run the component.""" ...` | Document [¶](#base.Document "Permanent link") ---------------------------------------------- Bases: `Document` Base document class, mostly inherited from Document class from llama-index. This class accept one positional argument `content` of an arbitrary type, which will store the raw content of the document. If specified, the class will use `content` to initialize the base llama\_index class. Attributes: | Name | Type | Description | | --- | --- | --- | | `content` | `Any` | raw content of the document, can be anything | | `source` | `Optional[str]` | id of the source of the Document. Optional. | | `channel` | `Optional[Literal['chat', 'info', 'index', 'debug', 'plot']]` | the channel to show the document. Optional.: - chat: show in chat message - info: show in information panel - index: show in index panel - debug: show in debug panel | Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84) | ``class Document(BaseDocument): """ Base document class, mostly inherited from Document class from llama-index. This class accept one positional argument `content` of an arbitrary type, which will store the raw content of the document. If specified, the class will use `content` to initialize the base llama_index class. Attributes: content: raw content of the document, can be anything source: id of the source of the Document. Optional. channel: the channel to show the document. Optional.: - chat: show in chat message - info: show in information panel - index: show in index panel - debug: show in debug panel """ content: Any = None source: Optional[str] = None channel: Optional[Literal["chat", "info", "index", "debug", "plot"]] = None def __init__(self, content: Optional[Any] = None, *args, **kwargs): if content is None: if kwargs.get("text", None) is not None: kwargs["content"] = kwargs["text"] elif kwargs.get("embedding", None) is not None: kwargs["content"] = kwargs["embedding"] # default text indicating this document only contains embedding kwargs["text"] = "
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81) | `def to_haystack_format(self) -> "HaystackDocument": """Convert struct to Haystack document format.""" from haystack.schema import Document as HaystackDocument metadata = self.metadata or {} text = self.text return HaystackDocument(content=text, meta=metadata)` | DocumentWithEmbedding [¶](#base.DocumentWithEmbedding "Permanent link") ------------------------------------------------------------------------ Bases: `Document` Subclass of Document which must contains embedding Use this if you want to enforce component's IOs to must contain embedding. Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95) | `class DocumentWithEmbedding(Document): """Subclass of Document which must contains embedding Use this if you want to enforce component's IOs to must contain embedding. """ def __init__(self, embedding: list[float], *args, **kwargs): kwargs["embedding"] = embedding super().__init__(*args, **kwargs)` | ExtractorOutput [¶](#base.ExtractorOutput "Permanent link") ------------------------------------------------------------ Bases: `Document` Represents the output of an extractor. Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151) | `class ExtractorOutput(Document): """ Represents the output of an extractor. """ matches: list[str]` | RetrievedDocument [¶](#base.RetrievedDocument "Permanent link") ---------------------------------------------------------------- Bases: `Document` Subclass of Document with retrieval-related information Attributes: | Name | Type | Description | | --- | --- | --- | | `score` | `float` | score of the document (from 0.0 to 1.0) | | `retrieval_metadata` | `dict` | metadata from the retrieval process, can be used by different components in a retrieved pipeline to communicate with each other | Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132) | `class RetrievedDocument(Document): """Subclass of Document with retrieval-related information Attributes: score (float): score of the document (from 0.0 to 1.0) retrieval_metadata (dict): metadata from the retrieval process, can be used by different components in a retrieved pipeline to communicate with each other """ score: float = Field(default=0.0) retrieval_metadata: dict = Field(default={})` | Back to top --- # Schema - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/base/schema.py "Edit this page") Schema ====== Document [¶](#base.schema.Document "Permanent link") ----------------------------------------------------- Bases: `Document` Base document class, mostly inherited from Document class from llama-index. This class accept one positional argument `content` of an arbitrary type, which will store the raw content of the document. If specified, the class will use `content` to initialize the base llama\_index class. Attributes: | Name | Type | Description | | --- | --- | --- | | `content` | `Any` | raw content of the document, can be anything | | `source` | `Optional[str]` | id of the source of the Document. Optional. | | `channel` | `Optional[Literal['chat', 'info', 'index', 'debug', 'plot']]` | the channel to show the document. Optional.: - chat: show in chat message - info: show in information panel - index: show in index panel - debug: show in debug panel | Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84) | ``class Document(BaseDocument): """ Base document class, mostly inherited from Document class from llama-index. This class accept one positional argument `content` of an arbitrary type, which will store the raw content of the document. If specified, the class will use `content` to initialize the base llama_index class. Attributes: content: raw content of the document, can be anything source: id of the source of the Document. Optional. channel: the channel to show the document. Optional.: - chat: show in chat message - info: show in information panel - index: show in index panel - debug: show in debug panel """ content: Any = None source: Optional[str] = None channel: Optional[Literal["chat", "info", "index", "debug", "plot"]] = None def __init__(self, content: Optional[Any] = None, *args, **kwargs): if content is None: if kwargs.get("text", None) is not None: kwargs["content"] = kwargs["text"] elif kwargs.get("embedding", None) is not None: kwargs["content"] = kwargs["embedding"] # default text indicating this document only contains embedding kwargs["text"] = "
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81) | `def to_haystack_format(self) -> "HaystackDocument": """Convert struct to Haystack document format.""" from haystack.schema import Document as HaystackDocument metadata = self.metadata or {} text = self.text return HaystackDocument(content=text, meta=metadata)` | DocumentWithEmbedding [¶](#base.schema.DocumentWithEmbedding "Permanent link") ------------------------------------------------------------------------------- Bases: `Document` Subclass of Document which must contains embedding Use this if you want to enforce component's IOs to must contain embedding. Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95) | `class DocumentWithEmbedding(Document): """Subclass of Document which must contains embedding Use this if you want to enforce component's IOs to must contain embedding. """ def __init__(self, embedding: list[float], *args, **kwargs): kwargs["embedding"] = embedding super().__init__(*args, **kwargs)` | RetrievedDocument [¶](#base.schema.RetrievedDocument "Permanent link") ----------------------------------------------------------------------- Bases: `Document` Subclass of Document with retrieval-related information Attributes: | Name | Type | Description | | --- | --- | --- | | `score` | `float` | score of the document (from 0.0 to 1.0) | | `retrieval_metadata` | `dict` | metadata from the retrieval process, can be used by different components in a retrieved pipeline to communicate with each other | Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132) | `class RetrievedDocument(Document): """Subclass of Document with retrieval-related information Attributes: score (float): score of the document (from 0.0 to 1.0) retrieval_metadata (dict): metadata from the retrieval process, can be used by different components in a retrieved pipeline to communicate with each other """ score: float = Field(default=0.0) retrieval_metadata: dict = Field(default={})` | ExtractorOutput [¶](#base.schema.ExtractorOutput "Permanent link") ------------------------------------------------------------------- Bases: `Document` Represents the output of an extractor. Source code in `libs/kotaemon/kotaemon/base/schema.py` | | | | --- | --- | | [146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151) | `class ExtractorOutput(Document): """ Represents the output of an extractor. """ matches: list[str]` | Back to top --- # Chatbot - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/chatbot/__init__.py "Edit this page") Chatbot ======= ChatConversation [¶](#chatbot.ChatConversation "Permanent link") ----------------------------------------------------------------- Bases: `SessionFunction` Base implementation of a chat bot component A chatbot component should * handle internal state, including history messages * return output for a given input Source code in `libs/kotaemon/kotaemon/chatbot/base.py` | | | | --- | --- | | [21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115) | `class ChatConversation(SessionFunction): """Base implementation of a chat bot component A chatbot component should: - handle internal state, including history messages - return output for a given input """ class Config: store_result = session_chat_storage system_message: str = "" bot: BaseChatBot def __init__(self, *args, **kwargs): self._history: List[BaseMessage] = [] self._store_result = ( f"{self.__module__}.{self.__class__.__name__},uninitiated_bot" ) super().__init__(*args, **kwargs) def run(self, message: HumanMessage) -> Optional[BaseMessage]: """Chat, given a message, return a response Args: message: The message to respond to Returns: The response to the message. If None, no response is sent. """ user_message = ( HumanMessage(content=message) if isinstance(message, str) else message ) self.history.append(user_message) output = self.bot(self.history).text output_message = None if output is not None: output_message = AIMessage(content=output) self.history.append(output_message) return output_message def start_session(self): self._store_result = self.bot.config.store_result super().start_session() if not self.history and self.system_message: system_message = SystemMessage(content=self.system_message) self.history.append(system_message) def end_session(self): super().end_session() self._history = [] def check_end( self, history: Optional[List[BaseMessage]] = None, user_message: Optional[HumanMessage] = None, bot_message: Optional[AIMessage] = None, ) -> bool: """Check if a conversation should end""" if user_message is not None and user_message.content == "": return True return False def terminal_session(self): """Create a terminal session""" self.start_session() print(">> Start chat:") while True: human = HumanMessage(content=input("Human: ")) if self.check_end(history=self.history, user_message=human): break output = self(human) if output is None: print("AI:
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62) | `def run(self, message: HumanMessage) -> Optional[BaseMessage]: """Chat, given a message, return a response Args: message: The message to respond to Returns: The response to the message. If None, no response is sent. """ user_message = ( HumanMessage(content=message) if isinstance(message, str) else message ) self.history.append(user_message) output = self.bot(self.history).text output_message = None if output is not None: output_message = AIMessage(content=output) self.history.append(output_message) return output_message` | ### check\_end [¶](#chatbot.ChatConversation.check_end "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3) | `check_end( history=None, user_message=None, bot_message=None )` | Check if a conversation should end Source code in `libs/kotaemon/kotaemon/chatbot/base.py` | | | | --- | --- | | [75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85) | `def check_end( self, history: Optional[List[BaseMessage]] = None, user_message: Optional[HumanMessage] = None, bot_message: Optional[AIMessage] = None, ) -> bool: """Check if a conversation should end""" if user_message is not None and user_message.content == "": return True return False` | ### terminal\_session [¶](#chatbot.ChatConversation.terminal_session "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `terminal_session()` | Create a terminal session Source code in `libs/kotaemon/kotaemon/chatbot/base.py` | | | | --- | --- | | [87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106) | `def terminal_session(self): """Create a terminal session""" self.start_session() print(">> Start chat:") while True: human = HumanMessage(content=input("Human: ")) if self.check_end(history=self.history, user_message=human): break output = self(human) if output is None: print("AI:
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11) | `class SimpleRespondentChatbot(BaseChatBot): """Simple text respondent chatbot that essentially wraps around a chat LLM""" llm: ChatLLM def _get_message(self) -> str: return self.llm(self.history).text` | Back to top --- # Simple Respondent - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/chatbot/simple_respondent.py "Edit this page") Simple Respondent ================= SimpleRespondentChatbot [¶](#chatbot.simple_respondent.SimpleRespondentChatbot "Permanent link") ------------------------------------------------------------------------------------------------- Bases: `BaseChatBot` Simple text respondent chatbot that essentially wraps around a chat LLM Source code in `libs/kotaemon/kotaemon/chatbot/simple_respondent.py` | | | | --- | --- | | [5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11) | `class SimpleRespondentChatbot(BaseChatBot): """Simple text respondent chatbot that essentially wraps around a chat LLM""" llm: ChatLLM def _get_message(self) -> str: return self.llm(self.history).text` | Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/base.py "Edit this page") Base ==== DocTransformer [¶](#indices.base.DocTransformer "Permanent link") ------------------------------------------------------------------ Bases: `BaseComponent` This is a base class for document transformers A document transformer transforms a list of documents into another list of documents. Transforming can mean splitting a document into multiple documents, reducing a large list of documents into a smaller list of documents, or adding metadata to each document in a list of documents, etc. Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26) | `class DocTransformer(BaseComponent): """This is a base class for document transformers A document transformer transforms a list of documents into another list of documents. Transforming can mean splitting a document into multiple documents, reducing a large list of documents into a smaller list of documents, or adding metadata to each document in a list of documents, etc. """ @abstractmethod def run( self, documents: list[Document], **kwargs, ) -> list[Document]: ...` | LlamaIndexDocTransformerMixin [¶](#indices.base.LlamaIndexDocTransformerMixin "Permanent link") ------------------------------------------------------------------------------------------------ Allow automatically wrapping a Llama-index component into kotaemon component Example class TokenSplitter(LlamaIndexMixin, BaseSplitter): def \_get\_li\_class(self): from llama\_index.core.text\_splitter import TokenTextSplitter return TokenTextSplitter To use this mixin, please: 1. Use this class as the 1st parent class, so that Python will prefer to use the attributes and methods of this class whenever possible. 2. Overwrite `_get_li_class` to return the relevant LlamaIndex component. Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103) | ``class LlamaIndexDocTransformerMixin: """Allow automatically wrapping a Llama-index component into kotaemon component Example: class TokenSplitter(LlamaIndexMixin, BaseSplitter): def _get_li_class(self): from llama_index.core.text_splitter import TokenTextSplitter return TokenTextSplitter To use this mixin, please: 1. Use this class as the 1st parent class, so that Python will prefer to use the attributes and methods of this class whenever possible. 2. Overwrite `_get_li_class` to return the relevant LlamaIndex component. """ def _get_li_class(self) -> Type[NodeParser]: raise NotImplementedError( "Please return the relevant LlamaIndex class in _get_li_class" ) def __init__(self, **params): self._li_cls = self._get_li_class() self._obj = self._li_cls(**params) self._kwargs = params super().__init__() def __repr__(self): kwargs = [] for key, value_obj in self._kwargs.items(): value = repr(value_obj) kwargs.append(f"{key}={value}") kwargs_repr = ", ".join(kwargs) return f"{self.__class__.__name__}({kwargs_repr})" def __str__(self): kwargs = [] for key, value_obj in self._kwargs.items(): value = str(value_obj) if len(value) > 20: value = f"{value[:15]}..." kwargs.append(f"{key}={value}") kwargs_repr = ", ".join(kwargs) return f"{self.__class__.__name__}({kwargs_repr})" def __setattr__(self, name: str, value: Any) -> None: if name.startswith("_") or name in self._protected_keywords(): return super().__setattr__(name, value) self._kwargs[name] = value return setattr(self._obj, name, value) def __getattr__(self, name: str) -> Any: if name in self._kwargs: return self._kwargs[name] return getattr(self._obj, name) def dump(self, *args, **kwargs): from theflow.utils.modules import serialize params = {key: serialize(value) for key, value in self._kwargs.items()} return { "__type__": f"{self.__module__}.{self.__class__.__qualname__}", **params, } def run( self, documents: list[Document], **kwargs, ) -> list[Document]: """Run Llama-index node parser and convert the output to Document from kotaemon """ docs = self._obj(documents, **kwargs) # type: ignore return [Document.from_dict(doc.to_dict()) for doc in docs]`` | ### run [¶](#indices.base.LlamaIndexDocTransformerMixin.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, **kwargs)` | Run Llama-index node parser and convert the output to Document from kotaemon Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103) | `def run( self, documents: list[Document], **kwargs, ) -> list[Document]: """Run Llama-index node parser and convert the output to Document from kotaemon """ docs = self._obj(documents, **kwargs) # type: ignore return [Document.from_dict(doc.to_dict()) for doc in docs]` | BaseIndexing [¶](#indices.base.BaseIndexing "Permanent link") -------------------------------------------------------------- Bases: `BaseComponent` Define the base interface for indexing pipeline Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115) | `class BaseIndexing(BaseComponent): """Define the base interface for indexing pipeline""" def to_retrieval_pipeline(self, **kwargs): """Convert the indexing pipeline to a retrieval pipeline""" raise NotImplementedError def to_qa_pipeline(self, **kwargs): """Convert the indexing pipeline to a QA pipeline""" raise NotImplementedError` | ### to\_retrieval\_pipeline [¶](#indices.base.BaseIndexing.to_retrieval_pipeline "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `to_retrieval_pipeline(**kwargs)` | Convert the indexing pipeline to a retrieval pipeline Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111) | `def to_retrieval_pipeline(self, **kwargs): """Convert the indexing pipeline to a retrieval pipeline""" raise NotImplementedError` | ### to\_qa\_pipeline [¶](#indices.base.BaseIndexing.to_qa_pipeline "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `to_qa_pipeline(**kwargs)` | Convert the indexing pipeline to a QA pipeline Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115) | `def to_qa_pipeline(self, **kwargs): """Convert the indexing pipeline to a QA pipeline""" raise NotImplementedError` | BaseRetrieval [¶](#indices.base.BaseRetrieval "Permanent link") ---------------------------------------------------------------- Bases: `BaseComponent` Define the base interface for retrieval pipeline Source code in `libs/kotaemon/kotaemon/indices/base.py` | | | | --- | --- | | [118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123) | `class BaseRetrieval(BaseComponent): """Define the base interface for retrieval pipeline""" @abstractmethod def run(self, *args, **kwargs) -> list[RetrievedDocument]: ...` | Back to top --- # Extractors - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/extractors/__init__.py "Edit this page") Extractors ========== Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/chatbot/base.py "Edit this page") Base ==== ChatConversation [¶](#chatbot.base.ChatConversation "Permanent link") ---------------------------------------------------------------------- Bases: `SessionFunction` Base implementation of a chat bot component A chatbot component should * handle internal state, including history messages * return output for a given input Source code in `libs/kotaemon/kotaemon/chatbot/base.py` | | | | --- | --- | | [21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115) | `class ChatConversation(SessionFunction): """Base implementation of a chat bot component A chatbot component should: - handle internal state, including history messages - return output for a given input """ class Config: store_result = session_chat_storage system_message: str = "" bot: BaseChatBot def __init__(self, *args, **kwargs): self._history: List[BaseMessage] = [] self._store_result = ( f"{self.__module__}.{self.__class__.__name__},uninitiated_bot" ) super().__init__(*args, **kwargs) def run(self, message: HumanMessage) -> Optional[BaseMessage]: """Chat, given a message, return a response Args: message: The message to respond to Returns: The response to the message. If None, no response is sent. """ user_message = ( HumanMessage(content=message) if isinstance(message, str) else message ) self.history.append(user_message) output = self.bot(self.history).text output_message = None if output is not None: output_message = AIMessage(content=output) self.history.append(output_message) return output_message def start_session(self): self._store_result = self.bot.config.store_result super().start_session() if not self.history and self.system_message: system_message = SystemMessage(content=self.system_message) self.history.append(system_message) def end_session(self): super().end_session() self._history = [] def check_end( self, history: Optional[List[BaseMessage]] = None, user_message: Optional[HumanMessage] = None, bot_message: Optional[AIMessage] = None, ) -> bool: """Check if a conversation should end""" if user_message is not None and user_message.content == "": return True return False def terminal_session(self): """Create a terminal session""" self.start_session() print(">> Start chat:") while True: human = HumanMessage(content=input("Human: ")) if self.check_end(history=self.history, user_message=human): break output = self(human) if output is None: print("AI:
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62) | `def run(self, message: HumanMessage) -> Optional[BaseMessage]: """Chat, given a message, return a response Args: message: The message to respond to Returns: The response to the message. If None, no response is sent. """ user_message = ( HumanMessage(content=message) if isinstance(message, str) else message ) self.history.append(user_message) output = self.bot(self.history).text output_message = None if output is not None: output_message = AIMessage(content=output) self.history.append(output_message) return output_message` | ### check\_end [¶](#chatbot.base.ChatConversation.check_end "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3) | `check_end( history=None, user_message=None, bot_message=None )` | Check if a conversation should end Source code in `libs/kotaemon/kotaemon/chatbot/base.py` | | | | --- | --- | | [75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85) | `def check_end( self, history: Optional[List[BaseMessage]] = None, user_message: Optional[HumanMessage] = None, bot_message: Optional[AIMessage] = None, ) -> bool: """Check if a conversation should end""" if user_message is not None and user_message.content == "": return True return False` | ### terminal\_session [¶](#chatbot.base.ChatConversation.terminal_session "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `terminal_session()` | Create a terminal session Source code in `libs/kotaemon/kotaemon/chatbot/base.py` | | | | --- | --- | | [87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106) | `def terminal_session(self): """Create a terminal session""" self.start_session() print(">> Start chat:") while True: human = HumanMessage(content=input("Human: ")) if self.check_end(history=self.history, user_message=human): break output = self(human) if output is None: print("AI:
[17](#__codelineno-0-17)
[18](#__codelineno-0-18) | `def session_chat_storage(obj): """Store using the bot location rather than the session location""" return obj._store_result` | Back to top --- # Doc Parsers - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/extractors/doc_parsers.py "Edit this page") Doc Parsers =========== Back to top --- # Ingests - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/ingests/__init__.py "Edit this page") Ingests ======= DocumentIngestor [¶](#indices.ingests.DocumentIngestor "Permanent link") ------------------------------------------------------------------------- Bases: `BaseComponent` Ingest common office document types into Document for indexing Document types * pdf * xlsx, xls * docx, doc Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `pdf_mode` | | mode for pdf extraction, one of "normal", "mathpix", "ocr" - normal: parse pdf text - mathpix: parse pdf text using mathpix - ocr: parse pdf image using flax | _required_ | | `doc_parsers` | | list of document parsers to parse the document | _required_ | | `text_splitter` | | splitter to split the document into text nodes | _required_ | | `override_file_extractors` | | override file extractors for specific file extensions The default file extractors are stored in `KH_DEFAULT_FILE_EXTRACTORS` | _required_ | Source code in `libs/kotaemon/kotaemon/indices/ingests/files.py` | | | | --- | --- | | [61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137) | ``class DocumentIngestor(BaseComponent): """Ingest common office document types into Document for indexing Document types: - pdf - xlsx, xls - docx, doc Args: pdf_mode: mode for pdf extraction, one of "normal", "mathpix", "ocr" - normal: parse pdf text - mathpix: parse pdf text using mathpix - ocr: parse pdf image using flax doc_parsers: list of document parsers to parse the document text_splitter: splitter to split the document into text nodes override_file_extractors: override file extractors for specific file extensions The default file extractors are stored in `KH_DEFAULT_FILE_EXTRACTORS` """ pdf_mode: str = "normal" # "normal", "mathpix", "ocr", "multimodal" doc_parsers: list[BaseDocParser] = Param(default_callback=lambda _: []) text_splitter: BaseSplitter = TokenSplitter.withx( chunk_size=1024, chunk_overlap=256, separator="\n\n", backup_separators=["\n", ".", " ", "\u200B"], ) override_file_extractors: dict[str, Type[BaseReader]] = {} def _get_reader(self, input_files: list[str \| Path]): """Get appropriate readers for the input files based on file extension""" file_extractors: dict[str, BaseReader] = { ext: reader for ext, reader in KH_DEFAULT_FILE_EXTRACTORS.items() } for ext, cls in self.override_file_extractors.items(): file_extractors[ext] = cls() if self.pdf_mode == "normal": file_extractors[".pdf"] = PDFReader() elif self.pdf_mode == "ocr": file_extractors[".pdf"] = OCRReader() elif self.pdf_mode == "multimodal": file_extractors[".pdf"] = AdobeReader() else: file_extractors[".pdf"] = MathpixPDFReader() main_reader = DirectoryReader( input_files=input_files, file_extractor=file_extractors, ) return main_reader def run(self, file_paths: list[str \| Path] \| str \| Path) -> list[Document]: """Ingest the file paths into Document Args: file_paths: list of file paths or a single file path Returns: list of parsed Documents """ if not isinstance(file_paths, list): file_paths = [file_paths] documents = self._get_reader(input_files=file_paths)() print(f"Read {len(file_paths)} files into {len(documents)} documents.") nodes = self.text_splitter(documents) print(f"Transform {len(documents)} documents into {len(nodes)} nodes.") self.log_progress(".num_docs", num_docs=len(nodes)) # document parsers call if self.doc_parsers: for parser in self.doc_parsers: nodes = parser(nodes) return nodes`` | ### run [¶](#indices.ingests.DocumentIngestor.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(file_paths)` | Ingest the file paths into Document Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `file_paths` | `list[str \| Path] \| str \| Path` | list of file paths or a single file path | _required_ | Returns: | Type | Description | | --- | --- | | `list[Document]` | list of parsed Documents | Source code in `libs/kotaemon/kotaemon/indices/ingests/files.py` | | | | --- | --- | | [114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137) | `def run(self, file_paths: list[str \| Path] \| str \| Path) -> list[Document]: """Ingest the file paths into Document Args: file_paths: list of file paths or a single file path Returns: list of parsed Documents """ if not isinstance(file_paths, list): file_paths = [file_paths] documents = self._get_reader(input_files=file_paths)() print(f"Read {len(file_paths)} files into {len(documents)} documents.") nodes = self.text_splitter(documents) print(f"Transform {len(documents)} documents into {len(nodes)} nodes.") self.log_progress(".num_docs", num_docs=len(nodes)) # document parsers call if self.doc_parsers: for parser in self.doc_parsers: nodes = parser(nodes) return nodes` | Back to top --- # Endpoint Based - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/endpoint_based.py "Edit this page") Endpoint Based ============== EndpointEmbeddings [¶](#embeddings.endpoint_based.EndpointEmbeddings "Permanent link") --------------------------------------------------------------------------------------- Bases: `BaseEmbeddings` An Embeddings component that uses an OpenAI API compatible endpoint. Attributes: | Name | Type | Description | | --- | --- | --- | | `endpoint_url` | `str` | The url of an OpenAI API compatible endpoint. | Source code in `libs/kotaemon/kotaemon/embeddings/endpoint_based.py` | | | | --- | --- | | [8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46) | `class EndpointEmbeddings(BaseEmbeddings): """ An Embeddings component that uses an OpenAI API compatible endpoint. Attributes: endpoint_url (str): The url of an OpenAI API compatible endpoint. """ endpoint_url: str def run( self, text: str \| list[str] \| Document \| list[Document] ) -> list[DocumentWithEmbedding]: """ Generate embeddings from text Args: text (str \| list[str] \| Document \| list[Document]): text to generate embeddings from Returns: list[DocumentWithEmbedding]: embeddings """ if not isinstance(text, list): text = [text] outputs = [] for item in text: response = requests.post( self.endpoint_url, json={"input": str(item)} ).json() outputs.append( DocumentWithEmbedding( text=str(item), embedding=response["data"][0]["embedding"], total_tokens=response["usage"]["total_tokens"], prompt_tokens=response["usage"]["prompt_tokens"], ) ) return outputs` | ### run [¶](#embeddings.endpoint_based.EndpointEmbeddings.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(text)` | Generate embeddings from text Args text (str | list\[str\] | Document | list\[Document\]): text to generate embeddings from Returns: list\[DocumentWithEmbedding\]: embeddings Source code in `libs/kotaemon/kotaemon/embeddings/endpoint_based.py` | | | | --- | --- | | [18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46) | `def run( self, text: str \| list[str] \| Document \| list[Document] ) -> list[DocumentWithEmbedding]: """ Generate embeddings from text Args: text (str \| list[str] \| Document \| list[Document]): text to generate embeddings from Returns: list[DocumentWithEmbedding]: embeddings """ if not isinstance(text, list): text = [text] outputs = [] for item in text: response = requests.post( self.endpoint_url, json={"input": str(item)} ).json() outputs.append( DocumentWithEmbedding( text=str(item), embedding=response["data"][0]["embedding"], total_tokens=response["usage"]["total_tokens"], prompt_tokens=response["usage"]["prompt_tokens"], ) ) return outputs` | Back to top --- # Fastembed - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/fastembed.py "Edit this page") Fastembed ========= FastEmbedEmbeddings [¶](#embeddings.fastembed.FastEmbedEmbeddings "Permanent link") ------------------------------------------------------------------------------------ Bases: `BaseEmbeddings` Utilize fastembed library for embeddings locally without GPU. Supported model: https://qdrant.github.io/fastembed/examples/Supported\_Models/ Code: https://github.com/qdrant/fastembed Source code in `libs/kotaemon/kotaemon/embeddings/fastembed.py` | | | | --- | --- | | [11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72) | ``class FastEmbedEmbeddings(BaseEmbeddings): """Utilize fastembed library for embeddings locally without GPU. Supported model: https://qdrant.github.io/fastembed/examples/Supported_Models/ Code: https://github.com/qdrant/fastembed """ model_name: str = Param( "BAAI/bge-small-en-v1.5", help=( "Model name for fastembed. Please refer " "[here](https://qdrant.github.io/fastembed/examples/Supported_Models/) " "for the list of supported models." ), required=True, ) batch_size: int = Param( 256, help="Batch size for embeddings. Higher values use more memory, but are faster", ) parallel: Optional[int] = Param( None, help=( "Number of threads to use for embeddings. " "If > 1, data-parallel encoding will be used. " "If 0, use all available CPUs. " "If None, use default onnxruntime threading. " "Defaults to None." ), ) @Param.auto() def client_(self) -> "TextEmbedding": try: from fastembed import TextEmbedding except ImportError: raise ImportError("Please install FastEmbed: `pip install fastembed`") return TextEmbedding(model_name=self.model_name) def invoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: input_ = self.prepare_input(text) embeddings = self.client_.embed( [_.content for _ in input_], batch_size=self.batch_size, parallel=self.parallel, ) return [ DocumentWithEmbedding( content=doc, embedding=list(embedding), ) for doc, embedding in zip(input_, embeddings) ] async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: """Fastembed does not support async API.""" return self.invoke(text, *args, **kwargs)`` | ### ainvoke `async` [¶](#embeddings.fastembed.FastEmbedEmbeddings.ainvoke "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `ainvoke(text, *args, **kwargs)` | Fastembed does not support async API. Source code in `libs/kotaemon/kotaemon/embeddings/fastembed.py` | | | | --- | --- | | [68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72) | `async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: """Fastembed does not support async API.""" return self.invoke(text, *args, **kwargs)` | Back to top --- # Langchain Based - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/langchain_based.py "Edit this page") Langchain Based =============== LCOpenAIEmbeddings [¶](#embeddings.langchain_based.LCOpenAIEmbeddings "Permanent link") ---------------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's OpenAI embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118) | `class LCOpenAIEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's OpenAI embedding, focusing on key parameters""" def __init__( self, model: str = "text-embedding-ada-002", openai_api_version: Optional[str] = None, openai_api_base: Optional[str] = None, openai_api_type: Optional[str] = None, openai_api_key: Optional[str] = None, request_timeout: Optional[float] = None, **params, ): super().__init__( model=model, openai_api_version=openai_api_version, openai_api_base=openai_api_base, openai_api_type=openai_api_type, openai_api_key=openai_api_key, request_timeout=request_timeout, **params, ) def _get_lc_class(self): try: from langchain_openai import OpenAIEmbeddings except ImportError: from langchain.embeddings import OpenAIEmbeddings return OpenAIEmbeddings` | LCAzureOpenAIEmbeddings [¶](#embeddings.langchain_based.LCAzureOpenAIEmbeddings "Permanent link") -------------------------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's AzureOpenAI embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148) | `class LCAzureOpenAIEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's AzureOpenAI embedding, focusing on key parameters""" def __init__( self, azure_endpoint: Optional[str] = None, deployment: Optional[str] = None, openai_api_key: Optional[str] = None, api_version: Optional[str] = None, request_timeout: Optional[float] = None, **params, ): super().__init__( azure_endpoint=azure_endpoint, deployment=deployment, api_version=api_version, openai_api_key=openai_api_key, request_timeout=request_timeout, **params, ) def _get_lc_class(self): try: from langchain_openai import AzureOpenAIEmbeddings except ImportError: from langchain.embeddings import AzureOpenAIEmbeddings return AzureOpenAIEmbeddings` | LCCohereEmbeddings [¶](#embeddings.langchain_based.LCCohereEmbeddings "Permanent link") ---------------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's Cohere embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190) | `class LCCohereEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's Cohere embedding, focusing on key parameters""" cohere_api_key: str = Param( help="API key (https://dashboard.cohere.com/api-keys)", default=None, required=True, ) model: str = Param( help="Model name to use (https://docs.cohere.com/docs/models)", default=None, required=True, ) user_agent: str = Param( help="User agent (leave default)", default="default", required=True ) def __init__( self, model: str = "embed-english-v2.0", cohere_api_key: Optional[str] = None, truncate: Optional[str] = None, request_timeout: Optional[float] = None, **params, ): super().__init__( model=model, cohere_api_key=cohere_api_key, truncate=truncate, request_timeout=request_timeout, **params, ) def _get_lc_class(self): try: from langchain_cohere import CohereEmbeddings except ImportError: from langchain.embeddings import CohereEmbeddings return CohereEmbeddings` | LCHuggingFaceEmbeddings [¶](#embeddings.langchain_based.LCHuggingFaceEmbeddings "Permanent link") -------------------------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's HuggingFace embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221) | `class LCHuggingFaceEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's HuggingFace embedding, focusing on key parameters""" model_name: str = Param( help=( "Model name to use (https://huggingface.co/models?" "pipeline_tag=sentence-similarity&sort=trending)" ), default=None, required=True, ) def __init__( self, model_name: str = "sentence-transformers/all-mpnet-base-v2", **params, ): super().__init__( model_name=model_name, **params, ) def _get_lc_class(self): try: from langchain_community.embeddings import HuggingFaceBgeEmbeddings except ImportError: from langchain.embeddings import HuggingFaceBgeEmbeddings return HuggingFaceBgeEmbeddings` | LCGoogleEmbeddings [¶](#embeddings.langchain_based.LCGoogleEmbeddings "Permanent link") ---------------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's Google GenAI embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256) | `class LCGoogleEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's Google GenAI embedding, focusing on key parameters""" google_api_key: str = Param( help="API key (https://aistudio.google.com/app/apikey)", default=None, required=True, ) model: str = Param( help="Model name to use (https://ai.google.dev/gemini-api/docs/models/gemini#text-embedding-and-embedding)", # noqa default="models/text-embedding-004", required=True, ) def __init__( self, model: str = "models/text-embedding-004", google_api_key: Optional[str] = None, **params, ): super().__init__( model=model, google_api_key=google_api_key, **params, ) def _get_lc_class(self): try: from langchain_google_genai import GoogleGenerativeAIEmbeddings except ImportError: raise ImportError("Please install langchain-google-genai") return GoogleGenerativeAIEmbeddings` | Back to top --- # CLI - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/cli.py "Edit this page") CLI === export [¶](#cli.export "Permanent link") ----------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `export(export_path, output)` | Export a pipeline to a config file Source code in `libs/kotaemon/kotaemon/cli.py` | | | | --- | --- | | [32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46) | `@promptui.command() @click.argument("export_path", nargs=1) @click.option("--output", default="promptui.yml", show_default=True, required=False) def export(export_path, output): """Export a pipeline to a config file""" import sys from theflow.utils.modules import import_dotted_string from kotaemon.contribs.promptui.config import export_pipeline_to_config sys.path.append(os.getcwd()) cls = import_dotted_string(export_path, safe=False) export_pipeline_to_config(cls, output) check_config_format(output)` | run [¶](#cli.run "Permanent link") ----------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `run(run_path, share, username, password, appname, port)` | Run the UI from a config file Examples: | | | | --- | --- | | 1
2
3
4
5
6
7
8
9
10
11
12
13
14 | `# Run with default config file $ kh promptui run # Run with username and password supplied $ kh promptui run --username admin --password password # Run with username and prompted password $ kh promptui run --username admin # Run and share to promptui # kh promptui run --username admin --password password --share --appname hey --port 7861` | Source code in `libs/kotaemon/kotaemon/cli.py` | | | | --- | --- | | [49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139) | `@promptui.command() @click.argument("run_path", required=False, default="promptui.yml") @click.option( "--share", is_flag=True, show_default=True, default=False, help="Share the app through Gradio. Requires --username to enable authentication.", ) @click.option( "--username", required=False, help=( "Username for the user. If not provided, the promptui will not have " "authentication." ), ) @click.option( "--password", required=False, help="Password for the user. If not provided, will be prompted.", ) @click.option( "--appname", required=False, help="The share app subdomain. Requires --share and --username", ) @click.option( "--port", required=False, help="Port to run the app. If not provided, will $GRADIO_SERVER_PORT (7860)", ) def run(run_path, share, username, password, appname, port): """Run the UI from a config file Examples: \b # Run with default config file $ kh promptui run \b # Run with username and password supplied $ kh promptui run --username admin --password password \b # Run with username and prompted password $ kh promptui run --username admin # Run and share to promptui # kh promptui run --username admin --password password --share --appname hey \ --port 7861 """ import sys from kotaemon.contribs.promptui.ui import build_from_dict sys.path.append(os.getcwd()) check_config_format(run_path) demo = build_from_dict(run_path) params: dict = {} if username is not None: if password is not None: auth = (username, password) else: auth = (username, click.prompt("Password", hide_input=True)) params["auth"] = auth port = int(port) if port else int(os.getenv("GRADIO_SERVER_PORT", "7860")) params["server_port"] = port if share: if username is None: raise ValueError( "Username must be provided to enable authentication for sharing" ) if appname: from kotaemon.contribs.promptui.tunnel import Tunnel tunnel = Tunnel( appname=str(appname), username=str(username), local_port=port ) url = tunnel.run() print(f"App is shared at {url}") else: params["share"] = True print("App is shared at Gradio") demo.launch(**params)` | makedoc [¶](#cli.makedoc "Permanent link") ------------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `makedoc(module, output, separation_level)` | Make documentation for module `module` Example: | | | | --- | --- | | 1
2
3 | `# Make component documentation for kotaemon library $ kh makedoc kotaemon` | Source code in `libs/kotaemon/kotaemon/cli.py` | | | | --- | --- | | [142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162) | ``@main.command() @click.argument("module", required=True) @click.option( "--output", default="docs.md", required=False, help="The output markdown file" ) @click.option( "--separation-level", required=False, default=1, help="Organize markdown layout" ) def makedoc(module, output, separation_level): """Make documentation for module `module` Example: \b # Make component documentation for kotaemon library $ kh makedoc kotaemon """ from kotaemon.contribs.docs import make_doc make_doc(module, output, separation_level) print(f"Documentation exported to {output}")`` | start\_project [¶](#cli.start_project "Permanent link") -------------------------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `start_project(template)` | Start a project from a template. Important: the value for --template corresponds to the name of the template folder, which is located at https://github.com/Cinnamon/kotaemon/tree/main/templates The default value is "project-default", which should work when you are starting a client project. Source code in `libs/kotaemon/kotaemon/cli.py` | | | | --- | --- | | [165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186) | `@main.command() @click.option( "--template", default="project-default", required=False, help="Template name", show_default=True, ) def start_project(template): """Start a project from a template. Important: the value for --template corresponds to the name of the template folder, which is located at https://github.com/Cinnamon/kotaemon/tree/main/templates The default value is "project-default", which should work when you are starting a client project. """ print("Retrieving template...") os.system( "cookiecutter git@github.com:Cinnamon/kotaemon.git " f"--directory='templates/{template}'" )` | Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/base.py "Edit this page") Base ==== Back to top --- # Files - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/ingests/files.py "Edit this page") Files ===== DocumentIngestor [¶](#indices.ingests.files.DocumentIngestor "Permanent link") ------------------------------------------------------------------------------- Bases: `BaseComponent` Ingest common office document types into Document for indexing Document types * pdf * xlsx, xls * docx, doc Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `pdf_mode` | | mode for pdf extraction, one of "normal", "mathpix", "ocr" - normal: parse pdf text - mathpix: parse pdf text using mathpix - ocr: parse pdf image using flax | _required_ | | `doc_parsers` | | list of document parsers to parse the document | _required_ | | `text_splitter` | | splitter to split the document into text nodes | _required_ | | `override_file_extractors` | | override file extractors for specific file extensions The default file extractors are stored in `KH_DEFAULT_FILE_EXTRACTORS` | _required_ | Source code in `libs/kotaemon/kotaemon/indices/ingests/files.py` | | | | --- | --- | | [61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137) | ``class DocumentIngestor(BaseComponent): """Ingest common office document types into Document for indexing Document types: - pdf - xlsx, xls - docx, doc Args: pdf_mode: mode for pdf extraction, one of "normal", "mathpix", "ocr" - normal: parse pdf text - mathpix: parse pdf text using mathpix - ocr: parse pdf image using flax doc_parsers: list of document parsers to parse the document text_splitter: splitter to split the document into text nodes override_file_extractors: override file extractors for specific file extensions The default file extractors are stored in `KH_DEFAULT_FILE_EXTRACTORS` """ pdf_mode: str = "normal" # "normal", "mathpix", "ocr", "multimodal" doc_parsers: list[BaseDocParser] = Param(default_callback=lambda _: []) text_splitter: BaseSplitter = TokenSplitter.withx( chunk_size=1024, chunk_overlap=256, separator="\n\n", backup_separators=["\n", ".", " ", "\u200B"], ) override_file_extractors: dict[str, Type[BaseReader]] = {} def _get_reader(self, input_files: list[str \| Path]): """Get appropriate readers for the input files based on file extension""" file_extractors: dict[str, BaseReader] = { ext: reader for ext, reader in KH_DEFAULT_FILE_EXTRACTORS.items() } for ext, cls in self.override_file_extractors.items(): file_extractors[ext] = cls() if self.pdf_mode == "normal": file_extractors[".pdf"] = PDFReader() elif self.pdf_mode == "ocr": file_extractors[".pdf"] = OCRReader() elif self.pdf_mode == "multimodal": file_extractors[".pdf"] = AdobeReader() else: file_extractors[".pdf"] = MathpixPDFReader() main_reader = DirectoryReader( input_files=input_files, file_extractor=file_extractors, ) return main_reader def run(self, file_paths: list[str \| Path] \| str \| Path) -> list[Document]: """Ingest the file paths into Document Args: file_paths: list of file paths or a single file path Returns: list of parsed Documents """ if not isinstance(file_paths, list): file_paths = [file_paths] documents = self._get_reader(input_files=file_paths)() print(f"Read {len(file_paths)} files into {len(documents)} documents.") nodes = self.text_splitter(documents) print(f"Transform {len(documents)} documents into {len(nodes)} nodes.") self.log_progress(".num_docs", num_docs=len(nodes)) # document parsers call if self.doc_parsers: for parser in self.doc_parsers: nodes = parser(nodes) return nodes`` | ### run [¶](#indices.ingests.files.DocumentIngestor.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(file_paths)` | Ingest the file paths into Document Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `file_paths` | `list[str \| Path] \| str \| Path` | list of file paths or a single file path | _required_ | Returns: | Type | Description | | --- | --- | | `list[Document]` | list of parsed Documents | Source code in `libs/kotaemon/kotaemon/indices/ingests/files.py` | | | | --- | --- | | [114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137) | `def run(self, file_paths: list[str \| Path] \| str \| Path) -> list[Document]: """Ingest the file paths into Document Args: file_paths: list of file paths or a single file path Returns: list of parsed Documents """ if not isinstance(file_paths, list): file_paths = [file_paths] documents = self._get_reader(input_files=file_paths)() print(f"Read {len(file_paths)} files into {len(documents)} documents.") nodes = self.text_splitter(documents) print(f"Transform {len(documents)} documents into {len(nodes)} nodes.") self.log_progress(".num_docs", num_docs=len(nodes)) # document parsers call if self.doc_parsers: for parser in self.doc_parsers: nodes = parser(nodes) return nodes` | Back to top --- # Embeddings - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/__init__.py "Edit this page") Embeddings ========== EndpointEmbeddings [¶](#embeddings.EndpointEmbeddings "Permanent link") ------------------------------------------------------------------------ Bases: `BaseEmbeddings` An Embeddings component that uses an OpenAI API compatible endpoint. Attributes: | Name | Type | Description | | --- | --- | --- | | `endpoint_url` | `str` | The url of an OpenAI API compatible endpoint. | Source code in `libs/kotaemon/kotaemon/embeddings/endpoint_based.py` | | | | --- | --- | | [8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46) | `class EndpointEmbeddings(BaseEmbeddings): """ An Embeddings component that uses an OpenAI API compatible endpoint. Attributes: endpoint_url (str): The url of an OpenAI API compatible endpoint. """ endpoint_url: str def run( self, text: str \| list[str] \| Document \| list[Document] ) -> list[DocumentWithEmbedding]: """ Generate embeddings from text Args: text (str \| list[str] \| Document \| list[Document]): text to generate embeddings from Returns: list[DocumentWithEmbedding]: embeddings """ if not isinstance(text, list): text = [text] outputs = [] for item in text: response = requests.post( self.endpoint_url, json={"input": str(item)} ).json() outputs.append( DocumentWithEmbedding( text=str(item), embedding=response["data"][0]["embedding"], total_tokens=response["usage"]["total_tokens"], prompt_tokens=response["usage"]["prompt_tokens"], ) ) return outputs` | ### run [¶](#embeddings.EndpointEmbeddings.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(text)` | Generate embeddings from text Args text (str | list\[str\] | Document | list\[Document\]): text to generate embeddings from Returns: list\[DocumentWithEmbedding\]: embeddings Source code in `libs/kotaemon/kotaemon/embeddings/endpoint_based.py` | | | | --- | --- | | [18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46) | `def run( self, text: str \| list[str] \| Document \| list[Document] ) -> list[DocumentWithEmbedding]: """ Generate embeddings from text Args: text (str \| list[str] \| Document \| list[Document]): text to generate embeddings from Returns: list[DocumentWithEmbedding]: embeddings """ if not isinstance(text, list): text = [text] outputs = [] for item in text: response = requests.post( self.endpoint_url, json={"input": str(item)} ).json() outputs.append( DocumentWithEmbedding( text=str(item), embedding=response["data"][0]["embedding"], total_tokens=response["usage"]["total_tokens"], prompt_tokens=response["usage"]["prompt_tokens"], ) ) return outputs` | FastEmbedEmbeddings [¶](#embeddings.FastEmbedEmbeddings "Permanent link") -------------------------------------------------------------------------- Bases: `BaseEmbeddings` Utilize fastembed library for embeddings locally without GPU. Supported model: https://qdrant.github.io/fastembed/examples/Supported\_Models/ Code: https://github.com/qdrant/fastembed Source code in `libs/kotaemon/kotaemon/embeddings/fastembed.py` | | | | --- | --- | | [11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72) | ``class FastEmbedEmbeddings(BaseEmbeddings): """Utilize fastembed library for embeddings locally without GPU. Supported model: https://qdrant.github.io/fastembed/examples/Supported_Models/ Code: https://github.com/qdrant/fastembed """ model_name: str = Param( "BAAI/bge-small-en-v1.5", help=( "Model name for fastembed. Please refer " "[here](https://qdrant.github.io/fastembed/examples/Supported_Models/) " "for the list of supported models." ), required=True, ) batch_size: int = Param( 256, help="Batch size for embeddings. Higher values use more memory, but are faster", ) parallel: Optional[int] = Param( None, help=( "Number of threads to use for embeddings. " "If > 1, data-parallel encoding will be used. " "If 0, use all available CPUs. " "If None, use default onnxruntime threading. " "Defaults to None." ), ) @Param.auto() def client_(self) -> "TextEmbedding": try: from fastembed import TextEmbedding except ImportError: raise ImportError("Please install FastEmbed: `pip install fastembed`") return TextEmbedding(model_name=self.model_name) def invoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: input_ = self.prepare_input(text) embeddings = self.client_.embed( [_.content for _ in input_], batch_size=self.batch_size, parallel=self.parallel, ) return [ DocumentWithEmbedding( content=doc, embedding=list(embedding), ) for doc, embedding in zip(input_, embeddings) ] async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: """Fastembed does not support async API.""" return self.invoke(text, *args, **kwargs)`` | ### ainvoke `async` [¶](#embeddings.FastEmbedEmbeddings.ainvoke "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `ainvoke(text, *args, **kwargs)` | Fastembed does not support async API. Source code in `libs/kotaemon/kotaemon/embeddings/fastembed.py` | | | | --- | --- | | [68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72) | `async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: """Fastembed does not support async API.""" return self.invoke(text, *args, **kwargs)` | LCAzureOpenAIEmbeddings [¶](#embeddings.LCAzureOpenAIEmbeddings "Permanent link") ---------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's AzureOpenAI embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148) | `class LCAzureOpenAIEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's AzureOpenAI embedding, focusing on key parameters""" def __init__( self, azure_endpoint: Optional[str] = None, deployment: Optional[str] = None, openai_api_key: Optional[str] = None, api_version: Optional[str] = None, request_timeout: Optional[float] = None, **params, ): super().__init__( azure_endpoint=azure_endpoint, deployment=deployment, api_version=api_version, openai_api_key=openai_api_key, request_timeout=request_timeout, **params, ) def _get_lc_class(self): try: from langchain_openai import AzureOpenAIEmbeddings except ImportError: from langchain.embeddings import AzureOpenAIEmbeddings return AzureOpenAIEmbeddings` | LCCohereEmbeddings [¶](#embeddings.LCCohereEmbeddings "Permanent link") ------------------------------------------------------------------------ Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's Cohere embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190) | `class LCCohereEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's Cohere embedding, focusing on key parameters""" cohere_api_key: str = Param( help="API key (https://dashboard.cohere.com/api-keys)", default=None, required=True, ) model: str = Param( help="Model name to use (https://docs.cohere.com/docs/models)", default=None, required=True, ) user_agent: str = Param( help="User agent (leave default)", default="default", required=True ) def __init__( self, model: str = "embed-english-v2.0", cohere_api_key: Optional[str] = None, truncate: Optional[str] = None, request_timeout: Optional[float] = None, **params, ): super().__init__( model=model, cohere_api_key=cohere_api_key, truncate=truncate, request_timeout=request_timeout, **params, ) def _get_lc_class(self): try: from langchain_cohere import CohereEmbeddings except ImportError: from langchain.embeddings import CohereEmbeddings return CohereEmbeddings` | LCGoogleEmbeddings [¶](#embeddings.LCGoogleEmbeddings "Permanent link") ------------------------------------------------------------------------ Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's Google GenAI embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256) | `class LCGoogleEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's Google GenAI embedding, focusing on key parameters""" google_api_key: str = Param( help="API key (https://aistudio.google.com/app/apikey)", default=None, required=True, ) model: str = Param( help="Model name to use (https://ai.google.dev/gemini-api/docs/models/gemini#text-embedding-and-embedding)", # noqa default="models/text-embedding-004", required=True, ) def __init__( self, model: str = "models/text-embedding-004", google_api_key: Optional[str] = None, **params, ): super().__init__( model=model, google_api_key=google_api_key, **params, ) def _get_lc_class(self): try: from langchain_google_genai import GoogleGenerativeAIEmbeddings except ImportError: raise ImportError("Please install langchain-google-genai") return GoogleGenerativeAIEmbeddings` | LCHuggingFaceEmbeddings [¶](#embeddings.LCHuggingFaceEmbeddings "Permanent link") ---------------------------------------------------------------------------------- Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's HuggingFace embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221) | `class LCHuggingFaceEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's HuggingFace embedding, focusing on key parameters""" model_name: str = Param( help=( "Model name to use (https://huggingface.co/models?" "pipeline_tag=sentence-similarity&sort=trending)" ), default=None, required=True, ) def __init__( self, model_name: str = "sentence-transformers/all-mpnet-base-v2", **params, ): super().__init__( model_name=model_name, **params, ) def _get_lc_class(self): try: from langchain_community.embeddings import HuggingFaceBgeEmbeddings except ImportError: from langchain.embeddings import HuggingFaceBgeEmbeddings return HuggingFaceBgeEmbeddings` | LCOpenAIEmbeddings [¶](#embeddings.LCOpenAIEmbeddings "Permanent link") ------------------------------------------------------------------------ Bases: `LCEmbeddingMixin`, `BaseEmbeddings` Wrapper around Langchain's OpenAI embedding, focusing on key parameters Source code in `libs/kotaemon/kotaemon/embeddings/langchain_based.py` | | | | --- | --- | | [89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118) | `class LCOpenAIEmbeddings(LCEmbeddingMixin, BaseEmbeddings): """Wrapper around Langchain's OpenAI embedding, focusing on key parameters""" def __init__( self, model: str = "text-embedding-ada-002", openai_api_version: Optional[str] = None, openai_api_base: Optional[str] = None, openai_api_type: Optional[str] = None, openai_api_key: Optional[str] = None, request_timeout: Optional[float] = None, **params, ): super().__init__( model=model, openai_api_version=openai_api_version, openai_api_base=openai_api_base, openai_api_type=openai_api_type, openai_api_key=openai_api_key, request_timeout=request_timeout, **params, ) def _get_lc_class(self): try: from langchain_openai import OpenAIEmbeddings except ImportError: from langchain.embeddings import OpenAIEmbeddings return OpenAIEmbeddings` | AzureOpenAIEmbeddings [¶](#embeddings.AzureOpenAIEmbeddings "Permanent link") ------------------------------------------------------------------------------ Bases: `BaseOpenAIEmbeddings` Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257) | `class AzureOpenAIEmbeddings(BaseOpenAIEmbeddings): azure_endpoint: str = Param( None, help=( "HTTPS endpoint for the Azure OpenAI model. The azure_endpoint, " "azure_deployment, and api_version parameters are used to construct " "the full URL for the Azure OpenAI model." ), required=True, ) azure_deployment: str = Param(None, help="Azure deployment name", required=True) api_version: str = Param(None, help="Azure model version", required=True) azure_ad_token: Optional[str] = Param(None, help="Azure AD token") azure_ad_token_provider: Optional[str] = Param(None, help="Azure AD token provider") @Param.auto(depends_on=["azure_ad_token_provider"]) def azure_ad_token_provider_(self): if isinstance(self.azure_ad_token_provider, str): return import_dotted_string(self.azure_ad_token_provider, safe=False) def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "azure_endpoint": self.azure_endpoint, "api_version": self.api_version, "api_key": self.api_key, "azure_ad_token": self.azure_ad_token, "azure_ad_token_provider": self.azure_ad_token_provider_, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncAzureOpenAI return AsyncAzureOpenAI(**params) from openai import AzureOpenAI return AzureOpenAI(**params) @retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.azure_deployment, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | ### prepare\_client [¶](#embeddings.AzureOpenAIEmbeddings.prepare_client "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `prepare_client(async_version=False)` | Get the OpenAI client Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `async_version` | `bool` | Whether to get the async version of the client | `False` | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239) | `def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "azure_endpoint": self.azure_endpoint, "api_version": self.api_version, "api_key": self.api_key, "azure_ad_token": self.azure_ad_token, "azure_ad_token_provider": self.azure_ad_token_provider_, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncAzureOpenAI return AsyncAzureOpenAI(**params) from openai import AzureOpenAI return AzureOpenAI(**params)` | ### openai\_response [¶](#embeddings.AzureOpenAIEmbeddings.openai_response "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `openai_response(client, **kwargs)` | Get the openai response Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257) | `@retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.azure_deployment, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | OpenAIEmbeddings [¶](#embeddings.OpenAIEmbeddings "Permanent link") -------------------------------------------------------------------- Bases: `BaseOpenAIEmbeddings` OpenAI chat model Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194) | `class OpenAIEmbeddings(BaseOpenAIEmbeddings): """OpenAI chat model""" base_url: Optional[str] = Param(None, help="OpenAI base URL") organization: Optional[str] = Param(None, help="OpenAI organization") model: str = Param( None, help=( "ID of the model to use. You can go to [Model overview](https://platform." "openai.com/docs/models/overview) to see the available models." ), required=True, ) def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "api_key": self.api_key, "organization": self.organization, "base_url": self.base_url, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncOpenAI return AsyncOpenAI(**params) from openai import OpenAI return OpenAI(**params) @retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.model, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | ### prepare\_client [¶](#embeddings.OpenAIEmbeddings.prepare_client "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `prepare_client(async_version=False)` | Get the OpenAI client Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `async_version` | `bool` | Whether to get the async version of the client | `False` | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176) | `def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "api_key": self.api_key, "organization": self.organization, "base_url": self.base_url, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncOpenAI return AsyncOpenAI(**params) from openai import OpenAI return OpenAI(**params)` | ### openai\_response [¶](#embeddings.OpenAIEmbeddings.openai_response "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `openai_response(client, **kwargs)` | Get the openai response Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194) | `@retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.model, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | TeiEndpointEmbeddings [¶](#embeddings.TeiEndpointEmbeddings "Permanent link") ------------------------------------------------------------------------------ Bases: `BaseEmbeddings` An Embeddings component that uses an TEI (Text-Embedding-Inference) API compatible endpoint. Ref: https://github.com/huggingface/text-embeddings-inference Attributes: | Name | Type | Description | | --- | --- | --- | | `endpoint_url` | `str` | The url of an TEI (Text-Embedding-Inference) API compatible endpoint. | | `normalize` | `bool` | Whether to normalize embeddings to unit length. | | `truncate` | `bool` | Whether to truncate embeddings to a fixed/default length. | Source code in `libs/kotaemon/kotaemon/embeddings/tei_endpoint_embed.py` | | | | --- | --- | | [11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105) | `class TeiEndpointEmbeddings(BaseEmbeddings): """An Embeddings component that uses an TEI (Text-Embedding-Inference) API compatible endpoint. Ref: https://github.com/huggingface/text-embeddings-inference Attributes: endpoint_url (str): The url of an TEI (Text-Embedding-Inference) API compatible endpoint. normalize (bool): Whether to normalize embeddings to unit length. truncate (bool): Whether to truncate embeddings to a fixed/default length. """ endpoint_url: str = Param(None, help="TEI embedding service api base URL") normalize: bool = Param( True, help="Normalize embeddings to unit length", ) truncate: bool = Param( True, help="Truncate embeddings to a fixed/default length", ) async def client_(self, inputs: list[str]): async with aiohttp.ClientSession() as session: async with session.post( url=self.endpoint_url, json={ "inputs": inputs, "normalize": self.normalize, "truncate": self.truncate, }, ) as resp: embeddings = await resp.json() return embeddings async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: if not isinstance(text, list): text = [text] text = self.prepare_input(text) outputs = [] batch_size = 6 num_batch = max(len(text) // batch_size, 1) for i in range(num_batch): if i == num_batch - 1: mini_batch = text[batch_size * i :] else: mini_batch = text[batch_size * i : batch_size * (i + 1)] mini_batch = [x.content for x in mini_batch] embeddings = await self.client_(mini_batch) # type: ignore outputs.extend( [ DocumentWithEmbedding(content=doc, embedding=embedding) for doc, embedding in zip(mini_batch, embeddings) ] ) return outputs def invoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: if not isinstance(text, list): text = [text] text = self.prepare_input(text) outputs = [] batch_size = 6 num_batch = max(len(text) // batch_size, 1) for i in range(num_batch): if i == num_batch - 1: mini_batch = text[batch_size * i :] else: mini_batch = text[batch_size * i : batch_size * (i + 1)] mini_batch = [x.content for x in mini_batch] embeddings = session.post( url=self.endpoint_url, json={ "inputs": mini_batch, "normalize": self.normalize, "truncate": self.truncate, }, ).json() outputs.extend( [ DocumentWithEmbedding(content=doc, embedding=embedding) for doc, embedding in zip(mini_batch, embeddings) ] ) return outputs` | Back to top --- # Tei Endpoint Embed - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/tei_endpoint_embed.py "Edit this page") Tei Endpoint Embed ================== TeiEndpointEmbeddings [¶](#embeddings.tei_endpoint_embed.TeiEndpointEmbeddings "Permanent link") ------------------------------------------------------------------------------------------------- Bases: `BaseEmbeddings` An Embeddings component that uses an TEI (Text-Embedding-Inference) API compatible endpoint. Ref: https://github.com/huggingface/text-embeddings-inference Attributes: | Name | Type | Description | | --- | --- | --- | | `endpoint_url` | `str` | The url of an TEI (Text-Embedding-Inference) API compatible endpoint. | | `normalize` | `bool` | Whether to normalize embeddings to unit length. | | `truncate` | `bool` | Whether to truncate embeddings to a fixed/default length. | Source code in `libs/kotaemon/kotaemon/embeddings/tei_endpoint_embed.py` | | | | --- | --- | | [11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105) | `class TeiEndpointEmbeddings(BaseEmbeddings): """An Embeddings component that uses an TEI (Text-Embedding-Inference) API compatible endpoint. Ref: https://github.com/huggingface/text-embeddings-inference Attributes: endpoint_url (str): The url of an TEI (Text-Embedding-Inference) API compatible endpoint. normalize (bool): Whether to normalize embeddings to unit length. truncate (bool): Whether to truncate embeddings to a fixed/default length. """ endpoint_url: str = Param(None, help="TEI embedding service api base URL") normalize: bool = Param( True, help="Normalize embeddings to unit length", ) truncate: bool = Param( True, help="Truncate embeddings to a fixed/default length", ) async def client_(self, inputs: list[str]): async with aiohttp.ClientSession() as session: async with session.post( url=self.endpoint_url, json={ "inputs": inputs, "normalize": self.normalize, "truncate": self.truncate, }, ) as resp: embeddings = await resp.json() return embeddings async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: if not isinstance(text, list): text = [text] text = self.prepare_input(text) outputs = [] batch_size = 6 num_batch = max(len(text) // batch_size, 1) for i in range(num_batch): if i == num_batch - 1: mini_batch = text[batch_size * i :] else: mini_batch = text[batch_size * i : batch_size * (i + 1)] mini_batch = [x.content for x in mini_batch] embeddings = await self.client_(mini_batch) # type: ignore outputs.extend( [ DocumentWithEmbedding(content=doc, embedding=embedding) for doc, embedding in zip(mini_batch, embeddings) ] ) return outputs def invoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: if not isinstance(text, list): text = [text] text = self.prepare_input(text) outputs = [] batch_size = 6 num_batch = max(len(text) // batch_size, 1) for i in range(num_batch): if i == num_batch - 1: mini_batch = text[batch_size * i :] else: mini_batch = text[batch_size * i : batch_size * (i + 1)] mini_batch = [x.content for x in mini_batch] embeddings = session.post( url=self.endpoint_url, json={ "inputs": mini_batch, "normalize": self.normalize, "truncate": self.truncate, }, ).json() outputs.extend( [ DocumentWithEmbedding(content=doc, embedding=embedding) for doc, embedding in zip(mini_batch, embeddings) ] ) return outputs` | Back to top --- # Indices - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/__init__.py "Edit this page") Indices ======= VectorIndexing [¶](#indices.VectorIndexing "Permanent link") ------------------------------------------------------------- Bases: `BaseIndexing` Ingest the document, run through the embedding, and store the embedding in a vector store. This pipeline supports the following set of inputs * List of documents * List of texts Source code in `libs/kotaemon/kotaemon/indices/vectorindex.py` | | | | --- | --- | | [21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113) | `class VectorIndexing(BaseIndexing): """Ingest the document, run through the embedding, and store the embedding in a vector store. This pipeline supports the following set of inputs: - List of documents - List of texts """ cache_dir: Optional[str] = getattr(flowsettings, "KH_CHUNKS_OUTPUT_DIR", None) vector_store: BaseVectorStore doc_store: Optional[BaseDocumentStore] = None embedding: BaseEmbeddings count_: int = 0 def to_retrieval_pipeline(self, *args, **kwargs): """Convert the indexing pipeline to a retrieval pipeline""" return VectorRetrieval( vector_store=self.vector_store, doc_store=self.doc_store, embedding=self.embedding, **kwargs, ) def write_chunk_to_file(self, docs: list[Document]): # save the chunks content into markdown format if self.cache_dir: file_name = docs[0].metadata.get("file_name") if not file_name: return file_name = Path(file_name) for i in range(len(docs)): markdown_content = "" if "page_label" in docs[i].metadata: page_label = str(docs[i].metadata["page_label"]) markdown_content += f"Page label: {page_label}" if "file_name" in docs[i].metadata: filename = docs[i].metadata["file_name"] markdown_content += f"\nFile name: {filename}" if "section" in docs[i].metadata: section = docs[i].metadata["section"] markdown_content += f"\nSection: {section}" if "type" in docs[i].metadata: if docs[i].metadata["type"] == "image": image_origin = docs[i].metadata["image_origin"] image_origin = f'
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43) | `def to_retrieval_pipeline(self, *args, **kwargs): """Convert the indexing pipeline to a retrieval pipeline""" return VectorRetrieval( vector_store=self.vector_store, doc_store=self.doc_store, embedding=self.embedding, **kwargs, )` | VectorRetrieval [¶](#indices.VectorRetrieval "Permanent link") --------------------------------------------------------------- Bases: `BaseRetrieval` Retrieve list of documents from vector store Source code in `libs/kotaemon/kotaemon/indices/vectorindex.py` | | | | --- | --- | | [116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298) | `class VectorRetrieval(BaseRetrieval): """Retrieve list of documents from vector store""" vector_store: BaseVectorStore doc_store: Optional[BaseDocumentStore] = None embedding: BaseEmbeddings rerankers: Sequence[BaseReranking] = [] top_k: int = 5 first_round_top_k_mult: int = 10 retrieval_mode: str = "hybrid" # vector, text, hybrid def _filter_docs( self, documents: list[RetrievedDocument], top_k: int \| None = None ): if top_k: documents = documents[:top_k] return documents def run( self, text: str \| Document, top_k: Optional[int] = None, **kwargs ) -> list[RetrievedDocument]: """Retrieve a list of documents from vector store Args: text: the text to retrieve similar documents top_k: number of top similar documents to return Returns: list[RetrievedDocument]: list of retrieved documents """ if top_k is None: top_k = self.top_k do_extend = kwargs.pop("do_extend", False) thumbnail_count = kwargs.pop("thumbnail_count", 3) if do_extend: top_k_first_round = top_k * self.first_round_top_k_mult else: top_k_first_round = top_k if self.doc_store is None: raise ValueError( "doc_store is not provided. Please provide a doc_store to " "retrieve the documents" ) result: list[RetrievedDocument] = [] # TODO: should declare scope directly in the run params scope = kwargs.pop("scope", None) emb: list[float] if self.retrieval_mode == "vector": emb = self.embedding(text)[0].embedding _, scores, ids = self.vector_store.query( embedding=emb, top_k=top_k_first_round, **kwargs ) docs = self.doc_store.get(ids) result = [ RetrievedDocument(**doc.to_dict(), score=score) for doc, score in zip(docs, scores) ] elif self.retrieval_mode == "text": query = text.text if isinstance(text, Document) else text docs = self.doc_store.query(query, top_k=top_k_first_round, doc_ids=scope) result = [RetrievedDocument(**doc.to_dict(), score=-1.0) for doc in docs] elif self.retrieval_mode == "hybrid": # similarity search section emb = self.embedding(text)[0].embedding vs_docs: list[RetrievedDocument] = [] vs_ids: list[str] = [] vs_scores: list[float] = [] def query_vectorstore(): nonlocal vs_docs nonlocal vs_scores nonlocal vs_ids assert self.doc_store is not None _, vs_scores, vs_ids = self.vector_store.query( embedding=emb, top_k=top_k_first_round, **kwargs ) if vs_ids: vs_docs = self.doc_store.get(vs_ids) # full-text search section ds_docs: list[RetrievedDocument] = [] def query_docstore(): nonlocal ds_docs assert self.doc_store is not None query = text.text if isinstance(text, Document) else text ds_docs = self.doc_store.query( query, top_k=top_k_first_round, doc_ids=scope ) vs_query_thread = threading.Thread(target=query_vectorstore) ds_query_thread = threading.Thread(target=query_docstore) vs_query_thread.start() ds_query_thread.start() vs_query_thread.join() ds_query_thread.join() result = [ RetrievedDocument(**doc.to_dict(), score=-1.0) for doc in ds_docs if doc not in vs_ids ] result += [ RetrievedDocument(**doc.to_dict(), score=score) for doc, score in zip(vs_docs, vs_scores) ] print(f"Got {len(vs_docs)} from vectorstore") print(f"Got {len(ds_docs)} from docstore") # use additional reranker to re-order the document list if self.rerankers and text: for reranker in self.rerankers: # if reranker is LLMReranking, limit the document with top_k items only if isinstance(reranker, LLMReranking): result = self._filter_docs(result, top_k=top_k) result = reranker.run(documents=result, query=text) result = self._filter_docs(result, top_k=top_k) print(f"Got raw {len(result)} retrieved documents") # add page thumbnails to the result if exists thumbnail_doc_ids: set[str] = set() # we should copy the text from retrieved text chunk # to the thumbnail to get relevant LLM score correctly text_thumbnail_docs: dict[str, RetrievedDocument] = {} non_thumbnail_docs = [] raw_thumbnail_docs = [] for doc in result: if doc.metadata.get("type") == "thumbnail": # change type to image to display on UI doc.metadata["type"] = "image" raw_thumbnail_docs.append(doc) continue if ( "thumbnail_doc_id" in doc.metadata and len(thumbnail_doc_ids) < thumbnail_count ): thumbnail_id = doc.metadata["thumbnail_doc_id"] thumbnail_doc_ids.add(thumbnail_id) text_thumbnail_docs[thumbnail_id] = doc else: non_thumbnail_docs.append(doc) linked_thumbnail_docs = self.doc_store.get(list(thumbnail_doc_ids)) print( "thumbnail docs", len(linked_thumbnail_docs), "non-thumbnail docs", len(non_thumbnail_docs), "raw-thumbnail docs", len(raw_thumbnail_docs), ) additional_docs = [] for thumbnail_doc in linked_thumbnail_docs: text_doc = text_thumbnail_docs[thumbnail_doc.doc_id] doc_dict = thumbnail_doc.to_dict() doc_dict["_id"] = text_doc.doc_id doc_dict["content"] = text_doc.content doc_dict["metadata"]["type"] = "image" for key in text_doc.metadata: if key not in doc_dict["metadata"]: doc_dict["metadata"][key] = text_doc.metadata[key] additional_docs.append(RetrievedDocument(**doc_dict, score=text_doc.score)) result = additional_docs + non_thumbnail_docs if not result: # return output from raw retrieved thumbnails result = self._filter_docs(raw_thumbnail_docs, top_k=thumbnail_count) return result` | ### run [¶](#indices.VectorRetrieval.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(text, top_k=None, **kwargs)` | Retrieve a list of documents from vector store Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `text` | `str \| Document` | the text to retrieve similar documents | _required_ | | `top_k` | `Optional[int]` | number of top similar documents to return | `None` | Returns: | Type | Description | | --- | --- | | `list[RetrievedDocument]` | list\[RetrievedDocument\]: list of retrieved documents | Source code in `libs/kotaemon/kotaemon/indices/vectorindex.py` | | | | --- | --- | | [134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298) | `def run( self, text: str \| Document, top_k: Optional[int] = None, **kwargs ) -> list[RetrievedDocument]: """Retrieve a list of documents from vector store Args: text: the text to retrieve similar documents top_k: number of top similar documents to return Returns: list[RetrievedDocument]: list of retrieved documents """ if top_k is None: top_k = self.top_k do_extend = kwargs.pop("do_extend", False) thumbnail_count = kwargs.pop("thumbnail_count", 3) if do_extend: top_k_first_round = top_k * self.first_round_top_k_mult else: top_k_first_round = top_k if self.doc_store is None: raise ValueError( "doc_store is not provided. Please provide a doc_store to " "retrieve the documents" ) result: list[RetrievedDocument] = [] # TODO: should declare scope directly in the run params scope = kwargs.pop("scope", None) emb: list[float] if self.retrieval_mode == "vector": emb = self.embedding(text)[0].embedding _, scores, ids = self.vector_store.query( embedding=emb, top_k=top_k_first_round, **kwargs ) docs = self.doc_store.get(ids) result = [ RetrievedDocument(**doc.to_dict(), score=score) for doc, score in zip(docs, scores) ] elif self.retrieval_mode == "text": query = text.text if isinstance(text, Document) else text docs = self.doc_store.query(query, top_k=top_k_first_round, doc_ids=scope) result = [RetrievedDocument(**doc.to_dict(), score=-1.0) for doc in docs] elif self.retrieval_mode == "hybrid": # similarity search section emb = self.embedding(text)[0].embedding vs_docs: list[RetrievedDocument] = [] vs_ids: list[str] = [] vs_scores: list[float] = [] def query_vectorstore(): nonlocal vs_docs nonlocal vs_scores nonlocal vs_ids assert self.doc_store is not None _, vs_scores, vs_ids = self.vector_store.query( embedding=emb, top_k=top_k_first_round, **kwargs ) if vs_ids: vs_docs = self.doc_store.get(vs_ids) # full-text search section ds_docs: list[RetrievedDocument] = [] def query_docstore(): nonlocal ds_docs assert self.doc_store is not None query = text.text if isinstance(text, Document) else text ds_docs = self.doc_store.query( query, top_k=top_k_first_round, doc_ids=scope ) vs_query_thread = threading.Thread(target=query_vectorstore) ds_query_thread = threading.Thread(target=query_docstore) vs_query_thread.start() ds_query_thread.start() vs_query_thread.join() ds_query_thread.join() result = [ RetrievedDocument(**doc.to_dict(), score=-1.0) for doc in ds_docs if doc not in vs_ids ] result += [ RetrievedDocument(**doc.to_dict(), score=score) for doc, score in zip(vs_docs, vs_scores) ] print(f"Got {len(vs_docs)} from vectorstore") print(f"Got {len(ds_docs)} from docstore") # use additional reranker to re-order the document list if self.rerankers and text: for reranker in self.rerankers: # if reranker is LLMReranking, limit the document with top_k items only if isinstance(reranker, LLMReranking): result = self._filter_docs(result, top_k=top_k) result = reranker.run(documents=result, query=text) result = self._filter_docs(result, top_k=top_k) print(f"Got raw {len(result)} retrieved documents") # add page thumbnails to the result if exists thumbnail_doc_ids: set[str] = set() # we should copy the text from retrieved text chunk # to the thumbnail to get relevant LLM score correctly text_thumbnail_docs: dict[str, RetrievedDocument] = {} non_thumbnail_docs = [] raw_thumbnail_docs = [] for doc in result: if doc.metadata.get("type") == "thumbnail": # change type to image to display on UI doc.metadata["type"] = "image" raw_thumbnail_docs.append(doc) continue if ( "thumbnail_doc_id" in doc.metadata and len(thumbnail_doc_ids) < thumbnail_count ): thumbnail_id = doc.metadata["thumbnail_doc_id"] thumbnail_doc_ids.add(thumbnail_id) text_thumbnail_docs[thumbnail_id] = doc else: non_thumbnail_docs.append(doc) linked_thumbnail_docs = self.doc_store.get(list(thumbnail_doc_ids)) print( "thumbnail docs", len(linked_thumbnail_docs), "non-thumbnail docs", len(non_thumbnail_docs), "raw-thumbnail docs", len(raw_thumbnail_docs), ) additional_docs = [] for thumbnail_doc in linked_thumbnail_docs: text_doc = text_thumbnail_docs[thumbnail_doc.doc_id] doc_dict = thumbnail_doc.to_dict() doc_dict["_id"] = text_doc.doc_id doc_dict["content"] = text_doc.content doc_dict["metadata"]["type"] = "image" for key in text_doc.metadata: if key not in doc_dict["metadata"]: doc_dict["metadata"][key] = text_doc.metadata[key] additional_docs.append(RetrievedDocument(**doc_dict, score=text_doc.score)) result = additional_docs + non_thumbnail_docs if not result: # return output from raw retrieved thumbnails result = self._filter_docs(raw_thumbnail_docs, top_k=thumbnail_count) return result` | Back to top --- # Openai - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/embeddings/openai.py "Edit this page") Openai ====== BaseOpenAIEmbeddings [¶](#embeddings.openai.BaseOpenAIEmbeddings "Permanent link") ----------------------------------------------------------------------------------- Bases: `BaseEmbeddings` Base interface for OpenAI embedding model, using the openai library. This class exposes the parameters in resources.Chat. To subclass this class: | | | | --- | --- | | 1
2
3 | ``- Implement the `prepare_client` method to return the OpenAI client - Implement the `openai_response` method to return the OpenAI response - Implement the params relate to the OpenAI client`` | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139) | ``class BaseOpenAIEmbeddings(BaseEmbeddings): """Base interface for OpenAI embedding model, using the openai library. This class exposes the parameters in resources.Chat. To subclass this class: - Implement the `prepare_client` method to return the OpenAI client - Implement the `openai_response` method to return the OpenAI response - Implement the params relate to the OpenAI client """ _dependencies = ["openai"] api_key: str = Param(None, help="API key", required=True) timeout: Optional[float] = Param(None, help="Timeout for the API request.") max_retries: Optional[int] = Param( None, help="Maximum number of retries for the API request." ) dimensions: Optional[int] = Param( None, help=( "The number of dimensions the resulting output embeddings should have. " "Only supported in `text-embedding-3` and later models." ), ) context_length: Optional[int] = Param( None, help="The maximum context length of the embedding model" ) @Param.auto(depends_on=["max_retries"]) def max_retries_(self): if self.max_retries is None: from openai._constants import DEFAULT_MAX_RETRIES return DEFAULT_MAX_RETRIES return self.max_retries def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ raise NotImplementedError def openai_response(self, client, **kwargs): """Get the openai response""" raise NotImplementedError def invoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: input_doc = self.prepare_input(text) client = self.prepare_client(async_version=False) input_: list[str \| list[int]] = [] splitted_indices = {} for idx, text in enumerate(input_doc): if self.context_length: chunks = split_text_by_chunk_size(text.text or " ", self.context_length) splitted_indices[idx] = (len(input_), len(input_) + len(chunks)) input_.extend(chunks) else: splitted_indices[idx] = (len(input_), len(input_) + 1) input_.append(text.text) resp = self.openai_response(client, input=input_, **kwargs).dict() output_ = list(sorted(resp["data"], key=lambda x: x["index"])) output = [] for idx, doc in enumerate(input_doc): embs = output_[splitted_indices[idx][0] : splitted_indices[idx][1]] if len(embs) == 1: output.append( DocumentWithEmbedding(embedding=embs[0]["embedding"], content=doc) ) continue chunk_lens = [ len(_) for _ in input_[splitted_indices[idx][0] : splitted_indices[idx][1]] ] vs: list[list[float]] = [_["embedding"] for _ in embs] emb = np.average(vs, axis=0, weights=chunk_lens) emb = emb / np.linalg.norm(emb) output.append(DocumentWithEmbedding(embedding=emb.tolist(), content=doc)) return output async def ainvoke( self, text: str \| list[str] \| Document \| list[Document], *args, **kwargs ) -> list[DocumentWithEmbedding]: input_ = self.prepare_input(text) client = self.prepare_client(async_version=True) resp = await self.openai_response( client, input=[_.text if _.text else " " for _ in input_], **kwargs ).dict() output_ = sorted(resp["data"], key=lambda x: x["index"]) return [ DocumentWithEmbedding(embedding=o["embedding"], content=i) for i, o in zip(input_, output_) ]`` | ### prepare\_client [¶](#embeddings.openai.BaseOpenAIEmbeddings.prepare_client "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `prepare_client(async_version=False)` | Get the OpenAI client Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `async_version` | `bool` | Whether to get the async version of the client | `False` | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81) | `def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ raise NotImplementedError` | ### openai\_response [¶](#embeddings.openai.BaseOpenAIEmbeddings.openai_response "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `openai_response(client, **kwargs)` | Get the openai response Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85) | `def openai_response(self, client, **kwargs): """Get the openai response""" raise NotImplementedError` | OpenAIEmbeddings [¶](#embeddings.openai.OpenAIEmbeddings "Permanent link") --------------------------------------------------------------------------- Bases: `BaseOpenAIEmbeddings` OpenAI chat model Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194) | `class OpenAIEmbeddings(BaseOpenAIEmbeddings): """OpenAI chat model""" base_url: Optional[str] = Param(None, help="OpenAI base URL") organization: Optional[str] = Param(None, help="OpenAI organization") model: str = Param( None, help=( "ID of the model to use. You can go to [Model overview](https://platform." "openai.com/docs/models/overview) to see the available models." ), required=True, ) def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "api_key": self.api_key, "organization": self.organization, "base_url": self.base_url, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncOpenAI return AsyncOpenAI(**params) from openai import OpenAI return OpenAI(**params) @retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.model, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | ### prepare\_client [¶](#embeddings.openai.OpenAIEmbeddings.prepare_client "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `prepare_client(async_version=False)` | Get the OpenAI client Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `async_version` | `bool` | Whether to get the async version of the client | `False` | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176) | `def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "api_key": self.api_key, "organization": self.organization, "base_url": self.base_url, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncOpenAI return AsyncOpenAI(**params) from openai import OpenAI return OpenAI(**params)` | ### openai\_response [¶](#embeddings.openai.OpenAIEmbeddings.openai_response "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `openai_response(client, **kwargs)` | Get the openai response Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194) | `@retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.model, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | AzureOpenAIEmbeddings [¶](#embeddings.openai.AzureOpenAIEmbeddings "Permanent link") ------------------------------------------------------------------------------------- Bases: `BaseOpenAIEmbeddings` Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257) | `class AzureOpenAIEmbeddings(BaseOpenAIEmbeddings): azure_endpoint: str = Param( None, help=( "HTTPS endpoint for the Azure OpenAI model. The azure_endpoint, " "azure_deployment, and api_version parameters are used to construct " "the full URL for the Azure OpenAI model." ), required=True, ) azure_deployment: str = Param(None, help="Azure deployment name", required=True) api_version: str = Param(None, help="Azure model version", required=True) azure_ad_token: Optional[str] = Param(None, help="Azure AD token") azure_ad_token_provider: Optional[str] = Param(None, help="Azure AD token provider") @Param.auto(depends_on=["azure_ad_token_provider"]) def azure_ad_token_provider_(self): if isinstance(self.azure_ad_token_provider, str): return import_dotted_string(self.azure_ad_token_provider, safe=False) def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "azure_endpoint": self.azure_endpoint, "api_version": self.api_version, "api_key": self.api_key, "azure_ad_token": self.azure_ad_token, "azure_ad_token_provider": self.azure_ad_token_provider_, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncAzureOpenAI return AsyncAzureOpenAI(**params) from openai import AzureOpenAI return AzureOpenAI(**params) @retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.azure_deployment, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | ### prepare\_client [¶](#embeddings.openai.AzureOpenAIEmbeddings.prepare_client "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `prepare_client(async_version=False)` | Get the OpenAI client Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `async_version` | `bool` | Whether to get the async version of the client | `False` | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239) | `def prepare_client(self, async_version: bool = False): """Get the OpenAI client Args: async_version (bool): Whether to get the async version of the client """ params = { "azure_endpoint": self.azure_endpoint, "api_version": self.api_version, "api_key": self.api_key, "azure_ad_token": self.azure_ad_token, "azure_ad_token_provider": self.azure_ad_token_provider_, "timeout": self.timeout, "max_retries": self.max_retries_, } if async_version: from openai import AsyncAzureOpenAI return AsyncAzureOpenAI(**params) from openai import AzureOpenAI return AzureOpenAI(**params)` | ### openai\_response [¶](#embeddings.openai.AzureOpenAIEmbeddings.openai_response "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `openai_response(client, **kwargs)` | Get the openai response Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257) | `@retry( retry=retry_if_not_exception_type( (openai.NotFoundError, openai.BadRequestError) ), wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(6), ) def openai_response(self, client, **kwargs): """Get the openai response""" params: dict = { "model": self.azure_deployment, } if self.dimensions: params["dimensions"] = self.dimensions params.update(kwargs) return client.embeddings.create(**params)` | split\_text\_by\_chunk\_size [¶](#embeddings.openai.split_text_by_chunk_size "Permanent link") ----------------------------------------------------------------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `split_text_by_chunk_size(text, chunk_size)` | Split the text into chunks of a given size Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `text` | `str` | text to split | _required_ | | `chunk_size` | `int` | size of each chunk | _required_ | Returns: | Type | Description | | --- | --- | | `list[list[int]]` | list of chunks (as tokens) | Source code in `libs/kotaemon/kotaemon/embeddings/openai.py` | | | | --- | --- | | [20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35) | `def split_text_by_chunk_size(text: str, chunk_size: int) -> list[list[int]]: """Split the text into chunks of a given size Args: text: text to split chunk_size: size of each chunk Returns: list of chunks (as tokens) """ encoding = tiktoken.get_encoding("cl100k_base") tokens = iter(encoding.encode(text)) result = [] while chunk := list(islice(tokens, chunk_size)): result.append(chunk) return result` | Back to top --- # Utils - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/qa/utils.py "Edit this page") Utils ===== Back to top --- # Qa - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/qa/__init__.py "Edit this page") Qa == CitationPipeline [¶](#indices.qa.CitationPipeline "Permanent link") -------------------------------------------------------------------- Bases: `BaseComponent` Citation pipeline to extract cited evidences from source (based on input question) Source code in `libs/kotaemon/kotaemon/indices/qa/citation.py` | | | | --- | --- | | [22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98) | `class CitationPipeline(BaseComponent): """Citation pipeline to extract cited evidences from source (based on input question)""" llm: BaseLLM def run(self, context: str, question: str): return self.invoke(context, question) def prepare_llm(self, context: str, question: str): schema = CiteEvidence.schema() function = { "name": schema["title"], "description": schema["description"], "parameters": schema, } llm_kwargs = { "tools": [{"type": "function", "function": function}], "tool_choice": "required", "tools_pydantic": [CiteEvidence], } messages = [ SystemMessage( content=( "You are a world class algorithm to answer " "questions with correct and exact citations." ) ), HumanMessage( content=( "Answer question using the following context. " "Use the provided function CiteEvidence() to cite your sources." ) ), HumanMessage(content=context), HumanMessage(content=f"Question: {question}"), HumanMessage( content=( "Tips: Make sure to cite your sources, " "and use the exact words from the context." ) ), ] return messages, llm_kwargs def invoke(self, context: str, question: str): messages, llm_kwargs = self.prepare_llm(context, question) try: print("CitationPipeline: invoking LLM") llm_output = self.get_from_path("llm").invoke(messages, **llm_kwargs) print("CitationPipeline: finish invoking LLM") if not llm_output.additional_kwargs.get("tool_calls"): return None first_func = llm_output.additional_kwargs["tool_calls"][0] if "function" in first_func: # openai and cohere format function_output = first_func["function"]["arguments"] else: # anthropic format function_output = first_func["args"] print("CitationPipeline:", function_output) if isinstance(function_output, str): output = CiteEvidence.parse_raw(function_output) else: output = CiteEvidence.parse_obj(function_output) except Exception as e: print(e) return None return output async def ainvoke(self, context: str, question: str): raise NotImplementedError()` | Back to top --- # Citation Qa Inline - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/qa/citation_qa_inline.py "Edit this page") Citation Qa Inline ================== InlineEvidence `dataclass` [¶](#indices.qa.citation_qa_inline.InlineEvidence "Permanent link") ----------------------------------------------------------------------------------------------- List of evidences to support the answer. Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa_inline.py` | | | | --- | --- | | [77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83) | `@dataclass class InlineEvidence: """List of evidences to support the answer.""" start_phrase: str \| None = None end_phrase: str \| None = None idx: int \| None = None` | AnswerWithInlineCitation [¶](#indices.qa.citation_qa_inline.AnswerWithInlineCitation "Permanent link") ------------------------------------------------------------------------------------------------------- Bases: `AnswerWithContextPipeline` Answer the question based on the evidence with inline citation Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa_inline.py` | | | | --- | --- | | [86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310)
[311](#__codelineno-0-311)
[312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317)
[318](#__codelineno-0-318)
[319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345)
[346](#__codelineno-0-346)
[347](#__codelineno-0-347)
[348](#__codelineno-0-348)
[349](#__codelineno-0-349)
[350](#__codelineno-0-350)
[351](#__codelineno-0-351)
[352](#__codelineno-0-352)
[353](#__codelineno-0-353)
[354](#__codelineno-0-354)
[355](#__codelineno-0-355)
[356](#__codelineno-0-356)
[357](#__codelineno-0-357)
[358](#__codelineno-0-358)
[359](#__codelineno-0-359)
[360](#__codelineno-0-360)
[361](#__codelineno-0-361) | `class AnswerWithInlineCitation(AnswerWithContextPipeline): """Answer the question based on the evidence with inline citation""" qa_citation_template: str = DEFAULT_QA_CITATION_PROMPT def get_prompt(self, question, evidence, evidence_mode: int): """Prepare the prompt and other information for LLM""" prompt_template = PromptTemplate(self.qa_citation_template) prompt = prompt_template.populate( context=evidence, question=question, safe=False, ) return prompt, evidence def answer_to_citations(self, answer) -> list[InlineEvidence]: citations: list[InlineEvidence] = [] lines = answer.split("\n") current_evidence = None for line in lines: # check citation idx using regex match = re.match(CITATION_PATTERN, line.lower()) if match: try: parsed_citation_idx = int(match.group(1)) except ValueError: parsed_citation_idx = None # conclude the current evidence if exists if current_evidence: citations.append(current_evidence) current_evidence = None current_evidence = InlineEvidence(idx=parsed_citation_idx) else: for keyword in [START_ANSWER_PATTERN, END_ANSWER_PATTERN]: if line.lower().startswith(keyword): matched_phrase = line[len(keyword) :].strip() if not current_evidence: current_evidence = InlineEvidence(idx=None) if keyword == START_ANSWER_PATTERN: current_evidence.start_phrase = matched_phrase else: current_evidence.end_phrase = matched_phrase break if ( current_evidence and current_evidence.end_phrase and current_evidence.start_phrase ): citations.append(current_evidence) current_evidence = None if current_evidence: citations.append(current_evidence) return citations def replace_citation_with_link(self, answer: str): # Define the regex pattern to match 【number】 pattern = r"【\d+】" alternate_pattern = r"\[\d+\]" # Regular expression to match merged citations multi_pattern = r"【([\d,\s]+)】" # Function to replace merged citations with independent ones def split_citations(match): # Extract the numbers, split by comma, and create individual citations numbers = match.group(1).split(",") return "".join(f"【{num.strip()}】" for num in numbers) # Replace merged citations in the text answer = re.sub(multi_pattern, split_citations, answer) # Find all citations in the answer matches = list(re.finditer(pattern, answer)) if not matches: matches = list(re.finditer(alternate_pattern, answer)) matched_citations = set() for match in matches: citation = match.group() matched_citations.add(citation) for citation in matched_citations: citation_id = citation[1:-1] answer = answer.replace( citation, ( "【{citation_id}】" ), ) answer = answer.replace(START_CITATION, "") return answer def stream( # type: ignore self, question: str, evidence: str, evidence_mode: int = 0, images: list[str] = [], **kwargs, ) -> Generator[Document, None, Document]: history = kwargs.get("history", []) print(f"Got {len(images)} images") # check if evidence exists, use QA prompt if evidence: prompt, evidence = self.get_prompt(question, evidence, evidence_mode) else: prompt = question output = "" logprobs = [] citation = None mindmap = None def mindmap_call(): nonlocal mindmap mindmap = self.create_mindmap_pipeline(context=evidence, question=question) mindmap_thread = None # execute function call in thread if evidence: if self.enable_mindmap: mindmap_thread = threading.Thread(target=mindmap_call) mindmap_thread.start() messages = [] if self.system_prompt: messages.append(SystemMessage(content=self.system_prompt)) for human, ai in history[-self.n_last_interactions :]: messages.append(HumanMessage(content=human)) messages.append(AIMessage(content=ai)) if self.use_multimodal and evidence_mode == EVIDENCE_MODE_FIGURE: # create image message: messages.append( HumanMessage( content=[ {"type": "text", "text": prompt}, ] + [ { "type": "image_url", "image_url": {"url": image}, } for image in images[:MAX_IMAGES] ], ) ) else: # append main prompt messages.append(HumanMessage(content=prompt)) final_answer = "" try: # try streaming first print("Trying LLM streaming") for out_msg in self.llm.stream(messages): if evidence: if START_ANSWER in output: if not final_answer: try: left_over_answer = output.split(START_ANSWER)[ 1 ].lstrip() except IndexError: left_over_answer = "" if left_over_answer: out_msg.text = left_over_answer + out_msg.text final_answer += ( out_msg.text.lstrip() if not final_answer else out_msg.text ) yield Document(channel="chat", content=out_msg.text) # check for the edge case of citation list is repeated # with smaller LLMs if START_CITATION in out_msg.text: break else: yield Document(channel="chat", content=out_msg.text) output += out_msg.text logprobs += out_msg.logprobs except NotImplementedError: print("Streaming is not supported, falling back to normal processing") output = self.llm(messages).text yield Document(channel="chat", content=output) if logprobs: qa_score = np.exp(np.average(logprobs)) else: qa_score = None citation = self.answer_to_citations(output) if mindmap_thread: mindmap_thread.join(timeout=CITATION_TIMEOUT) # convert citation to link answer = Document( text=final_answer, metadata={ "citation_viz": self.enable_citation_viz, "mindmap": mindmap, "citation": citation, "qa_score": qa_score, }, ) # yield the final answer final_answer = self.replace_citation_with_link(final_answer) if final_answer: yield Document(channel="chat", content=None) yield Document(channel="chat", content=final_answer) return answer def match_evidence_with_context(self, answer, docs) -> dict[str, list[dict]]: """Match the evidence with the context""" spans: dict[str, list[dict]] = defaultdict(list) if not answer.metadata["citation"]: return spans evidences = answer.metadata["citation"] for e_id, evidence in enumerate(evidences): start_phrase, end_phrase = evidence.start_phrase, evidence.end_phrase evidence_idx = evidence.idx if evidence_idx is None: evidence_idx = e_id + 1 best_match = None best_match_length = 0 best_match_doc_idx = None for doc in docs: match, match_length = find_start_end_phrase( start_phrase, end_phrase, doc.text ) if best_match is None or ( match is not None and match_length > best_match_length ): best_match = match best_match_length = match_length best_match_doc_idx = doc.doc_id if best_match is not None and best_match_doc_idx is not None: spans[best_match_doc_idx].append( { "start": best_match[0], "end": best_match[1], "idx": evidence_idx, } ) return spans` | ### get\_prompt [¶](#indices.qa.citation_qa_inline.AnswerWithInlineCitation.get_prompt "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `get_prompt(question, evidence, evidence_mode)` | Prepare the prompt and other information for LLM Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa_inline.py` | | | | --- | --- | | [91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101) | `def get_prompt(self, question, evidence, evidence_mode: int): """Prepare the prompt and other information for LLM""" prompt_template = PromptTemplate(self.qa_citation_template) prompt = prompt_template.populate( context=evidence, question=question, safe=False, ) return prompt, evidence` | ### match\_evidence\_with\_context [¶](#indices.qa.citation_qa_inline.AnswerWithInlineCitation.match_evidence_with_context "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `match_evidence_with_context(answer, docs)` | Match the evidence with the context Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa_inline.py` | | | | --- | --- | | [322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345)
[346](#__codelineno-0-346)
[347](#__codelineno-0-347)
[348](#__codelineno-0-348)
[349](#__codelineno-0-349)
[350](#__codelineno-0-350)
[351](#__codelineno-0-351)
[352](#__codelineno-0-352)
[353](#__codelineno-0-353)
[354](#__codelineno-0-354)
[355](#__codelineno-0-355)
[356](#__codelineno-0-356)
[357](#__codelineno-0-357)
[358](#__codelineno-0-358)
[359](#__codelineno-0-359)
[360](#__codelineno-0-360)
[361](#__codelineno-0-361) | `def match_evidence_with_context(self, answer, docs) -> dict[str, list[dict]]: """Match the evidence with the context""" spans: dict[str, list[dict]] = defaultdict(list) if not answer.metadata["citation"]: return spans evidences = answer.metadata["citation"] for e_id, evidence in enumerate(evidences): start_phrase, end_phrase = evidence.start_phrase, evidence.end_phrase evidence_idx = evidence.idx if evidence_idx is None: evidence_idx = e_id + 1 best_match = None best_match_length = 0 best_match_doc_idx = None for doc in docs: match, match_length = find_start_end_phrase( start_phrase, end_phrase, doc.text ) if best_match is None or ( match is not None and match_length > best_match_length ): best_match = match best_match_length = match_length best_match_doc_idx = doc.doc_id if best_match is not None and best_match_doc_idx is not None: spans[best_match_doc_idx].append( { "start": best_match[0], "end": best_match[1], "idx": evidence_idx, } ) return spans` | Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/rankings/base.py "Edit this page") Base ==== BaseReranking [¶](#indices.rankings.base.BaseReranking "Permanent link") ------------------------------------------------------------------------- Bases: `BaseComponent` Source code in `libs/kotaemon/kotaemon/indices/rankings/base.py` | | | | --- | --- | | [8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13) | `class BaseReranking(BaseComponent): @abstractmethod def run(self, documents: list[Document], query: str) -> list[Document]: """Main method to transform list of documents (re-ranking, filtering, etc)""" ...` | ### run `abstractmethod` [¶](#indices.rankings.base.BaseReranking.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Main method to transform list of documents (re-ranking, filtering, etc) Source code in `libs/kotaemon/kotaemon/indices/rankings/base.py` | | | | --- | --- | | [9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13) | `@abstractmethod def run(self, documents: list[Document], query: str) -> list[Document]: """Main method to transform list of documents (re-ranking, filtering, etc)""" ...` | Back to top --- # Citation - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/qa/citation.py "Edit this page") Citation ======== CiteEvidence [¶](#indices.qa.citation.CiteEvidence "Permanent link") --------------------------------------------------------------------- Bases: `BaseModel` List of evidences (maximum 5) to support the answer. Source code in `libs/kotaemon/kotaemon/indices/qa/citation.py` | | | | --- | --- | | [10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19) | `class CiteEvidence(BaseModel): """List of evidences (maximum 5) to support the answer.""" evidences: List[str] = Field( ..., description=( "Each source should be a direct quote from the context, " "as a substring of the original content (max 15 words)." ), )` | CitationPipeline [¶](#indices.qa.citation.CitationPipeline "Permanent link") ----------------------------------------------------------------------------- Bases: `BaseComponent` Citation pipeline to extract cited evidences from source (based on input question) Source code in `libs/kotaemon/kotaemon/indices/qa/citation.py` | | | | --- | --- | | [22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98) | `class CitationPipeline(BaseComponent): """Citation pipeline to extract cited evidences from source (based on input question)""" llm: BaseLLM def run(self, context: str, question: str): return self.invoke(context, question) def prepare_llm(self, context: str, question: str): schema = CiteEvidence.schema() function = { "name": schema["title"], "description": schema["description"], "parameters": schema, } llm_kwargs = { "tools": [{"type": "function", "function": function}], "tool_choice": "required", "tools_pydantic": [CiteEvidence], } messages = [ SystemMessage( content=( "You are a world class algorithm to answer " "questions with correct and exact citations." ) ), HumanMessage( content=( "Answer question using the following context. " "Use the provided function CiteEvidence() to cite your sources." ) ), HumanMessage(content=context), HumanMessage(content=f"Question: {question}"), HumanMessage( content=( "Tips: Make sure to cite your sources, " "and use the exact words from the context." ) ), ] return messages, llm_kwargs def invoke(self, context: str, question: str): messages, llm_kwargs = self.prepare_llm(context, question) try: print("CitationPipeline: invoking LLM") llm_output = self.get_from_path("llm").invoke(messages, **llm_kwargs) print("CitationPipeline: finish invoking LLM") if not llm_output.additional_kwargs.get("tool_calls"): return None first_func = llm_output.additional_kwargs["tool_calls"][0] if "function" in first_func: # openai and cohere format function_output = first_func["function"]["arguments"] else: # anthropic format function_output = first_func["args"] print("CitationPipeline:", function_output) if isinstance(function_output, str): output = CiteEvidence.parse_raw(function_output) else: output = CiteEvidence.parse_obj(function_output) except Exception as e: print(e) return None return output async def ainvoke(self, context: str, question: str): raise NotImplementedError()` | Back to top --- # Format Context - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/qa/format_context.py "Edit this page") Format Context ============== PrepareEvidencePipeline [¶](#indices.qa.format_context.PrepareEvidencePipeline "Permanent link") ------------------------------------------------------------------------------------------------- Bases: `BaseComponent` Prepare the evidence text from the list of retrieved documents This step usually happens after `DocumentRetrievalPipeline`. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `trim_func` | | a callback function or a BaseComponent, that splits a large chunk of text into smaller ones. The first one will be retained. | _required_ | Source code in `libs/kotaemon/kotaemon/indices/qa/format_context.py` | | | | --- | --- | | [15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114) | ``class PrepareEvidencePipeline(BaseComponent): """Prepare the evidence text from the list of retrieved documents This step usually happens after `DocumentRetrievalPipeline`. Args: trim_func: a callback function or a BaseComponent, that splits a large chunk of text into smaller ones. The first one will be retained. """ max_context_length: int = 32000 trim_func: TokenSplitter \| None = None def run(self, docs: list[RetrievedDocument]) -> Document: evidence = "" images = [] table_found = 0 evidence_modes = [] evidence_trim_func = ( self.trim_func if self.trim_func else TokenSplitter( chunk_size=self.max_context_length, chunk_overlap=0, separator=" ", tokenizer=partial( tiktoken.encoding_for_model("gpt-3.5-turbo").encode, allowed_special=set(), disallowed_special="all", ), ) ) for _, retrieved_item in enumerate(docs): retrieved_content = "" page = retrieved_item.metadata.get("page_label", None) source = filename = retrieved_item.metadata.get("file_name", "-") if page: source += f" (Page {page})" if retrieved_item.metadata.get("type", "") == "table": evidence_modes.append(EVIDENCE_MODE_TABLE) if table_found < 5: retrieved_content = retrieved_item.metadata.get( "table_origin", retrieved_item.text ) if retrieved_content not in evidence: table_found += 1 evidence += ( f"
Table from {source}\n" + retrieved_content + "\n
" ) elif retrieved_item.metadata.get("type", "") == "chatbot": evidence_modes.append(EVIDENCE_MODE_CHATBOT) retrieved_content = retrieved_item.metadata["window"] evidence += ( f"
Chatbot scenario from {filename} (Row {page})\n" + retrieved_content + "\n
" ) elif retrieved_item.metadata.get("type", "") == "image": evidence_modes.append(EVIDENCE_MODE_FIGURE) retrieved_content = retrieved_item.metadata.get("image_origin", "") retrieved_caption = html.escape(retrieved_item.get_content()) evidence += ( f"
Figure from {source}\n" + "
" ) images.append(retrieved_content) else: if "window" in retrieved_item.metadata: retrieved_content = retrieved_item.metadata["window"] else: retrieved_content = retrieved_item.text retrieved_content = retrieved_content.replace("\n", " ") if retrieved_content not in evidence: evidence += ( f"
Content from {source}: " + retrieved_content + " \n
" ) # resolve evidence mode evidence_mode = EVIDENCE_MODE_TEXT if EVIDENCE_MODE_FIGURE in evidence_modes: evidence_mode = EVIDENCE_MODE_FIGURE elif EVIDENCE_MODE_TABLE in evidence_modes: evidence_mode = EVIDENCE_MODE_TABLE # trim context by trim_len print("len (original)", len(evidence)) if evidence: texts = evidence_trim_func([Document(text=evidence)]) evidence = texts[0].text print("len (trimmed)", len(evidence)) return Document(content=(evidence_mode, evidence, images))`` | Back to top --- # Rankings - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/rankings/__init__.py "Edit this page") Rankings ======== BaseReranking [¶](#indices.rankings.BaseReranking "Permanent link") -------------------------------------------------------------------- Bases: `BaseComponent` Source code in `libs/kotaemon/kotaemon/indices/rankings/base.py` | | | | --- | --- | | [8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13) | `class BaseReranking(BaseComponent): @abstractmethod def run(self, documents: list[Document], query: str) -> list[Document]: """Main method to transform list of documents (re-ranking, filtering, etc)""" ...` | ### run `abstractmethod` [¶](#indices.rankings.BaseReranking.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Main method to transform list of documents (re-ranking, filtering, etc) Source code in `libs/kotaemon/kotaemon/indices/rankings/base.py` | | | | --- | --- | | [9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13) | `@abstractmethod def run(self, documents: list[Document], query: str) -> list[Document]: """Main method to transform list of documents (re-ranking, filtering, etc)""" ...` | CohereReranking [¶](#indices.rankings.CohereReranking "Permanent link") ------------------------------------------------------------------------ Bases: `BaseReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/cohere.py` | | | | --- | --- | | [10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60) | ``class CohereReranking(BaseReranking): model_name: str = "rerank-multilingual-v2.0" cohere_api_key: str = config("COHERE_API_KEY", "") use_key_from_ktem: bool = False def run(self, documents: list[Document], query: str) -> list[Document]: """Use Cohere Reranker model to re-order documents with their relevance score""" try: import cohere except ImportError: raise ImportError( "Please install Cohere `pip install cohere` to use Cohere Reranking" ) # try to get COHERE_API_KEY from embeddings if not self.cohere_api_key and self.use_key_from_ktem: try: from ktem.embeddings.manager import ( embedding_models_manager as embeddings, ) cohere_model = embeddings.get("cohere") ktem_cohere_api_key = cohere_model._kwargs.get( # type: ignore "cohere_api_key" ) if ktem_cohere_api_key != "your-key": self.cohere_api_key = ktem_cohere_api_key except Exception as e: print("Cannot get Cohere API key from `ktem`", e) if not self.cohere_api_key: print("Cohere API key not found. Skipping rerankings.") return documents cohere_client = cohere.Client(self.cohere_api_key) compressed_docs: list[Document] = [] if not documents: # to avoid empty api call return compressed_docs _docs = [d.content for d in documents] response = cohere_client.rerank( model=self.model_name, query=query, documents=_docs ) for r in response.results: doc = documents[r.index] doc.metadata["reranking_score"] = r.relevance_score compressed_docs.append(doc) return compressed_docs`` | ### run [¶](#indices.rankings.CohereReranking.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Use Cohere Reranker model to re-order documents with their relevance score Source code in `libs/kotaemon/kotaemon/indices/rankings/cohere.py` | | | | --- | --- | | [15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60) | ``def run(self, documents: list[Document], query: str) -> list[Document]: """Use Cohere Reranker model to re-order documents with their relevance score""" try: import cohere except ImportError: raise ImportError( "Please install Cohere `pip install cohere` to use Cohere Reranking" ) # try to get COHERE_API_KEY from embeddings if not self.cohere_api_key and self.use_key_from_ktem: try: from ktem.embeddings.manager import ( embedding_models_manager as embeddings, ) cohere_model = embeddings.get("cohere") ktem_cohere_api_key = cohere_model._kwargs.get( # type: ignore "cohere_api_key" ) if ktem_cohere_api_key != "your-key": self.cohere_api_key = ktem_cohere_api_key except Exception as e: print("Cannot get Cohere API key from `ktem`", e) if not self.cohere_api_key: print("Cohere API key not found. Skipping rerankings.") return documents cohere_client = cohere.Client(self.cohere_api_key) compressed_docs: list[Document] = [] if not documents: # to avoid empty api call return compressed_docs _docs = [d.content for d in documents] response = cohere_client.rerank( model=self.model_name, query=query, documents=_docs ) for r in response.results: doc = documents[r.index] doc.metadata["reranking_score"] = r.relevance_score compressed_docs.append(doc) return compressed_docs`` | LLMReranking [¶](#indices.rankings.LLMReranking "Permanent link") ------------------------------------------------------------------ Bases: `BaseReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/llm.py` | | | | --- | --- | | [23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66) | `class LLMReranking(BaseReranking): llm: BaseLLM prompt_template: PromptTemplate = PromptTemplate(template=RERANK_PROMPT_TEMPLATE) top_k: int = 3 concurrent: bool = True def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt).text)) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt).text) # use Boolean parser to extract relevancy output from LLM results = [output_parser.parse(result) for result in results] for include_doc, doc in zip(results, documents): if include_doc: filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | ### run [¶](#indices.rankings.LLMReranking.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Filter down documents based on their relevance to the query. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm.py` | | | | --- | --- | | [29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66) | `def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt).text)) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt).text) # use Boolean parser to extract relevancy output from LLM results = [output_parser.parse(result) for result in results] for include_doc, doc in zip(results, documents): if include_doc: filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | LLMScoring [¶](#indices.rankings.LLMScoring "Permanent link") -------------------------------------------------------------- Bases: `LLMReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_scoring.py` | | | | --- | --- | | [13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54) | `class LLMScoring(LLMReranking): def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs: list[Document] = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt))) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt)) for result, doc in zip(results, documents): score = np.exp(np.average(result.logprobs)) include_doc = output_parser.parse(result.text) if include_doc: doc.metadata["llm_reranking_score"] = score else: doc.metadata["llm_reranking_score"] = 1 - score filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | ### run [¶](#indices.rankings.LLMScoring.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Filter down documents based on their relevance to the query. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_scoring.py` | | | | --- | --- | | [14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54) | `def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs: list[Document] = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt))) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt)) for result, doc in zip(results, documents): score = np.exp(np.average(result.logprobs)) include_doc = output_parser.parse(result.text) if include_doc: doc.metadata["llm_reranking_score"] = score else: doc.metadata["llm_reranking_score"] = 1 - score filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | LLMTrulensScoring [¶](#indices.rankings.LLMTrulensScoring "Permanent link") ---------------------------------------------------------------------------- Bases: `LLMReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_trulens.py` | | | | --- | --- | | [96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182) | `class LLMTrulensScoring(LLMReranking): llm: BaseLLM system_prompt_template: PromptTemplate = SYSTEM_PROMPT_TEMPLATE user_prompt_template: PromptTemplate = USER_PROMPT_TEMPLATE concurrent: bool = True normalize: float = 10 trim_func: TokenSplitter = TokenSplitter.withx( chunk_size=MAX_CONTEXT_LEN, chunk_overlap=0, separator=" ", tokenizer=partial( tiktoken.encoding_for_model("gpt-3.5-turbo").encode, allowed_special=set(), disallowed_special="all", ), ) def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] documents = sorted(documents, key=lambda doc: doc.get_content()) if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: chunked_doc_content = self.trim_func( [ Document(content=doc.get_content()) # skip metadata which cause troubles ] )[0].text messages = [] messages.append( SystemMessage(self.system_prompt_template.populate()) ) messages.append( HumanMessage( self.user_prompt_template.populate( question=query, context=chunked_doc_content ) ) ) def llm_call(): return self.llm(messages).text futures.append(executor.submit(llm_call)) results = [future.result() for future in futures] else: results = [] for doc in documents: messages = [] messages.append(SystemMessage(self.system_prompt_template.populate())) messages.append( SystemMessage( self.user_prompt_template.populate( question=query, context=doc.get_content() ) ) ) results.append(self.llm(messages).text) # use Boolean parser to extract relevancy output from LLM results = [ (r_idx, float(re_0_10_rating(result)) / self.normalize) for r_idx, result in enumerate(results) ] results.sort(key=lambda x: x[1], reverse=True) for r_idx, score in results: doc = documents[r_idx] doc.metadata["llm_trulens_score"] = score filtered_docs.append(doc) print( "LLM rerank scores", [doc.metadata["llm_trulens_score"] for doc in filtered_docs], ) return filtered_docs` | ### run [¶](#indices.rankings.LLMTrulensScoring.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Filter down documents based on their relevance to the query. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_trulens.py` | | | | --- | --- | | [113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182) | `def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] documents = sorted(documents, key=lambda doc: doc.get_content()) if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: chunked_doc_content = self.trim_func( [ Document(content=doc.get_content()) # skip metadata which cause troubles ] )[0].text messages = [] messages.append( SystemMessage(self.system_prompt_template.populate()) ) messages.append( HumanMessage( self.user_prompt_template.populate( question=query, context=chunked_doc_content ) ) ) def llm_call(): return self.llm(messages).text futures.append(executor.submit(llm_call)) results = [future.result() for future in futures] else: results = [] for doc in documents: messages = [] messages.append(SystemMessage(self.system_prompt_template.populate())) messages.append( SystemMessage( self.user_prompt_template.populate( question=query, context=doc.get_content() ) ) ) results.append(self.llm(messages).text) # use Boolean parser to extract relevancy output from LLM results = [ (r_idx, float(re_0_10_rating(result)) / self.normalize) for r_idx, result in enumerate(results) ] results.sort(key=lambda x: x[1], reverse=True) for r_idx, score in results: doc = documents[r_idx] doc.metadata["llm_trulens_score"] = score filtered_docs.append(doc) print( "LLM rerank scores", [doc.metadata["llm_trulens_score"] for doc in filtered_docs], ) return filtered_docs` | Back to top --- # Citation Qa - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/qa/citation_qa.py "Edit this page") Citation Qa =========== AnswerWithContextPipeline [¶](#indices.qa.citation_qa.AnswerWithContextPipeline "Permanent link") -------------------------------------------------------------------------------------------------- Bases: `BaseComponent` Answer the question based on the evidence Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `llm` | | the language model to generate the answer | _required_ | | `citation_pipeline` | | generates citation from the evidence | _required_ | | `qa_template` | | the prompt template for LLM to generate answer (refer to evidence\_mode) | _required_ | | `qa_table_template` | | the prompt template for LLM to generate answer for table (refer to evidence\_mode) | _required_ | | `qa_chatbot_template` | | the prompt template for LLM to generate answer for pre-made scenarios (refer to evidence\_mode) | _required_ | | `lang` | | the language of the answer. Currently support English and Japanese | _required_ | Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa.py` | | | | --- | --- | | [80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185)
[186](#__codelineno-0-186)
[187](#__codelineno-0-187)
[188](#__codelineno-0-188)
[189](#__codelineno-0-189)
[190](#__codelineno-0-190)
[191](#__codelineno-0-191)
[192](#__codelineno-0-192)
[193](#__codelineno-0-193)
[194](#__codelineno-0-194)
[195](#__codelineno-0-195)
[196](#__codelineno-0-196)
[197](#__codelineno-0-197)
[198](#__codelineno-0-198)
[199](#__codelineno-0-199)
[200](#__codelineno-0-200)
[201](#__codelineno-0-201)
[202](#__codelineno-0-202)
[203](#__codelineno-0-203)
[204](#__codelineno-0-204)
[205](#__codelineno-0-205)
[206](#__codelineno-0-206)
[207](#__codelineno-0-207)
[208](#__codelineno-0-208)
[209](#__codelineno-0-209)
[210](#__codelineno-0-210)
[211](#__codelineno-0-211)
[212](#__codelineno-0-212)
[213](#__codelineno-0-213)
[214](#__codelineno-0-214)
[215](#__codelineno-0-215)
[216](#__codelineno-0-216)
[217](#__codelineno-0-217)
[218](#__codelineno-0-218)
[219](#__codelineno-0-219)
[220](#__codelineno-0-220)
[221](#__codelineno-0-221)
[222](#__codelineno-0-222)
[223](#__codelineno-0-223)
[224](#__codelineno-0-224)
[225](#__codelineno-0-225)
[226](#__codelineno-0-226)
[227](#__codelineno-0-227)
[228](#__codelineno-0-228)
[229](#__codelineno-0-229)
[230](#__codelineno-0-230)
[231](#__codelineno-0-231)
[232](#__codelineno-0-232)
[233](#__codelineno-0-233)
[234](#__codelineno-0-234)
[235](#__codelineno-0-235)
[236](#__codelineno-0-236)
[237](#__codelineno-0-237)
[238](#__codelineno-0-238)
[239](#__codelineno-0-239)
[240](#__codelineno-0-240)
[241](#__codelineno-0-241)
[242](#__codelineno-0-242)
[243](#__codelineno-0-243)
[244](#__codelineno-0-244)
[245](#__codelineno-0-245)
[246](#__codelineno-0-246)
[247](#__codelineno-0-247)
[248](#__codelineno-0-248)
[249](#__codelineno-0-249)
[250](#__codelineno-0-250)
[251](#__codelineno-0-251)
[252](#__codelineno-0-252)
[253](#__codelineno-0-253)
[254](#__codelineno-0-254)
[255](#__codelineno-0-255)
[256](#__codelineno-0-256)
[257](#__codelineno-0-257)
[258](#__codelineno-0-258)
[259](#__codelineno-0-259)
[260](#__codelineno-0-260)
[261](#__codelineno-0-261)
[262](#__codelineno-0-262)
[263](#__codelineno-0-263)
[264](#__codelineno-0-264)
[265](#__codelineno-0-265)
[266](#__codelineno-0-266)
[267](#__codelineno-0-267)
[268](#__codelineno-0-268)
[269](#__codelineno-0-269)
[270](#__codelineno-0-270)
[271](#__codelineno-0-271)
[272](#__codelineno-0-272)
[273](#__codelineno-0-273)
[274](#__codelineno-0-274)
[275](#__codelineno-0-275)
[276](#__codelineno-0-276)
[277](#__codelineno-0-277)
[278](#__codelineno-0-278)
[279](#__codelineno-0-279)
[280](#__codelineno-0-280)
[281](#__codelineno-0-281)
[282](#__codelineno-0-282)
[283](#__codelineno-0-283)
[284](#__codelineno-0-284)
[285](#__codelineno-0-285)
[286](#__codelineno-0-286)
[287](#__codelineno-0-287)
[288](#__codelineno-0-288)
[289](#__codelineno-0-289)
[290](#__codelineno-0-290)
[291](#__codelineno-0-291)
[292](#__codelineno-0-292)
[293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310)
[311](#__codelineno-0-311)
[312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317)
[318](#__codelineno-0-318)
[319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345)
[346](#__codelineno-0-346)
[347](#__codelineno-0-347)
[348](#__codelineno-0-348)
[349](#__codelineno-0-349)
[350](#__codelineno-0-350)
[351](#__codelineno-0-351)
[352](#__codelineno-0-352)
[353](#__codelineno-0-353)
[354](#__codelineno-0-354)
[355](#__codelineno-0-355)
[356](#__codelineno-0-356)
[357](#__codelineno-0-357)
[358](#__codelineno-0-358)
[359](#__codelineno-0-359)
[360](#__codelineno-0-360)
[361](#__codelineno-0-361)
[362](#__codelineno-0-362)
[363](#__codelineno-0-363)
[364](#__codelineno-0-364)
[365](#__codelineno-0-365)
[366](#__codelineno-0-366)
[367](#__codelineno-0-367)
[368](#__codelineno-0-368)
[369](#__codelineno-0-369)
[370](#__codelineno-0-370)
[371](#__codelineno-0-371)
[372](#__codelineno-0-372)
[373](#__codelineno-0-373)
[374](#__codelineno-0-374)
[375](#__codelineno-0-375)
[376](#__codelineno-0-376)
[377](#__codelineno-0-377)
[378](#__codelineno-0-378)
[379](#__codelineno-0-379)
[380](#__codelineno-0-380)
[381](#__codelineno-0-381)
[382](#__codelineno-0-382)
[383](#__codelineno-0-383)
[384](#__codelineno-0-384)
[385](#__codelineno-0-385)
[386](#__codelineno-0-386)
[387](#__codelineno-0-387)
[388](#__codelineno-0-388)
[389](#__codelineno-0-389)
[390](#__codelineno-0-390)
[391](#__codelineno-0-391)
[392](#__codelineno-0-392)
[393](#__codelineno-0-393)
[394](#__codelineno-0-394)
[395](#__codelineno-0-395)
[396](#__codelineno-0-396)
[397](#__codelineno-0-397)
[398](#__codelineno-0-398) | `class AnswerWithContextPipeline(BaseComponent): """Answer the question based on the evidence Args: llm: the language model to generate the answer citation_pipeline: generates citation from the evidence qa_template: the prompt template for LLM to generate answer (refer to evidence_mode) qa_table_template: the prompt template for LLM to generate answer for table (refer to evidence_mode) qa_chatbot_template: the prompt template for LLM to generate answer for pre-made scenarios (refer to evidence_mode) lang: the language of the answer. Currently support English and Japanese """ llm: ChatLLM = Node(default_callback=lambda _: llms.get_default()) vlm_endpoint: str = getattr(flowsettings, "KH_VLM_ENDPOINT", "") use_multimodal: bool = getattr(flowsettings, "KH_REASONINGS_USE_MULTIMODAL", True) citation_pipeline: CitationPipeline = Node( default_callback=lambda _: CitationPipeline(llm=llms.get_default()) ) create_mindmap_pipeline: CreateMindmapPipeline = Node( default_callback=lambda _: CreateMindmapPipeline(llm=llms.get_default()) ) qa_template: str = DEFAULT_QA_TEXT_PROMPT qa_table_template: str = DEFAULT_QA_TABLE_PROMPT qa_chatbot_template: str = DEFAULT_QA_CHATBOT_PROMPT qa_figure_template: str = DEFAULT_QA_FIGURE_PROMPT enable_citation: bool = False enable_mindmap: bool = False enable_citation_viz: bool = False system_prompt: str = "" lang: str = "English" # support English and Japanese n_last_interactions: int = 5 def get_prompt(self, question, evidence, evidence_mode: int): """Prepare the prompt and other information for LLM""" if evidence_mode == EVIDENCE_MODE_TEXT: prompt_template = PromptTemplate(self.qa_template) elif evidence_mode == EVIDENCE_MODE_TABLE: prompt_template = PromptTemplate(self.qa_table_template) elif evidence_mode == EVIDENCE_MODE_FIGURE: if self.use_multimodal: prompt_template = PromptTemplate(self.qa_figure_template) else: prompt_template = PromptTemplate(self.qa_template) else: prompt_template = PromptTemplate(self.qa_chatbot_template) prompt = prompt_template.populate( context=evidence, question=question, lang=self.lang, ) return prompt, evidence def run( self, question: str, evidence: str, evidence_mode: int = 0, **kwargs ) -> Document: return self.invoke(question, evidence, evidence_mode, **kwargs) def invoke( self, question: str, evidence: str, evidence_mode: int = 0, images: list[str] = [], **kwargs, ) -> Document: raise NotImplementedError async def ainvoke( # type: ignore self, question: str, evidence: str, evidence_mode: int = 0, images: list[str] = [], **kwargs, ) -> Document: """Answer the question based on the evidence In addition to the question and the evidence, this method also take into account evidence_mode. The evidence_mode tells which kind of evidence is. The kind of evidence affects: 1. How the evidence is represented. 2. The prompt to generate the answer. By default, the evidence_mode is 0, which means the evidence is plain text with no particular semantic representation. The evidence_mode can be: 1. "table": There will be HTML markup telling that there is a table within the evidence. 2. "chatbot": There will be HTML markup telling that there is a chatbot. This chatbot is a scenario, extracted from an Excel file, where each row corresponds to an interaction. Args: question: the original question posed by user evidence: the text that contain relevant information to answer the question (determined by retrieval pipeline) evidence_mode: the mode of evidence, 0 for text, 1 for table, 2 for chatbot """ raise NotImplementedError def stream( # type: ignore self, question: str, evidence: str, evidence_mode: int = 0, images: list[str] = [], **kwargs, ) -> Generator[Document, None, Document]: history = kwargs.get("history", []) print(f"Got {len(images)} images") # check if evidence exists, use QA prompt if evidence: prompt, evidence = self.get_prompt(question, evidence, evidence_mode) else: prompt = question # retrieve the citation citation = None mindmap = None def citation_call(): nonlocal citation citation = self.citation_pipeline(context=evidence, question=question) def mindmap_call(): nonlocal mindmap mindmap = self.create_mindmap_pipeline(context=evidence, question=question) citation_thread = None mindmap_thread = None # execute function call in thread if evidence: if self.enable_citation: citation_thread = threading.Thread(target=citation_call) citation_thread.start() if self.enable_mindmap: mindmap_thread = threading.Thread(target=mindmap_call) mindmap_thread.start() output = "" logprobs = [] messages = [] if self.system_prompt: messages.append(SystemMessage(content=self.system_prompt)) for human, ai in history[-self.n_last_interactions :]: messages.append(HumanMessage(content=human)) messages.append(AIMessage(content=ai)) if self.use_multimodal and evidence_mode == EVIDENCE_MODE_FIGURE: # create image message: messages.append( HumanMessage( content=[ {"type": "text", "text": prompt}, ] + [ { "type": "image_url", "image_url": {"url": image}, } for image in images[:MAX_IMAGES] ], ) ) else: # append main prompt messages.append(HumanMessage(content=prompt)) try: # try streaming first print("Trying LLM streaming") for out_msg in self.llm.stream(messages): output += out_msg.text logprobs += out_msg.logprobs yield Document(channel="chat", content=out_msg.text) except NotImplementedError: print("Streaming is not supported, falling back to normal processing") output = self.llm(messages).text yield Document(channel="chat", content=output) if logprobs: qa_score = np.exp(np.average(logprobs)) else: qa_score = None if citation_thread: citation_thread.join(timeout=CITATION_TIMEOUT) if mindmap_thread: mindmap_thread.join(timeout=CITATION_TIMEOUT) answer = Document( text=output, metadata={ "citation_viz": self.enable_citation_viz, "mindmap": mindmap, "citation": citation, "qa_score": qa_score, }, ) return answer def match_evidence_with_context(self, answer, docs) -> dict[str, list[dict]]: """Match the evidence with the context""" spans: dict[str, list[dict]] = defaultdict(list) if not answer.metadata["citation"]: return spans evidences = answer.metadata["citation"].evidences for quote in evidences: matched_excerpts = [] for doc in docs: matches = find_text(quote, doc.text) for start, end in matches: if "\|" not in doc.text[start:end]: spans[doc.doc_id].append( { "start": start, "end": end, } ) matched_excerpts.append(doc.text[start:end]) # print("Matched citation:", quote, matched_excerpts), return spans def prepare_citations(self, answer, docs) -> tuple[list[Document], list[Document]]: """Prepare the citations to show on the UI""" with_citation, without_citation = [], [] has_llm_score = any("llm_trulens_score" in doc.metadata for doc in docs) spans = self.match_evidence_with_context(answer, docs) id2docs = {doc.doc_id: doc for doc in docs} not_detected = set(id2docs.keys()) - set(spans.keys()) # render highlight spans for _id, ss in spans.items(): if not ss: not_detected.add(_id) continue cur_doc = id2docs[_id] highlight_text = "" ss = sorted(ss, key=lambda x: x["start"]) last_end = 0 text = cur_doc.text[: ss[0]["start"]] for idx, span in enumerate(ss): # prevent overlapping between span span_start = max(last_end, span["start"]) span_end = max(last_end, span["end"]) to_highlight = cur_doc.text[span_start:span_end] last_end = span_end # append to highlight on PDF viewer highlight_text += (" " if highlight_text else "") + to_highlight span_idx = span.get("idx", None) if span_idx is not None: to_highlight = f"【{span_idx}】" + to_highlight text += Render.highlight( to_highlight, elem_id=str(span_idx) if span_idx is not None else None, ) if idx < len(ss) - 1: text += cur_doc.text[span["end"] : ss[idx + 1]["start"]] text += cur_doc.text[ss[-1]["end"] :] # add to display list with_citation.append( Document( channel="info", content=Render.collapsible_with_header_score( cur_doc, override_text=text, highlight_text=highlight_text, open_collapsible=True, ), ) ) print("Got {} cited docs".format(len(with_citation))) sorted_not_detected_items_with_scores = [ (id_, id2docs[id_].metadata.get("llm_trulens_score", 0.0)) for id_ in not_detected ] sorted_not_detected_items_with_scores.sort(key=lambda x: x[1], reverse=True) for id_, _ in sorted_not_detected_items_with_scores: doc = id2docs[id_] doc_score = doc.metadata.get("llm_trulens_score", 0.0) is_open = not has_llm_score or ( doc_score > CONTEXT_RELEVANT_WARNING_SCORE and len(with_citation) == 0 ) without_citation.append( Document( channel="info", content=Render.collapsible_with_header_score( doc, open_collapsible=is_open ), ) ) return with_citation, without_citation` | ### get\_prompt [¶](#indices.qa.citation_qa.AnswerWithContextPipeline.get_prompt "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `get_prompt(question, evidence, evidence_mode)` | Prepare the prompt and other information for LLM Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa.py` | | | | --- | --- | | [118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138) | `def get_prompt(self, question, evidence, evidence_mode: int): """Prepare the prompt and other information for LLM""" if evidence_mode == EVIDENCE_MODE_TEXT: prompt_template = PromptTemplate(self.qa_template) elif evidence_mode == EVIDENCE_MODE_TABLE: prompt_template = PromptTemplate(self.qa_table_template) elif evidence_mode == EVIDENCE_MODE_FIGURE: if self.use_multimodal: prompt_template = PromptTemplate(self.qa_figure_template) else: prompt_template = PromptTemplate(self.qa_template) else: prompt_template = PromptTemplate(self.qa_chatbot_template) prompt = prompt_template.populate( context=evidence, question=question, lang=self.lang, ) return prompt, evidence` | ### ainvoke `async` [¶](#indices.qa.citation_qa.AnswerWithContextPipeline.ainvoke "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3) | `ainvoke( question, evidence, evidence_mode=0, images=[], **kwargs )` | Answer the question based on the evidence In addition to the question and the evidence, this method also take into account evidence\_mode. The evidence\_mode tells which kind of evidence is. The kind of evidence affects: 1. How the evidence is represented. 2. The prompt to generate the answer. By default, the evidence\_mode is 0, which means the evidence is plain text with no particular semantic representation. The evidence\_mode can be: 1. "table": There will be HTML markup telling that there is a table within the evidence. 2. "chatbot": There will be HTML markup telling that there is a chatbot. This chatbot is a scenario, extracted from an Excel file, where each row corresponds to an interaction. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `question` | `str` | the original question posed by user | _required_ | | `evidence` | `str` | the text that contain relevant information to answer the question (determined by retrieval pipeline) | _required_ | | `evidence_mode` | `int` | the mode of evidence, 0 for text, 1 for table, 2 for chatbot | `0` | Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa.py` | | | | --- | --- | | [155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182)
[183](#__codelineno-0-183)
[184](#__codelineno-0-184)
[185](#__codelineno-0-185) | `async def ainvoke( # type: ignore self, question: str, evidence: str, evidence_mode: int = 0, images: list[str] = [], **kwargs, ) -> Document: """Answer the question based on the evidence In addition to the question and the evidence, this method also take into account evidence_mode. The evidence_mode tells which kind of evidence is. The kind of evidence affects: 1. How the evidence is represented. 2. The prompt to generate the answer. By default, the evidence_mode is 0, which means the evidence is plain text with no particular semantic representation. The evidence_mode can be: 1. "table": There will be HTML markup telling that there is a table within the evidence. 2. "chatbot": There will be HTML markup telling that there is a chatbot. This chatbot is a scenario, extracted from an Excel file, where each row corresponds to an interaction. Args: question: the original question posed by user evidence: the text that contain relevant information to answer the question (determined by retrieval pipeline) evidence_mode: the mode of evidence, 0 for text, 1 for table, 2 for chatbot """ raise NotImplementedError` | ### match\_evidence\_with\_context [¶](#indices.qa.citation_qa.AnswerWithContextPipeline.match_evidence_with_context "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `match_evidence_with_context(answer, docs)` | Match the evidence with the context Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa.py` | | | | --- | --- | | [293](#__codelineno-0-293)
[294](#__codelineno-0-294)
[295](#__codelineno-0-295)
[296](#__codelineno-0-296)
[297](#__codelineno-0-297)
[298](#__codelineno-0-298)
[299](#__codelineno-0-299)
[300](#__codelineno-0-300)
[301](#__codelineno-0-301)
[302](#__codelineno-0-302)
[303](#__codelineno-0-303)
[304](#__codelineno-0-304)
[305](#__codelineno-0-305)
[306](#__codelineno-0-306)
[307](#__codelineno-0-307)
[308](#__codelineno-0-308)
[309](#__codelineno-0-309)
[310](#__codelineno-0-310)
[311](#__codelineno-0-311)
[312](#__codelineno-0-312)
[313](#__codelineno-0-313)
[314](#__codelineno-0-314)
[315](#__codelineno-0-315)
[316](#__codelineno-0-316)
[317](#__codelineno-0-317) | `def match_evidence_with_context(self, answer, docs) -> dict[str, list[dict]]: """Match the evidence with the context""" spans: dict[str, list[dict]] = defaultdict(list) if not answer.metadata["citation"]: return spans evidences = answer.metadata["citation"].evidences for quote in evidences: matched_excerpts = [] for doc in docs: matches = find_text(quote, doc.text) for start, end in matches: if "\|" not in doc.text[start:end]: spans[doc.doc_id].append( { "start": start, "end": end, } ) matched_excerpts.append(doc.text[start:end]) # print("Matched citation:", quote, matched_excerpts), return spans` | ### prepare\_citations [¶](#indices.qa.citation_qa.AnswerWithContextPipeline.prepare_citations "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `prepare_citations(answer, docs)` | Prepare the citations to show on the UI Source code in `libs/kotaemon/kotaemon/indices/qa/citation_qa.py` | | | | --- | --- | | [319](#__codelineno-0-319)
[320](#__codelineno-0-320)
[321](#__codelineno-0-321)
[322](#__codelineno-0-322)
[323](#__codelineno-0-323)
[324](#__codelineno-0-324)
[325](#__codelineno-0-325)
[326](#__codelineno-0-326)
[327](#__codelineno-0-327)
[328](#__codelineno-0-328)
[329](#__codelineno-0-329)
[330](#__codelineno-0-330)
[331](#__codelineno-0-331)
[332](#__codelineno-0-332)
[333](#__codelineno-0-333)
[334](#__codelineno-0-334)
[335](#__codelineno-0-335)
[336](#__codelineno-0-336)
[337](#__codelineno-0-337)
[338](#__codelineno-0-338)
[339](#__codelineno-0-339)
[340](#__codelineno-0-340)
[341](#__codelineno-0-341)
[342](#__codelineno-0-342)
[343](#__codelineno-0-343)
[344](#__codelineno-0-344)
[345](#__codelineno-0-345)
[346](#__codelineno-0-346)
[347](#__codelineno-0-347)
[348](#__codelineno-0-348)
[349](#__codelineno-0-349)
[350](#__codelineno-0-350)
[351](#__codelineno-0-351)
[352](#__codelineno-0-352)
[353](#__codelineno-0-353)
[354](#__codelineno-0-354)
[355](#__codelineno-0-355)
[356](#__codelineno-0-356)
[357](#__codelineno-0-357)
[358](#__codelineno-0-358)
[359](#__codelineno-0-359)
[360](#__codelineno-0-360)
[361](#__codelineno-0-361)
[362](#__codelineno-0-362)
[363](#__codelineno-0-363)
[364](#__codelineno-0-364)
[365](#__codelineno-0-365)
[366](#__codelineno-0-366)
[367](#__codelineno-0-367)
[368](#__codelineno-0-368)
[369](#__codelineno-0-369)
[370](#__codelineno-0-370)
[371](#__codelineno-0-371)
[372](#__codelineno-0-372)
[373](#__codelineno-0-373)
[374](#__codelineno-0-374)
[375](#__codelineno-0-375)
[376](#__codelineno-0-376)
[377](#__codelineno-0-377)
[378](#__codelineno-0-378)
[379](#__codelineno-0-379)
[380](#__codelineno-0-380)
[381](#__codelineno-0-381)
[382](#__codelineno-0-382)
[383](#__codelineno-0-383)
[384](#__codelineno-0-384)
[385](#__codelineno-0-385)
[386](#__codelineno-0-386)
[387](#__codelineno-0-387)
[388](#__codelineno-0-388)
[389](#__codelineno-0-389)
[390](#__codelineno-0-390)
[391](#__codelineno-0-391)
[392](#__codelineno-0-392)
[393](#__codelineno-0-393)
[394](#__codelineno-0-394)
[395](#__codelineno-0-395)
[396](#__codelineno-0-396)
[397](#__codelineno-0-397)
[398](#__codelineno-0-398) | `def prepare_citations(self, answer, docs) -> tuple[list[Document], list[Document]]: """Prepare the citations to show on the UI""" with_citation, without_citation = [], [] has_llm_score = any("llm_trulens_score" in doc.metadata for doc in docs) spans = self.match_evidence_with_context(answer, docs) id2docs = {doc.doc_id: doc for doc in docs} not_detected = set(id2docs.keys()) - set(spans.keys()) # render highlight spans for _id, ss in spans.items(): if not ss: not_detected.add(_id) continue cur_doc = id2docs[_id] highlight_text = "" ss = sorted(ss, key=lambda x: x["start"]) last_end = 0 text = cur_doc.text[: ss[0]["start"]] for idx, span in enumerate(ss): # prevent overlapping between span span_start = max(last_end, span["start"]) span_end = max(last_end, span["end"]) to_highlight = cur_doc.text[span_start:span_end] last_end = span_end # append to highlight on PDF viewer highlight_text += (" " if highlight_text else "") + to_highlight span_idx = span.get("idx", None) if span_idx is not None: to_highlight = f"【{span_idx}】" + to_highlight text += Render.highlight( to_highlight, elem_id=str(span_idx) if span_idx is not None else None, ) if idx < len(ss) - 1: text += cur_doc.text[span["end"] : ss[idx + 1]["start"]] text += cur_doc.text[ss[-1]["end"] :] # add to display list with_citation.append( Document( channel="info", content=Render.collapsible_with_header_score( cur_doc, override_text=text, highlight_text=highlight_text, open_collapsible=True, ), ) ) print("Got {} cited docs".format(len(with_citation))) sorted_not_detected_items_with_scores = [ (id_, id2docs[id_].metadata.get("llm_trulens_score", 0.0)) for id_ in not_detected ] sorted_not_detected_items_with_scores.sort(key=lambda x: x[1], reverse=True) for id_, _ in sorted_not_detected_items_with_scores: doc = id2docs[id_] doc_score = doc.metadata.get("llm_trulens_score", 0.0) is_open = not has_llm_score or ( doc_score > CONTEXT_RELEVANT_WARNING_SCORE and len(with_citation) == 0 ) without_citation.append( Document( channel="info", content=Render.collapsible_with_header_score( doc, open_collapsible=is_open ), ) ) return with_citation, without_citation` | Back to top --- # Cohere - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/rankings/cohere.py "Edit this page") Cohere ====== CohereReranking [¶](#indices.rankings.cohere.CohereReranking "Permanent link") ------------------------------------------------------------------------------- Bases: `BaseReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/cohere.py` | | | | --- | --- | | [10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60) | ``class CohereReranking(BaseReranking): model_name: str = "rerank-multilingual-v2.0" cohere_api_key: str = config("COHERE_API_KEY", "") use_key_from_ktem: bool = False def run(self, documents: list[Document], query: str) -> list[Document]: """Use Cohere Reranker model to re-order documents with their relevance score""" try: import cohere except ImportError: raise ImportError( "Please install Cohere `pip install cohere` to use Cohere Reranking" ) # try to get COHERE_API_KEY from embeddings if not self.cohere_api_key and self.use_key_from_ktem: try: from ktem.embeddings.manager import ( embedding_models_manager as embeddings, ) cohere_model = embeddings.get("cohere") ktem_cohere_api_key = cohere_model._kwargs.get( # type: ignore "cohere_api_key" ) if ktem_cohere_api_key != "your-key": self.cohere_api_key = ktem_cohere_api_key except Exception as e: print("Cannot get Cohere API key from `ktem`", e) if not self.cohere_api_key: print("Cohere API key not found. Skipping rerankings.") return documents cohere_client = cohere.Client(self.cohere_api_key) compressed_docs: list[Document] = [] if not documents: # to avoid empty api call return compressed_docs _docs = [d.content for d in documents] response = cohere_client.rerank( model=self.model_name, query=query, documents=_docs ) for r in response.results: doc = documents[r.index] doc.metadata["reranking_score"] = r.relevance_score compressed_docs.append(doc) return compressed_docs`` | ### run [¶](#indices.rankings.cohere.CohereReranking.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Use Cohere Reranker model to re-order documents with their relevance score Source code in `libs/kotaemon/kotaemon/indices/rankings/cohere.py` | | | | --- | --- | | [15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60) | ``def run(self, documents: list[Document], query: str) -> list[Document]: """Use Cohere Reranker model to re-order documents with their relevance score""" try: import cohere except ImportError: raise ImportError( "Please install Cohere `pip install cohere` to use Cohere Reranking" ) # try to get COHERE_API_KEY from embeddings if not self.cohere_api_key and self.use_key_from_ktem: try: from ktem.embeddings.manager import ( embedding_models_manager as embeddings, ) cohere_model = embeddings.get("cohere") ktem_cohere_api_key = cohere_model._kwargs.get( # type: ignore "cohere_api_key" ) if ktem_cohere_api_key != "your-key": self.cohere_api_key = ktem_cohere_api_key except Exception as e: print("Cannot get Cohere API key from `ktem`", e) if not self.cohere_api_key: print("Cohere API key not found. Skipping rerankings.") return documents cohere_client = cohere.Client(self.cohere_api_key) compressed_docs: list[Document] = [] if not documents: # to avoid empty api call return compressed_docs _docs = [d.content for d in documents] response = cohere_client.rerank( model=self.model_name, query=query, documents=_docs ) for r in response.results: doc = documents[r.index] doc.metadata["reranking_score"] = r.relevance_score compressed_docs.append(doc) return compressed_docs`` | Back to top --- # Retrievers - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/retrievers/__init__.py "Edit this page") Retrievers ========== Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/llms/base.py "Edit this page") Base ==== Back to top --- # Llm - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/rankings/llm.py "Edit this page") Llm === LLMReranking [¶](#indices.rankings.llm.LLMReranking "Permanent link") ---------------------------------------------------------------------- Bases: `BaseReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/llm.py` | | | | --- | --- | | [23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66) | `class LLMReranking(BaseReranking): llm: BaseLLM prompt_template: PromptTemplate = PromptTemplate(template=RERANK_PROMPT_TEMPLATE) top_k: int = 3 concurrent: bool = True def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt).text)) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt).text) # use Boolean parser to extract relevancy output from LLM results = [output_parser.parse(result) for result in results] for include_doc, doc in zip(results, documents): if include_doc: filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | ### run [¶](#indices.rankings.llm.LLMReranking.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Filter down documents based on their relevance to the query. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm.py` | | | | --- | --- | | [29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66) | `def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt).text)) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt).text) # use Boolean parser to extract relevancy output from LLM results = [output_parser.parse(result) for result in results] for include_doc, doc in zip(results, documents): if include_doc: filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | Back to top --- # Llm Scoring - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/rankings/llm_scoring.py "Edit this page") Llm Scoring =========== LLMScoring [¶](#indices.rankings.llm_scoring.LLMScoring "Permanent link") -------------------------------------------------------------------------- Bases: `LLMReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_scoring.py` | | | | --- | --- | | [13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54) | `class LLMScoring(LLMReranking): def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs: list[Document] = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt))) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt)) for result, doc in zip(results, documents): score = np.exp(np.average(result.logprobs)) include_doc = output_parser.parse(result.text) if include_doc: doc.metadata["llm_reranking_score"] = score else: doc.metadata["llm_reranking_score"] = 1 - score filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | ### run [¶](#indices.rankings.llm_scoring.LLMScoring.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Filter down documents based on their relevance to the query. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_scoring.py` | | | | --- | --- | | [14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54) | `def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs: list[Document] = [] output_parser = BooleanOutputParser() if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) futures.append(executor.submit(lambda: self.llm(_prompt))) results = [future.result() for future in futures] else: results = [] for doc in documents: _prompt = self.prompt_template.populate( question=query, context=doc.get_content() ) results.append(self.llm(_prompt)) for result, doc in zip(results, documents): score = np.exp(np.average(result.logprobs)) include_doc = output_parser.parse(result.text) if include_doc: doc.metadata["llm_reranking_score"] = score else: doc.metadata["llm_reranking_score"] = 1 - score filtered_docs.append(doc) # prevent returning empty result if len(filtered_docs) == 0: filtered_docs = documents[: self.top_k] return filtered_docs` | Back to top --- # Llm Trulens - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/rankings/llm_trulens.py "Edit this page") Llm Trulens =========== PATTERN\_INTEGER `module-attribute` [¶](#indices.rankings.llm_trulens.PATTERN_INTEGER "Permanent link") -------------------------------------------------------------------------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `PATTERN_INTEGER = compile('([+-]?[1-9][0-9]*\|0)')` | Regex that matches integers. LLMTrulensScoring [¶](#indices.rankings.llm_trulens.LLMTrulensScoring "Permanent link") ---------------------------------------------------------------------------------------- Bases: `LLMReranking` Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_trulens.py` | | | | --- | --- | | [96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182) | `class LLMTrulensScoring(LLMReranking): llm: BaseLLM system_prompt_template: PromptTemplate = SYSTEM_PROMPT_TEMPLATE user_prompt_template: PromptTemplate = USER_PROMPT_TEMPLATE concurrent: bool = True normalize: float = 10 trim_func: TokenSplitter = TokenSplitter.withx( chunk_size=MAX_CONTEXT_LEN, chunk_overlap=0, separator=" ", tokenizer=partial( tiktoken.encoding_for_model("gpt-3.5-turbo").encode, allowed_special=set(), disallowed_special="all", ), ) def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] documents = sorted(documents, key=lambda doc: doc.get_content()) if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: chunked_doc_content = self.trim_func( [ Document(content=doc.get_content()) # skip metadata which cause troubles ] )[0].text messages = [] messages.append( SystemMessage(self.system_prompt_template.populate()) ) messages.append( HumanMessage( self.user_prompt_template.populate( question=query, context=chunked_doc_content ) ) ) def llm_call(): return self.llm(messages).text futures.append(executor.submit(llm_call)) results = [future.result() for future in futures] else: results = [] for doc in documents: messages = [] messages.append(SystemMessage(self.system_prompt_template.populate())) messages.append( SystemMessage( self.user_prompt_template.populate( question=query, context=doc.get_content() ) ) ) results.append(self.llm(messages).text) # use Boolean parser to extract relevancy output from LLM results = [ (r_idx, float(re_0_10_rating(result)) / self.normalize) for r_idx, result in enumerate(results) ] results.sort(key=lambda x: x[1], reverse=True) for r_idx, score in results: doc = documents[r_idx] doc.metadata["llm_trulens_score"] = score filtered_docs.append(doc) print( "LLM rerank scores", [doc.metadata["llm_trulens_score"] for doc in filtered_docs], ) return filtered_docs` | ### run [¶](#indices.rankings.llm_trulens.LLMTrulensScoring.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(documents, query)` | Filter down documents based on their relevance to the query. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_trulens.py` | | | | --- | --- | | [113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154)
[155](#__codelineno-0-155)
[156](#__codelineno-0-156)
[157](#__codelineno-0-157)
[158](#__codelineno-0-158)
[159](#__codelineno-0-159)
[160](#__codelineno-0-160)
[161](#__codelineno-0-161)
[162](#__codelineno-0-162)
[163](#__codelineno-0-163)
[164](#__codelineno-0-164)
[165](#__codelineno-0-165)
[166](#__codelineno-0-166)
[167](#__codelineno-0-167)
[168](#__codelineno-0-168)
[169](#__codelineno-0-169)
[170](#__codelineno-0-170)
[171](#__codelineno-0-171)
[172](#__codelineno-0-172)
[173](#__codelineno-0-173)
[174](#__codelineno-0-174)
[175](#__codelineno-0-175)
[176](#__codelineno-0-176)
[177](#__codelineno-0-177)
[178](#__codelineno-0-178)
[179](#__codelineno-0-179)
[180](#__codelineno-0-180)
[181](#__codelineno-0-181)
[182](#__codelineno-0-182) | `def run( self, documents: list[Document], query: str, ) -> list[Document]: """Filter down documents based on their relevance to the query.""" filtered_docs = [] documents = sorted(documents, key=lambda doc: doc.get_content()) if self.concurrent: with ThreadPoolExecutor() as executor: futures = [] for doc in documents: chunked_doc_content = self.trim_func( [ Document(content=doc.get_content()) # skip metadata which cause troubles ] )[0].text messages = [] messages.append( SystemMessage(self.system_prompt_template.populate()) ) messages.append( HumanMessage( self.user_prompt_template.populate( question=query, context=chunked_doc_content ) ) ) def llm_call(): return self.llm(messages).text futures.append(executor.submit(llm_call)) results = [future.result() for future in futures] else: results = [] for doc in documents: messages = [] messages.append(SystemMessage(self.system_prompt_template.populate())) messages.append( SystemMessage( self.user_prompt_template.populate( question=query, context=doc.get_content() ) ) ) results.append(self.llm(messages).text) # use Boolean parser to extract relevancy output from LLM results = [ (r_idx, float(re_0_10_rating(result)) / self.normalize) for r_idx, result in enumerate(results) ] results.sort(key=lambda x: x[1], reverse=True) for r_idx, score in results: doc = documents[r_idx] doc.metadata["llm_trulens_score"] = score filtered_docs.append(doc) print( "LLM rerank scores", [doc.metadata["llm_trulens_score"] for doc in filtered_docs], ) return filtered_docs` | validate\_rating [¶](#indices.rankings.llm_trulens.validate_rating "Permanent link") ------------------------------------------------------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `validate_rating(rating)` | Validate a rating is between 0 and 10. Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_trulens.py` | | | | --- | --- | | [52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58) | `def validate_rating(rating) -> int: """Validate a rating is between 0 and 10.""" if not 0 <= rating <= 10: raise ValueError("Rating must be between 0 and 10") return rating` | re\_0\_10\_rating [¶](#indices.rankings.llm_trulens.re_0_10_rating "Permanent link") ------------------------------------------------------------------------------------- | | | | --- | --- | | [1](#__codelineno-0-1) | `re_0_10_rating(s)` | Extract a 0-10 rating from a string. If the string does not match an integer or matches an integer outside the 0-10 range, raises an error instead. If multiple numbers are found within the expected 0-10 range, the smallest is returned. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `s` | `str` | String to extract rating from. | _required_ | Returns: | Name | Type | Description | | --- | --- | --- | | `int` | `int` | Extracted rating. | Raises: | Type | Description | | --- | --- | | `ParseError` | If no integers between 0 and 10 are found in the string. | Source code in `libs/kotaemon/kotaemon/indices/rankings/llm_trulens.py` | | | | --- | --- | | [61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78)
[79](#__codelineno-0-79)
[80](#__codelineno-0-80)
[81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93) | `def re_0_10_rating(s: str) -> int: """Extract a 0-10 rating from a string. If the string does not match an integer or matches an integer outside the 0-10 range, raises an error instead. If multiple numbers are found within the expected 0-10 range, the smallest is returned. Args: s: String to extract rating from. Returns: int: Extracted rating. Raises: ParseError: If no integers between 0 and 10 are found in the string. """ matches = PATTERN_INTEGER.findall(s) if not matches: raise AssertionError vals = set() for match in matches: try: vals.add(validate_rating(int(match))) except ValueError: pass if not vals: raise AssertionError # Min to handle cases like "The rating is 8 out of 10." return min(vals)` | Back to top --- # Base - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/llms/chats/base.py "Edit this page") Base ==== Back to top --- # Splitters - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/splitters/__init__.py "Edit this page") Splitters ========= BaseSplitter [¶](#indices.splitters.BaseSplitter "Permanent link") ------------------------------------------------------------------- Bases: `DocTransformer` Represent base splitter class Source code in `libs/kotaemon/kotaemon/indices/splitters/__init__.py` | | | | --- | --- | | [4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7) | `class BaseSplitter(DocTransformer): """Represent base splitter class""" ...` | Back to top --- # Jina Web Search - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/retrievers/jina_web_search.py "Edit this page") Jina Web Search =============== WebSearch [¶](#indices.retrievers.jina_web_search.WebSearch "Permanent link") ------------------------------------------------------------------------------ Bases: `BaseComponent` WebSearch component for fetching data from the web using Jina API Source code in `libs/kotaemon/kotaemon/indices/retrievers/jina_web_search.py` | | | | --- | --- | | [10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60) | `class WebSearch(BaseComponent): """WebSearch component for fetching data from the web using Jina API """ def run( self, text: str, *args, **kwargs, ) -> list[RetrievedDocument]: if JINA_API_KEY == "": raise ValueError( "This feature requires JINA_API_KEY " "(get free one from https://jina.ai/reader)" ) # setup the request api_url = f"https://s.jina.ai/{text}" headers = {"X-With-Generated-Alt": "true", "Accept": "application/json"} if JINA_API_KEY: headers["Authorization"] = f"Bearer {JINA_API_KEY}" response = requests.get(api_url, headers=headers) response.raise_for_status() response_dict = response.json() return [ RetrievedDocument( text=( "###URL: [{url}]({url})\n\n" "####{title}\n\n" "{description}\n" "{content}" ).format( url=item["url"], title=item["title"], description=item["description"], content=item["content"], ), metadata={ "file_name": "Web search", "type": "table", "llm_trulens_score": 1.0, }, ) for item in response_dict["data"] ] def generate_relevant_scores(self, text, documents: list[RetrievedDocument]): return documents` | Back to top --- # Tavily Web Search - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/indices/retrievers/tavily_web_search.py "Edit this page") Tavily Web Search ================= WebSearch [¶](#indices.retrievers.tavily_web_search.WebSearch "Permanent link") -------------------------------------------------------------------------------- Bases: `BaseComponent` WebSearch component for fetching data from the web using Jina API Source code in `libs/kotaemon/kotaemon/indices/retrievers/tavily_web_search.py` | | | | --- | --- | | [8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57) | ``class WebSearch(BaseComponent): """WebSearch component for fetching data from the web using Jina API """ def run( self, text: str, *args, **kwargs, ) -> list[RetrievedDocument]: if TAVILY_API_KEY == "": raise ValueError( "This feature requires TAVILY_API_KEY " "(get free one from https://app.tavily.com/)" ) try: from tavily import TavilyClient except ImportError: raise ImportError( "Please install `pip install tavily-python` to use this feature" ) tavily_client = TavilyClient(api_key=TAVILY_API_KEY) results = tavily_client.search( query=text, search_depth="advanced", )["results"] context = "\n\n".join( "###URL: [{url}]({url})\n\n{content}".format( url=result["url"], content=result["content"], ) for result in results ) return [ RetrievedDocument( text=context, metadata={ "file_name": "Web search", "type": "table", "llm_trulens_score": 1.0, }, ) ] def generate_relevant_scores(self, text, documents: list[RetrievedDocument]): return documents`` | Back to top --- # Branching - kotaemon Docs [](https://github.com/Cinnamon/kotaemon/edit/main/kotaemon/llms/branching.py "Edit this page") Branching ========= SimpleBranchingPipeline [¶](#llms.branching.SimpleBranchingPipeline "Permanent link") -------------------------------------------------------------------------------------- Bases: `BaseComponent` A simple branching pipeline for executing multiple branches. Attributes: | Name | Type | Description | | --- | --- | --- | | `branches` | `List[BaseComponent]` | The list of branches to be executed. | Example | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3)
[4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32) | `from kotaemon.llms import ( LCAzureChatOpenAI, BasePromptComponent, GatedLinearPipeline, ) from kotaemon.parsers import RegexExtractor def identity(x): return x pipeline = SimpleBranchingPipeline() llm = LCAzureChatOpenAI( openai_api_base="your openai api base", openai_api_key="your openai api key", openai_api_version="your openai api version", deployment_name="dummy-q2-gpt35", temperature=0, request_timeout=600, ) for i in range(3): pipeline.add_branch( GatedLinearPipeline( prompt=BasePromptComponent(template=f"what is {i} in Japanese ?"), condition=RegexExtractor(pattern=f"{i}"), llm=llm, post_processor=identity, ) ) print(pipeline(condition_text="1")) print(pipeline(condition_text="2")) print(pipeline(condition_text="12"))` | Source code in `libs/kotaemon/kotaemon/llms/branching.py` | | | | --- | --- | | [8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31)
[32](#__codelineno-0-32)
[33](#__codelineno-0-33)
[34](#__codelineno-0-34)
[35](#__codelineno-0-35)
[36](#__codelineno-0-36)
[37](#__codelineno-0-37)
[38](#__codelineno-0-38)
[39](#__codelineno-0-39)
[40](#__codelineno-0-40)
[41](#__codelineno-0-41)
[42](#__codelineno-0-42)
[43](#__codelineno-0-43)
[44](#__codelineno-0-44)
[45](#__codelineno-0-45)
[46](#__codelineno-0-46)
[47](#__codelineno-0-47)
[48](#__codelineno-0-48)
[49](#__codelineno-0-49)
[50](#__codelineno-0-50)
[51](#__codelineno-0-51)
[52](#__codelineno-0-52)
[53](#__codelineno-0-53)
[54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61)
[62](#__codelineno-0-62)
[63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78) | `class SimpleBranchingPipeline(BaseComponent): """ A simple branching pipeline for executing multiple branches. Attributes: branches (List[BaseComponent]): The list of branches to be executed. Example: ```python from kotaemon.llms import ( LCAzureChatOpenAI, BasePromptComponent, GatedLinearPipeline, ) from kotaemon.parsers import RegexExtractor def identity(x): return x pipeline = SimpleBranchingPipeline() llm = LCAzureChatOpenAI( openai_api_base="your openai api base", openai_api_key="your openai api key", openai_api_version="your openai api version", deployment_name="dummy-q2-gpt35", temperature=0, request_timeout=600, ) for i in range(3): pipeline.add_branch( GatedLinearPipeline( prompt=BasePromptComponent(template=f"what is {i} in Japanese ?"), condition=RegexExtractor(pattern=f"{i}"), llm=llm, post_processor=identity, ) ) print(pipeline(condition_text="1")) print(pipeline(condition_text="2")) print(pipeline(condition_text="12")) ``` """ branches: List[BaseComponent] = Param(default_callback=lambda *_: []) def add_branch(self, component: BaseComponent): """ Add a new branch to the pipeline. Args: component (BaseComponent): The branch component to be added. """ self.branches.append(component) def run(self, **prompt_kwargs): """ Execute the pipeline by running each branch and return the outputs as a list. Args: **prompt_kwargs: Keyword arguments for the branches. Returns: List: The outputs of each branch as a list. """ output = [] for i, branch in enumerate(self.branches): self._prepare_child(branch, name=f"branch-{i}") output.append(branch(**prompt_kwargs)) return output` | ### add\_branch [¶](#llms.branching.SimpleBranchingPipeline.add_branch "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `add_branch(component)` | Add a new branch to the pipeline. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `component` | `BaseComponent` | The branch component to be added. | _required_ | Source code in `libs/kotaemon/kotaemon/llms/branching.py` | | | | --- | --- | | [54](#__codelineno-0-54)
[55](#__codelineno-0-55)
[56](#__codelineno-0-56)
[57](#__codelineno-0-57)
[58](#__codelineno-0-58)
[59](#__codelineno-0-59)
[60](#__codelineno-0-60)
[61](#__codelineno-0-61) | `def add_branch(self, component: BaseComponent): """ Add a new branch to the pipeline. Args: component (BaseComponent): The branch component to be added. """ self.branches.append(component)` | ### run [¶](#llms.branching.SimpleBranchingPipeline.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(**prompt_kwargs)` | Execute the pipeline by running each branch and return the outputs as a list. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `**prompt_kwargs` | | Keyword arguments for the branches. | `{}` | Returns: | Name | Type | Description | | --- | --- | --- | | `List` | | The outputs of each branch as a list. | Source code in `libs/kotaemon/kotaemon/llms/branching.py` | | | | --- | --- | | [63](#__codelineno-0-63)
[64](#__codelineno-0-64)
[65](#__codelineno-0-65)
[66](#__codelineno-0-66)
[67](#__codelineno-0-67)
[68](#__codelineno-0-68)
[69](#__codelineno-0-69)
[70](#__codelineno-0-70)
[71](#__codelineno-0-71)
[72](#__codelineno-0-72)
[73](#__codelineno-0-73)
[74](#__codelineno-0-74)
[75](#__codelineno-0-75)
[76](#__codelineno-0-76)
[77](#__codelineno-0-77)
[78](#__codelineno-0-78) | `def run(self, **prompt_kwargs): """ Execute the pipeline by running each branch and return the outputs as a list. Args: **prompt_kwargs: Keyword arguments for the branches. Returns: List: The outputs of each branch as a list. """ output = [] for i, branch in enumerate(self.branches): self._prepare_child(branch, name=f"branch-{i}") output.append(branch(**prompt_kwargs)) return output` | GatedBranchingPipeline [¶](#llms.branching.GatedBranchingPipeline "Permanent link") ------------------------------------------------------------------------------------ Bases: `SimpleBranchingPipeline` A simple gated branching pipeline for executing multiple branches based on a condition. This class extends the SimpleBranchingPipeline class and adds the ability to execute the branches until a branch returns a non-empty output based on a condition. Attributes: | Name | Type | Description | | --- | --- | --- | | `branches` | `List[BaseComponent]` | The list of branches to be executed. | Example | | | | --- | --- | | [1](#__codelineno-0-1)
[2](#__codelineno-0-2)
[3](#__codelineno-0-3)
[4](#__codelineno-0-4)
[5](#__codelineno-0-5)
[6](#__codelineno-0-6)
[7](#__codelineno-0-7)
[8](#__codelineno-0-8)
[9](#__codelineno-0-9)
[10](#__codelineno-0-10)
[11](#__codelineno-0-11)
[12](#__codelineno-0-12)
[13](#__codelineno-0-13)
[14](#__codelineno-0-14)
[15](#__codelineno-0-15)
[16](#__codelineno-0-16)
[17](#__codelineno-0-17)
[18](#__codelineno-0-18)
[19](#__codelineno-0-19)
[20](#__codelineno-0-20)
[21](#__codelineno-0-21)
[22](#__codelineno-0-22)
[23](#__codelineno-0-23)
[24](#__codelineno-0-24)
[25](#__codelineno-0-25)
[26](#__codelineno-0-26)
[27](#__codelineno-0-27)
[28](#__codelineno-0-28)
[29](#__codelineno-0-29)
[30](#__codelineno-0-30)
[31](#__codelineno-0-31) | `from kotaemon.llms import ( LCAzureChatOpenAI, BasePromptComponent, GatedLinearPipeline, ) from kotaemon.parsers import RegexExtractor def identity(x): return x pipeline = GatedBranchingPipeline() llm = LCAzureChatOpenAI( openai_api_base="your openai api base", openai_api_key="your openai api key", openai_api_version="your openai api version", deployment_name="dummy-q2-gpt35", temperature=0, request_timeout=600, ) for i in range(3): pipeline.add_branch( GatedLinearPipeline( prompt=BasePromptComponent(template=f"what is {i} in Japanese ?"), condition=RegexExtractor(pattern=f"{i}"), llm=llm, post_processor=identity, ) ) print(pipeline(condition_text="1")) print(pipeline(condition_text="2"))` | Source code in `libs/kotaemon/kotaemon/llms/branching.py` | | | | --- | --- | | [81](#__codelineno-0-81)
[82](#__codelineno-0-82)
[83](#__codelineno-0-83)
[84](#__codelineno-0-84)
[85](#__codelineno-0-85)
[86](#__codelineno-0-86)
[87](#__codelineno-0-87)
[88](#__codelineno-0-88)
[89](#__codelineno-0-89)
[90](#__codelineno-0-90)
[91](#__codelineno-0-91)
[92](#__codelineno-0-92)
[93](#__codelineno-0-93)
[94](#__codelineno-0-94)
[95](#__codelineno-0-95)
[96](#__codelineno-0-96)
[97](#__codelineno-0-97)
[98](#__codelineno-0-98)
[99](#__codelineno-0-99)
[100](#__codelineno-0-100)
[101](#__codelineno-0-101)
[102](#__codelineno-0-102)
[103](#__codelineno-0-103)
[104](#__codelineno-0-104)
[105](#__codelineno-0-105)
[106](#__codelineno-0-106)
[107](#__codelineno-0-107)
[108](#__codelineno-0-108)
[109](#__codelineno-0-109)
[110](#__codelineno-0-110)
[111](#__codelineno-0-111)
[112](#__codelineno-0-112)
[113](#__codelineno-0-113)
[114](#__codelineno-0-114)
[115](#__codelineno-0-115)
[116](#__codelineno-0-116)
[117](#__codelineno-0-117)
[118](#__codelineno-0-118)
[119](#__codelineno-0-119)
[120](#__codelineno-0-120)
[121](#__codelineno-0-121)
[122](#__codelineno-0-122)
[123](#__codelineno-0-123)
[124](#__codelineno-0-124)
[125](#__codelineno-0-125)
[126](#__codelineno-0-126)
[127](#__codelineno-0-127)
[128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154) | ``class GatedBranchingPipeline(SimpleBranchingPipeline): """ A simple gated branching pipeline for executing multiple branches based on a condition. This class extends the SimpleBranchingPipeline class and adds the ability to execute the branches until a branch returns a non-empty output based on a condition. Attributes: branches (List[BaseComponent]): The list of branches to be executed. Example: ```python from kotaemon.llms import ( LCAzureChatOpenAI, BasePromptComponent, GatedLinearPipeline, ) from kotaemon.parsers import RegexExtractor def identity(x): return x pipeline = GatedBranchingPipeline() llm = LCAzureChatOpenAI( openai_api_base="your openai api base", openai_api_key="your openai api key", openai_api_version="your openai api version", deployment_name="dummy-q2-gpt35", temperature=0, request_timeout=600, ) for i in range(3): pipeline.add_branch( GatedLinearPipeline( prompt=BasePromptComponent(template=f"what is {i} in Japanese ?"), condition=RegexExtractor(pattern=f"{i}"), llm=llm, post_processor=identity, ) ) print(pipeline(condition_text="1")) print(pipeline(condition_text="2")) ``` """ def run(self, *, condition_text: Optional[str] = None, **prompt_kwargs): """ Execute the pipeline by running each branch and return the output of the first branch that returns a non-empty output based on the provided condition. Args: condition_text (str): The condition text to evaluate for each branch. Default to None. **prompt_kwargs: Keyword arguments for the branches. Returns: Union[OutputType, None]: The output of the first branch that satisfies the condition, or None if no branch satisfies the condition. Raises: ValueError: If condition_text is None """ if condition_text is None: raise ValueError("`condition_text` must be provided.") for i, branch in enumerate(self.branches): self._prepare_child(branch, name=f"branch-{i}") output = branch(condition_text=condition_text, **prompt_kwargs) if output: return output return Document(None)`` | ### run [¶](#llms.branching.GatedBranchingPipeline.run "Permanent link") | | | | --- | --- | | [1](#__codelineno-0-1) | `run(*, condition_text=None, **prompt_kwargs)` | Execute the pipeline by running each branch and return the output of the first branch that returns a non-empty output based on the provided condition. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `condition_text` | `str` | The condition text to evaluate for each branch. Default to None. | `None` | | `**prompt_kwargs` | | Keyword arguments for the branches. | `{}` | Returns: | Type | Description | | --- | --- | | | Union\[OutputType, None\]: The output of the first branch that satisfies the | | | condition, or None if no branch satisfies the condition. | Raises: | Type | Description | | --- | --- | | `ValueError` | If condition\_text is None | Source code in `libs/kotaemon/kotaemon/llms/branching.py` | | | | --- | --- | | [128](#__codelineno-0-128)
[129](#__codelineno-0-129)
[130](#__codelineno-0-130)
[131](#__codelineno-0-131)
[132](#__codelineno-0-132)
[133](#__codelineno-0-133)
[134](#__codelineno-0-134)
[135](#__codelineno-0-135)
[136](#__codelineno-0-136)
[137](#__codelineno-0-137)
[138](#__codelineno-0-138)
[139](#__codelineno-0-139)
[140](#__codelineno-0-140)
[141](#__codelineno-0-141)
[142](#__codelineno-0-142)
[143](#__codelineno-0-143)
[144](#__codelineno-0-144)
[145](#__codelineno-0-145)
[146](#__codelineno-0-146)
[147](#__codelineno-0-147)
[148](#__codelineno-0-148)
[149](#__codelineno-0-149)
[150](#__codelineno-0-150)
[151](#__codelineno-0-151)
[152](#__codelineno-0-152)
[153](#__codelineno-0-153)
[154](#__codelineno-0-154) | ``def run(self, *, condition_text: Optional[str] = None, **prompt_kwargs): """ Execute the pipeline by running each branch and return the output of the first branch that returns a non-empty output based on the provided condition. Args: condition_text (str): The condition text to evaluate for each branch. Default to None. **prompt_kwargs: Keyword arguments for the branches. Returns: Union[OutputType, None]: The output of the first branch that satisfies the condition, or None if no branch satisfies the condition. Raises: ValueError: If condition_text is None """ if condition_text is None: raise ValueError("`condition_text` must be provided.") for i, branch in enumerate(self.branches): self._prepare_child(branch, name=f"branch-{i}") output = branch(condition_text=condition_text, **prompt_kwargs) if output: return output return Document(None)`` | Back to top ---