# Table of Contents

- [👋 Welcome to Gaia | Gaia](#-welcome-to-gaia-gaia)
- [💻 For Developers | Gaia](#-for-developers-gaia)
- [🛠️ For Node Operators | Gaia](#-for-node-operators-gaia)
- [🧠 Knowledge Bases | Gaia](#-knowledge-bases-gaia)
- [Overview | Gaia](#overview-gaia)
- [🔌 Agent Integrations & Plugins | Gaia](#-agent-integrations-plugins-gaia)
- [🔗 Domain Operator Guide | Gaia](#-domain-operator-guide-gaia)
- [🌐 Public Gaia Domain | Gaia](#-public-gaia-domain-gaia)
- [What is a Gaia node? | Gaia](#what-is-a-gaia-node-gaia)
- [📚 Guides & Tutorials | Gaia](#-guides-tutorials-gaia)
- [❓ Frequently Asked Questions | Gaia](#-frequently-asked-questions-gaia)
- [Setting up your own node | Gaia](#setting-up-your-own-node-gaia)
- [Get your API key | Gaia](#get-your-api-key-gaia)
- [Using your Gaia Node | Gaia](#using-your-gaia-node-gaia)
- [📄 Litepaper | Gaia](#-litepaper-gaia)
- [Customizing Your Gaia Node | Gaia](#customizing-your-gaia-node-gaia)
- [OpenAI ecosystem apps | Gaia](#openai-ecosystem-apps-gaia)
- [Gaia CLI options | Gaia](#gaia-cli-options-gaia)
- [Joining the Gaia Protocol | Gaia](#joining-the-gaia-protocol-gaia)
- [Troubleshooting | Gaia](#troubleshooting-gaia)
- [Quick Start with Launching Gaia Domain | Gaia](#quick-start-with-launching-gaia-domain-gaia)
- [Install or Uninstall the CLI | Gaia](#install-or-uninstall-the-cli-gaia)
- [Fine-tune an open-source LLM with llama.cpp | Gaia](#fine-tune-an-open-source-llm-with-llama-cpp-gaia)
- [System requirements | Gaia](#system-requirements-gaia)
- [Advanced Deployment Options | Gaia](#advanced-deployment-options-gaia)
- [Working with Coinbase AgentKit | Gaia](#working-with-coinbase-agentkit-gaia)
- [Working with eliza | Gaia](#working-with-eliza-gaia)
- [API Reference | Gaia](#api-reference-gaia)
- [Run a local-only node | Gaia](#run-a-local-only-node-gaia)
- [Install multiple nodes on a single machine | Gaia](#install-multiple-nodes-on-a-single-machine-gaia)
- [Calling external tools | Gaia](#calling-external-tools-gaia)
- [Ensuring Gaia Node Reliability | Gaia](#ensuring-gaia-node-reliability-gaia)
- [User Guide | Gaia](#user-guide-gaia)
- [Intro to knowledge bases | Gaia](#intro-to-knowledge-bases-gaia)
- [Agentic translation on Gaia | Gaia](#agentic-translation-on-gaia-gaia)
- [Node Operator Guide | Gaia](#node-operator-guide-gaia)
- [How to create a knowledge base | Gaia](#how-to-create-a-knowledge-base-gaia)
- [Creator Guide | Gaia](#creator-guide-gaia)
- [Build a knowledge base using Gaia web tool | Gaia](#build-a-knowledge-base-using-gaia-web-tool-gaia)
- [Domain Operator Guide | Gaia](#domain-operator-guide-gaia)
- [Tutorial | Gaia](#tutorial-gaia)
- [GaiaNet: GenAI Agent Network | Gaia](#gaianet-genai-agent-network-gaia)
- [Public GaiaNet nodes | Gaia](#public-gaianet-nodes-gaia)
- [OpenAI ecosystem apps | Gaia](#openai-ecosystem-apps-gaia)
- [Quick start with GaiaNet Node | Gaia](#quick-start-with-gaianet-node-gaia)
- [Agent frameworks and apps | Gaia](#agent-frameworks-and-apps-gaia)
- [Joining the Gaia Protocol | Gaia](#joining-the-gaia-protocol-gaia)
- [Customize Your GaiaNet Node | Gaia](#customize-your-gaianet-node-gaia)
- [Quick Start with Launching Gaia Domain | Gaia](#quick-start-with-launching-gaia-domain-gaia)
- [Knowledge bases | Gaia](#knowledge-bases-gaia)
- [GaiaNet Node with finetuned LLMs | Gaia](#gaianet-node-with-finetuned-llms-gaia)
- [Dify + Gaia | Gaia](#dify-gaia-gaia)
- [Open WebUI | Gaia](#open-webui-gaia)
- [Anything LLM | Gaia](#anything-llm-gaia)
- [LlamaTutor | Gaia](#llamatutor-gaia)
- [Cursor AI IDE | Gaia](#cursor-ai-ide-gaia)
- [Gaia nodes with long-term knowledge | Gaia](#gaia-nodes-with-long-term-knowledge-gaia)
- [Stockbot | Gaia](#stockbot-gaia)
- [Prompt Engineering Tool | Gaia](#prompt-engineering-tool-gaia)
- [FlowiseAI RAG chat | Gaia](#flowiseai-rag-chat-gaia)
- [Zed | Gaia](#zed-gaia)
- [FlowiseAI tool call | Gaia](#flowiseai-tool-call-gaia)
- [LobeChat | Gaia](#lobechat-gaia)
- [LlamaCloud | Gaia](#llamacloud-gaia)
- [Obsidian | Gaia](#obsidian-gaia)
- [AI coding assistant: Continue | Gaia](#ai-coding-assistant-continue-gaia)
- [CodeGPT | Gaia](#codegpt-gaia)
- [LlamaCoder | Gaia](#llamacoder-gaia)
- [Translation Agent + Gaia | Gaia](#translation-agent-gaia-gaia)
- [Agent Zero | Gaia](#agent-zero-gaia)
- [A planning agent | Gaia](#a-planning-agent-gaia)
- [Knowledge base from a plain text file | Gaia](#knowledge-base-from-a-plain-text-file-gaia)
- [LlamaEdgeBook | Gaia](#llamaedgebook-gaia)
- [Knowledge base from source / summary pairs | Gaia](#knowledge-base-from-source-summary-pairs-gaia)
- [Knowledge base from a markdown file | Gaia](#knowledge-base-from-a-markdown-file-gaia)
- [Knowledge base from a PDF file | Gaia](#knowledge-base-from-a-pdf-file-gaia)
- [Knowledge base from a URL | Gaia](#knowledge-base-from-a-url-gaia)
- [Install CUDA on Linux | Gaia](#install-cuda-on-linux-gaia)
- [Use my GaiaNet node | Gaia](#use-my-gaianet-node-gaia)
- [Working with Coinbase AgentKit | Gaia](#working-with-coinbase-agentkit-gaia)
- [Working with eliza | Gaia](#working-with-eliza-gaia)
- [Agentic translation on GaiaNet | Gaia](#agentic-translation-on-gaianet-gaia)
- [API Reference | Gaia](#api-reference-gaia)
- [Start a node on AWS using AMI images | Gaia](#start-a-node-on-aws-using-ami-images-gaia)
- [Calling external tools | Gaia](#calling-external-tools-gaia)
- [System requirements | Gaia](#system-requirements-gaia)
- [GaiaNet CLI options | Gaia](#gaianet-cli-options-gaia)
- [Start a node with Docker | Gaia](#start-a-node-with-docker-gaia)
- [Install and uninstall | Gaia](#install-and-uninstall-gaia)
- [Troubleshooting | Gaia](#troubleshooting-gaia)
- [How do I ... | Gaia](#how-do-i-gaia)
- [Protect the server process | Gaia](#protect-the-server-process-gaia)
- [Dify + GaiaNet | Gaia](#dify-gaianet-gaia)
- [Anything LLM | Gaia](#anything-llm-gaia)
- [llama.cpp | Gaia](#llama-cpp-gaia)
- [Open WebUI | Gaia](#open-webui-gaia)
- [FlowiseAI RAG chat | Gaia](#flowiseai-rag-chat-gaia)
- [Cursor AI IDE | Gaia](#cursor-ai-ide-gaia)
- [Stockbot | Gaia](#stockbot-gaia)
- [Zed | Gaia](#zed-gaia)
- [FlowiseAI tool call | Gaia](#flowiseai-tool-call-gaia)
- [LlamaCloud | Gaia](#llamacloud-gaia)
- [LobeChat | Gaia](#lobechat-gaia)
- [CodeGPT | Gaia](#codegpt-gaia)
- [Obsidian | Gaia](#obsidian-gaia)
- [AI coding assistant: Continue | Gaia](#ai-coding-assistant-continue-gaia)
- [Translation Agent  + GaiaNet | Gaia](#translation-agent-gaianet-gaia)
- [Agent Zero | Gaia](#agent-zero-gaia)
- [LlamaCoder | Gaia](#llamacoder-gaia)
- [A planning agent | Gaia](#a-planning-agent-gaia)
- [LlamaEdgeBook | Gaia](#llamaedgebook-gaia)
- [Gaia nodes with long-term knowledge | Gaia](#gaia-nodes-with-long-term-knowledge-gaia)
- [LlamaTutor | Gaia](#llamatutor-gaia)
- [Fine-tune LLMs | Gaia](#fine-tune-llms-gaia)
- [Knowledge base from a plain text file | Gaia](#knowledge-base-from-a-plain-text-file-gaia)
- [Knowledge base from source / summary pairs | Gaia](#knowledge-base-from-source-summary-pairs-gaia)
- [Build a knowledge base using Gaia web tool | Gaia](#build-a-knowledge-base-using-gaia-web-tool-gaia)
- [Knowledge base from a URL | Gaia](#knowledge-base-from-a-url-gaia)
- [Knowledge base from a PDF file | Gaia](#knowledge-base-from-a-pdf-file-gaia)
- [Knowledge base from a markdown file | Gaia](#knowledge-base-from-a-markdown-file-gaia)
- [Install multiple nodes on a single machine | Gaia](#install-multiple-nodes-on-a-single-machine-gaia)
- [Install CUDA on Linux | Gaia](#install-cuda-on-linux-gaia)
- [Start a node with Docker | Gaia](#start-a-node-with-docker-gaia)
- [Start a node on AWS using AMI images | Gaia](#start-a-node-on-aws-using-ami-images-gaia)
- [Run a local-only node | Gaia](#run-a-local-only-node-gaia)

---

# 👋 Welcome to Gaia | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Gaia is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise.

It allows individuals and businesses to create AI agents. Each Gaia node provides:

*   a web-based chatbot UI [Chat with a Gaia node](https://rustcoder.gaia.domains/chatbot-ui/index.html)
     that is an expert on the Rust programming language.
*   an OpenAI compatible API. [See how](/agent-integrations/intro)
     to use a Gaia node as a drop-in OpenAI replacement in your favorite AI agent app.

100% of today's AI agents are applications in the OpenAI ecosystem. With our API approach, Gaia is an alternative to OpenAI. Each Gaia node has the ability to be customized with a fine-tuned model supplemented by domain knowledge which eliminates the generic responses many have come to expect. For example, a Gaia node for a financial analyst agent can write SQL code to query SEC 10K filings to respond to user questions.

Similar Gaia nodes are organized into Gaia domains, to provide stable services by load balancing across the nodes. Gaia domains have public-facing URLs and promote agent services to their communities. When a user or an agent app sends an API request to the domain's API endpoint URL, the domain is responsible for directing the request to a node that is ready.

Next steps:[​](#next-steps "Direct link to Next steps:")

---------------------------------------------------------

### Users[​](#users "Direct link to Users")

If you are an end user of AI agent applications, you can:

*   [Find a list of interesting Gaia nodes you can chat with on the web, or access via API](/nodes)
    .
*   [Use a Gaia node as the backend AI engine for your favorite AI agent apps](/agent-integrations)
    .

### Node operators[​](#node-operators "Direct link to Node operators")

If you are interested in running Gaia nodes, you can

*   [Get started with a Gaia node](/getting-started/quick-start)
    .
*   [Customize the Gaia node with a finetuned model and custom knowledge base](/getting-started/customize)
    .
*   [Join the Gaia Protocol](/getting-started/register)
    

### Domain operators[​](#domain-operators "Direct link to Domain operators")

If you are a Gaia Domain Name owner, you can

*   [Launch your domain](/domain-guide/quick-start)
    .

### Creators[​](#creators "Direct link to Creators")

If you are a creator or knowledge worker interested in creating your own AI agent service, you can:

*   [Create your own knowledge base](/knowledge-bases)
    .
*   [Finetune a model to "speak" like you](/tutorial/llamacpp)
    .

*   [Next steps:](#next-steps)
    *   [Users](#users)
        
    *   [Node operators](#node-operators)
        
    *   [Domain operators](#domain-operators)
        
    *   [Creators](#creators)

---

# 💻 For Developers | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ What is a Gaia node?\
------------------------\
\
A Gaia node is an open-source developer platform that lets anyone build, launch, scale and monetize AI agents. It's like having your own personal AI assistant that you can customize and share with others.](/getting-started/what-is-a-node)

[📄️ Setting up your own node\
----------------------------\
\
This guide provides the requisite knowledge necessary to quickly get started with installing a Gaia node.](/getting-started/quick-start)

[📄️ Get your API key\
--------------------\
\
We're introducing API keys for authentication. You can create API keys by following the steps below:](/getting-started/authentication)

[📄️ Using your Gaia Node\
------------------------\
\
When you start a Gaia node, or you find a node on the web, you could use it as a](/getting-started/mynode)

[📄️ Customizing Your Gaia Node\
------------------------------\
\
A key goal of the Gaia project is to enable each individual to create and run his or her own agent service node using finetuned LLMs and proprietary knowledge. In all likelihood, you are not going to run a node with the default Llama 3.2 LLM and Paris guidebook knowledge base.](/getting-started/customize)

[📄️ Gaia CLI options\
--------------------\
\
After installing the Gaia software, you can use the gaianet CLI to manage the node. The following are the CLI options.](/getting-started/cli-options)

[📄️ Joining the Gaia Protocol\
-----------------------------\
\
After successfully running a Gaia node on your machine, it's time to join the Gaia protocol network and get rewards for sharing computing power with the world by binding your node ID and device ID to a Metamask account.](/getting-started/register)

[📄️ Troubleshooting\
-------------------\
\
The system cannot find CUDA libraries](/getting-started/troubleshooting)

[📄️ Install or Uninstall the CLI\
--------------------------------\
\
The Gaia node utilizes version control from its source GitHub repo. You can check out the Gaia node versions from the release page.](/getting-started/install)

[📄️ System requirements\
-----------------------\
\
You can install the Gaia on a wide variety of devices and operating systems with or without GPUs. The node installing and operating instructions work on devices ranging from Raspberry Pi, MacBooks, Linux servers, Windows Desktop, to cloud-based Nvidia H100 clusters. For institutional operators, we recommend EITHER of the following for a Gaia node.](/getting-started/system-requirements)

[📄️ API Reference\
-----------------\
\
Introduction](/getting-started/api-reference)

[🗃️ Advanced Deployment Options\
-------------------------------\
\
6 items](/category/advanced-deployment-options)

---

# 🛠️ For Node Operators | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Node Operators focus on setting up, running, and maintaining Gaia nodes without requiring coding expertise.

Running a GaiaNet node allows you to participate in the decentralized AI network and potentially earn rewards. The process is straightforward and can be completed in just a few minutes.

Before you begin, ensure your system meets these [minimum requirements](/getting-started/system-requirements)

Here you'll find everything you need to get started with running and managing your Gaia node.

Getting Started[​](#getting-started "Direct link to Getting Started")

----------------------------------------------------------------------

Learn how to get started with your first GaiaNet node.

[Learn More →](/getting-started/what-is-a-node)

Node Setup & Authentication[​](#node-setup--authentication "Direct link to Node Setup & Authentication")

---------------------------------------------------------------------------------------------------------

Step-by-step guide for node installation, setup, and security.

*   [Setting up your own node](/getting-started/quick-start)
    
*   [Authentication guide](/getting-started/authentication)
    

Node Operations[​](#node-operations "Direct link to Node Operations")

----------------------------------------------------------------------

Learn about day-to-day operations and customization.

*   [Using your Gaia Node](/getting-started/mynode)
    
*   [Customizing Your Gaia Node](/getting-started/customize)
    

Network & Protocol[​](#network--protocol "Direct link to Network & Protocol")

------------------------------------------------------------------------------

Join the network and start earning rewards.

*   [Gaia CLI options](/getting-started/cli-options)
    
*   [Joining the Gaia Protocol](/getting-started/register)
    

System Requirements & Troubleshooting[​](#system-requirements--troubleshooting "Direct link to System Requirements & Troubleshooting")

---------------------------------------------------------------------------------------------------------------------------------------

Make sure your system is ready and get help when needed.

*   [System requirements](/getting-started/system-requirements)
    
*   [Troubleshooting guide](/getting-started/troubleshooting)
    

Advanced Deployment Options[​](#advanced-deployment-options "Direct link to Advanced Deployment Options")

----------------------------------------------------------------------------------------------------------

Learn about different deployment options.

*   [Install or Uninstall the CLI](/getting-started/install)
    
*   [Install multiple nodes](/getting-started/advanced-deployment-options/multiple)
    
*   [Run a local-only node](/getting-started/advanced-deployment-options/local)
    
*   [Protect the server process](/getting-started/advanced-deployment-options/protect)
    

*   [Getting Started](#getting-started)
    
*   [Node Setup & Authentication](#node-setup--authentication)
    
*   [Node Operations](#node-operations)
    
*   [Network & Protocol](#network--protocol)
    
*   [System Requirements & Troubleshooting](#system-requirements--troubleshooting)
    
*   [Advanced Deployment Options](#advanced-deployment-options)

---

# 🧠 Knowledge Bases | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ Intro to knowledge bases\
----------------------------\
\
Knowledge bases are crucial components in Gaia's context for creating effective AI agents. They serve as structured repositories of information that agents can draw upon to answer questions, make decisions, and perform tasks.](/knowledge-bases/intro)

[🗃️ How to create a knowledge base\
----------------------------------\
\
5 items](/knowledge-bases/how-to)

[📄️ Build a knowledge base using Gaia web tool\
----------------------------------------------\
\
Gaia has developed a tool for making vector collection snapshot files, so everyone can easily create their own knowledge base.](/knowledge-bases/web-tool)

---

# Overview | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

GaiaNet is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise. It allows individuals and businesses to create AI agents. Each GaiaNet node provides:

*   a web-based chatbot UI [Chat with a GaiaNet node](https://rustcoder.gaia.domains/chatbot-ui/index.html)
     that is an expert on the Rust programming language.
*   an OpenAI compatible API. [See how](/1.0.0/user-guide/apps/intro)
     to use a GaiaNet node as a drop-in OpenAI replacement in your favorite AI agent app.

100% of today's AI agents are applications in the OpenAI ecosystem. With our API approach, GaiaNet is an alternative to OpenAI. Each GaiaNet node has the ability to be customized with a fine-tuned model supplemented by domain knowledge which eliminates the generic responses many have come to expect. For example, a GaiaNet node for a financial analyst agent can write SQL code to query SEC 10K filings to respond to user questions.

Similar GaiaNet nodes are organized into GaiaNet domains, to provide stable services by load balancing across the nodes. GaiaNet domains have public-facing URLs and promote agent services to their communities. When a user or an agent app sends an API request to the domain's API endpoint URL, the domain is responsible for directing the request to a node that is ready.

Next steps:[​](#next-steps "Direct link to Next steps:")

---------------------------------------------------------

### Users[​](#users "Direct link to Users")

If you are an end user of AI agent applications, you can:

*   [Find a list of interesting GaiaNet nodes you can chat with on the web, or access via API](/1.0.0/user-guide/nodes)
    .
*   [Use a GaiaNet node as the backend AI engine for your favorite AI agent apps](/1.0.0/category/agent-frameworks-and-apps)
    .

### Node operators[​](#node-operators "Direct link to Node operators")

If you are interested in running GaiaNet nodes, you can

*   [Get started with a GaiaNet node](/1.0.0/node-guide/quick-start)
    .
*   [Customize the GaiaNet node with a finetuned model and custom knowledge base](/1.0.0/node-guide/customize)
    .
*   [Join the Gaia Protocol](/1.0.0/node-guide/register)
    

### Domain operators[​](#domain-operators "Direct link to Domain operators")

If you are a Gaia Domain Name owner, you can

*   [Launch your domain](/1.0.0/domain-guide/quick-start)
    .

### Creators[​](#creators "Direct link to Creators")

If you are a creator or knowledge worker interested in creating your own AI agent service, you can:

*   [Create your own knowledge base](/1.0.0/category/knowledge-bases)
    .
*   [Finetune a model to "speak" like you](/1.0.0/category/gaianet-node-with-finetuned-llms)
    .

*   [Next steps:](#next-steps)
    *   [Users](#users)
        
    *   [Node operators](#node-operators)
        
    *   [Domain operators](#domain-operators)
        
    *   [Creators](#creators)

---

# 🔌 Agent Integrations & Plugins | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ OpenAI ecosystem apps\
-------------------------\
\
Since each Gaia node provides an OpenAI-compatible API service, it can be a drop-in replacement for OpenAI in almost all LLM applications and frameworks. Checkout the articles in this section for instructions and examples for how to use Gaia in popular LLM apps.](/agent-integrations/intro)

[📄️ Dify + Gaia\
---------------\
\
You can configure the Dify framework using any Gaia node as the backend LLM API. That allows you to use your own or community Gaia nodes in any application built on Dify. It supports](/agent-integrations/dify)

[📄️ Open WebUI\
--------------\
\
You can configure the Open WebUI framework, a self-hosted WebUI, using any Gaia node as the backend LLM API. That allows you to use your own or community Gaia nodes in any application built on Open WebUI.](/agent-integrations/openwebui)

[📄️ Anything LLM\
----------------\
\
Anything LLM is the all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities. You can configure Anything LLM using the Gaia node as the LLM backend.](/agent-integrations/anything_llm)

[📄️ Cursor AI IDE\
-----------------\
\
Cursor is an AI-powered code editor / IDE. Using LLMs to generate and review code, Cursor is an alternative to the very popular GitHub Copilot.](/agent-integrations/cursor)

[📄️ Stockbot\
------------\
\
Stockbot is a lightning fast AI Chatbot that responds with live interactive stock charts, financials, news, screeners, and more. You can configure a Gaia node as the LLM backend.](/agent-integrations/stockbot)

[📄️ FlowiseAI RAG chat\
----------------------\
\
FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use Gaia nodes as LLM service providers.](/agent-integrations/flowiseai)

[📄️ FlowiseAI tool call\
-----------------------\
\
FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents.](/agent-integrations/flowiseai-tool-call)

[📄️ LobeChat\
------------\
\
You can configure LobeChat to use a Gaia node as its backend LLM API. It provides a richer and more customizable UI than the default Gaia chatbot UI.](/agent-integrations/lobechat)

[📄️ LlamaCloud\
--------------\
\
LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. LlamaParse can support different kinds of files, like pdf, doc, .ppt, and other formats.](/agent-integrations/llamaparse)

[📄️ Zed\
-------\
\
Zed is a next-generation code editor designed for high-performance collaboration with humans and AI, and it is written in Rust. You can use Zed with your own Gaia node as the LLM backend. There are two big reasons for that](/agent-integrations/zed)

[📄️ Obsidian\
------------\
\
Obsidian is a note-taking application that enables users to create, link, and visualize ideas directly on their devices. With Obsidian, you can seamlessly sync notes across devices, publish your work, and collaborate with others. The app is highly customizable, allowing users to enhance functionality through a wide range of plugins and themes. Its unique features include a graph view to visualize connections between notes, making it ideal for managing complex information and fostering creativity. Obsidian also emphasizes data privacy by storing notes locally.](/agent-integrations/obsidian)

[📄️ CodeGPT\
-----------\
\
CodeGPT is a pair-programming partner for developers. It offers AI chat assistance, auto-completion, code explanation, error-checking, and much more. You can find the CodeGPT extension in VScode and Jetbrains. You can easily configure it to use Gaia nodes as LLM backends.](/agent-integrations/codegpt)

[📄️ AI coding assistant: Continue\
---------------------------------\
\
Continue is the leading open-source AI code assistant.](/agent-integrations/continue)

[📄️ LlamaCoder\
--------------\
\
LlamaCoder is an open-source tool designed to generate small apps with a single prompt. It leverages LLM to help you quickly create and enhance React applications.](/agent-integrations/llamacoder)

[📄️ Agent Zero\
--------------\
\
Agent Zero is a general purpose AI agent application. You can simply ask it to accomplish tasks on the command line.](/agent-integrations/agent-zero)

[📄️ Translation Agent + Gaia\
----------------------------\
\
The Translation Agent originally built by Prof. Andrew Ng, designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use any Gaia node as the LLM backend.](/agent-integrations/translation-agent)

[📄️ A planning agent\
--------------------\
\
The gpt planner is a Python application that demonstrate the planning capabilities of LLMs. When you run it, it will ask the LLM to generate multiple action plans for a goal or a query. It will then ask the LLM to compare and select the best plan, and then rewrite it to answer the user query.](/agent-integrations/gpt-planner)

[📄️ LlamaEdgeBook\
-----------------\
\
LlamaEdgeBook, forked from GroqBook, is an open-source tool that scaffolds the creation of books from a one-line prompt using open-source LLMs. You can configure the LlamaEdgeBook framework using any Gaia node as the backend LLM API.](/agent-integrations/llamaedgebook)

[📄️ LlamaTutor\
--------------\
\
The LlamaTutor is a TypeScript](/agent-integrations/llamatutor)

---

# 🔗 Domain Operator Guide | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ Quick Start with Launching Gaia Domain\
------------------------------------------\
\
This guide provides all the information you need to quickly set up and run a Gaia Domain.](/domain-guide/quick-start)

---

# 🌐 Public Gaia Domain | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Each Gaia node provides a web-based chatbot UI and an OpenAI compatible web service. Here are some popular nodes. Please refer to the [agent apps](/agent-integrations/intro)
 section to see how to use the Gaia API in your favorite agent frameworks or apps.

Gaia nodes are organized into Gaia domains to provide public services. Each domain has a single API endpoint that load-balances across multiple nodes to ensure service availability.

Below are some Gaia domains we provide for free to the public.

Public Gaia domains[​](#public-gaia-domains "Direct link to Public Gaia domains")

----------------------------------------------------------------------------------

### LLM: Llama 8b[​](#llm-llama-8b "Direct link to LLM: Llama 8b")

This domain runs Llama 3.1 8b agent nodes without any additional knowledge. It is capable of tool / function calling. [Learn more](/tutorial/tool-call)
 how to use tool call models in your agent app. [Chat with it](https://llama8b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) |
| Model Name (for LLM) | llama |
| Model Name (for Text embedding) | nomic-embed |
| API key | [Get your API Key here](/getting-started/authentication) |

### Voice-to-text: Whisper v2 large[​](#voice-to-text-whisper-v2-large "Direct link to Voice-to-text: Whisper v2 large")

This domain runs Whisper v2 Large agent nodes for voice to text transcription and translation. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://whisper.gaia.domains/v1](https://whisper.gaia.domains/v1) |
| Model Name | whisper |
| API key | [Get your API Key here](/getting-started/authentication) |

### Text-to-image: Realistic vision[​](#text-to-image-realistic-vision "Direct link to Text-to-image: Realistic vision")

This domain runs stable diffusion agent nodes fine-tuned to generate realistic portraits. The model is [Realistic Vision V6.0 B1](https://civitai.com/models/4201/realistic-vision-v60-b1)
. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://portrait.gaia.domains/v1](https://portrait.gaia.domains/v1) |
| Model Name | portrait |
| API key | [Get your API Key here](/getting-started/authentication) |

### Text-to-voice: GPT-SoVITS[​](#text-to-voice-gpt-sovits "Direct link to Text-to-voice: GPT-SoVITS")

Coming soon!

Coding assistant agents[​](#coding-assistant-agents "Direct link to Coding assistant agents")

----------------------------------------------------------------------------------------------

### Coder[​](#coder "Direct link to Coder")

A general coding agent running a STOA coder LLM. It is a highly capable but slow model. [Chat with it](https://coder.gaia.domains/chatbot-ui/index.html)
 or use it from an IDE tool, such as [Cursor AI](/agent-integrations/cursor)
 or [Zed](/agent-integrations/zed)
. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://coder.gaia.domains/v1](https://coder.gaia.domains/v1) |
| Model Name (for LLM) | coder |
| Model Name (for Text embedding) | nomic-embed |
| API key | [Get your API Key here](/getting-started/authentication) |

### Rust Coder[​](#rust-coder "Direct link to Rust Coder")

A coding agent supplemented with open-source [Rust content](https://huggingface.co/datasets/gaianet/learn-rust)
 from the Rust Foundation and published books. [Chat with it](https://rustcoder.gaia.domains/chatbot-ui/index.html)
 or use it from an IDE tool, such as [Cursor AI](/agent-integrations/cursor)
 or [Zed](/agent-integrations/zed)
. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://rustcoder.gaia.domains/v1](https://rustcoder.gaia.domains/v1) |
| Model Name (for LLM) | rustcoder |
| Model Name (for Text embedding) | nomic-embed |
| API key | [Get your API Key here](/getting-started/authentication) |

Alternative LLM domains[​](#alternative-llm-domains "Direct link to Alternative LLM domains")

----------------------------------------------------------------------------------------------

### Llama 3b[​](#llama-3b "Direct link to Llama 3b")

This domain runs plain Llama 3.2 3b agent nodes without any additional knowledge. [Chat with it](https://llama3b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://llama3b.gaia.domains/v1](https://llama3b.gaia.domains/v1) |
| Model Name (for LLM) | llama3b |
| Model Name (for Text embedding) | nomic-embed |
| API key | [Get your API Key here](/getting-started/authentication) |

### Qwen 7b[​](#qwen-7b "Direct link to Qwen 7b")

The Qwen 7b LLM is great for non-English tasks such as translation. [Chat with it](https://qwen7b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://qwen7b.gaia.domains/v1](https://qwen7b.gaia.domains/v1) |
| Model Name (for LLM) | qwen7b |
| Model Name (for Text embedding) | nomic-embed |
| API key | [Get your API Key here](/getting-started/authentication) |

### Qwen 72b[​](#qwen-72b "Direct link to Qwen 72b")

The Qwen 72b LLM is a top open source LLM. It is very capable but could also be slow. It is capable of tool / function calling. [Learn more](/tutorial/tool-call)
 how to use tool call models in your agent app. [Chat with it](https://qwen72b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/agent-integrations/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://qwen72b.gaia.domains/v1](https://qwen72b.gaia.domains/v1) |
| Model Name (for LLM) | qwen72b |
| Model Name (for Text embedding) | nomic-embed |
| API key | [Get your API Key here](/getting-started/authentication) |

*   [Public Gaia domains](#public-gaia-domains)
    *   [LLM: Llama 8b](#llm-llama-8b)
        
    *   [Voice-to-text: Whisper v2 large](#voice-to-text-whisper-v2-large)
        
    *   [Text-to-image: Realistic vision](#text-to-image-realistic-vision)
        
    *   [Text-to-voice: GPT-SoVITS](#text-to-voice-gpt-sovits)
        
*   [Coding assistant agents](#coding-assistant-agents)
    *   [Coder](#coder)
        
    *   [Rust Coder](#rust-coder)
        
*   [Alternative LLM domains](#alternative-llm-domains)
    *   [Llama 3b](#llama-3b)
        
    *   [Qwen 7b](#qwen-7b)
        
    *   [Qwen 72b](#qwen-72b)

---

# What is a Gaia node? | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

A Gaia node is an open-source developer platform that lets anyone build, launch, scale and monetize AI agents. It's like having your own personal AI assistant that you can customize and share with others.

Here's a breakdown of the key components inside a Gaia node:

*   **WasmEdge Runtime**: WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime for cloud native, edge, and decentralized applications. It powers serverless apps, embedded functions, microservices, smart contracts, and IoT devices. It is the easiest and the fastest way to run LLMs on your own devices. Check the Github of WasmEdge [here](https://github.com/WasmEdge/WasmEdge)
    .
*   **LLM (Large Language Models)**: You can use any LLM from Huggingface, which is a platform hosting thousands of open-source models. Gaia has its own organization page on Huggingface where you can find optimized models for Gaia nodes. You can choose or fine-tune these models for specific tasks or knowledge areas. For example, you could select a model that's an expert in chemistry or one that mimics a particular writing style. Checkout Gaia on Huggingface [here](https://huggingface.co/gaianet)
    .
*   **RAG (Retrieval-Augmented Generation)**: This component helps the AI access and use relevant information from a knowledge base. It's like giving the AI the ability to quickly look up facts in a specialized encyclopedia before answering questions.
*   **Vector Database**: This stores information in a format that the AI can easily search and understand. It's similar to how a library organizes books, but for AI-friendly data.
*   **Multimodal | Embedding Models**: These allow the node to understand and work with different types of data, like text, images, or even audio. The embedding part helps convert this information into a format the AI can process efficiently.
*   **API Server**: This is the interface that allows users or applications to interact with the Gaia node. It's like a reception desk that takes requests and returns answers.
*   **Tool Usage and Function Calling**: These components allow the AI to use external tools or perform specific actions. For instance, the AI could use a calculator tool to solve math problems or call a weather API to get current weather information.
*   **Prompt Selection and Management**: This helps guide the AI's responses by providing context or instructions. It's like giving the AI a script to follow for different scenarios.
*   **Node ID**: This is a unique identifier for each Gaia node, allowing it to be recognized and connected to the larger Gaia system.

Each Gaia node provides a specialized API service that encapsulates a unique combination of

*   a specialized and fine-tuned LLM (e.g., an LLM that excels in answering questions about the Rust programming language)
*   a domain-specific knowledge base (e.g., knowledge about the WasmEdge project)
*   an inference app that manages the context and history of conversations (e.g., RAG and MemGPT prompt injection) compute resources required to run the LLM app as an API service (e.g., a Nvidia GPU or a Mac M3 device)

The Gaia node API service is fully compatible with the OpenAI JSON spec, and hence each Gaia node can function as an alternative backend for OpenAI-based frontends or agents.

Gaia Protocol[​](#gaia-protocol "Direct link to Gaia Protocol")

----------------------------------------------------------------

The Gaia protocol connects and incentivizes Gaia nodes and domains to form a coherent network of web services for AI agents. It provides a mechanism to discover, connect to, and pay for Gaia node services through a decentralized marketplace. It also incentivizes domains to manage node agents through a staking program. Furthermore, the Gaia protocol connects model creators (i.e., people who have skills to finetune models) and knowledge providers to node operators through a marketplace.

![Gaia Protocol](/assets/images/gaia-protocol-82f49ecf93cb30699fc1168070630fce.png)

*   [Gaia Protocol](#gaia-protocol)

---

# 📚 Guides & Tutorials | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ Gaia nodes with long-term knowledge\
---------------------------------------\
\
The LLM app requires both long-term and short-term memories. Long-term memory includes factual knowledge, historical facts, background stories etc. They are best added to the context as complete chapters instead of small chunks of text to maintain the internal consistency of the knowledge.](/tutorial/concepts)

[📄️ Calling external tools\
--------------------------\
\
Tool calling is one of the truly "LLM native" interaction modes that has never existed before.](/tutorial/tool-call)

[📄️ Agentic translation on Gaia\
-------------------------------\
\
Prof. Andrew Ng's agentic translation is a great demonstration on how to coordinate multiple LLM "agents" to work on a single task. It allows multiple smaller LLMs (like Llama-3 or Gemma-2) to work together and produce better results than a single large LLM (like ChatGPT).](/tutorial/translator-agent)

[📄️ Fine-tune an open-source LLM with llama.cpp\
-----------------------------------------------\
\
You could fine-tune an open-source LLM to](/tutorial/llamacpp)

[📄️ Working with Coinbase AgentKit\
----------------------------------\
\
You can use a Gaia node to power the Coinbase AgentKit.](/tutorial/coinbase)

[📄️ Working with eliza\
----------------------\
\
Eliza is a simple, fast, and lightweight AI agent framework. Recently, eliza has integrated Gaia as one of the model service provider. This means you can now use Gaia as the LLM service backend for the Eliza framework.](/tutorial/eliza)

[📄️ Prompt Engineering Tool\
---------------------------\
\
Video Guide](/tutorial/prompt-engineering-tool)

---

# ❓ Frequently Asked Questions | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

What is Gaia?

Gaia is a platform for creating and deploying AI agents. It provides tools and infrastructure to build, train, and manage custom AI solutions for various applications.

How do I create a knowledge base in Gaia?

You can create a knowledge base in Gaia through several methods:

1.  [From a plain text file](/knowledge-bases/how-to/text)
    
2.  [From a markdown file](/knowledge-bases/how-to/markdown)
    
3.  [From Source/Summary Pairs (or a CSV)](/knowledge-bases/how-to/csv)
    
4.  [From a PDF file](/knowledge-bases/how-to/pdf)
    
5.  [From a URL](/knowledge-bases/how-to/firecrawl)
    

Each method is detailed in our ["How to create a knowledge base"](/knowledge-bases)
 guide.

Is Gaia suitable for enterprise use?

Yes, Gaia is designed with enterprise needs in mind. It offers:

*   Scalable infrastructure
*   Security features for sensitive data
*   Integration capabilities with existing systems
*   Customization options for specific industry needs

---

# Setting up your own node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

This guide provides the requisite knowledge necessary to quickly get started with installing a Gaia node.

### Prerequisites[​](#prerequisites "Direct link to Prerequisites")

Before you get started, ensure that you have the following on your system:

| System | Minimum Requirements |
| --- | --- |
| OSX with Apple Silicon (M1-M4 chip) | 16GB RAM (32GB recommended) |
| Ubuntu Linux 20.04 with Nvidia CUDA 12 SDK | 8GB VRAM on GPU |
| Azure/AWS | Nvidia T4 GPU Instance |

Learn more about [system requirements](/getting-started/system-requirements)
.

### Installing the node[​](#installing-the-node "Direct link to Installing the node")

1.  Use the following command to download the latest version of the Gaia node:

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash

2.  Run the command printed on the terminal to set up the environment path, it is started with `source`. ![](/assets/images/quick-start-96fed073147105b82f4885934602852b.png)
    
3.  Use the following command to initialize the Gaia node according to the configuration options in `$HOME/gaianet/config.json`. By default, the Gaia is initialized with a [Llama 3.2](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
     LLM and a [knowledge base about Paris](https://huggingface.co/datasets/gaianet/paris)
    . This command could take some time since it will download a very large LLM file.
    

    gaianet init

4.  Use the following command to start your node:

    gaianet start

#### After starting your node[​](#after-starting-your-node "Direct link to After starting your node")

*   A successful start prints a public URL for the node. Opening a browser to that URL will display the node information and allow you to chat with the AI agent on the node.

    ... ... https://0xf63939431ee11267f4855a166e11cc44d24960c0.gaianet.xyz

*   To stop the node: `gaianet stop`

Video Guide[​](#video-guide "Direct link to Video Guide")

----------------------------------------------------------

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Customize](/getting-started/customize)
     your node. You probably do not wish to just run the default Phi-3 model and Paris knowledge base.
*   [Join the Gaia Protocol](/getting-started/register)
     and join the Gaia protocol network to receive payments.
*   [Ensure Node Reliability](/getting-started/advanced-deployment-options/protect)
     the node server to ensure stable service.

*   [Prerequisites](#prerequisites)
    
*   [Installing the node](#installing-the-node)
    
*   [Video Guide](#video-guide)
    
*   [Next steps](#next-steps)

---

# Get your API key | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

We're introducing API keys for authentication. You can create API keys by following the steps below:

1.  Go to [https://gaianet.ai](https://gaianet.ai)
     and click on **Launch App**
2.  Click **CONNECT** and connect your Metamask Wallet
3.  After connecting your wallet, click on the profile drop down and then click **Setting**

![](/assets/images/settings-for-api-12fe9b183d793b1b6c67c2836b49c790.png)

4.  Under **Setting**, click on **Gaia API Keys** and then **Create API Key**

![](/assets/images/settings-for-api-keys-9796ca33d6aa6b8ce205bf388c3e07eb.png)

5.  Give your API Key a name and click **Create**

Important

Remember that your API key is a secret! Do not share it with others or expose it in any client-side code (browsers, apps). Production requests must be routed through your own backend server where your API key can be securely loaded from an environment variable or key management service.

![](/assets/images/create-api-key-5911b944e10616af0e31670920beff5c.png)

![](/assets/images/api-key-created-bbfb92a8c7df41f38a6ac3f7d448e757.png)

Tip

Your API Key is like a password and helps verify your usage. This will be one of the last times you‘ll see it displayed, so remember to save it in a safe place.

Important[​](#important "Direct link to Important")

----------------------------------------------------

#### API Keys are now available for all your applications and usage.[​](#api-keys-are-now-available-for-all-your-applications-and-usage "Direct link to API Keys are now available for all your applications and usage.")

Currently, we don't charge anything for usage or API key creation. However, it is highly recommendation that you start using and updating your existing applications or new ones with your own API keys to avoid any disruption in the future.

Please refer to the [API Reference](/getting-started/api-reference)
 page for the updated examples with the API keys usage.

*   [Important](#important)

---

# Using your Gaia Node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

When you [start a Gaia node](/getting-started/quick-start)
, or you find a node on the web, you could use it as a web-based chatbot UI and an OpenAI compatible web service. Just load the node's public URL in the browser to open its dashboard. Let's say the URL is as follows.

    https://0x1234...xyz.gaianet.network/

> Please refer to the [agent apps](/agent-integrations/intro)
>  section to see how to use the Gaia node API in your favorite agent frameworks or apps.

Web-based chatbot[​](#web-based-chatbot "Direct link to Web-based chatbot")

----------------------------------------------------------------------------

On the Gaia node dashboard, you will see a "Chat with this node" button.

![](/assets/images/chat_button-8557659182506ceed736a17b449dd9d8.png)

OpenAI API replacement[​](#openai-api-replacement "Direct link to OpenAI API replacement")

-------------------------------------------------------------------------------------------

The Gaia node is a drop-in replacement for OpenAI API in [agent and LLM apps](/agent-integrations/intro)
. On the Gaia node dashboard, you will see a table that shows how to replace OpenAI parameters in those apps.

![](/assets/images/openai_api_options-6610435c06758b955dcd1f73bcfd1d52.png)

*   [Web-based chatbot](#web-based-chatbot)
    
*   [OpenAI API replacement](#openai-api-replacement)

---

# 📄 Litepaper | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

GaiaNet: GenAI Agent Network
============================

Abstract[​](#abstract "Direct link to Abstract")

-------------------------------------------------

Specialized, finetuned and RAG-enhanced open-source Large Language Models are key elements in emerging AI agent applications. However, those agent apps also present unique challenges to the traditional cloud computing and SaaS infrastructure, including new requirements for application portability, virtualization, security isolation, costs, data privacy, and ownership.

GaiaNet is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise. A GaiaNet node consists of a high-performance and cross-platform application runtime, a finetuned LLM, a knowledge embedding model, a vector database, a prompt manager, an open API server, and a plugin system for calling external tools and functions using LLM outputs. It can be deployed by any knowledge worker as a digital twin and offered as a web API service. A new class of tradeable assets and a marketplace could be created from individualized knowledge bases and components. Similar GaiaNet nodes are organized into GaiaNet domains, which offer trusted and reliable AI agent services to the public. The GaiaNet node and domains are governed by the GaiaNet DAO (Decentralized Autonomous Organization). Through Purpose Bound Money smart contracts, the GaiaNet network is a decentralized marketplace for AI agent services.

Introduction[​](#introduction "Direct link to Introduction")

-------------------------------------------------------------

The emergence of ChatGPT and Large Language Model (LLM) has revolutionized how humans produce and consume knowledge. Within a year, AI-native applications have evolved from chatbots to copilots, to agents.

> AI agents would increasingly evolve from supportive tools (akin to Copilots) to autonomous entities capable of completing tasks independently. — Dr. Andrew Ng at Sequoia Capital AI Ascent 2024 Summit

Agents are software applications that can complete tasks on its own autonomously like a human. The agent can understand the task, plan the steps to complete the task, execute all the steps, handle errors and exceptions, and deliver the results. While a powerful LLM could act as the “brain” for the agent, we need to connect to external data sources (eyes and ears), domain-specific knowledge base and prompts (skills), context stores (memory), and external tools (hands). For agent tasks, we often need to customize the LLM itself

*   to reduce hallucinations in a specific domain.
*   to generate responses in a specific format (e.g., a JSON schema).
*   to answer “politically incorrect” questions (e.g., to analyze CVE exploits for an agent in the security domain).
*   and to answer requests in a specific style (e.g., to mimic a person).

![What is a GaiaNet agent](/assets/images/gaianet_agent-f614c7c96d8253e1b4024e396b96169e.png)

Agents are complex software that require significant amount of engineering and resources. Today, most agents are close-source and hosted on SaaS-based LLMs. Popular examples include GPTs and Microsoft/GitHub copilots on OpenAI LLMs, and Duet on Google’s Gemini LLMs.

However, as we discussed, a key requirement for agents is to customize and adapt its underlying LLM and software stack for domain-specific tasks — an area where centralized SaaS perform very poorly. For example, with ChatGPT, every small task must be handled by a very large model. It is also enormously expensive to fine-tune or modify any ChatGPT models. The one-size-fits-all LLMs are detrimental to the agent use case in capabilities, alignment, and cost structure. Furthermore, the SaaS hosted LLMs lack privacy controls on how the agent’s private knowledge might be used and shared. Because of these shortcomings, it is difficult for individual knowledge workers to create and monetize agents for his or her own domain and tasks on SaaS platforms like OpenAI, Google, Anthropic, Microsoft and AWS.

In this paper, we propose a decentralized software platform and protocol network for AI agents for everyone. Specifically, our goals are two-folds.

**Goal #1:** Empower individuals to incorporate his/her private knowledge and expertise into personal LLM agent apps. Those apps aim to perform knowledge tasks and use tools just as the individual would, but also reflect the individual’s style and values.

**Goal #2:** Enable individuals to provide and scale their LLM agents as services, and get compensated for their expertise and work.

> GaiaNet is “YouTube for knowledge and skills.”

Open-source and decentralization[​](#open-source-and-decentralization "Direct link to Open-source and decentralization")

-------------------------------------------------------------------------------------------------------------------------

As of April 2024, there are over 6000 open-source LLMs published on Hugging face. Compared with close-source LLMs, such as GPT-4, open-source LLMs offer advantages in privacy, cost, and systematic bias. Even with general QA performance, open-source LLMs are closing the gap with close-source counterparties quickly.

![Open vs close source LLMs](/assets/images/closed_vs_open-7465b42b1c6b33cfebee85ed6c721e5a.jpg)

For AI agent use cases, it has been demonstrated that smaller but task-specific LLMs often outperform larger general models.

However, it is difficult for individuals and businesses to deploy and orchestrate multiple finetuned LLMs on their own heterogeneous GPU infrastructure. The complex software stack for agents, as well as the complex interaction with external tools, are fragile and error-prone.

Furthermore, LLM agents have entirely different scaling characteristics than past application servers. LLM is extremely computationally intensive. A LLM agent server can typically only serve one user at a time, and it often blocks for seconds at a time. The scaling need is no longer to handle many async requests on a single server, but to load balance among many discrete servers on the internet scale.

The GaiaNet project provides a cross-platform and highly efficient SDK and runtime for finetuned open-source LLMs with proprietary knowledge bases, customized prompts, structured responses, and external tools for function calling. A GaiaNet node can be started in minutes on any personal, cloud, or edge device. It can then offer services through an incentivized web3 network.

GaiaNet node[​](#gaianet-node "Direct link to GaiaNet node")

-------------------------------------------------------------

The basic operational unit in the GaiaNet network is a node. A GaiaNet node is a streamlined software stack that allows any technically competent person to run an AI agent of his own. The software stack on the GaiaNet node consists of the following 7 key components.

![GaiaNet node architecture](/assets/images/gaianet_node-91f817661a1d3e1f4f7628f809ed6c70.png)

**1 Application runtime.** GaiaNet applications run in a lightweight, secure and high-performance sandbox called WasmEdge. As an open-source project managed by the Linux Foundation and CNCF, WasmEdge runtime works seamlessly with leading cloud native tools such as Docker, containerd, CRI-O, Podman and Kubernetes. It is also the virtual machine of choice by leading public blockchains to securely and efficiently execute on-chain and off-chain smart contracts.

WasmEdge is a high-performance and cross-platform runtime. It can run AI models on almost all CPUs, GPUs, and AI accelerators at native speed, making it an ideal runtime for decentralized AI agents.

**2 Finetuned LLM.** The GaiaNet node supports almost all open-source LLMs, multimodal models (eg Large Vision Models or LVMs), text-to-image models (eg Stable Diffusion) and text-to-video models. That includes all finetuned models using personal or proprietary data.

The node owner can finetune open-source models using a wide variety of tools. For example, the node owner can finetune an LLM using personal chat histories so that the finetuned LLM can mimic his own speaking style. He can also finetune an LLM to focus it on a specific knowledge domain to reduce hallucinations and improve answer quality for questions in that domain. A finetuned LLM can guarantee to output JSON text that matches a pre-determined schema for use with external tools.

Besides LLMs, the node owner could finetune Stable Diffusion models with her own photos to generate images that look like her.

**3 Embedding model.** The GaiaNet node needs to manage a body of public or proprietary knowledge for the AI agent. It is a key feature that enables the agent to specialize and outperform much larger models in a specific domain. The embedding models are specially trained LLMs that turns input sentences into a vector representation, instead of generating completions. Since the embedding models are trained from LLMs, they can “embed” the “meaning” of the sentences into the vectors so that similar sentences are located close together in the high dimensional space occupied by those vectors.

With the embedding model, a GaiaNet node can ingest a body of text, images, PDFs, web links, audio and video files, and generate a collection of embedding vectors based on their contents. The embedding model also turns user questions and conversations into vectors, which allows the GaiaNet node to quickly identify contents in its knowledge base that are relevant to the current conversation.

**4 Vector database.** The embedding vectors that form GaiaNet node’s knowledge base are stored on the node itself for optimal performance and maximum privacy. The GaiaNet node includes a Qdrant vector database.

**5 Custom prompts.** Besides finetuning and knowledge arguments, the easiest way to customize an LLM for new applications is simply to prompt it. Like humans, LLMs are remarkable one-shot learners. You can simply give it an example of how to accomplish a task, and it will learn and do similar tasks on its own. Prompt engineering is a practical field to research and develop such prompts.

Furthermore, effective prompts could be highly dependent on the model in use. A prompt that works well for a large model, such as Mixtral 8x22b, is probably not going to work well for a small model like Mistral 7b.

The GaiaNet node can support several different prompts that are dynamically chosen and used in applications. For example,

*   The `system_prompt` is a general introduction to the agent task the node is supposed to perform. It often contains a persona to help the LLM respond with the right tone. For example, the `system_prompt` for a college teaching assistant could be: “You are a teaching assistant for UC Berkeley’s computer science 101 class. Please explain concepts and answer questions in detail. Do not answer any question that is not related to math or computer science.”
*   The `rag_prompt` is a prefix prompt to be dynamically inserted in front of knowledge base search results in an RAG chat. It could be something like this: “Please answer the question based on facts and opinions in the context below. Do not make anything that is not in the context. ---------”

The LLM community has developed many useful prompts for different application use cases. GaiaNet node allows you to easily manage and experiment with them.

Through the our developer SDK, GaiaNet owners and operators could customize the logic of dynamic prompt generation in their own way. For example, a GaiaNet node could perform a Google search for any user question, and add the search results into the prompt as context.

**6 Function calls and tool use.** The LLM not only is great at generating human language, but also excels at generating machine instructions. Through finetuning and prompt engineering, we could get some LLMs to consistently generate structured JSON objects or computer code in many language tasks, such as summarizing and extracting key elements from a paragraph of text.

The GaiaNet node allows you to specify the output format of the generated text. You can give it a grammar specification file to enforce that responses will always conform to a pre-defined JSON schema.

Once the LLM returns a structured JSON response, the agent typically need to pass the JSON to a tool that performs the task and comes back with an answer. For example, the user question might be.

    What is the weather like in Singapore?

The LLM generates the following JSON response.

    {"tool":"get_current_weather", "location":"Singapore","unit":"celsius"}

The GaiaNet node must know what is the tool associated with get\_current\_weather and then invoke it. GaiaNet node owners and operators can configure any number of external tools by mapping a tool name with a web service endpoint. In the above example, the get\_current\_weather tool might be associated with a web service that takes this JSON data. The GaiaNet node sends the JSON to the web service endpoint via HTTPS POST and receives an answer.

    42

It then optionally feeds the answer to the LLM to generate a human language answer.

    The current weather in Singapore is 42C. 

Through the GaiaNet node SDK, developers are not limited to using web services. They can write plugins to process LLM responses locally on the node. For example, the LLM might return Python code, which can be executed locally in a sandbox and for the GaiaNet node to perform a complex operation.

**7 The API server.** All GaiaNet nodes must have the same API for questions and answers. That allows front-end applications to work with, and potentially be load-balanced to any GaiaNet node. We choose to support the OpenAI API specification, which enables GaiaNet nodes to become drop-in replacements for OpenAI API endpoints for a large ecosystem of applications.

The API server runs securely and cross-platform on the WasmEdge runtime. It ties together all the other components in the GaiaNet node. It receives user requests, generates an embedding from the request, searches the vector database, adds search results to the prompt context, generates an LLM response, and then optionally uses the response to perform function calling. The API server also provides a web-based chatbot UI for users to chat with the RAG-enhanced finetuned LLM on the node.

GaiaNet network[​](#gaianet-network "Direct link to GaiaNet network")

----------------------------------------------------------------------

While each GaiaNet node is already a powerful AI agent capable of answering complex questions and performing actions, individual nodes are not suitable for providing public services. There are several important reasons.

*   For the public consumers and users, it is very hard to judge the trustworthiness of individual GaiaNet nodes. Harmful misinformation could be spread by malicious node operators.
*   For GaiaNet node owners and operators, there is no economic incentive to provide such services to the public, which could be very costly to run.
*   The AI agent servers have very different scaling characteristics than traditional internet application servers. When the agent is processing a user request, it typically takes up all the computing resources on the hardware. Instead of using software to scale concurrent users on a single server, the challenge of GaiaNet is to scale to many different identical nodes for a large application.

Those challenges have given rise to the GaiaNet domain, which forms the basis of the GaiaNet web3 network. A GaiaNet domain is a collection of GaiaNet nodes available under a single Internet domain name. The domain operator decides which GaiaNet nodes can be registered under the domain and makes the node services available to the public. For example, a GaiaNet domain might be a Computer Science teaching assistant for UC Berkeley. The domain could provide services through `https://cs101.gaianet.berkeley.edu`. The domain operator needs to do the following.

*   Verify and admit individual nodes to be registered under the domain. Those nodes must all meet requirements, such as the LLM, knowledge base, and prompts, set by the domain operator to ensure service quality. The node registration on a domain could be done via a whitelist or blacklist. It is up to the domain operator.
*   Monitor each node’s performance at real time and remove inactive ones.
*   Promotes the “teaching assistant” chatbot apps to the target audience.
*   Set the price for the API services.
*   Load balance between active nodes.
*   Getting paid by users.
*   Pay nodes for their services.

![GaiaNet network architecture](/assets/images/gaianet_eco-cd68b47237f0227986c6317281b31ccb.png)

Each GaiaNet node has an unique node ID in the form of an ETH address. The private key associated with the ETH address is stored on the node. Once a node is successfully registered with a domain, it is entitled to receive payments from both service revenue and network awards from the domain. The domain could send payments directly to the node's ETH address. Or, the domain could provide a mechanism for a node operator to register multiple nodes under a single Metamask address, such as signing a challenge phrase using the node private keys. In that case, the node operator will receive aggregated payments in his Metamask account for all associated nodes.

Each GaiaNet domain has an associated smart contract that is used for escrow payments. It is similar to OpenAI’s credit payment model, where users purchase credits first, and then consume them over time. When the user pays into the smart contract, an access token will be automatically issued to him. He uses this token to make API calls to the domain, which is then load-balanced to random nodes in the domain. As the user consumes those services, his fund in the contract depletes and the access token stops working if he no longer has any balance.

The pricing and payment of the API service are determined by the domain operator. It is typically denominated in USD stable coins. The domain operator pays a share of the revenue to node operators who provided the services. The GaiaNet network is a decentralized marketplace of agent services.

> The funds locked in GaiaNet domain contracts are for a single purpose of consuming API services. It is called Purpose Bound Money.

A key aspect of the GaiaNet protocol is that the domain operators are “trust providers” in the ecosystem of decentralized nodes. The protocol network is designed to incentivize the trust of the operators through tokenomics designs such as mining and staking. GaiaNet nodes, domains, users, and developers form a DAO to grow the network and benefit all contributors.

GaiaNet token[​](#gaianet-token "Direct link to GaiaNet token")

----------------------------------------------------------------

The GaiaNet token is a utility token designed to facilitate transactions, support governance, and foster trust in the network. It serves three primary purposes.

*   As a DAO governance token, holders can participate in setting the rules of the network.
*   As a staking token, holders vouch for domain operators’ trustworthiness. Stakers get a cut from the domain operator’s service revenue. But they could also be slashed if the domain operator misbehave, such as spreading misinformation or providing unreliable services.
*   As a payment token, the GaiaNet token could be deposited into the domain’s escrow contract and be used to pay for services over time.

The payment utility of the GaiaNet token is designed to balance the network supply and demand. The value of the GaiaNet token asset is determined at the time when it enters or leaves the escrow smart contract based on real-time exchange rates.

Service consumers could lock in savings from the potential appreciation of the token. For example, if a user deposits $100 worth of GaiaNet tokens into the contract, and when the domain and nodes get paid, the token value has gone up to $110, he would have received $110 worth of agent services.

Conversely, if the token price drops, the service providers (domains and nodes) now have an opportunity to “mine” the tokens on the cheap. If the $100 initial tokens is only worth $90 now, service providers will get more tokens for each unit of electricity and compute they provide. That incentivizes more nodes to join the network and speculate on a later rise in token value.

> An exercise: OpenAI is projected to reach $5 billion in ARR in 2024. Assume that most enterprise customers pay quarterly, that is $1.25 billion of circulation market cap in addition to OpenAI’s current enterprise value if they were to issue a payment token. The overall AI services market size is projected to reach $2 trillion in a few years. That translates to $500 billion market cap for a payment utility token alone.

Component marketplace for AI assets[​](#component-marketplace-for-ai-assets "Direct link to Component marketplace for AI assets")

----------------------------------------------------------------------------------------------------------------------------------

GaiaNet is a developer platform to create your agent services. We provide tools for you to do these.

*   Tools to generate finetuning datasets and perform finetuning on CPU and GPU machines.
*   Tools to ingest documents and create vector embeddings for the knowledge base.
*   Rust-based SDK to dynamically generate and manage prompts.
*   Rust-based SDK to extend the agent’s capability for invoking tools and software on the node.

For developers who do not wish to operate nodes, we are building a marketplace for

*   finetuned models
*   knowledge bases and datasets
*   function-calling plugins

All those components are blockchain-based assets represented by NFTs. A node operator could purchase NFTs for the components it wishes to use, and share service revenue with the component developers. That enables diverse and cashflow-generating assets to be issued from the GaiaNet ecosystem.

Conclusion[​](#conclusion "Direct link to Conclusion")

-------------------------------------------------------

GaiaNet provides open-source tools for individuals and teams to create agent services using their proprietary knowledge and skills. Developers could create finetuned LLMs, knowledge collections, and plugins for the agent, and issue assets based on those components. The GaiaNet protocol makes those nodes discoverable and accessible through GaiaNet domains.

*   [Abstract](#abstract)
    
*   [Introduction](#introduction)
    
*   [Open-source and decentralization](#open-source-and-decentralization)
    
*   [GaiaNet node](#gaianet-node)
    
*   [GaiaNet network](#gaianet-network)
    
*   [GaiaNet token](#gaianet-token)
    
*   [Component marketplace for AI assets](#component-marketplace-for-ai-assets)
    
*   [Conclusion](#conclusion)

---

# Customizing Your Gaia Node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

A key goal of the Gaia project is to enable each individual to create and run his or her own agent service node using finetuned LLMs and proprietary knowledge. In all likelihood, you are not going to run a node with the [default](/getting-started/quick-start)
 Llama 3.2 LLM and Paris guidebook knowledge base.

In this chapter, we will discuss ways to customize your node.

Pre-set configurations[​](#pre-set-configurations "Direct link to Pre-set configurations")

-------------------------------------------------------------------------------------------

All the node configuration options, such as LLM settings, vector collection for the knowledge base, and prompts, are all in the `gaianet/config.json` file. You can edit this file directly to use your models and vector collections.

Or, you can select a different `config.json` when you initialize the node. Just pass in a URL to the `config.json` file in your `gaianet init` command.

We have several pre-set `config.json` files to choose from [in this repo](https://github.com/GaiaNet-AI/node-configs)
.

For example, the following command initialize a Gaia node with a Llama 3 8B model.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json

> The URL to the `config.json` must point to the actual text file. (i.e., the `raw.githubusercontent.com` URL for GitHub links) instead of the GitHub HTML page for that file.

The config subcommand[​](#the-config-subcommand "Direct link to The config subcommand")

----------------------------------------------------------------------------------------

After you have initialized the node, you can still make changes to its configuration by editing the `config.json` file directly. But it is easier and safer to use the `gaianet` CLI to make changes.

note

You MUST run `gaianet init` and `gaianet start` again after you make any changes to the node configuration.

The following command shows the `config.json` fields you can make changes to.

    gaianet config list

Now, let's look at some examples.

### Select an LLM[​](#select-an-llm "Direct link to Select an LLM")

There are over 10,000 finetuned open-source LLMs you can choose from on Huggingface. They each have different sizes (larger models are more capable but more expensive to run), unique capabilities (e.g., uncensored, to excel in math or reasoning, to support large context length etc), domain expertise (e.g., medicine, coding), and / or styles (e.g., to speak like a teacher or a pirate, to respond in code, to follow conversations).

To replace Gaia node's default LLM with an alternative finetuned model, you will need to make changes to the model file, prompt template, and model context length parameters. Those parameters vary depending on the model, but they can be found on the [Gaia Huggingface organization's](https://huggingface.co/gaianet)
 model cards. For example, the following command changes the LLM to a Llama 3 8B model.

    gaianet config \  --chat-url https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf \  --chat-ctx-size 4096 \  --prompt-template llama-3-chat 

> The llama 3 8B model requires at least 16GB of RAM.

If none of the published finetuned models are perfect for your use case, you can also finetune your own LLM by following [these guides](/tutorial/llamacpp)
. Your Gaia node can run your own finetuned models.

> The `--chat-url` argument could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a privately trained or finetuned LLM model file.

### Select a knowledge base[​](#select-a-knowledge-base "Direct link to Select a knowledge base")

A key feature of Gaia is that users can create and deploy proprietary knowledge base on the node to supplement the LLM. Each knowledge base is a snapshot file for a vector collection. We encourage you to [create your own knowledge base](/knowledge-bases/how-to)
. But you can also use ready-made knowledge bases. You will need to do the following.

*   specify the URL to the vector collection (i.e., the `snapshot` or `snapshot.tar.gz` file) in the `snapshot` option.
*   use the same embedding model that generated this vector collection.
*   modify the `system_prompt` to give the model background knowledge.
*   modify the `rag_prompt` to instruct the model to answer the question when context is retrieved from the vector collection.

The following example changes the knowledge base in the node from "Paris guidebook" to "London guidebook".

    gaianet config \  --snapshot https://huggingface.co/datasets/gaianet/london/resolve/main/london_768_nomic-embed-text-v1.5-f16.snapshot.tar.gz \  --embedding-url https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf \  --embedding-ctx-size 8192 \  --system-prompt "You are a tour guide in London, UK. Please answer the question from a London visitor accurately." \  --rag-prompt "The following text is the context for the user question.\n----------------\n"

> The `--snapshot` could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a private vector collection snapshot.

Depending on the quality and size of the vectors, you might also need to change the `qdrant-` options to customize the retrieval behavior.

*   `qdrant-limit` sets the max number of relevant context to add to the prompt. If your knowledge base consists of large sections of text (i.e., each book chapter is a vector), you should probably make this 1 or 2 to limit the prompt length to a reasonable size.
*   `qdrant-score-threshold` is the min match "score" the knowledge content must meet in order to be considered "relevant". This depends on the quality of the knowledge text and the embedding model. In general, this score should be over 0.5 to reduce irrelevant context in the prompt.

> The embedding model encodes and transforms text into vectors so that the can be stored, searched and retrieved. For different context material, you might need a different embedding model to achieve the optimal performance. The [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
>  is a good place to see the performance benchmarks of embedding models. You can find many of them in the [gaia organization on Huggingface](https://huggingface.co/gaianet)
> .

### Customize prompts[​](#customize-prompts "Direct link to Customize prompts")

In `config.json`, you can also customize the prompts. Prompts are often tailored for the finetuned LLM or the knowledge base to generate optimal responses from the node.

The `--system-prompt` option sets a system prompt. It provides the background and "personality" of the node. Each API request can set its own system prompt.

The `--rag-prompt` is the prompt to be appended after the system prompt (or user query). It introduces the RAG context retrieved from the vector database, which follows it.

The `--rag-policy` option specifies where the `rag-prompt` and context should go. By default, its value is `system-message` and it puts the context in the system prompt. But you could also set it to `last-user-message`, which puts the `rag-prompt` and context in front of the latest message from the user.

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

Remember to re-initialize and re-start the node after you make configuration changes.

    # If the node is running# gaianet stopgaianet initgaianet start

Next, you can

*   [Create a knowledge base](/knowledge-bases/how-to)
     from your proprietary knowledge or skills.
*   [Finetune](/tutorial/llamacpp)
     your own LLM.

Have fun!

*   [Pre-set configurations](#pre-set-configurations)
    
*   [The config subcommand](#the-config-subcommand)
    *   [Select an LLM](#select-an-llm)
        
    *   [Select a knowledge base](#select-a-knowledge-base)
        
    *   [Customize prompts](#customize-prompts)
        
*   [Next steps](#next-steps)

---

# OpenAI ecosystem apps | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Since each Gaia node provides an OpenAI-compatible API service, it can be a drop-in replacement for OpenAI in almost all LLM applications and frameworks. Checkout the articles in this section for instructions and examples for how to use Gaia in popular LLM apps.

The OpenAI Python library[​](#the-openai-python-library "Direct link to The OpenAI Python library")

----------------------------------------------------------------------------------------------------

note

Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](/getting-started/authentication)
 tutorial.

You can install the [official OpenAI Python library](https://pypi.org/project/openai/)
 as follows.

    pip install openai

When you create an OpenAI client using the library, you can pass in the API endpoint point as the `base_url`. Remember to append the `/v1` after the host name. You can find a list of public nodes [here](/nodes)
.

    import openaiclient = openai.OpenAI(base_url="https://YOUR-NODE-ID.us.gaianet.network/v1", api_key="YOUR_API_KEY_GOES_HERE")

Alternatively, you could set an environment variables at the OS level.

    export OPENAI_API_BASE=https://YOUR-NODE-ID.us.gaianet.network/v1export OPENAI_API_KEY=YOUR_API_KEY_GOES_HERE

Then, when you make API calls from the `client`, make sure that the `model` is set to the model name available on your node.

    response = client.chat.completions.create(    model="Meta-Llama-3-8B-Instruct-Q5_K_M",    messages=[        {"role": "system", "content": "You are a strategic reasoner."},            {"role": "user", "content": "What is the purpose of life?"}        ],        temperature=0.7,        max_tokens=500    ])

That's it! You can now take any application built with the official OpenAI Python library and use a Gaia node as its backend!

The OpenAI Node API library[​](#the-openai-node-api-library "Direct link to The OpenAI Node API library")

----------------------------------------------------------------------------------------------------------

note

Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](/getting-started/authentication)
 tutorial.

You can install the [OpenAI Node library](https://www.npmjs.com/package/openai)
 which provides convenient access to the OpenAI REST API from TypeScript or JavaScript as follows:

    npm install openai

Import it into your project as:

    // Example usage in Node.jsconst OpenAI = require('openai');

Create an OpenAI client with a custom base URL. Remember to append the `/v1` after the host name.

    const client = new OpenAI({  baseURL: 'https://YOUR-NODE-ID.us.gaianet.network/v1',  apiKey: 'YOUR_API_KEY_GOES_HERE'});

Alternatively, you can set an environment variable using `dotenv` in Node.

    process.env.OPENAI_API_BASE = 'https://YOUR-NODE-ID.us.gaianet.network/v1';

Then, when you make API calls from the `client`, make sure that the `model` is set to the model name available on your node.

    async function callOpenAI() {  try {    const response = await client.chat.completions.create({      model: "Meta-Llama-3-8B-Instruct-Q5_K_M",      messages: [        { role: "system", content: "You are a strategic reasoner." },        { role: "user", content: "What is the purpose of life?" }      ],      temperature: 0.7,      max_tokens: 500    });    console.log(response.choices[0].message.content);  } catch (error) {    console.error('Error:', error);  }}//UsagecallOpenAI();

*   [The OpenAI Python library](#the-openai-python-library)
    
*   [The OpenAI Node API library](#the-openai-node-api-library)

---

# Gaia CLI options | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

After installing the Gaia software, you can use the `gaianet` CLI to manage the node. The following are the CLI options.

help[​](#help "Direct link to help")

-------------------------------------

You can use `gaianet --help` to check all the available CLI options.

    gaianet --help## OutputUsage: gaianet {config|init|run|stop|OPTIONS}Subcommands:  config             Update the configuration.  init               Initialize the GaiaNet node.  run|start          Start the GaiaNet node.  stop               Stop the GaiaNet node.Options:  --help             Show this help message

version[​](#version "Direct link to version")

----------------------------------------------

You can use `gaianet --version` to check your GaiaNet version.

    gaianet --version

init[​](#init "Direct link to init")

-------------------------------------

The `gaianet init` command initializes the node according to the `$HOME/gaianet/config.json` file. You can use some of our [pre-set configurations](https://github.com/GaiaNet-AI/node-configs)
.

*   `gaianet init` will init the default node. It's an RAG application with Gaia knowledge.
*   `gaianet init --config mua` will init a node with the MUA project knowledge.
*   `gaianet init --base <path>` will init a node in an alternative directory.

You can also use `gaianet init url_your_config_json` to init your customized settings for the node. You can customize your node using the Gaia node link. If you're familiar with the Gaia config.json, you can create your own manually. See an example [here](https://github.com/GaiaNet-AI/gaianet-node/blob/main/config.json)
.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/pure-llama-3-8b/config.json

start[​](#start "Direct link to start")

----------------------------------------

The `gaianet start` is to start running the node.

*   Use `gaianet start` to start the node according to the `$HOME/gaianet/config.json` file.
*   Use `gaianet start --base $HOME/gaianet-2.alt` to start the node according to the `$HOME/gaianet-2/config.json` file.
*   Use `gaianet start --local-only` to start the node for local use according to the `$HOME/gaianet/config.json` file.

stop[​](#stop "Direct link to stop")

-------------------------------------

The `gaianet stop` is to stop the running node.

*   Use `gaianet stop` to stop running the node.
*   Use `gaianet stop --force` to force stop the Gaia node.
*   Use `gaianet stop --base $HOME/gaianet-2.alt` to stop the node according to the `$HOME/gaianet-2/config.json` file.

config[​](#config "Direct link to config")

-------------------------------------------

The `gaianet config` can update the key fields defined in the `config.json` file.

*   `gaianet config --help` will list all the available arguments
*   `gaianet config --chat-url <url>` will change the download link of the chat model.
*   `gaianet config --prompt-template <template>` will change the prompt\_template of the chat model.
*   `gaianet config --chat-ctx-size <size>` will change the context size of the chat model. The default value is 4096.
*   `gaianet config --embedding-url <url>` will change the download link of the embedding model.
*   `gaianet config --embedding-ctx-size <size>` will change the context size of the embedding model. The value here is associated with the embedding model you choose.
*   `gaianet config --port <port>` will change the port of the Gaia node API server.
*   `gaianet config --system-prompt "<prompt>"` will change the system prompt.
*   `gaianet config --rag-prompt "<prompt>"` will change the rag prompt.
*   `gaianet config --reverse-prompt "<prompt>"` will change the reverse prompt.
*   `gainet config --base <path>` will modify the `<path>/config.json` parameters.

After you use `gaianet conifg` to change some parameters, please

1.  use `gaianet init` to make your settings take effect.
2.  use `gaianet start` to start your new node.

If you use `gaianet config --base $HOME/gaianet-2.alt` to update some settings, please

1.  use `gaianet init --base $HOME/gaianet-2.alt` to make your settings take effect.
2.  use `gaianet start --base $HOME/gaianet-2.alt` to start your new node.

The `gaianet config` supports multiple parameters in one command. The example below will change the download link and prompt template of the chat model at the same time.

    gaianet config --chat-url https://huggingface.co/gaianet/gemma-1.1-2b-it-GGUF/resolve/main/gemma-1.1-2b-it-Q5_K_M.gguf --prompt-template gemma-chat

The output is the following.

    [+] Updating the url of chat model ...    * Old url: https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf    * New url: https://huggingface.co/gaianet/gemma-1.1-2b-it-GGUF/resolve/main/gemma-1.1-2b-it-Q5_K_M.gguf[+] Updating the prompt template of chat model ...    * Old template: llama-3-chat    * New template: gemma-chat[+] COMPLETED! The config.json is updated successfully.

base[​](#base "Direct link to base")

-------------------------------------

The `--base` option is global. You can combine it with other subcommands to specify a base directory for the Gaia node other than the `$HOME/gaianet`.

*   [help](#help)
    
*   [version](#version)
    
*   [init](#init)
    
*   [start](#start)
    
*   [stop](#stop)
    
*   [config](#config)
    
*   [base](#base)

---

# Joining the Gaia Protocol | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

After successfully running a Gaia node on your machine, it's time to join the Gaia protocol network and get rewards for sharing computing power with the world by binding your node ID and device ID to a Metamask account.

When you run a node with the Getting Started guide, you may notice that the Gaia software has generated a node ID for you. The node ID is an ETH address. The easiest way to find the node ID is to use `gaianet info` to print the node ID and device ID on the terminal.

    gaianet info

The output will be the following:

    Node ID: 0x80642b1----9a76a284efeb2bb6d142eDevice ID: device-e77---1446f1b51

![](/assets/images/register-01-fcaa9730a879aeaf3f91746bce5b1c68.png)

To receive rewards, bind your node and device IDs with a Metamask account using the Gaia web portal:

1.  Open [https://www.gaianet.ai/](https://www.gaianet.ai/)
     on your browser and click **"Connect."**
2.  Log into Metamask using the QR code and connect your account.
3.  Hover on your profile and click **"Setting"** and click **"Nodes."**
4.  Click **"Connect new node"** and enter your node and device IDs in the boxes.
5.  Click **"Join"**  
    ![](/assets/images/node-register-57acac5fa455af6b6b35b500149bd9a2.png)

After your node has successfully joined the network, it will be displayed in the list of nodes on the Node Page.

note

If you are running multiple nodes, you can bind the node IDs and their corresponding device IDs to your MetaMask wallet.

### Protect your node ID and device ID[​](#protect-your-node-id-and-device-id "Direct link to Protect your node ID and device ID")

The Gaia installer generates a pair of ETH address and keystore and password for your node automatically. This information is stored in the `gaianet/nodeid.json` file. Please keep the JSON file carefully.

*   The ETH address is your node ID. You will use this ETH address to join the Gaia network.
*   The keystore stores the private key associated with the ETH address encrypted by the password.

tip

The `nodeid.json` file is the only proof that your node belongs to you. In many protocol operations, you will need this private key to sign request messages to send to the protocol smart contracts.

note

The device ID is only visible to yourself.

Join a Domain[​](#join-a-domain "Direct link to Join a Domain")

----------------------------------------------------------------

Once your node is successfully bound, you can proceed to join a Gaia Domain. There are two ways to join a domain:

*   Join a domain from your nodes management page
*   Join a domain from the [AI Agent Domains page](https://www.gaianet.ai/agents)
    

### Steps to Join a Domain from Your Node Management Page[​](#steps-to-join-a-domain-from-your-node-management-page "Direct link to Steps to Join a Domain from Your Node Management Page")

1.  **Locate Your Nodes**  
    Navigate to **Profile → Nodes** to view the list of nodes you already registered.
    
2.  **Initiate the Join Process**  
    Click the `...` button next to your node and select **Join Domain**. This will guide you through the steps to join a Gaia Domain.
    

![image](https://github.com/user-attachments/assets/e7b0613b-332f-46bd-ae35-88ab5e0e8280)

3.  **Follow the Join Steps**
    
    *   **Update Node Domain**  
        Change your node's domain to `gaia.domains`, which is necessary. The following command line will change your node's domain to `gaia.domains`. If your node has already used `gaia.domains`, then you can skip this step.
        
            gaianet stopgaianet config --domain gaia.domainsgaianet initgaianet start
        
    *   **Select a Domain**  
        Choose a domain from the available online domain list. You can review essential details for each domain, such as:
        
        *   Required LLM
        *   Join policy
        *   Number of joined nodes
    
    ![image](https://github.com/user-attachments/assets/fa99b0ef-e25d-4e51-933a-30062cb480d5)
    
    *   **Verify Node Status**  
        Ensure your node meets the domain's requirements. If everything checks out, you can submit a join request to the domain.
    
    ![image](https://github.com/user-attachments/assets/caafe74c-9393-43a2-a428-1b008b7ce717)
    
    > Normally, your node needs to be online and meet the specific LLM requirement.
    

### Steps to Join a Domain from the AI Agent Domains page[​](#steps-to-join-a-domain-from-the-ai-agent-domains-page "Direct link to Steps to Join a Domain from the AI Agent Domains page")

The AI agent Domains page featured lots of domains various from crypto knowledge to useful tools. Each has options for "Chat Now" or "Join Now". You can click on "Join Now" to join that domain.

![image](https://github.com/user-attachments/assets/6dbcc6fd-f6d8-452a-8e1c-2581bd637c9f)

### Important Notes[​](#important-notes "Direct link to Important Notes")

*   Some Gaia Domains require approval for new nodes to join.
*   Ensure you comply with the domain's rules before your node becomes publicly accessible under the domain's URL.

Following these steps will seamlessly integrate your node into a Gaia Domain.

*   [Protect your node ID and device ID](#protect-your-node-id-and-device-id)
    
*   [Join a Domain](#join-a-domain)
    *   [Steps to Join a Domain from Your Node Management Page](#steps-to-join-a-domain-from-your-node-management-page)
        
    *   [Steps to Join a Domain from the AI Agent Domains page](#steps-to-join-a-domain-from-the-ai-agent-domains-page)
        
    *   [Important Notes](#important-notes)

---

# Troubleshooting | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

The system cannot find CUDA libraries[​](#the-system-cannot-find-cuda-libraries "Direct link to The system cannot find CUDA libraries")

----------------------------------------------------------------------------------------------------------------------------------------

Sometimes, the CUDA toolkit is installed in a non-standard location. The error message here is often not able to find `libcu*12`. For example, you might have CUDA installed with your Python setup. The following command would install CUDA into Python's environment.

    sudo apt install python3-pip -ypip3 install --upgrade fschat accelerate autoawq vllm

The easiest way to fix is simply to link those non-standard CUDA libraries to the standard location, like this.

    ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cublas/lib/libcublas.so.12 /usr/lib/libcublas.so.12ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cuda_runtime/lib/libcudart.so.12 /usr/lib/libcudart.so.12ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cublas/lib/libcublasLt.so.12 /usr/lib/libcublasLt.so.12

Failed to recover from collection snapshot on Windows WSL[​](#failed-to-recover-from-collection-snapshot-on-windows-wsl "Direct link to Failed to recover from collection snapshot on Windows WSL")

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

On Windows WSL, you could see this error while running `gaianet init`.

       * Import the Qdrant collection snapshot ...      The process may take a few minutes. Please wait ...    * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Tokio task join error: task 1242 panicked"},"time":0.697784244}

When you look into the `~/gaianet/log/init-qdrant.log` file, you could see this line of error

    2024-05-20T07:24:52.900895Z ERROR qdrant::startup: Panic occurred in file /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cgroups-rs-0.3.4/src/memory.rs at line 587: called `Result::unwrap()` on an `Err` value: Error { kind: ReadFailed("/sys/fs/cgroup/memory.high"), cause: Some(Os { code: 2, kind: NotFound, message: "No such file or directory" }) }  

The solution is to disable the `autoMemoryReclaim` feature in WSL. Step to turn on/off this feature:

1.  Edit `C:\Users<Your user name>.wslconfig`
2.  Remove or comment out `autoMemoryReclaim` in `[experimental]` section.

![](/assets/images/disable_autoMemoryReclaim_wsl-af7a70038bb2a972c29c0e1b2bc44cf8.png)

Thanks to [RoggeOhta](https://github.com/RoggeOhta)
 for discovering this. You can learn more about it [here](https://github.com/GaiaNet-AI/gaianet-node/issues/46)
.

Failed to start the node with an error message `Port 8080 is in use. Exit ...`[​](#failed-to-start-the-node-with-an-error-message-port-8080-is-in-use-exit- "Direct link to failed-to-start-the-node-with-an-error-message-port-8080-is-in-use-exit-")

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

You may see the following error when you run `gaianet start`.

    gaianet start[+] Checking the config.json file ...You already have a private key.[+] Starting LlamaEdge API Server ...    Port 8080 is in use. Exit ...

The solution is to run `gaianet stop` first to kill all processes, and then run `gaianet start` to start the node.

Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory[​](#load-library-failed-libgompso1-cannot-open-shared-object-file-no-such-file-or-directory "Direct link to Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory")

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

On Windows WSL, you may see this error when running `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash`

    * Generate node IS[2024-07-02 17:50:55.175] [error] loading failed: invalid path, Code: 0x20[2024-07-02 17:50:55.175] [error]   Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory[2024-07-02 17:50:55.176] [error] loading failed: invalid path, Code: 0x20[2024-07-02 17:50:55.176] [error]   Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory

The error is caused by the lack of `libgomp.so.1`, a library that should be automatically installed on Ubuntu by default.

To solve this, you must install the `libgomp.so.1` library.

    sudo apt-get updatesudo apt-get install libgomp1

If you're using CentOS, you can use

    yum install libgomp

This issue was fixed in `version 0.2.2`.

Failed to remove the default collection[​](#failed-to-remove-the-default-collection "Direct link to Failed to remove the default collection")

----------------------------------------------------------------------------------------------------------------------------------------------

    Failed to remove the default collection. {"status":{"error":"Service internal error: No such file or directory (os error 2)"},"time":0.050924542}

It typically indicates that the Qdrant instance was not shut down properly before you try to init it again with a new snapshot. The solution is to stop the GaiaNet node first.

    gaianet stop

Alternatively, you could manually kill the processes from the terminal or in the OS's Activity Monitor.

    sudo pkill -9 qdrantsudo pkill -9 wasmedgesudo pkill -9 frpc

Then you can run `gaianet init` and then `gaianet start` again.

File I/O error[​](#file-io-error "Direct link to File I/O error")

------------------------------------------------------------------

        * Import the Qdrant collection snapshot ...      The process may take a few minutes. Please wait ...    * [Error] Failed to recover from the collection snapshot. An error occurred processing field `snapshot`: File I/O error: Operation not permitted (os error 1) 

It typically indicates that the Qdrant instance was not shut down properly before you try to init it again with a new snapshot. The solution is to stop the GaiaNet node first.

    gaianet stop

Alternatively, you could manually kill the processes from the terminal or in the OS's Activity Monitor.

    sudo pkill -9 qdrantsudo pkill -9 wasmedgesudo pkill -9 frpc

Then you can run `gaianet init` and then `gaianet start` again.

The "Failed to open the file" Error[​](#the-failed-to-open-the-file-error "Direct link to The "Failed to open the file" Error")

--------------------------------------------------------------------------------------------------------------------------------

    Warning: Failed to open the file Warning: https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolveWarning: /main/consensus/consensus.snapshot: No such file or directorycurl: (23) Failure writing output to destination

The reason for this type of error is a mis-configured `config.json` file. The solution is to delete the comments in `config.json` and re-run the `gaianet init` command.

The "Too many open files" Error on macOS[​](#the-too-many-open-files-error-on-macos "Direct link to The "Too many open files" Error on macOS")

-----------------------------------------------------------------------------------------------------------------------------------------------

When running `gaianet init` to initialize a new node on macOS, you may encounter an error related to snapshot recovery if your snapshot contains a large amount of text. The error message may be the following:

     * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Too many open files (os error 24)"},"time":1.574064833}    * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Too many open files (os error 24)"},"time":1.574064833}

This issue is caused by the default file descriptor (FD) limit on macOS, which is set to a relatively low value of 256.

To resolve this issue, you can increase the default FD limit on your system. To do so, run the following command:

    ulimit -n 10000

This will temporarily set the FD limit to 10,000. Next, use `gaianet init` and `gaianet start` commands in the SAME terminal.

Permission denied when use the installer script to install WasmEdge[​](#permission-denied-when-use-the-installer-script-to-install-wasmedge "Direct link to Permission denied when use the installer script to install WasmEdge")

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

When running `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash` to install GaiaNet node software, you may meet the permission denied error especially installing the WasmEdge runtime.

![](/assets/images/troubleshooting-01-38a832c2c9b16055602256ce4af65948.png)

This error is caused by the lack of `/tmp` write permission. You can use `--tmpdir` to specify where you want to install the WasmEdge runtime. Please note, you will need to have written permission to the `<YOUR_PATH>` folder.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --tmpdir YOUR_PATH

This problem is fixed in `version 0.2.3`.

*   [The system cannot find CUDA libraries](#the-system-cannot-find-cuda-libraries)
    
*   [Failed to recover from collection snapshot on Windows WSL](#failed-to-recover-from-collection-snapshot-on-windows-wsl)
    
*   [Failed to start the node with an error message `Port 8080 is in use. Exit ...`](#failed-to-start-the-node-with-an-error-message-port-8080-is-in-use-exit-)
    
*   [Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory](#load-library-failed-libgompso1-cannot-open-shared-object-file-no-such-file-or-directory)
    
*   [Failed to remove the default collection](#failed-to-remove-the-default-collection)
    
*   [File I/O error](#file-io-error)
    
*   [The "Failed to open the file" Error](#the-failed-to-open-the-file-error)
    
*   [The "Too many open files" Error on macOS](#the-too-many-open-files-error-on-macos)
    
*   [Permission denied when use the installer script to install WasmEdge](#permission-denied-when-use-the-installer-script-to-install-wasmedge)

---

# Quick Start with Launching Gaia Domain | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

This guide provides all the information you need to quickly set up and run a Gaia Domain.

> **Note:** Ensure that you are the owner of a Gaia Domain Name before proceeding. You can verify your Gaia Domain Name in the "Assets" section of your profile.

Gaia simplifies the process for domain operators to launch and host a Gaia Domain service in just a few clicks.

### Steps to Launch Your Gaia Domain[​](#steps-to-launch-your-gaia-domain "Direct link to Steps to Launch Your Gaia Domain")

1.  **Access the Create Gaia Domain Page**  
    Click **LAUNCH DOMAIN** in the "Domain" or "Assets" section under [your profile](https://www.gaianet.ai/)
    . This will take you to the Create Gaia Domain page.
    
2.  **Fill in Domain Details**  
    Enter the general information for your domain, including:
    
    *   Domain profile
    *   Domain Name
    *   Description
    *   System Prompt
3.  **Choose a Gaia Domain Name**  
    Select a Gaia domain name from your assets.
    
4.  **Select a Supplier**  
    Currently, **Gaia Cloud** is the only supplier.
    
5.  **Pick a Gaia Domain Tier**  
    Choose a tier to enhance your domain's rewards, which is necessary.
    
6.  **Configure Server and Management Options**
    
    *   Confirm the server configuration for running your domain.
    *   Set management preferences, such as whether nodes can join automatically and the specific LLM to use.

After completing these six steps, your Gaia Domain will be successfully launched and other nodes can join your domain.

*   [Steps to Launch Your Gaia Domain](#steps-to-launch-your-gaia-domain)

---

# Install or Uninstall the CLI | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

The Gaia node utilizes version control from [its source GitHub repo](https://github.com/GaiaNet-AI/gaianet-node)
. You can check out the Gaia node versions from [the release page](https://github.com/GaiaNet-AI/gaianet-node/releases)
.

Install[​](#install "Direct link to Install")

----------------------------------------------

You can install the WasmEdge Runtime on any generic Linux and MacOS platform.

### Install the latest version of Gaia node[​](#install-the-latest-version-of-gaia-node "Direct link to Install the latest version of Gaia node")

To install the most recent version of Gaia node, run the following command line.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash

The Gaia node will be installed in your `$HOME/gaianet` folder by default.

> If you want to install gaianet in a different directory, you can use `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --base <Full_Path>` to specify where you want to install GaiaNet. Once you use `--base` to define a different directory, you should always add `--base <Full_Path>` to init and start your node. Here is an example:
> 
>     # Assume that you're in the root directorymkdir testcurl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s --  --base $HOME/testgaianet init --base $HOME/testgaianet start --base $HOME/testgaianet stop --base $HOME/test

### Install the specific version of Gaia Node[​](#install-the-specific-version-of-gaia-node "Direct link to Install the specific version of Gaia Node")

If you want to install a particular Gaia node version, change the version number in the following command line.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/download/0.1.3/install.sh' | bash

Check out the release log [here](https://github.com/GaiaNet-AI/gaianet-node/releases)
.

Update the current Gaia node[​](#update-the-current-gaia-node "Direct link to Update the current Gaia node")

-------------------------------------------------------------------------------------------------------------

Simply run the following command to upgrade your node.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --upgrade

tip

The `upgrade` option will keep your node id.

What's installed[​](#whats-installed "Direct link to What's installed")

------------------------------------------------------------------------

If you install the Gaia node in the `$HOME/gaianet` directory by default, you will have the following directories and files after installation:

*   The `$HOME/gaianet/bin` directory contains the Gaia CLI tool, frpc binary and Qdrant Vector database binary.
*   The `$HOME/gaianet/` directory contains the `llamaedge-api-server.wasm` and `rag-api-server.wasm` for the LLM inference, dashboard (chatbot ui), nodeid.json for the registering your node, and gaianet-domain binary.
*   The `$HOME/.wasmedge/bin` directory contains the WasmEdge Runtime CLI executable files, which serve as the LLM runtime.

CLI options for the installer[​](#cli-options-for-the-installer "Direct link to CLI options for the installer")

----------------------------------------------------------------------------------------------------------------

You can use the following command line to check out all the available CLI options

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --help

The output should be as follows. You can use the following options to customize your installation.

    Usage:  ./install.sh [Options]Options:  --config <Url>     Specify a url to the config file  --base <Path>      Specify a path to the gaianet base directory  --reinstall        Install and download all required deps  --tmpdir <Path>    Specify a path to the temporary directory [default: /tmp]  --ggmlcuda [11/12] Install a specific CUDA enabled GGML plugin version [Possible values: 11, 12].  --enable-vector:   Install vector log aggregator  --version          Print version  --help             Print usage

Uninstalling the Gaia CLI[​](#uninstalling-the-gaia-cli "Direct link to Uninstalling the Gaia CLI")

----------------------------------------------------------------------------------------------------

To uninstall or clear the environment, run the following command.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/uninstall.sh' | bash

danger

**Important Reminder: This command will remove all the Gaia-related files, including the `nodeid.json`. It's your responsibility to keep your nodeid.json safe. If you want to run the same node after reinstalling, please save the `nodeid.json` file and `frpc.toml` file carefully.**

*   [Install](#install)
    *   [Install the latest version of Gaia node](#install-the-latest-version-of-gaia-node)
        
    *   [Install the specific version of Gaia Node](#install-the-specific-version-of-gaia-node)
        
*   [Update the current Gaia node](#update-the-current-gaia-node)
    
*   [What's installed](#whats-installed)
    
*   [CLI options for the installer](#cli-options-for-the-installer)
    
*   [Uninstalling the Gaia CLI](#uninstalling-the-gaia-cli)

---

# Fine-tune an open-source LLM with llama.cpp | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Fine-tune LLMs
==============

You could fine-tune an open-source LLM to

*   Teach it to follow conversations.
*   Teach it to respect and follow instructions.
*   Make it refuse to answer certain questions.
*   Give it a specific "speaking" style.
*   Make it response in certain formats (e.g., JSON).
*   Give it focus on a specific domain area.
*   Teach it certain knowledge.

To do that, you need to create a set of question and answer pairs to show the model the prompt and the expected response. Then, you can use a fine-tuning tool to perform the training and make the model respond the expected answer for each question.

How to fine-tune an open-source LLM with llama.cpp
==================================================

The popular llama.cpp tool comes with a `finetune` utility. It works well on CPUs! This fine-tune guide is reproduced with permission from Tony Yuan's [Finetune an open-source LLM for the chemistry subject](https://github.com/YuanTony/chemistry-assistant/tree/main/fine-tune-model)
 project.

Build the fine-tune utility from llama.cpp[​](#build-the-fine-tune-utility-from-llamacpp "Direct link to Build the fine-tune utility from llama.cpp")

------------------------------------------------------------------------------------------------------------------------------------------------------

The `finetune` utility in llama.cpp can work with quantized GGUF files on CPUs, and hence dramatically reducing the hardware requirements and expenses for fine-tuning LLMs.

Check out and download the llama.cpp source code.

    git clone https://github.com/ggerganov/llama.cppcd llama.cpp

Build the llama.cpp binary.

    mkdir buildcd buildcmake ..cmake --build . --config Release

If you have NVIDIA GPU and CUDA toolkit installed, you should build llama.cpp with CUDA support.

    mkdir buildcd buildcmake .. -DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcccmake --build . --config Release

Get the base model[​](#get-the-base-model "Direct link to Get the base model")

-------------------------------------------------------------------------------

We are going to use Meta's Llama2 chat 13B model as the base model. Note that we are using a Q5 quantized GGUF model file directly to save computing resources. You can use any of the Llama2 compatible GGUF models on Hugging Face.

    cd .. # change to the llama.cpp directorycd models/curl -LO https://huggingface.co/gaianet/Llama-2-13B-Chat-GGUF/resolve/main/llama-2-13b-chat.Q5_K_M.gguf

Create a question and answer set for fine-tuning[​](#create-a-question-and-answer-set-for-fine-tuning "Direct link to Create a question and answer set for fine-tuning")

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Next we came up with 1700+ pairs of QAs for the chemistry subject. It is like the following in a [CSV file](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.csv)
.

| Question | Answer |
| --- | --- |
| What is unique about hydrogen? | It's the most abundant element in the universe, making up over 75% of all matter. |
| What is the main component of Jupiter? | Hydrogen is the main component of Jupiter and the other gas giant planets. |
| Can hydrogen be used as fuel? | Yes, hydrogen is used as rocket fuel. It can also power fuel cells to generate electricity. |
| What is mercury's atomic number? | The atomic number of mercury is 80 |
| What is Mercury? | Mercury is a silver colored metal that is liquid at room temperature. It has an atomic number of 80 on the periodic table. It is toxic to humans. |

> We used GPT-4 to help me come up many of these QAs.

Then, we wrote a [Python script](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/convert.py)
 to convert each row in the CSV file into a sample QA in the Llama2 chat template format. Notice that each QA pair starts with `<SFT>` as an indicator for the fine-tune program to start a sample. The result [train.txt](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.txt)
 file can now be used in fine-tuning.

Put the [train.txt](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.txt)
 file in the `llama.cpp/models` directory with the GGUF base model.

Finetune![​](#finetune "Direct link to Finetune!")

---------------------------------------------------

Use the following command to start the fine-tuning process on your CPUs. I am putting it in the background so that it can run continuously now. It could take several days or even a couple of weeks depending on how many CPUs you have.

    nohup ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --lora-out lora.bin --train-data train.txt --sample-start '<SFT>' --adam-iter 1024 &

You can check the process every few hours in the `nohup.out` file. It will report the `loss` for each iteration. You can stop the process when the `loss` goes consistently under `0.1`.

**Note 1** If you have multiple CPUs (or CPU cores), you can speed up the fine-tuning process by adding a `-t` parameter to the above command to use more threads. For example, if you have 60 CPU cores, you could do `-t 60` to use all of them.

**Note 2** If your fine-tuning process is interrupted, you can restart it from `checkpoint-250.gguf`. The next file it outputs is `checkpoint-260.gguf`.

    nohup ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in checkpoint-250.gguf --lora-out lora.bin --train-data train.txt --sample-start '<SFT>' --adam-iter 1024 &

Merge[​](#merge "Direct link to Merge")

----------------------------------------

The fine-tuning process updates several layers of the LLM's neural network. Those updated layers are saved in a file called `lora.bin` and you can now merge them back to the base LLM to create the new fine-tuned LLM.

    ../build/bin/export-lora --model-base llama-2-13b-chat.Q5_K_M.gguf --lora lora.bin --model-out chemistry-assistant-13b-q5_k_m.gguf

The result is this file.

    curl -LO https://huggingface.co/juntaoyuan/chemistry-assistant-13b/resolve/main/chemistry-assistant-13b-q5_k_m.gguf

**Note 3** If you want to use a checkpoint to generate a `lora.bin` file, use the following command. This is needed when you believe the final `lora.bin` is an overfit.

    ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in checkpoint-250.gguf --only-write-lora --lora-out lora.bin

*   [Build the fine-tune utility from llama.cpp](#build-the-fine-tune-utility-from-llamacpp)
    
*   [Get the base model](#get-the-base-model)
    
*   [Create a question and answer set for fine-tuning](#create-a-question-and-answer-set-for-fine-tuning)
    
*   [Finetune!](#finetune)
    
*   [Merge](#merge)

---

# System requirements | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

You can install the Gaia on a wide variety of devices and operating systems with or without GPUs. The node installing and operating instructions work on devices ranging from Raspberry Pi, MacBooks, Linux servers, Windows Desktop, to cloud-based Nvidia H100 clusters. For institutional operators, we recommend EITHER of the following for a Gaia node.

*   Mac desktop or server computers (i.e., iMac, Mini, Studio or Pro) with Apple Silicon (M1 to M4), and at least 16GB of RAM (32GB or more recommended).
*   Ubuntu Linux 22.04 server with NVIDIA CUDA 12 SDK installed. At least 8GB of VRAM on the GPU is required (24GB or more recommended). On AWS and Azure, that means GPU instances with at least the Nvidia T4 GPU installed.

> Check out our [tutorial](/getting-started/advanced-deployment-options/cuda)
>  on how to install the NVIDIA driver and the CUDA toolkit on a Ubuntu 22.04 machine.

If you are hosting the node in your home or office, it needs access to the public Internet to join the Gaia network.

Supported on[​](#supported-on "Direct link to Supported on")

-------------------------------------------------------------

Gaia node software is designed to be cross-platform, allowing it to run on various CPU and GPU architectures. The Gaia installer automatically detects the presence of NVIDIA CUDA drivers and leverages the power of GPU accelerators on the device. More hardware support is on the way.

### GPU[​](#gpu "Direct link to GPU")

The Gaia node can run on all types of NVIDIA GPU products from H100 to NVIDIA Jetson series of hardware. It also runs on all Apple Silicon M-series GPUs.

### CPU[​](#cpu "Direct link to CPU")

*   Arm-64 based on CPU chips
*   X86 based on CPU chips
*   Apple M1 chips
*   Apple M1 Pro chips
*   Apple M1 Max chips
*   Apple M1 Ultra chips
*   Apple M2 chips
*   Apple M2 Pro chips
*   Apple M2 Max chips
*   Apple M2 Ultra chips
*   Apple M3 chips
*   Apple M3 Pro chips
*   Apple M3 Max chips
*   Apple M3 Ultra chips

### Oses[​](#oses "Direct link to Oses")

*   macOS
*   Linux-like OS

*   [Supported on](#supported-on)
    *   [GPU](#gpu)
        
    *   [CPU](#cpu)
        
    *   [Oses](#oses)

---

# Advanced Deployment Options | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ Start a node with Docker\
----------------------------\
\
You can run all the commands in this document without any change on any machine with the latest Docker and at least 8GB of RAM available to the container.](/getting-started/advanced-deployment-options/docker)

[📄️ Start a node on AWS using AMI images\
----------------------------------------\
\
We have created a series of public AMIs for you to start Gaia nodes in AWS with just a few clicks.](/getting-started/advanced-deployment-options/aws)

[📄️ Install CUDA on Linux\
-------------------------\
\
If you are using an Nvidia-enabled VM instance from a public cloud, you should probably use the VM image provided by the cloud. It typically has the correct versions of Nvidia driver and CUDA toolkit already installed.](/getting-started/advanced-deployment-options/cuda)

[📄️ Install multiple nodes on a single machine\
----------------------------------------------\
\
The default Gaia installer installs the node into the $HOME/gaianet base directory.](/getting-started/advanced-deployment-options/multiple)

[📄️ Run a local-only node\
-------------------------\
\
By default, the Gaia node registers itself with a Gaia domain and is accesible from the public.](/getting-started/advanced-deployment-options/local)

[📄️ Ensuring Gaia Node Reliability\
----------------------------------\
\
Protect the server process](/getting-started/advanced-deployment-options/protect)

---

# Working with Coinbase AgentKit | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

You can use a Gaia node to power the [Coinbase AgentKit](https://github.com/coinbase/cdp-agentkit)
. The Gaia node must run an LLM that is optimized for [tool calling](/tutorial/tool-call)
. Or, you could simply use our public node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llamatool.us.gaianet.network/v1](https://llamatool.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Quickstart[​](#quickstart "Direct link to Quickstart")

-------------------------------------------------------

First, you need a [Coinbase Developer Platform account](https://www.coinbase.com/developer-platform)
 and then [create an API key](https://docs.cdp.coinbase.com/advanced-trade/docs/auth/#creating-api-keys)
.

Next, check out the AgentKit example code.

    git clone https://github.com/coinbase/cdp-agentkitcd cdp-agentkit/cdp-langchain/examples/chatbot

Set the environment variables for your API key.

    export CDP_API_KEY_NAME='organizations/.../apiKeys/...'export CDP_API_KEY_PRIVATE_KEY='-----BEGIN EC...END EC PRIVATE KEY-----\n'

Edit the `chatbot.py` file to configure the agent to use the Gaia node above.

    llm = ChatOpenAI(model="llama", api_key="GAIA", base_url="https://llamatool.us.gaianet.network/v1")

Finally, run the agent using Python.

    python chatbot.py

You can see a [video demo here](https://x.com/juntao/status/1858634152599224828)
.

A Telegram bot for AgentKit[​](#a-telegram-bot-for-agentkit "Direct link to A Telegram bot for AgentKit")

----------------------------------------------------------------------------------------------------------

We have also built a Telegram bot that allows you to interact with your own wallet through text and voice messages. You will need to bring your own Coinbase credentials and wallet for your bot to operate on. Go to chat the with bot on Telegram.

[https://t.me/agentkit\_bot](https://t.me/agentkit_bot)

It asks you to use slash commands to set your wallet credentials for your bot. It is like this.

    /name organizations/.../apiKeys/.../pk -----BEGIN EC...END EC PRIVATE KEY-----\n/wallet {"wallet_id": "...

Then, you can just tell the Telegram bot what you want to do on-chain, and the agent will do it for you.

*   [Quickstart](#quickstart)
    
*   [A Telegram bot for AgentKit](#a-telegram-bot-for-agentkit)

---

# Working with eliza | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Eliza is a simple, fast, and lightweight AI agent framework. Recently, eliza has integrated Gaia as one of the [model service provider](https://github.com/elizaOS/eliza/pull/762)
. This means you can now use Gaia as the LLM service backend for the Eliza framework.

### Build a Trump agent with eliza and Gaia[​](#build-a-trump-agent-with-eliza-and-gaia "Direct link to Build a Trump agent with eliza and Gaia")

This guide demonstrates how to create an agent with Trump-like characteristics using [the Get Started guide from eliza](https://elizaos.github.io/eliza/docs/quickstart/)
.

#### Set up the environment[​](#set-up-the-environment "Direct link to Set up the environment")

> Note: Ensure your Node.js version is above 0.23.0 before proceeding.

Firstly, clone the Eliza repository:

    git clone https://github.com/elizaos/eliza.gitcd elizagit checkout v0.1.7-alpha.1

Next, install the required dependencies.

    pnpm install

Then, build the local libraries.

    pnpm build

#### Choose Gaia as the model service provider[​](#choose-gaia-as-the-model-service-provider "Direct link to Choose Gaia as the model service provider")

After that, we will need to configure the environment and use a Gaia node as model service provider.

    cp .env.example .env

Then, edit the `.env` file to include Gaia-related configuration values:

    # Gaianet ConfigurationGAIANET_MODEL=llama3bGAIANET_SERVER_URL=https://llama3b.gaia.domains/v1SMALL_GAIANET_MODEL=            # Default: llama3bSMALL_GAIANET_SERVER_URL=       # Default: https://llama3b.gaia.domains/v1MEDIUM_GAIANET_MODEL=           # Default: llamaMEDIUM_GAIANET_SERVER_URL=      # Default: https://llama8b.gaia.domains/v1LARGE_GAIANET_MODEL=            # Default: qwen72bLARGE_GAIANET_SERVER_URL=       # Default: https://qwen72b.gaia.domains/v1GAIANET_EMBEDDING_MODEL=nomic-embedUSE_GAIANET_EMBEDDING=TRUE          # Set to TRUE for GAIANET/768, leave blank for local

By using this configuration, the system will utilize the Llama 3b Gaia domain as the LLM backend. You can replace `GAIANET_SERVER_URL` with a URL for your custom node or domain.

#### Create the Trump agent[​](#create-the-trump-agent "Direct link to Create the Trump agent")

The default character templates are located in the `characters` folder.

Update the `modelProvider` for the desired character. For Gaia, the provider name is `gaianet`.

        "name": "trump",    "clients": [],    "modelProvider": "gaianet",    "settings": {        "secrets": {},        "voice": {            "model": "en_US-male-medium"        }    },    "plugins": [],

Then, we can use the following command line to start running the agent.

    pnpm run dev --character="characters/trump.character.json"

After the service runs successfully, we can launch the client UI to interact with the agent:

    pnpm start:client

Finally, open `http://localhost:5174/` on your browser to start chatting with the agent.

![image](https://github.com/user-attachments/assets/48474b35-1d40-4334-8ed0-c97f373c2b0b)

### Advanced use case[​](#advanced-use-case "Direct link to Advanced use case")

For more inspiration, refer to [Nader Dabit's example on building a Twitter AI bot](https://x.com/dabit3/status/1863772029565981144)
.

*   [Build a Trump agent with eliza and Gaia](#build-a-trump-agent-with-eliza-and-gaia)
    
*   [Advanced use case](#advanced-use-case)

---

# API Reference | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Introduction[​](#introduction "Direct link to Introduction")

-------------------------------------------------------------

Each Gaia node is an OpenAI compatible API server. You can build your application based on the Gaia node API. You can also replace OpenAI API configuration with the Gaia node API in other AI agent frameworks.

The base URL to send all API requests is `https://node_id.gaianet.network/v1`.

note

Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](/getting-started/authentication)
 tutorial.

Endpoints[​](#endpoints "Direct link to Endpoints")

----------------------------------------------------

### Chat[​](#chat "Direct link to Chat")

The `chat/completions` endpoint returns an LLM response based on the system prompt and user query.

#### Non-streaming[​](#non-streaming "Direct link to Non-streaming")

By default, the API responds with a full answer in the HTTP response.

**Request**

    curl -X POST https://node_id.gaianet.network/v1/chat/completions \  -H 'accept:application/json' \  -H 'Content-Type: application/json' \  -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the capital of France?"}], "model": "model_name"}'

**Response:**

    {"id":"chatcmpl-bcfeebe0-5372-42c0-ac92-0615213e1c97","object":"chat.completion","created":1716380086,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","choices":[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish_reason":"stop"}],"usage":{"prompt_tokens":61,"completion_tokens":4,"total_tokens":65}}%  

#### streaming[​](#streaming "Direct link to streaming")

Add `"stream":true` in your request to make the API send back partial responses as the LLM generates its answer.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/chat/completions \  -H 'accept:application/json' \  -H 'Content-Type: application/json' \  -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the capital of France?"}], "model": "model_name", "stream":true}'

**Response:**

    data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":"I"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" am"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" a"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}...data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" an"},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" AI"},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: [DONE]

#### Request body[​](#request-body "Direct link to Request body")

| Field | Type | Required | Description | Default | Example |
| --- | --- | --- | --- | --- | --- |
| messages | List | Required | A list of messages for the conversation.  <br>1 . System message (depends on the large language mode you use)  <br>\* `content` of the system messages is required  <br>\* `"role":"system"` is required  <br>2\. User message (required)  <br>\* `content` is required.  <br>\* `"role":"user"` is required | N/A | "messages": \["role": "system","content": "You are a helpful assistant."},{"role": "user",  <br>"content": "Hello!"}\] |
| model | String | Required | The chat model you used | N/A | Llama-3-8B-262k-Q5\_K\_M |
| top\_p | Number | Optional | An alternative to sampling with temperature. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | 1   | Number between 0 and 1. |
| temperature | Number | Optional | Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | 1   | Number between 0 and 2. |
| presence\_penalty | Number | Optional | Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. | 0   | Number between -2.0 and 2.0. |
| stream | boolean | Optional | Make the answer streaming output | FALSE | "stream":true |
| frequency\_penalty | Number | Optional | Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood of repeating the same line verbatim. | 0   | Number between -2.0 and 2.0. |

#### Response body[​](#response-body "Direct link to Response body")

| Field | Type | Streaming or non-streaming | Description | Default | Example |
| --- | --- | --- | --- | --- | --- |
| id  | string | Both | A unique identifier for the chat completion. | Generated randomly | chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4 |
| object | string | Both | The object type | `chat.completion.chunk` in the streaming mode.  <br>`chat.completion` in the non-streaming mode. | `chat.completion.chunk` in the streaming mode.  <br>`chat.completion` in the non-streaming mode. |
| choices | array | Both | A list of chat completion choices. |     | "choices":\[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish\_reason":"stop"}\] |
| created | integer | Both | The Unix timestamp (in seconds) of when the chat completion was created. | N/A | 1716380086 |
| model | string | Both | The model used for the chat completion. | Depends on the model you use. | Llama-3-8B-Instruct-Q5\_K\_M |
| usage | object | Both | Usage statistics for the completion request, including completion\_tokens, prompt\_tokens, and total\_tokens. | N/A | "usage":{"prompt\_tokens":61,"completion\_tokens":4,"total\_tokens":65} |

### Embedding[​](#embedding "Direct link to Embedding")

The `embeddings` endpoint computes embeddings for user queries or file chunks.

**Request**

    curl -X POST https://node_id.gaianet.network/v1/embeddings \    -H 'accept:application/json' \    -H 'Content-Type: application/json' \    -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \    -d '{"model": "nomic-embed-text-v1.5.f16", "input":["Paris, city and capital of France, ..., for Paris has retained its importance as a centre for education and intellectual pursuits.", "Paris’s site at a crossroads ..., drawing to itself much of the talent and vitality of the provinces."]}'

**Response:**

    {    "object": "list",    "data": [        {            "index": 0,            "object": "embedding",            "embedding": [                0.1428378969,                -0.0447309874,                0.007660218049,                ...                -0.0128974719,                -0.03543198109,                0.03974733502,                0.00946635101,                -0.01531364303            ]        },        {            "index": 1,            "object": "embedding",            "embedding": [                0.0697753951,                -0.0001159032545,                0.02073983476,                ...                0.03565846011,                -0.04550019652,                0.02691745944,                0.02498772368,                -0.003226313973            ]        }    ],    "model": "nomic-embed-text-v1.5.f16",    "usage": {        "prompt_tokens": 491,        "completion_tokens": 0,        "total_tokens": 491    }}

### Retrieve[​](#retrieve "Direct link to Retrieve")

The `retrieve` endpoint can retrieve text from the node's vector collection based on the user's query.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/retrieve \    -H 'accept:application/json' \    -H 'Content-Type: application/json' \    -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \    -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the location of Paris?"}], "model":"nomic-embed-text-v1.5.f16"}'

**Response:**

    {    "points": [        {            "source": "\"Paris is located in northern central France, in a north-bending arc of the river Seine whose crest includes two islands, the Île Saint-Louis and the larger Île de la Cité, which form the oldest part of the city. The river's mouth on the English Channel is about 233 mi downstream from the city. The city is spread widely on both banks of the river. Overall, the city is relatively flat, and the lowest point is 35 m above sea level. Paris has several prominent hills, the highest of which is Montmartre at 130 m.\\n\"",            "score": 0.74011195        },        {            "source": "\"The Paris region is the most active water transport area in France, with most of the cargo handled by Ports of Paris in facilities located around Paris. The rivers Loire, Rhine, Rhône, Me\\n\"",            "score": 0.63990676        },        {            "source": "\"Paris\\nCountry\\tFrance\\nRegion\\nÎle-de-France\\r\\nDepartment\\nParis\\nIntercommunality\\nMétropole du Grand Paris\\nSubdivisions\\n20 arrondissements\\nGovernment\\n • Mayor (2020–2026)\\tAnne Hidalgo (PS)\\r\\nArea\\n1\\t105.4 km2 (40.7 sq mi)\\n • Urban\\n (2020)\\t2,853.5 km2 (1,101.7 sq mi)\\n • Metro\\n (2020)\\t18,940.7 km2 (7,313.0 sq mi)\\nPopulation\\n (2023)\\n2,102,650\\n • Rank\\t9th in Europe\\n1st in France\\r\\n • Density\\t20,000/km2 (52,000/sq mi)\\n • Urban\\n (2019)\\n10,858,852\\n • Urban density\\t3,800/km2 (9,900/sq mi)\\n • Metro\\n (Jan. 2017)\\n13,024,518\\n • Metro density\\t690/km2 (1,800/sq mi)\\nDemonym(s)\\nParisian(s) (en) Parisien(s) (masc.), Parisienne(s) (fem.) (fr), Parigot(s) (masc.), \\\"Parigote(s)\\\" (fem.) (fr, colloquial)\\nTime zone\\nUTC+01:00 (CET)\\r\\n • Summer (DST)\\nUTC+02:00 (CEST)\\r\\nINSEE/Postal code\\t75056 /75001-75020, 75116\\r\\nElevation\\t28–131 m (92–430 ft)\\n(avg. 78 m or 256 ft)\\nWebsite\\twww.paris.fr\\r\\n1 French Land Register data, which excludes lakes, ponds, glaciers > 1 km2 (0.386 sq mi or 247 acres) and river estuaries.\\n\"",            "score": 0.62259054        },        {            "source": "\" in Paris\\n\"",            "score": 0.6152092        },        {            "source": "\"The Parisii, a sub-tribe of the Celtic Senones, inhabited the Paris area from around the middle of the 3rd century BC. One of the area's major north–south trade routes crossed the Seine on the île de la Cité, which gradually became an important trading centre. The Parisii traded with many river towns (some as far away as the Iberian Peninsula) and minted their own coins.\\n\"",            "score": 0.5720232        }    ],    "limit": 5,    "score_threshold": 0.4}

### Get the model[​](#get-the-model "Direct link to Get the model")

The `models` endpoint provides the chat and embedding models available on the node.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/models

**Response:**

    {"object":"list","data":[{"id":"Llama-3-8B-Instruct-262k-Q5_K_M","created":1716383261,"object":"model","owned_by":"Not specified"},{"id":"nomic-embed-text-v1.5.f16","created":1716383261,"object":"model","owned_by":"Not specified"}]}%   

### Get the node info[​](#get-the-node-info "Direct link to Get the node info")

The `info` endpoint provides detailed information about the node.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/info

**Response:**

    {    "version": "0.5.0",    "plugin_version": "b2694 (commit 0d56246f)",    "port": "8080",    "models": [        {            "name": "Llama-2-7b-chat-hf-Q5_K_M",            "type": "chat",            "prompt_template": "Llama2Chat",            "n_predict": 1024,            "n_gpu_layers": 100,            "ctx_size": 4096,            "batch_size": 512,            "temperature": 1.0,            "top_p": 1.0,            "repeat_penalty": 1.1,            "presence_penalty": 0.0,            "frequency_penalty": 0.0        },        {            "name": "all-MiniLM-L6-v2-ggml-model-f16",            "type": "embedding",            "prompt_template": "Llama2Chat",            "n_predict": 1024,            "n_gpu_layers": 100,            "ctx_size": 384,            "batch_size": 512,            "temperature": 1.0,            "top_p": 1.0,            "repeat_penalty": 1.1,            "presence_penalty": 0.0,            "frequency_penalty": 0.0        }    ],    "qdrant_config": {        "url": "http://localhost:6333",        "collection_name": "default",        "limit": 5,        "score_threshold": 0.4    }} 

Status Codes[​](#status-codes "Direct link to Status Codes")

-------------------------------------------------------------

| HTTP response code | Description | Reason | Solutions |
| --- | --- | --- | --- |
| 404 | Not found | The endpoint URL is invalid | Please check the endpoint URL |
| 500 | Internal Server Error | Model is not found. | Please check out the model name. |
| 400 | Bad request |     |     |

*   [Introduction](#introduction)
    
*   [Endpoints](#endpoints)
    *   [Chat](#chat)
        
    *   [Embedding](#embedding)
        
    *   [Retrieve](#retrieve)
        
    *   [Get the model](#get-the-model)
        
    *   [Get the node info](#get-the-node-info)
        
*   [Status Codes](#status-codes)

---

# Run a local-only node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

By default, the Gaia node registers itself with a Gaia domain and is accesible from the public.

For many users, it could also be important to start a local server for testing. To do that, you just need to pass the `--local-only` option.

    gaianet start --local-only

---

# Install multiple nodes on a single machine | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

The [default Gaia installer](/getting-started/quick-start)
 installs the node into the `$HOME/gaianet` base directory. You could install multiple nodes on the same machine. Each node has its own "base directory". To do that, you just need to use the `--base` option.

Let's say that the base directory for your second node is `$HOME/node-2`. You can install Gaia node software using the following command.

    curl -sSfL 'https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/install.sh' | bash -s -- --base $HOME/node-2

After that, you can specify the `gaianet` CLI to operate on `node-2` by appending the `--base` option as well.

    gaianet init --base $HOME/node-2gaianet start --base $HOME/node-2

---

# Calling external tools | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Tool calling is one of the truly "LLM native" interaction modes that has never existed before. It gives the "thinking" LLMs the ability to "act" -- both in acquiring new knowledge and in performing real world actions. It is a crucial part of any agentic application.

Open source LLMs are increasingly good at using tools. The Llama 3 models have now made it possible to have reliable tool calling performance on 8b class of LLMs running on your own laptop!

In this tutorial, we will show you a simple Python program that allows a local LLM to run code and manipulate data on the local computer!

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    . You will need to start a Gaia node for the [Llama-3-Groq model](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool)
     or the [Mistral-7B-v0.3 Instruct model](https://github.com/GaiaNet-AI/node-configs/tree/main/mistral-0.3-7b-instruct-tool-call)
     . You can then use the node's API URL endpoint and model name in your tool call apps.
*   [use a public node](/nodes)
    

In this tutorial, we will use a public Llama3 node with the function call support.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llamatool.us.gaianet.network/v1](https://llamatool.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the demo agent[​](#run-the-demo-agent "Direct link to Run the demo agent")

-------------------------------------------------------------------------------

The [agent app](https://github.com/second-state/llm_todo)
 is written in Python. It demonstrates how the LLM could use tools to operate a SQL database. In this case, it starts and operates an in-memory SQLite database. The database stores a list of todo items.

Download the code and install the Python dependencies as follows.

    git clone https://github.com/second-state/llm_todocd llm_todopip install -r requirements.txt

Set the environment variables for the API server and model name we just set up.

    export OPENAI_MODEL_NAME="llama"export OPENAI_BASE_URL= "https://llamatool.us.gaianet.network/v1"

Run the `main.py` application and bring up the command line chat interface.

    python main.py

Use the agent[​](#use-the-agent "Direct link to Use the agent")

----------------------------------------------------------------

Now, you can ask the LLM to perform tasks. For example, you can say

    User: Help me to write down it I'm going to have a meeting with the marketing team.

The LLM understands that you need to insert a record into the database and returns a tool call response in JSON.

    Assistant:<tool_call>{"id": 0, "name": "create_task", "arguments": {"task": "have a meeting with the marketing team"}}</tool_call>

The agent app (i.e., `main.py`) executes the tool call `create_task` in the JSON response, and sends back the results as role `Tool`. You do not need to do anything here as it happens automatically in `main.py`. The SQLite database is updated when the agent app executes the tool call.

    Tool:[{'result': 'ok'}]

The LLM receives the execution result and then answers you.

    Assistant:I've added "have a meeting with the marketing team" to your task list. Is there anything else you'd like to do?

You can continue the conversation.

To learn more about how tool calling works, see [this article](https://github.com/LlamaEdge/LlamaEdge/blob/main/api-server/ToolUse.md)
.

Make it robust[​](#make-it-robust "Direct link to Make it robust")

-------------------------------------------------------------------

One of the major challenges for LLM applications is the frequent unreliability of their responses. For example:

_If the LLM generates an incorrect tool call that fails to address the user’s query,_

you can refine and optimize the descriptions for each tool call function. The LLM chooses its tools based on these descriptions, so it's vital to craft them in a way that aligns with typical user queries.

_If the LLM hallucinates and produces tool calls with non-existent function names or incorrect parameters,_

the agent app should identify this issue and prompt the LLM to create a new response.

Tool calling is a fundamental feature in the evolving field of agentic LLM applications. We’re eager to see the innovative ideas you bring forward!

*   [Prerequisites](#prerequisites)
    
*   [Run the demo agent](#run-the-demo-agent)
    
*   [Use the agent](#use-the-agent)
    
*   [Make it robust](#make-it-robust)

---

# Ensuring Gaia Node Reliability | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Protect the server process[​](#protect-the-server-process "Direct link to Protect the server process")

-------------------------------------------------------------------------------------------------------

Sometimes, the OS could kill the `wasmedge` process on the Gaia node if it consumes too much resources. For production servers, you should protect the server process.

Use Supervise[​](#use-supervise "Direct link to Use Supervise")

----------------------------------------------------------------

The `supervise` tool can help us monitor the `wasmedge` process, and automatically restart the process in case the process dies or is killed. The `gaianet` CLI will make use of `supervise` automatically if it is installed on the system.

For macOS users, you can install `supervise` via the daemontools tool by running the following command.

    brew install daemontools

For Linux users, please refer to [the installation guide](https://cr.yp.to/daemontools/install.html)
 to install the `daemontools` tool.

Reduce the nice value[​](#reduce-the-nice-value "Direct link to Reduce the nice value")

----------------------------------------------------------------------------------------

If the `supervise` tool is too heavy handed, we could also increase the priority of the `wasmedge` process. The OS will try NOT to kill high priority processes until it absolutely has to. We do that by reducing the `nice` value of the `wasmedge` process.

    sudo renice -n -19 $(cat $HOME/gaianet/llamaedge.pid)

*   [Protect the server process](#protect-the-server-process)
    
*   [Use Supervise](#use-supervise)
    
*   [Reduce the nice value](#reduce-the-nice-value)

---

# User Guide | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Public GaiaNet nodes\
------------------------\
\
Each GaiaNet node provides a web-based chatbot UI and an OpenAI compatible web service.](/1.0.0/user-guide/nodes)

[📄️ Use my GaiaNet node\
-----------------------\
\
When you start a GaiaNet node, or you find a node on the web, you could use it as a](/1.0.0/user-guide/mynode)

[🗃️ Agent frameworks and apps\
-----------------------------\
\
20 items](/1.0.0/category/agent-frameworks-and-apps)

[📄️ API Reference\
-----------------\
\
Introduction](/1.0.0/user-guide/api-reference)

---

# Intro to knowledge bases | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Knowledge bases are crucial components in Gaia's context for creating effective AI agents. They serve as structured repositories of information that agents can draw upon to answer questions, make decisions, and perform tasks.

In Gaia's ecosystem, knowledge bases are important for several reasons:

*   **Customization:** They allow developers to tailor their AI agents with specific domain knowledge, making them more useful for particular applications or industries.
*   **Improved Accuracy:** By providing agents with curated, relevant information, knowledge bases help ensure more accurate and contextually appropriate responses.
*   **Efficiency:** Agents can quickly access pre-organized information rather than having to process large amounts of unstructured data in real-time.
*   **Scalability:** As the knowledge base grows, the agent's capabilities can expand without requiring retraining of the entire model.

To leverage knowledge bases in creating AI agents using Gaia:

*   **Create a Knowledge Base:** Gaia supports multiple formats for creating knowledge bases, including plain text files, markdown files, PDF documents, CSV files with source/summary pairs, and even web URLs. This flexibility allows you to use the most convenient format for your data.
*   **Organize Information:** Structure your knowledge base logically, focusing on the key information your agent will need to access.
*   **Import to Gaia:** Use Gaia's tools to import your knowledge base into the system. The platform likely provides interfaces or APIs for this purpose.
*   **Configure Agent:** When setting up your AI agent in Gaia, specify which knowledge base(s) it should use as a reference.
*   **Test and Refine:** Interact with your agent, observe its performance, and iteratively refine your knowledge base to improve outcomes.
*   **Update Regularly:** Keep your knowledge base current by adding new information or updating existing entries as needed.

### Knowledge Base Format[​](#knowledge-base-format "Direct link to Knowledge Base Format")

Gaia employs a specific format for knowledge bases to optimize performance and integration with its AI agents. While the exact details may vary, here are some key aspects:

*   **Structured Data:** Knowledge bases in Gaia are likely organized in a structured format that allows for efficient indexing and retrieval.
*   **Metadata:** Each entry in the knowledge base may include metadata such as timestamps, categories, or tags to enhance searchability.

### Vector Database Integration[​](#vector-database-integration "Direct link to Vector Database Integration")

Gaia utilizes Qdrant as its default vector database to enhance the capabilities of its knowledge bases:

*   **Qdrant Integration:** [Qdrant](https://qdrant.tech/)
     is a high-performance, open-source vector database optimized for similarity search and machine learning applications.
*   **Embedding Storage:** Text from knowledge bases is converted into high-dimensional vectors (embeddings) and efficiently stored in Qdrant.
*   **Fast Similarity Search:** Qdrant enables rapid similarity searches, allowing AI agents to quickly find the most relevant information from the knowledge base.
*   **Scalability:** Qdrant is designed to handle large-scale vector search problems, supporting extensive knowledge bases with millions of entries.
*   **Flexible Filtering:** Qdrant allows for complex filtering during search operations, enabling more precise information retrieval based on metadata or other attributes.
*   **CRUD Operations:** Gaia can leverage Qdrant's support for real-time updates, allowing for dynamic modifications to the knowledge base without significant performance impact.
*   **Cloud-Native Architecture:** Qdrant's design aligns well with cloud environments, facilitating easy scaling and deployment of Gaia's knowledge base system.

By using Qdrant, Gaia provides a robust and efficient vector search capability, enabling AI agents to quickly access and utilize relevant information from large and complex knowledge bases.

### Leveraging the System[​](#leveraging-the-system "Direct link to Leveraging the System")

To make the most of Gaia's knowledge base and vector database system:

*   **Optimize Content:** Structure your knowledge base entries to align with Gaia's preferred format for best performance.
*   **Use Appropriate Queries:** When designing your AI agent, formulate queries that take advantage of the vector search capabilities.
*   **Regular Updates:** Keep your knowledge base current, as the vector database can be used to update embeddings for new or changed content.
*   **Performance Monitoring:** Pay attention to retrieval speed and accuracy, adjusting your knowledge base structure if needed.

By understanding and effectively using Gaia's specific knowledge base format and vector database system, you can create more powerful and efficient AI agents that leverage information retrieval capabilities to their fullest extent.

*   [Knowledge Base Format](#knowledge-base-format)
    
*   [Vector Database Integration](#vector-database-integration)
    
*   [Leveraging the System](#leveraging-the-system)

---

# Agentic translation on Gaia | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Prof. Andrew Ng's [agentic translation](https://www.linkedin.com/posts/andrewyng_github-andrewyngtranslation-agent-activity-7206347897938866176-5tDJ/)
 is a great demonstration on how to coordinate multiple LLM "agents" to work on a single task. It allows multiple smaller LLMs (like Llama-3 or Gemma-2) to work together and produce better results than a single large LLM (like ChatGPT).

[Gaia](https://www.gaianet.ai/)
, with 2000+ nodes running all kinds of finetuned LLms and knowledge bases, provides a huge opportunity for agentic apps to choose and use their own LLM backends.

Introduction to the LLM Translation Agent[​](#introduction-to-the-llm-translation-agent "Direct link to Introduction to the LLM Translation Agent")

----------------------------------------------------------------------------------------------------------------------------------------------------

This LLM Translation Agent is designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use your own fine-tuned models or any LLMs on Hugging Face like Meta's Llama 3.

> For detailed commands on starting and running this agent, please visit [GitHub - Second State/translation-agent](https://github.com/second-state/translation-agent/blob/use_llamaedge/step-by-step-use-LocalAI.md)
> .

To get started, clone the Translation Agent.

    git clone https://github.com/second-state/translation-agent.git    cd translation-agentgit checkout use_llamaedge

Next, we will install a local Gaia node, which provides the backend API services required by the agent. You can, of course, use [Gaia nodes from the community](/nodes)
 if you do not want to start your own.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash

You will also need the following configurations and prerequisites to run the agent app. If you are using a public Gaia node instead of your local node, replace the `http://localhost:8080` with `https://node_id.us.gaianet.network`.

    export OPENAI_BASE_URL="http://localhost:8080/v1"export PYTHONPATH=${PWD}/srcexport OPENAI_API_KEY="GAIANET"pip install python-dotenvpip install openai tiktoken icecream langchain_text_splitters

Demo 1: Running Translation Agents with Llama-3-8B[​](#demo-1-running-translation-agents-with-llama-3-8b "Direct link to Demo 1: Running Translation Agents with Llama-3-8B")

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

First, let's run the translation agent with Meta AI's popular Llama-3 model. We select the smallest Llama-3 model (the 8b model) for this demo. The translation task is from Chinese to English. Our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text)
 is in Chinese, a brief intro to the ancient Chinese royal palace, the Forbidden City.

### Step 1.1: Run a Llama-3-8B Gaia node[​](#step-11-run-a-llama-3-8b-gaia-node "Direct link to Step 1.1: Run a Llama-3-8B Gaia node")

Configure and download the model. Since the size of the model is 5.73 GB. It can take a while to download.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json

Next, use the following command to start the Gaia node.

    gaianet start

### Step 1.2 Run the Translation Agent on top of Llama-3-8B[​](#step-12-run-the-translation-agent-on-top-of-llama-3-8b "Direct link to Step 1.2 Run the Translation Agent on top of Llama-3-8B")

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `Meta-Llama-3-8B-Instruct-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import osimport translation_agent as ta        if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/forbiddencity.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="Meta-Llama-3-8B-Instruct-Q5_K_M",    )        print(f"Translation:\n\n{translation}")

Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here, because we named our source text `forbiddencity.txt`, the relative path to the document would be `sample-texts/forbiddencity.txt`.

Run the below commands to have your text file translated into English.

    cd examplespython example_script.py

Wait for several minutes and you will have [a fully translated version](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-Llama-3-8B)
 appear on your terminal screen.

Demo 2: Running Translation Agents with gemma-2-27b[​](#demo-2-running-translation-agents-with-gemma-2-27b "Direct link to Demo 2: Running Translation Agents with gemma-2-27b")

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The benefit of running the Translation Agent with Gaia is the ability for users to choose and embed different LLMs for different agentic tasks. To demonstrate this point, we will now change the translation agent LLM from Llama-3-8b to Google's gemma-2-27b, which is of similar size but scores higher on many language-related benchmarks.

The translation task is the same as before. Our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text)
 is in Chinese, a brief intro to the ancient Chinese royal palace, the Forbidden City. The translation target is English.

### Step 2.1 Run a gemma-2-27b Gaia node[​](#step-21-run-a-gemma-2-27b-gaia-node "Direct link to Step 2.1 Run a gemma-2-27b Gaia node")

Configure and download the model. Since the size of the model is 6.40G, it could take a while to download.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/gemma-2-27b-it/config.json

Next, use the following command to start the Gaia node.

    gaianet start

### Step 2.2 Run the Translation Agent to run on top of gemma-2-27b[​](#step-22-run-the-translation-agent-to-run-on-top-of-gemma-2-27b "Direct link to Step 2.2 Run the Translation Agent to run on top of gemma-2-27b")

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `gemma-2-27b-it-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import os  import translation_agent as ta      if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/forbiddencity.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="gemma-2-27b-it-Q5_K_M",    )        print(f"Translation:\n\n{translation}")

Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here,because we named our source text `forbiddencity.txt`, the relative path to the document would be `sample-texts/forbiddencity.txt`.

Run the below commands to have your text file translated into English.

    cd examples    python example_script.py

You can find the translated result in English [here](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-gemma-2-27b)
.

Demo 3: Running Translation Agents with Phi-3-Medium long context model[​](#demo-3-running-translation-agents-with-phi-3-medium-long-context-model "Direct link to Demo 3: Running Translation Agents with Phi-3-Medium long context model")

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The Llama-3 and Gemma-2 models are great LLMs, but they have relatively small context windows. The agent requires all text to fit into the LLM context window, and that limits the size of articles they can translate. To fix this problem, we could select an open source LLM with a large context window. For this demo, we choose Microsoft's Phi-3-medium-128k model, which has a massive 128k (over 100 thousand words or the length of several books) context window.

We run [a lengthy Chinese article on Forbidden City's collaboration with the Varsaille Palace](https://hackmd.io/vuFYZTVsQZyKmkeQ3ThZQw?view#Source-text)
 through our Translation Agent powered by a Phi-3-medium-128k model we start locally.

### Step 3.1: Run a Phi-3-medium-128k Gaia node[​](#step-31-run-a-phi-3-medium-128k-gaia-node "Direct link to Step 3.1: Run a Phi-3-medium-128k Gaia node")

Configure and download the model.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/phi-3-medium-instruct-128k/config_full.json

Next, use the following command to start the Gaia node with a 128k context window.

    gaianet start

### Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k[​](#step-32-clone-and-run-the-translation-agent-on-top-of-phi-3-medium-128k "Direct link to Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k")

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `Phi-3-medium-128k-instruct-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import os  import translation_agent as ta      if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/long_article.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="Phi-3-medium-128k-instruct-Q5_K_M",    )        print(f"Translation:\n\n{translation}")

Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here, because we named our source text `long_article.txt`, the relative path to the document would be `sample-texts/long_article.txt`.

    cd examplespython example_script.py

[The translated results were impressive,](https://hackmd.io/vuFYZTVsQZyKmkeQ3ThZQw?view#Source-text)
 with the translation capturing the nuances and context of the original text with high fidelity.

Evaluation of Translation Quality[​](#evaluation-of-translation-quality "Direct link to Evaluation of Translation Quality")

----------------------------------------------------------------------------------------------------------------------------

The three models, Llama-3-8B, gemma-2-27b, and Phi-3-medium, have exhibited varying levels of performance in translating complex historical and cultural content from Chinese to English.

Llama-3-8B provides a translation that effectively captures the factual content but shows occasional stiffness in language, possibly indicating a direct translation approach that doesn't fully adapt idiomatic expressions. It does not keep section title and the format of the original text and left certain part untranslated.

In contrast, The translation by gemma-2-27b is quite accurate and retains the original meaning of the short intro article of Forbidden city. gemma-2-27b's translation exhibits a smooth and natural English flow, suggesting a sophisticated understanding of both the source language and the target language’s grammatical structures. The choice of words and sentence structures in gemma-2-27b's output demonstrates a high degree of linguistic finesse, suggesting it might be well-suited for translating formal and historically nuanced texts.

The Phi-3-medium-128k model can translate book-length text from Chinese to English. It demonstrates robust capabilities in handling large volumes of complex content, suggesting advanced memory handling and contextual awareness. The quality of translation remains consistent even with increased text length, indicating Phi's utility in projects requiring extensive, detailed translations. But you can see it makes certain mistakes like mistaken "Wenhua Hall" as "also known as Forbidden City" in the first paragraph.

Overall, each model has its strengths, with gemma-2-27b standing out for linguistic finesse and Phi-3-medium-128k for handling lengthy texts.

Conclusion[​](#conclusion "Direct link to Conclusion")

-------------------------------------------------------

[Gaia](https://github.com/GaiaNet-AI)
 provides an easy way to select and use different open-source LLMs in your agentic applications to fully take advantage of their finetuned capabilities for specific tasks.

Once you have a local Gaia node up and running, you could share it with others and make $$$ by joining the [Gaia network](https://www.gaianet.ai/)
!

*   [Introduction to the LLM Translation Agent](#introduction-to-the-llm-translation-agent)
    
*   [Demo 1: Running Translation Agents with Llama-3-8B](#demo-1-running-translation-agents-with-llama-3-8b)
    *   [Step 1.1: Run a Llama-3-8B Gaia node](#step-11-run-a-llama-3-8b-gaia-node)
        
    *   [Step 1.2 Run the Translation Agent on top of Llama-3-8B](#step-12-run-the-translation-agent-on-top-of-llama-3-8b)
        
*   [Demo 2: Running Translation Agents with gemma-2-27b](#demo-2-running-translation-agents-with-gemma-2-27b)
    *   [Step 2.1 Run a gemma-2-27b Gaia node](#step-21-run-a-gemma-2-27b-gaia-node)
        
    *   [Step 2.2 Run the Translation Agent to run on top of gemma-2-27b](#step-22-run-the-translation-agent-to-run-on-top-of-gemma-2-27b)
        
*   [Demo 3: Running Translation Agents with Phi-3-Medium long context model](#demo-3-running-translation-agents-with-phi-3-medium-long-context-model)
    *   [Step 3.1: Run a Phi-3-medium-128k Gaia node](#step-31-run-a-phi-3-medium-128k-gaia-node)
        
    *   [Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k](#step-32-clone-and-run-the-translation-agent-on-top-of-phi-3-medium-128k)
        
*   [Evaluation of Translation Quality](#evaluation-of-translation-quality)
    
*   [Conclusion](#conclusion)

---

# Node Operator Guide | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Quick start with GaiaNet Node\
---------------------------------\
\
This guide provides the requisite knowledge necessary to quickly get started with installing a GaiaNet node.](/1.0.0/node-guide/quick-start)

[📄️ Install and uninstall\
-------------------------\
\
The GaiaNet node utilizes version control from its source GitHub repo. You can check out the GaiaNet node versions from the release page.](/1.0.0/node-guide/install_uninstall)

[📄️ System requirements\
-----------------------\
\
You can install the GaiaNet on a wide variety of devices and operating systems with or without GPUs. The node installing and operating instructions work on devices ranging from Raspberry Pi, MacBooks, Linux servers, Windows Desktop, to cloud-based Nvidia H100 clusters. For institutional operators, we recommend EITHER of the following for a GaiaNet node.](/1.0.0/node-guide/system-requirements)

[📄️ Customize Your GaiaNet Node\
-------------------------------\
\
A key goal of the GaiaNet project is to enable each individual to create and run his or her own](/1.0.0/node-guide/customize)

[📄️ Joining the Gaia Protocol\
-----------------------------\
\
To join the Gaia protocol, you will need to complete the following two tasks.](/1.0.0/node-guide/register)

[📄️ GaiaNet CLI options\
-----------------------\
\
After installing the GaiaNet software, you can use the gaianet CLI to manage the node. The following are the CLI options.](/1.0.0/node-guide/cli-options)

[📄️ Troubleshooting\
-------------------\
\
The system cannot find CUDA libraries](/1.0.0/node-guide/troubleshooting)

[🗃️ How do I ...\
----------------\
\
6 items](/1.0.0/category/how-do-i-)

---

# How to create a knowledge base | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

[📄️ Knowledge base from a plain text file\
-----------------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a plain text file. The](/knowledge-bases/how-to/text)

[📄️ Knowledge base from a markdown file\
---------------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a markdown file. The snapshot file can then be loaded by a Gaia node as its knowledge base.](/knowledge-bases/how-to/markdown)

[📄️ Knowledge base from source / summary pairs\
----------------------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot for optimal retrieval of long-form text documents. The approach is to create two columns of text in a CSV file.](/knowledge-bases/how-to/csv)

[📄️ Knowledge base from a PDF file\
----------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a PDF file. First, we will parse the unstructured PDF file to a structured markdown file. Then, we will follow the steps from Knowledge base from a markdown file to create embedding for your PDF files.](/knowledge-bases/how-to/pdf)

[📄️ Knowledge base from a URL\
-----------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a Web URL. First, we will parse the URL to a structured markdown file. Then, we will follow the steps from Knowledge base from a markdown file to create embedding for your URL.](/knowledge-bases/how-to/firecrawl)

---

# Creator Guide | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[🗃️ Knowledge bases\
-------------------\
\
7 items](/1.0.0/category/knowledge-bases)

[🗃️ GaiaNet Node with finetuned LLMs\
------------------------------------\
\
2 items](/1.0.0/category/gaianet-node-with-finetuned-llms)

---

# Build a knowledge base using Gaia web tool | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Gaia has developed a tool for making vector collection snapshot files, so everyone can easily create their own knowledge base.

Access it here: [https://tools.gaianet.xyz/](https://tools.gaianet.xyz/)

Segment your text file[​](#segment-your-text-file "Direct link to Segment your text file")

-------------------------------------------------------------------------------------------

First, copy unformatted text into a txt file. Then follow the two rules to chunk your content, ie putting similar content together.

*   Each title and related content are a chunk. There is no blank lines in one chunk.
*   Use a blank line to recognize different chunks.

After that, save it as a txt file.

For example, below is your source.

![The input knowledge in a text file](/assets/images/web_tool_input-1b1323226ee6bfd7974431e25296b961.png)

After formatted, it will look like the following.

    What is a blockchain?A blockchain is a distributed, cryptographically-secure database structure that allows network participants to establish a trusted and immutable record of transactional data without the need for intermediaries. A blockchain can execute a variety of functions beyond transaction settlement, such as smart contracts. Smart contracts are digital agreements that are embedded in code and can have limitless formats and conditions. Blockchains have proven themselves as superior solutions for securely coordinating data, but they are capable of much more, including tokenization, incentive design, attack-resistance, and reducing counterparty risk. The very first blockchain was the Bitcoin blockchain, which was itself a culmination of over a century of advancements in cryptography and database technology.What is blockchain software?Blockchain software is like any other software. The first of its kind was Bitcoin, which was released as open source software, making it available to anyone to use or change. There are a wide variety of efforts across the blockchain ecosystem to improve upon Bitcoin's original software. Ethereum has its own open source blockchain software. Some blockchain software is proprietary and not available to the public.

Generate the snapshot file[​](#generate-the-snapshot-file "Direct link to Generate the snapshot file")

-------------------------------------------------------------------------------------------------------

1.  Visit this URL: [https://tools.gaianet.xyz/](https://tools.gaianet.xyz/)
    , upload the above prepared txt file.
2.  Edit your `dbname` . _**Note: Do not include spaces or special characters in the dbname**_.
3.  Choose Embedding model, we suggest use `nomic-embed-text-v1.5.f16`.
4.  Click the "Make RAG" button and wait.

When finished, the chatbot will display Gaia Node config info. It is a JSON format as follows.

    {  "embedding": "https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf",  "embedding_ctx_size": 768,  "snapshot": "https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolve/main/test/test.snapshot"}

Update the node config[​](#update-the-node-config "Direct link to Update the node config")

-------------------------------------------------------------------------------------------

Run the following

    gaianet config \  --snapshot https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolve/main/test/test.snapshot \  --embedding-url https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf \  --embedding-ctx-size 768

and then

    gaianet initgaianet start

Have fun!

*   [Segment your text file](#segment-your-text-file)
    
*   [Generate the snapshot file](#generate-the-snapshot-file)
    
*   [Update the node config](#update-the-node-config)

---

# Domain Operator Guide | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Quick Start with Launching Gaia Domain\
------------------------------------------\
\
This guide provides all the information you need to quickly set up and run a Gaia Domain.](/1.0.0/domain-guide/quick-start)

---

# Tutorial | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Calling external tools\
--------------------------\
\
Tool calling is one of the truly "LLM native" interaction modes that has never existed before.](/1.0.0/tutorial/tool-call)

[📄️ Agentic translation on GaiaNet\
----------------------------------\
\
Prof. Andrew Ng's agentic translation is a great demonstration on how to coordinate multiple LLM "agents" to work on a single task. It allows multiple smaller LLMs (like Llama-3 or Gemma-2) to work together and produce better results than a single large LLM (like ChatGPT).](/1.0.0/tutorial/translator-agent)

[📄️ Working with Coinbase AgentKit\
----------------------------------\
\
You can use a Gaia node to power the Coinbase AgentKit.](/1.0.0/tutorial/coinbase)

[📄️ Working with eliza\
----------------------\
\
eliza is a simple, fast, and lightweight AI agent framework. Recently, eliza has integrated Gaia as one of the model service provider. This means you can now use Gaia as the LLM service backend for the Eliza framework.](/1.0.0/tutorial/eliza)

---

# GaiaNet: GenAI Agent Network | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Abstract[​](#abstract "Direct link to Abstract")

-------------------------------------------------

Specialized, finetuned and RAG-enhanced open-source Large Language Models are key elements in emerging AI agent applications. However, those agent apps also present unique challenges to the traditional cloud computing and SaaS infrastructure, including new requirements for application portability, virtualization, security isolation, costs, data privacy, and ownership.

GaiaNet is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise. A GaiaNet node consists of a high-performance and cross-platform application runtime, a finetuned LLM, a knowledge embedding model, a vector database, a prompt manager, an open API server, and a plugin system for calling external tools and functions using LLM outputs. It can be deployed by any knowledge worker as a digital twin and offered as a web API service. A new class of tradeable assets and a marketplace could be created from individualized knowledge bases and components. Similar GaiaNet nodes are organized into GaiaNet domains, which offer trusted and reliable AI agent services to the public. The GaiaNet node and domains are governed by the GaiaNet DAO (Decentralized Autonomous Organization). Through Purpose Bound Money smart contracts, the GaiaNet network is a decentralized marketplace for AI agent services.

Introduction[​](#introduction "Direct link to Introduction")

-------------------------------------------------------------

The emergence of ChatGPT and Large Language Model (LLM) has revolutionized how humans produce and consume knowledge. Within a year, AI-native applications have evolved from chatbots to copilots, to agents.

> AI agents would increasingly evolve from supportive tools (akin to Copilots) to autonomous entities capable of completing tasks independently. — Dr. Andrew Ng at Sequoia Capital AI Ascent 2024 Summit

Agents are software applications that can complete tasks on its own autonomously like a human. The agent can understand the task, plan the steps to complete the task, execute all the steps, handle errors and exceptions, and deliver the results. While a powerful LLM could act as the “brain” for the agent, we need to connect to external data sources (eyes and ears), domain-specific knowledge base and prompts (skills), context stores (memory), and external tools (hands). For agent tasks, we often need to customize the LLM itself

*   to reduce hallucinations in a specific domain.
*   to generate responses in a specific format (e.g., a JSON schema).
*   to answer “politically incorrect” questions (e.g., to analyze CVE exploits for an agent in the security domain).
*   and to answer requests in a specific style (e.g., to mimic a person).

![What is a GaiaNet agent](/assets/images/gaianet_agent-f614c7c96d8253e1b4024e396b96169e.png)

Agents are complex software that require significant amount of engineering and resources. Today, most agents are close-source and hosted on SaaS-based LLMs. Popular examples include GPTs and Microsoft/GitHub copilots on OpenAI LLMs, and Duet on Google’s Gemini LLMs.

However, as we discussed, a key requirement for agents is to customize and adapt its underlying LLM and software stack for domain-specific tasks — an area where centralized SaaS perform very poorly. For example, with ChatGPT, every small task must be handled by a very large model. It is also enormously expensive to fine-tune or modify any ChatGPT models. The one-size-fits-all LLMs are detrimental to the agent use case in capabilities, alignment, and cost structure. Furthermore, the SaaS hosted LLMs lack privacy controls on how the agent’s private knowledge might be used and shared. Because of these shortcomings, it is difficult for individual knowledge workers to create and monetize agents for his or her own domain and tasks on SaaS platforms like OpenAI, Google, Anthropic, Microsoft and AWS.

In this paper, we propose a decentralized software platform and protocol network for AI agents for everyone. Specifically, our goals are two-folds.

**Goal #1:** Empower individuals to incorporate his/her private knowledge and expertise into personal LLM agent apps. Those apps aim to perform knowledge tasks and use tools just as the individual would, but also reflect the individual’s style and values.

**Goal #2:** Enable individuals to provide and scale their LLM agents as services, and get compensated for their expertise and work.

> GaiaNet is “YouTube for knowledge and skills.”

Open-source and decentralization[​](#open-source-and-decentralization "Direct link to Open-source and decentralization")

-------------------------------------------------------------------------------------------------------------------------

As of April 2024, there are over 6000 open-source LLMs published on Hugging face. Compared with close-source LLMs, such as GPT-4, open-source LLMs offer advantages in privacy, cost, and systematic bias. Even with general QA performance, open-source LLMs are closing the gap with close-source counterparties quickly.

![Open vs close source LLMs](/assets/images/closed_vs_open-7465b42b1c6b33cfebee85ed6c721e5a.jpg)

For AI agent use cases, it has been demonstrated that smaller but task-specific LLMs often outperform larger general models.

However, it is difficult for individuals and businesses to deploy and orchestrate multiple finetuned LLMs on their own heterogeneous GPU infrastructure. The complex software stack for agents, as well as the complex interaction with external tools, are fragile and error-prone.

Furthermore, LLM agents have entirely different scaling characteristics than past application servers. LLM is extremely computationally intensive. A LLM agent server can typically only serve one user at a time, and it often blocks for seconds at a time. The scaling need is no longer to handle many async requests on a single server, but to load balance among many discrete servers on the internet scale.

The GaiaNet project provides a cross-platform and highly efficient SDK and runtime for finetuned open-source LLMs with proprietary knowledge bases, customized prompts, structured responses, and external tools for function calling. A GaiaNet node can be started in minutes on any personal, cloud, or edge device. It can then offer services through an incentivized web3 network.

GaiaNet node[​](#gaianet-node "Direct link to GaiaNet node")

-------------------------------------------------------------

The basic operational unit in the GaiaNet network is a node. A GaiaNet node is a streamlined software stack that allows any technically competent person to run an AI agent of his own. The software stack on the GaiaNet node consists of the following 7 key components.

![GaiaNet node architecture](/assets/images/gaianet_node-91f817661a1d3e1f4f7628f809ed6c70.png)

**1 Application runtime.** GaiaNet applications run in a lightweight, secure and high-performance sandbox called WasmEdge. As an open-source project managed by the Linux Foundation and CNCF, WasmEdge runtime works seamlessly with leading cloud native tools such as Docker, containerd, CRI-O, Podman and Kubernetes. It is also the virtual machine of choice by leading public blockchains to securely and efficiently execute on-chain and off-chain smart contracts.

WasmEdge is a high-performance and cross-platform runtime. It can run AI models on almost all CPUs, GPUs, and AI accelerators at native speed, making it an ideal runtime for decentralized AI agents.

**2 Finetuned LLM.** The GaiaNet node supports almost all open-source LLMs, multimodal models (eg Large Vision Models or LVMs), text-to-image models (eg Stable Diffusion) and text-to-video models. That includes all finetuned models using personal or proprietary data.

The node owner can finetune open-source models using a wide variety of tools. For example, the node owner can finetune an LLM using personal chat histories so that the finetuned LLM can mimic his own speaking style. He can also finetune an LLM to focus it on a specific knowledge domain to reduce hallucinations and improve answer quality for questions in that domain. A finetuned LLM can guarantee to output JSON text that matches a pre-determined schema for use with external tools.

Besides LLMs, the node owner could finetune Stable Diffusion models with her own photos to generate images that look like her.

**3 Embedding model.** The GaiaNet node needs to manage a body of public or proprietary knowledge for the AI agent. It is a key feature that enables the agent to specialize and outperform much larger models in a specific domain. The embedding models are specially trained LLMs that turns input sentences into a vector representation, instead of generating completions. Since the embedding models are trained from LLMs, they can “embed” the “meaning” of the sentences into the vectors so that similar sentences are located close together in the high dimensional space occupied by those vectors.

With the embedding model, a GaiaNet node can ingest a body of text, images, PDFs, web links, audio and video files, and generate a collection of embedding vectors based on their contents. The embedding model also turns user questions and conversations into vectors, which allows the GaiaNet node to quickly identify contents in its knowledge base that are relevant to the current conversation.

**4 Vector database.** The embedding vectors that form GaiaNet node’s knowledge base are stored on the node itself for optimal performance and maximum privacy. The GaiaNet node includes a Qdrant vector database.

**5 Custom prompts.** Besides finetuning and knowledge arguments, the easiest way to customize an LLM for new applications is simply to prompt it. Like humans, LLMs are remarkable one-shot learners. You can simply give it an example of how to accomplish a task, and it will learn and do similar tasks on its own. Prompt engineering is a practical field to research and develop such prompts.

Furthermore, effective prompts could be highly dependent on the model in use. A prompt that works well for a large model, such as Mixtral 8x22b, is probably not going to work well for a small model like Mistral 7b.

The GaiaNet node can support several different prompts that are dynamically chosen and used in applications. For example,

*   The `system_prompt` is a general introduction to the agent task the node is supposed to perform. It often contains a persona to help the LLM respond with the right tone. For example, the `system_prompt` for a college teaching assistant could be: “You are a teaching assistant for UC Berkeley’s computer science 101 class. Please explain concepts and answer questions in detail. Do not answer any question that is not related to math or computer science.”
*   The `rag_prompt` is a prefix prompt to be dynamically inserted in front of knowledge base search results in an RAG chat. It could be something like this: “Please answer the question based on facts and opinions in the context below. Do not make anything that is not in the context. ---------”

The LLM community has developed many useful prompts for different application use cases. GaiaNet node allows you to easily manage and experiment with them.

Through the our developer SDK, GaiaNet owners and operators could customize the logic of dynamic prompt generation in their own way. For example, a GaiaNet node could perform a Google search for any user question, and add the search results into the prompt as context.

**6 Function calls and tool use.** The LLM not only is great at generating human language, but also excels at generating machine instructions. Through finetuning and prompt engineering, we could get some LLMs to consistently generate structured JSON objects or computer code in many language tasks, such as summarizing and extracting key elements from a paragraph of text.

The GaiaNet node allows you to specify the output format of the generated text. You can give it a grammar specification file to enforce that responses will always conform to a pre-defined JSON schema.

Once the LLM returns a structured JSON response, the agent typically need to pass the JSON to a tool that performs the task and comes back with an answer. For example, the user question might be.

    What is the weather like in Singapore?

The LLM generates the following JSON response.

    {"tool":"get_current_weather", "location":"Singapore","unit":"celsius"}

The GaiaNet node must know what is the tool associated with get\_current\_weather and then invoke it. GaiaNet node owners and operators can configure any number of external tools by mapping a tool name with a web service endpoint. In the above example, the get\_current\_weather tool might be associated with a web service that takes this JSON data. The GaiaNet node sends the JSON to the web service endpoint via HTTPS POST and receives an answer.

    42

It then optionally feeds the answer to the LLM to generate a human language answer.

    The current weather in Singapore is 42C. 

Through the GaiaNet node SDK, developers are not limited to using web services. They can write plugins to process LLM responses locally on the node. For example, the LLM might return Python code, which can be executed locally in a sandbox and for the GaiaNet node to perform a complex operation.

**7 The API server.** All GaiaNet nodes must have the same API for questions and answers. That allows front-end applications to work with, and potentially be load-balanced to any GaiaNet node. We choose to support the OpenAI API specification, which enables GaiaNet nodes to become drop-in replacements for OpenAI API endpoints for a large ecosystem of applications.

The API server runs securely and cross-platform on the WasmEdge runtime. It ties together all the other components in the GaiaNet node. It receives user requests, generates an embedding from the request, searches the vector database, adds search results to the prompt context, generates an LLM response, and then optionally uses the response to perform function calling. The API server also provides a web-based chatbot UI for users to chat with the RAG-enhanced finetuned LLM on the node.

GaiaNet network[​](#gaianet-network "Direct link to GaiaNet network")

----------------------------------------------------------------------

While each GaiaNet node is already a powerful AI agent capable of answering complex questions and performing actions, individual nodes are not suitable for providing public services. There are several important reasons.

*   For the public consumers and users, it is very hard to judge the trustworthiness of individual GaiaNet nodes. Harmful misinformation could be spread by malicious node operators.
*   For GaiaNet node owners and operators, there is no economic incentive to provide such services to the public, which could be very costly to run.
*   The AI agent servers have very different scaling characteristics than traditional internet application servers. When the agent is processing a user request, it typically takes up all the computing resources on the hardware. Instead of using software to scale concurrent users on a single server, the challenge of GaiaNet is to scale to many different identical nodes for a large application.

Those challenges have given rise to the GaiaNet domain, which forms the basis of the GaiaNet web3 network. A GaiaNet domain is a collection of GaiaNet nodes available under a single Internet domain name. The domain operator decides which GaiaNet nodes can be registered under the domain and makes the node services available to the public. For example, a GaiaNet domain might be a Computer Science teaching assistant for UC Berkeley. The domain could provide services through `https://cs101.gaianet.berkeley.edu`. The domain operator needs to do the following.

*   Verify and admit individual nodes to be registered under the domain. Those nodes must all meet requirements, such as the LLM, knowledge base, and prompts, set by the domain operator to ensure service quality. The node registration on a domain could be done via a whitelist or blacklist. It is up to the domain operator.
*   Monitor each node’s performance at real time and remove inactive ones.
*   Promotes the “teaching assistant” chatbot apps to the target audience.
*   Set the price for the API services.
*   Load balance between active nodes.
*   Getting paid by users.
*   Pay nodes for their services.

![GaiaNet network architecture](/assets/images/gaianet_eco-cd68b47237f0227986c6317281b31ccb.png)

Each GaiaNet node has an unique node ID in the form of an ETH address. The private key associated with the ETH address is stored on the node. Once a node is successfully registered with a domain, it is entitled to receive payments from both service revenue and network awards from the domain. The domain could send payments directly to the node's ETH address. Or, the domain could provide a mechanism for a node operator to register multiple nodes under a single Metamask address, such as signing a challenge phrase using the node private keys. In that case, the node operator will receive aggregated payments in his Metamask account for all associated nodes.

Each GaiaNet domain has an associated smart contract that is used for escrow payments. It is similar to OpenAI’s credit payment model, where users purchase credits first, and then consume them over time. When the user pays into the smart contract, an access token will be automatically issued to him. He uses this token to make API calls to the domain, which is then load-balanced to random nodes in the domain. As the user consumes those services, his fund in the contract depletes and the access token stops working if he no longer has any balance.

The pricing and payment of the API service are determined by the domain operator. It is typically denominated in USD stable coins. The domain operator pays a share of the revenue to node operators who provided the services. The GaiaNet network is a decentralized marketplace of agent services.

> The funds locked in GaiaNet domain contracts are for a single purpose of consuming API services. It is called Purpose Bound Money.

A key aspect of the GaiaNet protocol is that the domain operators are “trust providers” in the ecosystem of decentralized nodes. The protocol network is designed to incentivize the trust of the operators through tokenomics designs such as mining and staking. GaiaNet nodes, domains, users, and developers form a DAO to grow the network and benefit all contributors.

GaiaNet token[​](#gaianet-token "Direct link to GaiaNet token")

----------------------------------------------------------------

The GaiaNet token is a utility token designed to facilitate transactions, support governance, and foster trust in the network. It serves three primary purposes.

*   As a DAO governance token, holders can participate in setting the rules of the network.
*   As a staking token, holders vouch for domain operators’ trustworthiness. Stakers get a cut from the domain operator’s service revenue. But they could also be slashed if the domain operator misbehave, such as spreading misinformation or providing unreliable services.
*   As a payment token, the GaiaNet token could be deposited into the domain’s escrow contract and be used to pay for services over time.

The payment utility of the GaiaNet token is designed to balance the network supply and demand. The value of the GaiaNet token asset is determined at the time when it enters or leaves the escrow smart contract based on real-time exchange rates.

Service consumers could lock in savings from the potential appreciation of the token. For example, if a user deposits $100 worth of GaiaNet tokens into the contract, and when the domain and nodes get paid, the token value has gone up to $110, he would have received $110 worth of agent services.

Conversely, if the token price drops, the service providers (domains and nodes) now have an opportunity to “mine” the tokens on the cheap. If the $100 initial tokens is only worth $90 now, service providers will get more tokens for each unit of electricity and compute they provide. That incentivizes more nodes to join the network and speculate on a later rise in token value.

> An exercise: OpenAI is projected to reach $5 billion in ARR in 2024. Assume that most enterprise customers pay quarterly, that is $1.25 billion of circulation market cap in addition to OpenAI’s current enterprise value if they were to issue a payment token. The overall AI services market size is projected to reach $2 trillion in a few years. That translates to $500 billion market cap for a payment utility token alone.

Component marketplace for AI assets[​](#component-marketplace-for-ai-assets "Direct link to Component marketplace for AI assets")

----------------------------------------------------------------------------------------------------------------------------------

GaiaNet is a developer platform to create your agent services. We provide tools for you to do these.

*   Tools to generate finetuning datasets and perform finetuning on CPU and GPU machines.
*   Tools to ingest documents and create vector embeddings for the knowledge base.
*   Rust-based SDK to dynamically generate and manage prompts.
*   Rust-based SDK to extend the agent’s capability for invoking tools and software on the node.

For developers who do not wish to operate nodes, we are building a marketplace for

*   finetuned models
*   knowledge bases and datasets
*   function-calling plugins

All those components are blockchain-based assets represented by NFTs. A node operator could purchase NFTs for the components it wishes to use, and share service revenue with the component developers. That enables diverse and cashflow-generating assets to be issued from the GaiaNet ecosystem.

Conclusion[​](#conclusion "Direct link to Conclusion")

-------------------------------------------------------

GaiaNet provides open-source tools for individuals and teams to create agent services using their proprietary knowledge and skills. Developers could create finetuned LLMs, knowledge collections, and plugins for the agent, and issue assets based on those components. The GaiaNet protocol makes those nodes discoverable and accessible through GaiaNet domains.

*   [Abstract](#abstract)
    
*   [Introduction](#introduction)
    
*   [Open-source and decentralization](#open-source-and-decentralization)
    
*   [GaiaNet node](#gaianet-node)
    
*   [GaiaNet network](#gaianet-network)
    
*   [GaiaNet token](#gaianet-token)
    
*   [Component marketplace for AI assets](#component-marketplace-for-ai-assets)
    
*   [Conclusion](#conclusion)

---

# Public GaiaNet nodes | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Each GaiaNet node provides a web-based chatbot UI and an OpenAI compatible web service. Here are some popular nodes. Please refer to the [agent apps](/1.0.0/user-guide/apps/intro)
 section to see how to use the GaiaNet API in your favorite agent frameworks or apps.

Gaia nodes are organized into Gaia domains to provide public services. Each domain has a single API endpoint that load-balances across multiple nodes to ensure service availability. Below are some Gaia domains we provider for free to the public.

Public Gaia domains[​](#public-gaia-domains "Direct link to Public Gaia domains")

----------------------------------------------------------------------------------

### LLM: Llama 8b[​](#llm-llama-8b "Direct link to LLM: Llama 8b")

This domain runs Llama 3.1 8b agent nodes without any additional knowledge. It is capable of tool / function calling. [Learn more](/1.0.0/tutorial/tool-call)
 how to use tool call models in your agent app. [Chat with it](https://llama8b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) |
| Model Name (for LLM) | llama |
| Model Name (for Text embedding) | nomic-embed |
| API key | Empty or any value |

### Voice-to-text: Whisper v2 large[​](#voice-to-text-whisper-v2-large "Direct link to Voice-to-text: Whisper v2 large")

This domain runs Whisper v2 Large agent nodes for voice to text transcription and translation. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://whisper.gaia.domains/v1](https://whisper.gaia.domains/v1) |
| Model Name | whisper |
| API key | Empty or any value |

### Text-to-image: Realistic vision[​](#text-to-image-realistic-vision "Direct link to Text-to-image: Realistic vision")

This domain runs stable diffusion agent nodes fine-tuned to generate realistic portraits. The model is [Realistic Vision V6.0 B1](https://civitai.com/models/4201/realistic-vision-v60-b1)
. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://portrait.gaia.domains/v1](https://portrait.gaia.domains/v1) |
| Model Name | portrait |
| API key | Empty or any value |

### Text-to-voice: GPT-SoVITS[​](#text-to-voice-gpt-sovits "Direct link to Text-to-voice: GPT-SoVITS")

Coming soon!

Coding assistant agents[​](#coding-assistant-agents "Direct link to Coding assistant agents")

----------------------------------------------------------------------------------------------

### Coder[​](#coder "Direct link to Coder")

A general coding agent running a STOA coder LLM. It is a highly capable but slow model. [Chat with it](https://coder.gaia.domains/chatbot-ui/index.html)
 or use it from an IDE tool, such as [Cursor AI](/1.0.0/user-guide/apps/cursor)
 or [Zed](/1.0.0/user-guide/apps/zed)
. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://coder.gaia.domains/v1](https://coder.gaia.domains/v1) |
| Model Name (for LLM) | coder |
| Model Name (for Text embedding) | nomic-embed |
| API key | Empty or any value |

### Rust Coder[​](#rust-coder "Direct link to Rust Coder")

A coding agent supplemented with open-source [Rust content](https://huggingface.co/datasets/gaianet/learn-rust)
 from the Rust Foundation and published books. [Chat with it](https://rustcoder.gaia.domains/chatbot-ui/index.html)
 or use it from an IDE tool, such as [Cursor AI](/1.0.0/user-guide/apps/cursor)
 or [Zed](/1.0.0/user-guide/apps/zed)
. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://rustcoder.gaia.domains/v1](https://rustcoder.gaia.domains/v1) |
| Model Name (for LLM) | rustcoder |
| Model Name (for Text embedding) | nomic-embed |
| API key | Empty or any value |

Alternative LLM domains[​](#alternative-llm-domains "Direct link to Alternative LLM domains")

----------------------------------------------------------------------------------------------

### Llama 3b[​](#llama-3b "Direct link to Llama 3b")

This domain runs plain Llama 3.2 3b agent nodes without any additional knowledge. [Chat with it](https://llama3b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://llama3b.gaia.domains/v1](https://llama3b.gaia.domains/v1) |
| Model Name (for LLM) | llama3b |
| Model Name (for Text embedding) | nomic-embed |
| API key | Empty or any value |

### Qwen 7b[​](#qwen-7b "Direct link to Qwen 7b")

The Qwen 7b LLM is great for non-English tasks such as translation. [Chat with it](https://qwen7b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://qwen7b.gaia.domains/v1](https://qwen7b.gaia.domains/v1) |
| Model Name (for LLM) | qwen7b |
| Model Name (for Text embedding) | nomic-embed |
| API key | Empty or any value |

### Qwen 72b[​](#qwen-72b "Direct link to Qwen 72b")

The Qwen 72b LLM is a top open source LLM. It is very capable but could also be slow. It is capable of tool / function calling. [Learn more](/1.0.0/tutorial/tool-call)
 how to use tool call models in your agent app. [Chat with it](https://qwen72b.gaia.domains/chatbot-ui/index.html)
 or use it from another app. Replace OpenAI configuration in [your app](/1.0.0/user-guide/apps/intro)
 with the following.

| Config option | Value |
| --- | --- |
| API endpoint URL | [https://qwen72b.gaia.domains/v1](https://qwen72b.gaia.domains/v1) |
| Model Name (for LLM) | qwen72b |
| Model Name (for Text embedding) | nomic-embed |
| API key | Empty or any value |

*   [Public Gaia domains](#public-gaia-domains)
    *   [LLM: Llama 8b](#llm-llama-8b)
        
    *   [Voice-to-text: Whisper v2 large](#voice-to-text-whisper-v2-large)
        
    *   [Text-to-image: Realistic vision](#text-to-image-realistic-vision)
        
    *   [Text-to-voice: GPT-SoVITS](#text-to-voice-gpt-sovits)
        
*   [Coding assistant agents](#coding-assistant-agents)
    *   [Coder](#coder)
        
    *   [Rust Coder](#rust-coder)
        
*   [Alternative LLM domains](#alternative-llm-domains)
    *   [Llama 3b](#llama-3b)
        
    *   [Qwen 7b](#qwen-7b)
        
    *   [Qwen 72b](#qwen-72b)

---

# OpenAI ecosystem apps | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Since each GaiaNet node provides an OpenAI-compatible API service, it can be a drop-in replacement for OpenAI in almost all LLM applications and frameworks. Checkout the articles in this section for instructions and examples for how to use GaiaNet in popular LLM apps.

The OpenAI Python library[​](#the-openai-python-library "Direct link to The OpenAI Python library")

----------------------------------------------------------------------------------------------------

You can install the [official OpenAI Python library](https://pypi.org/project/openai/)
 as follows.

    pip install openai

When you create an OpenAI client using the library, you can pass in the API endpoint point as the `base_url`. Remember to append the `/v1` after the host name. You can find a list of public nodes [here](/1.0.0/user-guide/nodes)
.

    import openaiclient = openai.OpenAI(base_url="https://YOUR-NODE-ID.us.gaianet.network/v1", api_key="")

Alternatively, you could set an environment variable at the OS level.

    export OPENAI_API_BASE=https://YOUR-NODE-ID.us.gaianet.network/v1

Then, when you make API calls from the `client`, make sure that the `model` is set to the model name available on your node.

    response = client.chat.completions.create(    model="Meta-Llama-3-8B-Instruct-Q5_K_M",    messages=[        {"role": "system", "content": "You are a strategic reasoner."},            {"role": "user", "content": "What is the purpose of life?"}        ],        temperature=0.7,        max_tokens=500    ])

That's it! You can now take any application built with the official OpenAI Python library and use a Gaia node as its backend!

The OpenAI Node API library[​](#the-openai-node-api-library "Direct link to The OpenAI Node API library")

----------------------------------------------------------------------------------------------------------

You can install the OpenAI Node library which provides convenient access to the OpenAI REST API from TypeScript or JavaScript as follows:

    npm install openai

Import it into your project as:

    // Example usage in Node.jsconst OpenAI = require('openai');

Create an OpenAI client with a custom base URL. Remember to append the `/v1` after the host name.

    const client = new OpenAI({  baseURL: 'https://YOUR-NODE-ID.us.gaianet.network/v1',  apiKey: '' // Leave this empty when using Gaia});

Alternatively, you can set an environment variable using `dotenv` in Node.

    process.env.OPENAI_API_BASE = 'https://YOUR-NODE-ID.us.gaianet.network/v1';

Then, when you make API calls from the `client`, make sure that the `model` is set to the model name available on your node.

    async function callOpenAI() {  try {    const response = await client.chat.completions.create({      model: "Meta-Llama-3-8B-Instruct-Q5_K_M",      messages: [        { role: "system", content: "You are a strategic reasoner." },        { role: "user", content: "What is the purpose of life?" }      ],      temperature: 0.7,      max_tokens: 500    });    console.log(response.choices[0].message.content);  } catch (error) {    console.error('Error:', error);  }}//UsagecallOpenAI();

*   [The OpenAI Python library](#the-openai-python-library)
    
*   [The OpenAI Node API library](#the-openai-node-api-library)

---

# Quick start with GaiaNet Node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

This guide provides the requisite knowledge necessary to quickly get started with installing a GaiaNet node.

### Prerequisites[​](#prerequisites "Direct link to Prerequisites")

Before you get started, ensure that you have the following on your system:

| System | Minimum Requirements |
| --- | --- |
| OSX with Apple Silicon (M1-M4 chip) | 8GB RAM (16GB recommended) |
| Ubuntu Linux 20.04 with Nvidia CUDA 12 SDK | 8GB VRAM on GPU |
| Azure/AWS | Nvidia T4 GPU Instance |

Learn more about [system requirements](/1.0.0/node-guide/system-requirements)
.

### Installing the node[​](#installing-the-node "Direct link to Installing the node")

1.  Use the following command to download the latest version of the GaiaNet node:

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash

2.  Run the command printed on the terminal to set up the environment path, it is started with `source`. ![](/assets/images/quick-start-96fed073147105b82f4885934602852b.png)
    
3.  Use the following command to initialize the GaiaNet node according to the configuration options in `$HOME/gaianet/config.json`. By default, the GaiaNet is initialized with a [Llama 3.2 3B](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/)
     LLM and a [knowledge base about Paris](https://huggingface.co/datasets/gaianet/paris)
    . This command could take some time since it will download a very large LLM file.
    

    gaianet init

4.  Use the following command to start your node:

    gaianet start

#### After starting your node[​](#after-starting-your-node "Direct link to After starting your node")

*   A successful start prints a public URL for the node. Opening a browser to that URL will display the node information and allow you to chat with the AI agent on the node.

    ... ... https://0xf63939431ee11267f4855a166e11cc44d24960c0.gaia.domains

*   To stop the node: `gaianet stop`

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Customize](/1.0.0/node-guide/customize)
     your node. You probably do not wish to just run the default Phi-3 model and Paris knowledge base.
*   [Register](/1.0.0/node-guide/register)
     and join the GaiaNet protocol network to receive payments.
*   [Protect](/1.0.0/node-guide/tasks/protect)
     the node server to ensure stable service.
*   Install and manage [multiple nodes](/1.0.0/node-guide/tasks/multiple)
     on a single machine.

*   [Prerequisites](#prerequisites)
    
*   [Installing the node](#installing-the-node)
    
*   [Next steps](#next-steps)

---

# Agent frameworks and apps | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ OpenAI ecosystem apps\
-------------------------\
\
Since each GaiaNet node provides an OpenAI-compatible API service, it can be a drop-in replacement for](/1.0.0/user-guide/apps/intro)

[📄️ Dify + GaiaNet\
------------------\
\
You can configure the Dify framework using any GaiaNet node as the backend LLM API. That allows you to use your own](/1.0.0/user-guide/apps/dify)

[📄️ Open WebUI\
--------------\
\
You can configure the Open WebUI framework, a self-hosted WebUI, using any GaiaNet node as the backend LLM API. That allows you to use your own](/1.0.0/user-guide/apps/openwebui)

[📄️ Anything LLM\
----------------\
\
Anything LLM is the all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities. You can configure Anything LLM using the GaiaNet node as the LLM backend.](/1.0.0/user-guide/apps/anything_llm)

[📄️ Cursor AI IDE\
-----------------\
\
Cursor is an AI-powered code editor / IDE. Using LLMs to generate and review code, Cursor is an alternative to the very popular GitHub Copilot.](/1.0.0/user-guide/apps/cursor)

[📄️ Stockbot\
------------\
\
Stockbot is a lightning fast AI Chatbot that responds with live interactive stock charts, financials, news, screeners, and more. You can configure a Gaia node as the LLM backend.](/1.0.0/user-guide/apps/stockbot)

[📄️ FlowiseAI RAG chat\
----------------------\
\
FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use Gaia nodes as LLM service providers.](/1.0.0/user-guide/apps/flowiseai)

[📄️ FlowiseAI tool call\
-----------------------\
\
FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents.](/1.0.0/user-guide/apps/flowiseai-tool-call)

[📄️ LobeChat\
------------\
\
You can configure LobeChat to use a GaiaNet node as its backend LLM API. It provides a richer and more customizable UI than the default GaiaNet chatbot UI.](/1.0.0/user-guide/apps/lobechat)

[📄️ LlamaCloud\
--------------\
\
LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. LlamaParse can support different kinds of files, like pdf, doc, .ppt, and other formats.](/1.0.0/user-guide/apps/llamaparse)

[📄️ Zed\
-------\
\
Zed is a next-generation code editor designed for high-performance collaboration with humans and AI, and it is written in Rust. You can use Zed with your own Gaia node as the LLM backend. There are two big reasons for that](/1.0.0/user-guide/apps/zed)

[📄️ Obsidian\
------------\
\
Obsidian is a note-taking application that enables users to create, link, and visualize ideas directly on their devices. With Obsidian, you can seamlessly sync notes across devices, publish your work, and collaborate with others. The app is highly customizable, allowing users to enhance functionality through a wide range of plugins and themes. Its unique features include a graph view to visualize connections between notes, making it ideal for managing complex information and fostering creativity. Obsidian also emphasizes data privacy by storing notes locally.](/1.0.0/user-guide/apps/obsidian)

[📄️ CodeGPT\
-----------\
\
CodeGPT is a pair-programming partner for developers. It offers AI chat assistance, auto-completion, code explanation, error-checking, and much more. You can find the CodeGPT extension in VScode and Jetbrains. You can easily configure it to use Gaia nodes as LLM backends.](/1.0.0/user-guide/apps/codegpt)

[📄️ AI coding assistant: Continue\
---------------------------------\
\
Continue is the leading open-source AI code assistant.](/1.0.0/user-guide/apps/continue)

[📄️ LlamaCoder\
--------------\
\
LlamaCoder is an open-source tool designed to generate small apps with a single prompt. It leverages LLM to help you quickly create and enhance React applications.](/1.0.0/user-guide/apps/llamacoder)

[📄️ Agent Zero\
--------------\
\
Agent Zero is a general purpose AI agent application. You can simply ask it to accomplish tasks on the command line.](/1.0.0/user-guide/apps/agent-zero)

[📄️ Translation Agent + GaiaNet\
-------------------------------\
\
The Translation Agent originally built by Prof. Andrew Ng, designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use any GaiaNet node as the LLM backend.](/1.0.0/user-guide/apps/translation-agent)

[📄️ A planning agent\
--------------------\
\
The gpt planner is a Python](/1.0.0/user-guide/apps/gpt-planner)

[📄️ LlamaEdgeBook\
-----------------\
\
LlamaEdgeBook, forked from GroqBook, is an open-source tool that scaffolds the creation of books from a one-line prompt using open-source LLMs. You can configure the LlamaEdgeBook framework using any GaiaNet node as the backend LLM API.](/1.0.0/user-guide/apps/llamaedgebook)

[📄️ LlamaTutor\
--------------\
\
The LlamaTutor is a TypeScript](/1.0.0/user-guide/apps/llamatutor)

---

# Joining the Gaia Protocol | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

To join the Gaia protocol, you will need to complete the following two tasks.

*   Bind your node by connecting your node ID and device ID.
*   Join a Gaia domain

Bind your node[​](#bind-your-node "Direct link to Bind your node")

-------------------------------------------------------------------

After successfully running a Gaia node on your machine, it's time to join the Gaia protocol network and get rewards for sharing computing power with the world by binding your node ID and device ID to a Metamask account.

When you run a node with the Getting Started guide, you may notice that the Gaia software has generated a node ID for you. The node ID is an ETH address. The easiest way to find the node ID is to use `gaianet info` to print the node ID and device ID on the terminal.

    gaianet info

The output will be the following:

    Node ID: 0x80642b1----9a76a284efeb2bb6d142eDevice ID: device-e77---1446f1b51

![](/assets/images/register-01-fcaa9730a879aeaf3f91746bce5b1c68.png)

To receive rewards, bind your node and device IDs with a Metamask account using the Gaia web portal:

1.  Open [https://www.gaianet.ai/](https://www.gaianet.ai/)
     on your browser and click **"Connect."**
2.  Log into Metamask using the QR code and connect your account.
3.  Hover on your profile and click **"Setting"** and click **"Nodes."**
4.  Click **"Connect new node"** and enter your node and device IDs in the boxes.
5.  Click **"Join"**  
    ![](/assets/images/node-register-57acac5fa455af6b6b35b500149bd9a2.png)

After your node has successfully joined the network, it will be displayed in the list of nodes on the Node Page.

**If you are running multiple nodes, you can bind the node IDs and their corresponding device IDs to your MetaMask wallet.**

### Protect your node ID and device ID[​](#protect-your-node-id-and-device-id "Direct link to Protect your node ID and device ID")

The Gaia installer generates a pair of ETH address and keystore and password for your node automatically. This information is stored in the `gaianet/nodeid.json` file. Please keep the JSON file carefully.

*   The ETH address is your node ID. You will use this ETH address to join the Gaia network.
*   The keystore stores the private key associated with the ETH address encrypted by the password.

The `nodeid.json` file is the only proof that your node belongs to you. In many protocol operations, you will need this private key to sign request messages to send to the protocol smart contracts.

The device ID is only visible to you.

Join a Domain[​](#join-a-domain "Direct link to Join a Domain")

----------------------------------------------------------------

Once your node is successfully bound, you can proceed to join a Gaia Domain. There are two ways to join a domain:

*   Join a domain from your nodes management page
*   Join a domain from the [AI Agent Domains page](https://www.gaianet.ai/agents)
    

### Steps to Join a Domain from Your Node Management Page[​](#steps-to-join-a-domain-from-your-node-management-page "Direct link to Steps to Join a Domain from Your Node Management Page")

1.  **Locate Your Nodes**  
    Navigate to **Profile → Nodes** to view the list of nodes you already registered.
    
2.  **Initiate the Join Process**  
    Click the `...` button next to your node and select **Join Domain**. This will guide you through the steps to join a Gaia Domain.
    

![image](https://github.com/user-attachments/assets/e7b0613b-332f-46bd-ae35-88ab5e0e8280)

3.  **Follow the Join Steps**
    
    *   **Update Node Domain**  
        Change your node's domain to `gaia.domains`, which is necessary. The following command line will change your node's domain to `gaia.domains`. If your node has already used `gaia.domains`, then you can skip this step.
        
            gaianet stopgaianet config --domain gaia.domainsgaianet initgaianet start
        
    *   **Select a Domain**  
        Choose a domain from the available online domain list. You can review essential details for each domain, such as:
        
        *   Required LLM
        *   Join policy
        *   Number of joined nodes
    
    ![image](https://github.com/user-attachments/assets/fa99b0ef-e25d-4e51-933a-30062cb480d5)
    
    *   **Verify Node Status**  
        Ensure your node meets the domain's requirements. If everything checks out, you can submit a join request to the domain.
    
    ![image](https://github.com/user-attachments/assets/caafe74c-9393-43a2-a428-1b008b7ce717)
    
    > Normally, your node needs to be online and meet the specific LLM requirement.
    

### Steps to Join a Domain from the AI Agent Domains page[​](#steps-to-join-a-domain-from-the-ai-agent-domains-page "Direct link to Steps to Join a Domain from the AI Agent Domains page")

The AI agent Domains page featured lots of domains various from crypto knowledge to useful tools. Each has options for "Chat Now" or "Join Now". You can click on "Join Now" to join that domain.

![image](https://github.com/user-attachments/assets/6dbcc6fd-f6d8-452a-8e1c-2581bd637c9f)

### Important Notes[​](#important-notes "Direct link to Important Notes")

*   Some Gaia Domains require approval for new nodes to join.
*   Ensure you comply with the domain's rules before your node becomes publicly accessible under the domain's URL.

Following these steps will seamlessly integrate your node into a Gaia Domain.

*   [Bind your node](#bind-your-node)
    *   [Protect your node ID and device ID](#protect-your-node-id-and-device-id)
        
*   [Join a Domain](#join-a-domain)
    *   [Steps to Join a Domain from Your Node Management Page](#steps-to-join-a-domain-from-your-node-management-page)
        
    *   [Steps to Join a Domain from the AI Agent Domains page](#steps-to-join-a-domain-from-the-ai-agent-domains-page)
        
    *   [Important Notes](#important-notes)

---

# Customize Your GaiaNet Node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

A key goal of the GaiaNet project is to enable each individual to create and run his or her own agent service node using finetuned LLMs and proprietary knowledge. In all likelihood, you are not going to run a node with the [default](/1.0.0/node-guide/quick-start)
 Llama 3.2 3B LLM and Paris guidebook knowledge base. In this chapter, we will discuss ways to customize your node.

Pre-set configurations[​](#pre-set-configurations "Direct link to Pre-set configurations")

-------------------------------------------------------------------------------------------

All the node configuration options, such as LLM settings, vector collection for the knowledge base, and prompts, are all in the `gaianet/config.json` file. You can edit this file directly to use your models and vector collections. Or, you can select a different `config.json` when you initialize the node. Just pass in a URL to the `config.json` file in your `gaianet init` command. We have several pre-set `config.json` files to choose from [in this repo](https://github.com/GaiaNet-AI/node-configs)
. For example, the following command initializes a GaiaNet node with a Llama 3 8B model.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json

> The URL to the `config.json` must point to the actual text file. (i.e., the `raw.githubusercontent.com` URL for GitHub links) instead of the GitHub HTML page for that file.

The config subcommand[​](#the-config-subcommand "Direct link to The config subcommand")

----------------------------------------------------------------------------------------

After you have initialized the node, you can still make changes to its configuration by editing the `config.json` file directly. But it is easier and safer to use the `gaianet` CLI to make changes.

> You MUST run `gaianet init` again after you make any changes to the node configuration.

The following command shows the `config.json` fields you can make changes to.

    gaianet config list

Now, let's look at some examples.

### Select an LLM[​](#select-an-llm "Direct link to Select an LLM")

There are over 10,000 finetuned open-source LLMs you can choose from on Huggingface. They each have different sizes (larger models are more capable but more expensive to run), unique capabilities (e.g., uncensored, to excel in math or reasoning, to support large context length etc), domain expertise (e.g., medicine, coding), and / or styles (e.g., to speak like a teacher or a pirate, to respond in code, to follow conversations).

To replace GaiaNet node's default LLM with an alternative finetuned model, you will need to make changes to the model file, prompt template, and model context length parameters. Those parameters vary depending on the model, but they can be found on the [GaiaNet Huggingface organization's](https://huggingface.co/gaianet)
 model cards. For example, the following command changes the LLM to a Llama 3 8B model.

    gaianet config \  --chat-url https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf \  --chat-ctx-size 4096 \  --prompt-template llama-3-chat 

> The llama 3 8B model requires at least 16GB of RAM.

If none of the published finetuned models are perfect for your use case, you can also finetune your own LLM by following [these guides](/1.0.0/creator-guide/finetune/intro)
. Your GaiaNet node can run your own finetuned models.

> The `--chat-url` argument could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a privately trained or finetuned LLM model file.

### Select a knowledge base[​](#select-a-knowledge-base "Direct link to Select a knowledge base")

A key feature of GaiaNet is that users can create and deploy proprietary knowledge base on the node to supplement the LLM. Each knowledge base is a snapshot file for a vector collection. We encourage you to [create your own knowledge base](/1.0.0/creator-guide/knowledge/concepts)
. But you can also use ready-made knowledge bases. You will need to do the following.

*   specify the URL to the vector collection (i.e., the `snapshot` or `snapshot.tar.gz` file) in the `snapshot` option.
*   use the same embedding model that generated this vector collection.
*   modify the `system_prompt` to give the model background knowledge.
*   modify the `rag_prompt` to instruct the model to answer the question when context is retrieved from the vector collection.

The following example changes the knowledge base in the node from "Paris guidebook" to "London guidebook".

    gaianet config \  --snapshot https://huggingface.co/datasets/gaianet/london/resolve/main/london_768_nomic-embed-text-v1.5-f16.snapshot.tar.gz \  --embedding-url https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf \  --embedding-ctx-size 8192 \  --system-prompt "You are a tour guide in London, UK. Please answer the question from a London visitor accurately." \  --rag-prompt "The following text is the context for the user question.\n----------------\n"

> The `--snapshot` could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a private vector collection snapshot.

Depending on the quality and size of the vectors, you might also need to change the `qdrant-` options to customize the retrieval behavior.

*   `qdrant-limit` sets the max number of relevant context to add to the prompt. If your knowledge base consists of large sections of text (i.e., each book chapter is a vector), you should probably make this 1 or 2 to limit the prompt length to a reasonable size.
*   `qdrant-score-threshold` is the min match "score" the knowledge content must meet in order to be considered "relevant". This depends on the quality of the knowledge text and the embedding model. In general, this score should be over 0.5 to reduce irrelevant context in the prompt.

> The embedding model encodes and transforms text into vectors so that they can be stored, searched and retrieved. For different context material, you might need a different embedding model to achieve the optimal performance. The [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
>  is a good place to see the performance benchmarks of embedding models. You can find many of them in the [gaianet organization on Huggingface](https://huggingface.co/gaianet)
> .

### Customize prompts[​](#customize-prompts "Direct link to Customize prompts")

In `config.json`, you can also customize the prompts. Prompts are often tailored for the finetuned LLM or the knowledge base to generate optimal responses from the node.

The `--system-prompt` option sets a system prompt. It provides the background and "personality" of the node. Each API request can set its own system prompt.

The `--rag-prompt` is the prompt to be appended after the system prompt (or user query). It introduces the RAG context retrieved from the vector database, which follows it.

The `--rag-policy` option specifies where the `rag-prompt` and context should go. By default, its value is `system-message` and it puts the context in the system prompt. But you could also set it to `last-user-message`, which puts the `rag-prompt` and context in front of the latest message from the user.

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

Remember to re-initialize and re-start the node after you make configuration changes.

    # If the node is running# gaianet stopgaianet initgaianet start

Next, you can

*   [Create a knowledge base](/1.0.0/creator-guide/knowledge/concepts)
     from your proprietary knowledge or skills.
*   [Finetune](/1.0.0/creator-guide/finetune/intro)
     your own LLM.

Have fun!

*   [Pre-set configurations](#pre-set-configurations)
    
*   [The config subcommand](#the-config-subcommand)
    *   [Select an LLM](#select-an-llm)
        
    *   [Select a knowledge base](#select-a-knowledge-base)
        
    *   [Customize prompts](#customize-prompts)
        
*   [Next steps](#next-steps)

---

# Quick Start with Launching Gaia Domain | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

This guide provides all the information you need to quickly set up and run a Gaia Domain.

> **Note:** Ensure that you are the owner of a Gaia Domain Name before proceeding. You can verify your Gaia Domain Name in the "Assets" section of your profile.

Gaia simplifies the process for domain operators to launch and host a Gaia Domain service in just a few clicks.

### Steps to Launch Your Gaia Domain[​](#steps-to-launch-your-gaia-domain "Direct link to Steps to Launch Your Gaia Domain")

1.  **Access the Create Gaia Domain Page**  
    Click **LAUNCH DOMAIN** in the "Domain" or "Assets" section under [your profile](https://www.gaianet.ai/)
    . This will take you to the Create Gaia Domain page.
    
2.  **Fill in Domain Details**  
    Enter the general information for your domain, including:
    
    *   Domain profile
    *   Domain Name
    *   Description
    *   System Prompt
3.  **Choose a Gaia Domain Name**  
    Select a Gaia domain name from your assets.
    
4.  **Select a Supplier**  
    Currently, **Gaia Cloud** is the only supplier.
    
5.  **Pick a Gaia Domain Tier**  
    Choose a tier to enhance your domain's rewards, which is necessary.
    
6.  **Configure Server and Management Options**
    
    *   Confirm the server configuration for running your domain.
    *   Set management preferences, such as whether nodes can join automatically and the specific LLM to use.

After completing these six steps, your Gaia Domain will be successfully launched and other nodes can join your domain.

*   [Steps to Launch Your Gaia Domain](#steps-to-launch-your-gaia-domain)

---

# Knowledge bases | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Gaia nodes with long-term knowledge\
---------------------------------------\
\
The LLM app requires both long-term and short-term memories. Long-term memory includes factual knowledge, historical facts, background stories etc. They are best added to the context as complete chapters instead of small chunks of text to maintain the internal consistency of the knowledge.](/1.0.0/creator-guide/knowledge/concepts)

[📄️ Knowledge base from a plain text file\
-----------------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a plain text file. The](/1.0.0/creator-guide/knowledge/text)

[📄️ Knowledge base from a markdown file\
---------------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a markdown file. The](/1.0.0/creator-guide/knowledge/markdown)

[📄️ Knowledge base from source / summary pairs\
----------------------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot for optimal retrieval of](/1.0.0/creator-guide/knowledge/csv)

[📄️ Knowledge base from a PDF file\
----------------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a PDF file. First, we will parse the unstructured PDF file to a structured markdown file. Then, we will follow the steps from Knowledge base from a markdown file to create embedding for your PDF files.](/1.0.0/creator-guide/knowledge/pdf)

[📄️ Knowledge base from a URL\
-----------------------------\
\
In this section, we will discuss how to create a vector collection snapshot from a Web URL. First, we will parse the URL to a structured markdown file. Then, we will follow the steps from Knowledge base from a markdown file to create embedding for your URL.](/1.0.0/creator-guide/knowledge/firecrawl)

[📄️ Build a knowledge base using Gaia web tool\
----------------------------------------------\
\
GaiaNet has developed a tool for making vector collection snapshot files, so everyone can easily create their own knowledge base.](/1.0.0/creator-guide/knowledge/web-tool)

---

# GaiaNet Node with finetuned LLMs | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Fine-tune LLMs\
------------------\
\
You could fine-tune an open-source LLM to](/1.0.0/creator-guide/finetune/intro)

[📄️ llama.cpp\
-------------\
\
The popular llama.cpp tool comes with a finetune utility. It works well on CPUs! This fine-tune guide is reproduced with](/1.0.0/creator-guide/finetune/llamacpp)

---

# Dify + Gaia | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

You can configure the Dify framework using any Gaia node as the backend LLM API. That allows you to use your own or community Gaia nodes in any application built on Dify. It supports

*   The hosted [Dify.ai](https://dify.ai/)
     service.
*   Products and services with embedded Dify framework, such as the [Terminus](https://www.jointerminus.com/)
     project.
*   Any product that is built on the open source [Dify framework](https://github.com/langgenius/dify)
    .

Steps[​](#steps "Direct link to Steps")

----------------------------------------

First, log into Dify's web portal and select `Settings | Model Provider`. From the list, you can add an OpenAI-API-compatible provider.

Add an LLM model with the modela name and API endpoint listed on your Gaia node's web dashboard. Or, you can just add [a popular Gaia node](/nodes)
. Leave the API Key field empty.

![Configure a Gaia Llama3 8b model in Dify](/assets/images/dify_chat-5ed526b07c37f2ec1eac1628ba724b77.png)

Most Dify applications also require an embedding model to search text in the vector space. Add an mebedding model with the modela name and API endpoint listed on your Gaia node's web dashboard. Or, you can just add [a popular Gaia node](/nodes)
. Leave the API Key field empty.

![Configure a Gaia embedding model in Dify](/assets/images/dify_embedding-4e1036868a26c71ea5da20fb1eb8b819.png)

That's it. You can now see that the new models are available at the top panel of Dify for every chatbot or agent. Just select your Gaia models for chat or embedding, and the Dify app will automagically use it!

![Select a Gaia node as backend model in Dify](/assets/images/dify_select-ca2df00e17b8049965bd0543bff1c3be.png)

![Chat with the Gaia Llama3 8b model in Dify](/assets/images/dify_chatbot_ui-eabc342c591397b0f2c4746bfd64e09a.png)

*   [Steps](#steps)

---

# Open WebUI | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

You can configure the Open WebUI framework, a self-hosted WebUI, using any Gaia node as the backend LLM API. That allows you to use your own or community Gaia nodes in any application built on Open WebUI.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) | llama |
| Embedding | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) | nomic |

Start the Open WebUI on your machine[​](#start-the-open-webui-on-your-machine "Direct link to Start the Open WebUI on your machine")

-------------------------------------------------------------------------------------------------------------------------------------

After successfully starting the Gaia node, you can use `docker run` to start the Open WebUI.

    docker run -d -p 3000:8080 \  -v open-webui:/app/backend/data \  -e OPENAI_API_BASE_URL="https://llama8b.gaia.domains/v1" \  -e OPENAI_API_KEYS="gaianet" \  --name open-webui \  --restart always \  ghcr.io/open-webui/open-webui:main

Then, open `http://localhost:3000` in your browser and you will see the Open WebUI page.

You can also configure your own node when the webUI is started.

*   Click on your profile on the top right corner and choose **Setting**.
*   Then choose Connections. In the OpenAI API field, type your node base URL and enter several random characters.
*   Click on Save to make the change take effective.

![](/assets/images/openwebui-02-931678d7734f60414bc258b7859c4928.png)

Use Open WebUI as a Chatbot UI[​](#use-open-webui-as-a-chatbot-ui "Direct link to Use Open WebUI as a Chatbot UI")

-------------------------------------------------------------------------------------------------------------------

Simply choose the chat model under **Select a model** and then you can send messages to the Gaia node.

![](/assets/images/openwebui-01-1a26ed6523b22a60f60f40dd656cb407.png)

Use Open WebUI as a client-side RAG tool[​](#use-open-webui-as-a-client-side-rag-tool "Direct link to Use Open WebUI as a client-side RAG tool")

-------------------------------------------------------------------------------------------------------------------------------------------------

Open WebUI also offers a way to implement RAG application. Since the Gaia nodes have OpenAI-compatible embedding APIs, you can also use this feature. However, to use this feature, it's recommend to start a node without any snapshots, like [this one](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-8b-instruct)
.

**Step 1:** Use Gaia node as the embedding API

*   Click on **Workspace** on left top and choose **Documents** tab. This is where you manage the uploaded documents.
*   Click on **Document Settings** to configure the embedding setting.
*   In the **General Settings**, choose OpenAI as the Embedding Model Engine. Enter the node API base URL and several random characters. Then, enter the embedding model name in the Embedding Model field. Click Save to apply the changes.

![](/assets/images/openwebui-04-1b648b9e231f053f6f761981134cdf81.png)

**Step 2:** Use Gaia node as the embedding API

Click on **+** to upload your documentations.

**Step 3:** Chat

Then go back to the chat page. Before you send a message, type **#** to choose the document you want to as the context.

![](/assets/images/openwebui-05-ba004a234279045cee6af7fa304126c8.png)

That's it.

*   [Prerequisites](#prerequisites)
    
*   [Start the Open WebUI on your machine](#start-the-open-webui-on-your-machine)
    
*   [Use Open WebUI as a Chatbot UI](#use-open-webui-as-a-chatbot-ui)
    
*   [Use Open WebUI as a client-side RAG tool](#use-open-webui-as-a-client-side-rag-tool)

---

# Anything LLM | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Anything LLM is the all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities. You can configure Anything LLM using the Gaia node as the LLM backend.

It's recommended to start a node without any snapshots, like [this one](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-8b-instruct)
.

Steps[​](#steps "Direct link to Steps")

----------------------------------------

First, we will need to add Gaia Node as the LLM chat model and embedding model.

*   Go to Settings
*   Choose Local AI as the LLM provider in the LLM inference section. Then copy and paste the Gaia node API base URL in the LocalAI Base URL. The chat model and embedding model along with your node will be loaded automatically. Choose the chat model here and input the context length. Remember to click on Save Changes to make the changes take effect.

![](/assets/images/anything-llm-01-f6c30d16fad64e2e6d8d2c0735914a25.png)

*   Then go to the Embedding Preference section and choose LocalAI as the embedding provider. Then copy and paste the Gaia node API base URL in the LocalAI Base URL. The chat model and embedding model along with your node will be loaded automatically. Choose the embedding model here and input the max embedding chunk length. Don't forget to click on Save Changes to make the changes take effect.

The above operations make the Gaia node as the LLM backend.

Second, let's set up the data source. You can upload a file or use the data connectors provided by Anything LLM.

![](/assets/images/anything-llm-02-3764f4ddb2aeef48cbd11eead1a2aa57.png)

When you move a data source to the workspace, Anything LLM will call the Gaia node's embedding API to chunk and compute embeddings for your documentation, which may take some minutes.

That's it. When the embedding is done, go back to your workspace and ask a question.

![](/assets/images/anything-llm-03-3630c1f82a45461237e6a1fcf825a8d3.png)

> You can check out the `start-llamaedge.log` to check what happens.

*   [Steps](#steps)

---

# LlamaTutor | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

The [LlamaTutor](https://github.com/Nutlope/llamatutor)
 is a TypeScript application using Llama 3.1 to act as an open-source AI personal tutor. When you run it, it will ask the LLM to search on the web based on your inquiry topic and then generate content for you to learn.

The program didn't use the official OpenAI library, we can use the forked one that supports [using a Gaia node](/intro)
 as the LLM backend.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [Run your own node](/getting-started/quick-start)
    
*   [Use a public node](/nodes)
    

In this tutorial, we will use a public Llama3 node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, we will need to get the source code of the forked LlamaTutor

    git clone https://github.com/JYC0413/llamatutor.gitcd llamatutor

Next, configure the `.env` file.

    cp .example.env .env

You will need to configure four parameters here.

*   SERPER\_API\_KEY: The [serper API key](https://serper.dev/)
     for searching content online. You can also use BING\_API\_KEY here.
*   LLAMAEDGE\_BASE\_URL: URL for the LLM API base URL.
*   LLAMAEDGE\_MODEL\_NAME: Name of the model to be used.
*   LLAMAEDGE\_API\_KEY: API key for accessing the LLM services.

Then, we will need to install the required dependencies.

    npm install

Next, let's run the application locally.

    npm run dev

Finally, open [http://localhost:3000](http://localhost:3000)
 in your browser and start to learn.

![](/assets/images/llamatutor-01-f14f9e39e0ece9f4ae961504fe4d5eda.png)

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# Cursor AI IDE | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

[Cursor](https://www.cursor.com/)
 is an AI-powered code editor / IDE. Using LLMs to generate and review code, Cursor is an alternative to the very popular GitHub Copilot. You can use Cursor with your own Gaia node as the LLM backend. There are two big reasons for that

*   Your Gaia node could be supplemented by a knowledge base that is specific to your proprietary code repository, programming language choices, and coding guidelines / styles.
*   Your Gaia node could ensure that your code stays private within your organization.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node to provide LLM API services. You can

*   [Run your own node](/getting-started/quick-start)
    
*   [Use a public node](/nodes)
    

In this tutorial, we will use public [Qwen 2.5 Coder](https://github.com/QwenLM/Qwen2.5-Coder)
 nodes to power Cursor.

| Model type | API base URL | Model name |
| --- | --- | --- |
| General coding assistant | `https://coder.gaia.domains/v1` | coder |
| Coding assistant with Rust knowledge | `https://rustcoder.gaia.domains/v1` | rustcoder |
| Rust expert (slower but more accurate) | `https://rustexpert.gaia.domains/v1` | rustexpert |

> A limitation of Cursor is that it does not support local LLM services. A Gaia node comes with a default networking tunnel that turns your local LLM service into a HTTPS service accessible from the Internet. That allows Cursor to use your own private LLM for coding. Start your own [Qwen Coder](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct)
>  or [Qwen Coder with Rust](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct_rustlang)
>  nodes today!

Configure Cursor[​](#configure-cursor "Direct link to Configure Cursor")

-------------------------------------------------------------------------

First, download and install [Cursor](https://www.cursor.com/)
. Click on the **Settings** button on the top right. Then, click on **Models** to configure the backend LLM service.

Second, add a model named `coder` and turn off all the other models like `gpt-4o`.

Third, go to the OpenAI API Key section,

*   Click on **Override OpenAI Base URL**. Type `https://coder.gaia.domains/v1` here.
*   For the OpenAI API key, you can use any random chars such as `GAIA`. Click on **Verfiy** to test if the connection is correct.

![](/assets/images/cursor-01-fad7a1003a06d1009e698e9df24fde0c.png)

Use Cursor[​](#use-cursor "Direct link to Use Cursor")

-------------------------------------------------------

You can use

*   **command + K** to edit the highlighted code
*   **command + L** to open the chat room and ask questions about the code.

![](/assets/images/cursor-02-28b91f7d5e5c8be2cf75d3b8d3333184.png)

Video Guide[​](#video-guide "Direct link to Video Guide")

----------------------------------------------------------

*   [Prerequisites](#prerequisites)
    
*   [Configure Cursor](#configure-cursor)
    
*   [Use Cursor](#use-cursor)
    
*   [Video Guide](#video-guide)

---

# Gaia nodes with long-term knowledge | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

The LLM app requires both long-term and short-term memories. Long-term memory includes factual knowledge, historical facts, background stories etc. They are best added to the context as complete chapters instead of small chunks of text to maintain the internal consistency of the knowledge.

[RAG](https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/)
 is an important technique to inject contextual knowledge into an LLM application. It improves accuracy and reduces the hallucination of LLMs. An effective RAG application combines real-time and user-specific short-term memory (chunks) with stable long-term memory (chapters) in the prompt context.

Since the application's long-term memory is stable (even immutable), we package it in a vector database tightly coupled with the LLM. The client app assembles the short-term memory in the prompt and is supplemented with the long-term memory on the LLM server. We call the approach "server-side RAG".

> The long context length supported by modern LLMs are especially well-suited for long-term knowledge that are best represented by chapters of text.

A Gaia node is an OpenAI compatible LLM service that is grounded by long-term knowledge on the server side. The client application can simply chat with it or provide realtime / short-term memory since the LLM is already aware of the domain or background.

> For example, if you ask ChatGPT the question What is Layer 2, the answer is that Layer 2 is a concept from the computer network. However, if you ask a blockchain person, they answer that Layer 2 is a way to scale the original Ethereum network. That's the difference between a generic LLM and knowledge-supplemented LLMs.

We will cover the external knowledge preparation and how a knowledge-supplemented LLM completes a conversation. If you have learned how a RAG application works, go to [Build a RAG application with Gaia](/knowledge-bases/web-tool)
 to start building one.

1.  Create embeddings for your own knowledge as the long-term memory.
2.  Lifecycle of a user query on a knowledge-supplemented LLM.

For this solution, we will use

*   a chat model like Llama-3-8B for generating responses to the user.
*   a text embedding model like [nomic-embed-text](https://huggingface.co/second-state/Nomic-embed-text-v1.5-Embedding-GGUF)
     for creating and retrieving embeddings.
*   a Vector DB like Qdrant for storing embeddings.

Workflow for creating knowledge embeddings[​](#workflow-for-creating-knowledge-embeddings "Direct link to Workflow for creating knowledge embeddings")

-------------------------------------------------------------------------------------------------------------------------------------------------------

The first step is to create embeddings for our knowledge base and store the embeddings in a vector DB.

![create-embedding](https://github.com/GaiaNet-AI/docs/assets/45785633/2ff40178-64f4-4e2e-bbd9-f12ce35186b7)

First of all, we split the long text into sections (i.e, chunks). All LLMs have a maximum context length. The model can't read the context if the text is too long. The most used rule for a Gaia node is to put the content in one chapter together. Remember, insert a blank line between two chunks. You can also use other algorithms to chunk your text.

After chunking the document, we can convert these chunks into embeddings leveraging the embedding model. The embedding model is trained to create embeddings based on text and search for similar embeddings. We will use the latter function in the process of user query.

Additionally, we will need a vector DB to store the embeddings so that we can retrieve these embeddings quickly at any time.

On a Gaia node, we will get a database snapshot with the embeddings to use at last. Check out how to create your embeddings using [Gaia web tool](/knowledge-bases/web-tool)
, [from a plain text file](/knowledge-bases/how-to/text)
, and [from a markdown file](/knowledge-bases/how-to/markdown)
.

Lifecycle of a user query on a knowledge-supplemented LLM[​](#lifecycle-of-a-user-query-on-a-knowledge-supplemented-llm "Direct link to Lifecycle of a user query on a knowledge-supplemented LLM")

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Next, let's learn the lifecycle of a user query on a knowledge-supplemented LLM. We will take [a Gaia Node with Gaia knowledge](https://knowledge.gaianet.network/chatbot-ui/index.html)
 as an example.

![user-query-rag](https://github.com/GaiaNet-AI/docs/assets/45785633/c64b85ea-65f0-43d2-8ab3-78889d21c248)

### Ask a question[​](#ask-a-question "Direct link to Ask a question")

when you send a question in human language to the node, the embedding model will first convert your question to embedding.

### Retrieve similar embeddings[​](#retrieve-similar-embeddings "Direct link to Retrieve similar embeddings")

Then, the embedding model will search all the embeddings stored in the Qdrant vector DB and retrieve the embeddings that are similar to the question embeddings.

### Response to the user query[​](#response-to-the-user-query "Direct link to Response to the user query")

The embedding node will return the retrieved embeddings to the chat model. The chat model will use the retrieved embeddings plus your input questions as context to answer your queries finally.

*   [Workflow for creating knowledge embeddings](#workflow-for-creating-knowledge-embeddings)
    
*   [Lifecycle of a user query on a knowledge-supplemented LLM](#lifecycle-of-a-user-query-on-a-knowledge-supplemented-llm)
    *   [Ask a question](#ask-a-question)
        
    *   [Retrieve similar embeddings](#retrieve-similar-embeddings)
        
    *   [Response to the user query](#response-to-the-user-query)

---

# Stockbot | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Stockbot is a lightning fast AI Chatbot that responds with live interactive stock charts, financials, news, screeners, and more. You can configure a Gaia node as the LLM backend.

> Please note, the Stockbot leverages function call to call the external API. You will need to use the model which supports function call, like [llama-3.1-8b](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool)
> .

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use a public Llama3 node with the function call support.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llamatool.us.gaianet.network/v1](https://llamatool.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, we will need to get the source code of the forked Stockbot.

    git clone https://github.com/JYC0413/stockbot-on-groq.gitcd stockbot-on-groq

Next, configure the `.env` file.

    cp .env.example .env.local

You will need to configure four parameters here.

*   LLAMAEDGE\_BASE\_URL: URL for the LLM API base URL.
*   LLAMAEDGE\_MODEL\_NAME: Name of the model to be used.
*   LLAMAEDGE\_API\_KEY: API key for accessing the LLM services.

Then, we will need to install the required dependencies.

    npm install

Next, let's run the application locally.

    npm run dev

Finally, you can open [http://localhost:3000](http://localhost:3000)
 and ask the stock related questions.

![](/assets/images/stockbot-01-fa5ede68e47bee6abc9b86c9d0d1441c.png)

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# Prompt Engineering Tool | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Video Guide[​](#video-guide "Direct link to Video Guide")

----------------------------------------------------------

### Test your AI Agents with multiple LLMs using Gaia's Prompt Engineering Tool[​](#test-your-ai-agents-with-multiple-llms-using-gaias-prompt-engineering-tool "Direct link to Test your AI Agents with multiple LLMs using Gaia's Prompt Engineering Tool")

Are you working with AI language models? This tool is a game-changer! It lets you:

*   ✨ Test Prompts on Multiple AI Models: Quickly compare results side-by-side to see which model performs best.
*   🔄 Save and Reuse Prompts: Store your favorite prompts for easy access anytime.
*   🔧 Use Variables for Flexibility: Customize your prompts with variables to suit any use case.

Whether you're fine-tuning, experimenting, or just exploring the capabilities of different models, this tool will make your workflow smoother and more efficient.

Try it out now and see how easy it is to get the most out of your AI language models! 🔗 [https://prompt-engineering-toolkit-rho.vercel.app/](https://prompt-engineering-toolkit-rho.vercel.app/)

*   [Video Guide](#video-guide)
    *   [Test your AI Agents with multiple LLMs using Gaia's Prompt Engineering Tool](#test-your-ai-agents-with-multiple-llms-using-gaias-prompt-engineering-tool)

---

# FlowiseAI RAG chat | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use Gaia nodes as LLM service providers.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) | llama |
| Embedding | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) | nomic |

Start a FlowiseAI server[​](#start-a-flowiseai-server "Direct link to Start a FlowiseAI server")

-------------------------------------------------------------------------------------------------

Follow [the FlowiseAI guide](https://docs.flowiseai.com/getting-started)
 to install Flowise locally

    npm install -g flowisenpx flowise start

After running successfully, you can open [http://localhost:3000](http://localhost:3000)
 to check out the Flowise AI tool.

Build a documents QnA chatbot[​](#build-a-documents-qna-chatbot "Direct link to Build a documents QnA chatbot")

----------------------------------------------------------------------------------------------------------------

FlowiseAI allows you to visually set up all the workflow components for an AI agent. If you're new to FlowiseAI, it's recommended to use a template quick start. In fact, there are lots of templates around OpenAI in the Flowise marketplace. All we need to do is to replace the ChatOpenAI component with the ChatLocalAI component.

Let's take the **Flowise Docs QnA** as an example. You can build a QnA chatbot based on your documents. In this example, we would like to chat with a set of documents in a GitHub repo. The default template was built with OpenAI and we will now change it to use an open-source LLM on a Gaia node.

### Get the **Flowise Docs QnA** template[​](#get-the-flowise-docs-qna-template "Direct link to get-the-flowise-docs-qna-template")

![](/assets/images/flowise-01-38ce1dc6e85edf44ce942f3c032751bc.png)

Click on Marketplaces on the left tab to browse all the templates. The template **Flowise Docs QnA** we will use is the first one.

![](/assets/images/flowise-02-5e4c3abe45028dea057cadc04cae301b.png)

Then, click on Use this template button on the left top corner to open the visual editor.

### Connect the chat model API[​](#connect-the-chat-model-api "Direct link to Connect the chat model API")

You will need to delete the ChatOpenAI component and click the + button to search ChatLocalAI, and then drag the ChatLocalAI to the screen.

![](/assets/images/flowise-03-00ebd2de39ebd00533f63659e11b1e48.png)

Then, you will need to input

*   the Gaia node base URL `https://llama8b.gaia.domains/v1`
*   the model name `llama`

Next, connect the ChatLocalAI component with the field `Chat model` in the **Conversational Retrieval QA Chain** component.

### Connect the embedding model API[​](#connect-the-embedding-model-api "Direct link to Connect the embedding model API")

The default template uses the OpenAI Embeddings component to create embeddings for your documents. We need to replace the **OpenAI Embeddings** component with the **LocalAI Embeddings** component.

*   Use the Gaia node base URL `https://llama8b.gaia.domains/v1` in the Base Path field.
*   Input the model name `nomic-embed-text-v1.5.f16` in the Model Name field.

Next, connect the **LocalAI Embeddings** component with the field `embedding` in the **In-Memory Vector Store** component.

### Set up your documents[​](#set-up-your-documents "Direct link to Set up your documents")

Then, let's go through the GitHub component to connect the chat application to our documents on GitHub. You will need to put your docs GitHub link into the **Repo Link** field. For example, you can put Gaia's docs link: `https://github.com/GaiaNet-AI/docs/tree/main/docs`.

Give it a try[​](#give-it-a-try "Direct link to Give it a try")

----------------------------------------------------------------

You can send a question like "How to install a Gaia node" after saving the current chatflow.

![](/assets/images/flowise-04-7ee98ecd7039fbf54776e2300c1e9ca9.png)

You will get the answer based on the Gaia docs, which are more accurate.

More examples[​](#more-examples "Direct link to More examples")

----------------------------------------------------------------

There are lots of examples on the Flowise marketplace. To build a Flowise agent based on Gaia, simply replace the **Chat OpenAI** and **OpenAI Embeddings** component with the Gaia base URL.

*   [Prerequisites](#prerequisites)
    
*   [Start a FlowiseAI server](#start-a-flowiseai-server)
    
*   [Build a documents QnA chatbot](#build-a-documents-qna-chatbot)
    *   [Get the **Flowise Docs QnA** template](#get-the-flowise-docs-qna-template)
        
    *   [Connect the chat model API](#connect-the-chat-model-api)
        
    *   [Connect the embedding model API](#connect-the-embedding-model-api)
        
    *   [Set up your documents](#set-up-your-documents)
        
*   [Give it a try](#give-it-a-try)
    
*   [More examples](#more-examples)

---

# Zed | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

[Zed](https://zed.dev/)
 is a next-generation code editor designed for high-performance collaboration with humans and AI, and it is written in Rust. You can use Zed with your own Gaia node as the LLM backend. There are two big reasons for that

*   Your Gaia node could be supplemented by a knowledge base that is specific to your proprietary code repository, programming language choices, and coding guidelines/styles.
*   Your Gaia node could ensure that your code stays private within your organization.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node to provide LLM services to Zed. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use public [Qwen 2.5 Coder](https://github.com/QwenLM/Qwen2.5-Coder)
 nodes to power Cursor.

| Model type | API base URL | Model name |
| --- | --- | --- |
| General coding assistant | `https://coder.gaia.domains/v1` | coder |
| Coding assistant with Rust knowledge | `https://rustcoder.gaia.domains/v1` | rustcoder |
| Rust expert (slower but more accurate) | `https://rustexpert.gaia.domains/v1` | rustexpert |

> A limitation of Cursor is that it does not support local LLM services. A Gaia node comes with a default networking tunnel that turns your local LLM service into a HTTPS service accessible from the Internet. That allows Cursor to use your own private LLM for coding. Start your own [Qwen Coder](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct)
>  or [Qwen Coder with Rust](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct_rustlang)
>  nodes today!

Configure Zed[​](#configure-zed "Direct link to Configure Zed")

----------------------------------------------------------------

First, download and install [Zed](https://zed.dev/)
. Click on your profile on the top right and choose **Setting**. Then a new tab called `settings.json` will be opened. You can configure your Zed by editing this file.

![](/assets/images/zed-01-cf0cb473f9d2facf71d318043fca0179.png)

Below is the `settings.json` we used. You can copy and paste sections `language_models` and `assistant` to your own. They configure Zed to use an OpenAI-compatible API provider and then specify the API endpoint URL and model name for that provider.

    {  "features": {    "inline_completion_provider": "none"  },  "language_models": {    "openai": {      "version": "1",      "api_url": "https://rustcoder.gaia.domains/v1",      "low_speed_timeout_in_seconds": 60,      "available_models": [        {          "name": "yicoder9b",          "max_tokens": 8096        }      ]    }  },  "assistant": {    "provider": "openai",    "default_model": {      "provider": "openai",      "model": "yicoder9b"    },    "version": "2"  },  "ui_font_size": 16,  "buffer_font_size": 16,  "theme": {    "mode": "system",    "light": "One Light",    "dark": "One Dark"  }}

Next we will configure the API key to access this Gaia node.

Go back to the folder you opened. Click on the Star icon at the bottom to turn on the Assistant panel.

![](/assets/images/zed-02-0120fdb612302c83426a76807ee18115.png)

Click on **Open configuration** to set up the API Key.

![](/assets/images/zed-03-71751fef612aad8f6d0b69312847f495.png)

Since we are using a free public Gaia node, you can use any API key in the OpenAI section at the bottom of the screen. For example, you can enter `GAIA`.

Now, we have everything ready.

Use Zed
=======

You can

*   Edit the highlighted code by selecting the code and clicking on the **Inline Assistant** button.

![](/assets/images/zed-04-693b6322554fe5bb3d0c1972ab22c709.png)

*   Open the Assistant panel by clicking on the **Assistant** icon at the bottom to turn on the Assistant panel.

![](/assets/images/zed-05-6785a1b71e9a38dee8d0d505d9296d2b.png)

*   [Prerequisites](#prerequisites)
    
*   [Configure Zed](#configure-zed)

---

# FlowiseAI tool call | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use a Gaia node that supports [LLM tool calling](https://github.com/LlamaEdge/LlamaEdge/blob/main/api-server/ToolUse.md)
.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. In this tutorial, you will need to [set up a public node with tool call support](https://github.com/GaiaNet-AI/node-configs/blob/main/mistral-0.3-7b-instruct-tool-call/README.md)
.

Start a FlowiseAI server[​](#start-a-flowiseai-server "Direct link to Start a FlowiseAI server")

-------------------------------------------------------------------------------------------------

Follow [the FlowiseAI guide](https://docs.flowiseai.com/getting-started)
 to install Flowise locally

    npm install -g flowisenpx flowise start

After running successfully, you can open `http://localhost:3000` to check out the Flowise AI tool.

Build a chatbot for realtime IP lookup[​](#build-a-chatbot-for-realtime-ip-lookup "Direct link to Build a chatbot for realtime IP lookup")

-------------------------------------------------------------------------------------------------------------------------------------------

Step 1: Create a new **Chatflow** from the UI.

![](/assets/images/flowise-tool-01-54bc5131cf142cebbfc44675f276c962.png)

Step 2: On the **Chatflow** canvas, add a node called **ChatLocalAI**.

![](/assets/images/flowise-tool-02-82e1dcf92b2aceeccff81ac1e7e891de.png)

Step 3: Configure the **ChatLocalAI** widget to use the Gaia node with tool call support you have created.

*   Base path: `https://YOUR-NODE-ID.us.gaianet.network/v1`
*   Model name: e.g., `Mistral-7B-Instruct-v0.3.Q5_K_M`

Step 4: Add a node called **Custom Tool**

Create a function named `get_ip_address_geo_location`. The function requires a `string` parameter called `ip`.

The **Tool description** field is the "prompt" that tells the LLM when to use this function. In this example, if the LLM detects that the user is asking about the city or country of an IP address, it will return a tool call response asking FlowiseAI to perform this function call first.

![](/assets/images/flowise-tool-03-93283784efe5a1f1343443da51c35c88.png)

Now you can add JavaScript code for this function. It looks up the location of the input `ip` parameter.

    const fetch = require("node-fetch")const url = "http://ipwho.is/"+$iptry {  const response = await fetch(url)  const result = await response.text()  console.log(result)  return result} catch(error) {  console.error(error)}

![](/assets/images/flowise-tool-04-9217ad7610930f56bc859d250c1538ec.png)

Step 5: Add a node called **Buffer Memory** to the canvas.

Step 6: Add a node called **Tool Agent**.

Step 7: Connect the nodes.

Connect the **Custom Tool** and **Buffer Memory** nodes to the appropriate connectors on the **Tool Agent** node. Connect the **ChatLocalAI** node to the **Custom Tool**.

![](/assets/images/flowise-tool-05-8cf21165d8466b203f3f521ba9e55b61.png)

Step 8: Save the **Chatflow**.

Give it a try[​](#give-it-a-try "Direct link to Give it a try")

----------------------------------------------------------------

From the FlowiseAI UI, you can open a chat window to chat with the **ChatLocalAI** you just created. Let's ask a question:

    What's the location of this address 35.222.115.181

The LLM understands that the request is to find a location for an IP address, and sees that we have a function called `get_ip_address_geo_location` in tools, which has a description that matches this task. So, it responses with a JSON message to call this function with the IP address it extracts from the user query.

This tool calling JSON message is NOT displayed to the user in the chatbot. Instead, the FlowiseAI **Custom Tool** node captures it and executes the JavaScript code associated with this tool call. The result of the tool call is then sent back to the LLM together with the original query, which is why we need the **Buffer Memory** node BTW, and the LLM formulates a human readable response to the original question.

![](/assets/images/flowise-tool-06-3691b227be09ee5bd6e7c6badf78c2ff.png)

*   [Prerequisites](#prerequisites)
    
*   [Start a FlowiseAI server](#start-a-flowiseai-server)
    
*   [Build a chatbot for realtime IP lookup](#build-a-chatbot-for-realtime-ip-lookup)
    
*   [Give it a try](#give-it-a-try)

---

# LobeChat | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

You can configure [LobeChat](https://lobehub.com/)
 to use a Gaia node as its backend LLM API. It provides a richer and more customizable UI than the default Gaia chatbot UI.

Steps[​](#steps "Direct link to Steps")

----------------------------------------

**Step 1: Set up the Gaia node API base url as the OpenAI provider**

Go to the [Language Model Setting page](https://chat-preview.lobehub.com/settings/modal?agent=&session=inbox&tab=llm)
 and choose OpenAI.

1.  Enter a random string in the OpenAI API Key field. It does not matter what you enter here since we are going to ignore it on the backend.
2.  Enter the Gaia node API base URL in the API Proxy Address field. For example, you can use `https://llama8b.gaia.domains/v1` here.
3.  Enable Use Client-Side Fetching Mode
4.  Click on the Get Model List text and it will automatically fetch LLMs available on the Gaia node. Choose the chat model `llama` here.
5.  Optional: click on the Check button to check the connection status.

![](/assets/images/lobechat-gaianet-01-bf6655ad5eac47bd6eb47de87c3a6759.png)

**Step 2: Start chatting via the LobeChat UI**

Next, let's go back to the chat page. Before starting chatting, choose the model you just chose in the previous step around **Just Chat**.

Now you can chat with the Gaia node via LobeChat.

![](/assets/images/lobechat-gaianet-02-f15f6809e36ea088a2a9c2abd89adc2b.png)

*   [Steps](#steps)

---

# LlamaCloud | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. LlamaParse can support different kinds of files, like pdf, doc, .ppt, and other formats.

You can configure LlamaParse to use the Gaia node as the LLM backend, hence you can create a RAG application based on your PDF files locally.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) | llama |
| Embedding | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) | nomic-embed |

Steps[​](#steps "Direct link to Steps")

----------------------------------------

We will use an open-sourced GitHub repo, called `llamaparse-integration`, to make LlamaPase easy to use. The `llamaparse-integration` application supports

*   Multiple file formats, like `.pdf` and `.doc`,
*   Multiple files

We will need to get the source code in your terminal first.

    git clone https://github.com/alabulei1/llamaparse-integration.gitcd llamaparse-integration

Next, install the required mode packages.

    npm install llamaindexnpm install dotenv

Start a Qdant instance. The Qdrant instance is to store the embeddings.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Then, we will need to set up the LLM model settings. We can configure the model setting in the `.env` file.

    OPENAI_BASE_URL=https://llama8b.gaia.domains/v1/OPENAI_API_KEY=gaianetLLAMAEDGE_CHAT_MODEL=llamaLLAMAEDGE_EMBEDDING_MODEL=nomicLLAMA_CLOUD_API_KEY=Your_Own_KEYFILE_PATH=FILE_DIR=./pdf_dirCOLLECTION_NAME=defaultQDRANT_URL=http://127.0.0.1:6333SAVE_MARKDOWN_PATH=output.md

Here are some notes about the `.env` setting:

*   You can get the LlamaCloud key from [https://cloud.llamaindex.ai](https://cloud.llamaindex.ai)
    
*   You may need to make changes according to your model setting and file path.
*   If you put your file name in the `FILE_PATH=`, the program will build a RAG application with this single pdf file.
*   If the `FILE_PATH=` is empty, the program will build a RAG application with the files under the `FILE_DIR=./pdf_dir`. You can include multiple files in the folder.

Next, we can run the program to build an RAG application based on the PDF file

    npx tsx pdfRender.ts

After it runs successfully, you can send a query via the command line.

![](/assets/images/llamaparse-01-07933ece762716a4ea93c06f31abb0d5.png)

*   [Prerequisites](#prerequisites)
    
*   [Steps](#steps)

---

# Obsidian | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

Obsidian is a note-taking application that enables users to create, link, and visualize ideas directly on their devices. With Obsidian, you can seamlessly sync notes across devices, publish your work, and collaborate with others. The app is highly customizable, allowing users to enhance functionality through a wide range of plugins and themes. Its unique features include a graph view to visualize connections between notes, making it ideal for managing complex information and fostering creativity. Obsidian also emphasizes data privacy by storing notes locally.

**Obsidian-local-gpt is a plugin that** allows users to run a local large language model within Obsidian note-taking application. This plugin enables various AI-powered features directly in Obsidian, such as text generation, summarization, spelling and grammar checks, and task extraction.

A key feature of this plugin is that it supports a large number of open source LLMs. You can choose an LLM that is finetuned for your specific task — eg if you take a lot of coding notes, you could choose a Codestral or CodeLlama or DeepSeek LLM. Furthermore, if you choose to run the LLM locally on your own computer, the plugin would support private and offline use of the LLM features. For more details, you can visit the [obsidian-local-gpt GitHub page](https://github.com/pfrankov/obsidian-local-gpt)
.

This guide explains how to set up and use the plugin with a Gaia node as an alternative to OpenAI or Ollama.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [Run your own node](/getting-started/quick-start)
    
*   [Use a public node](/nodes)
    

In this tutorial, we will use a public node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) |
| Model Name | llama |

Obsidian-local-gpt Plugin Setup[​](#obsidian-local-gpt-plugin-setup "Direct link to Obsidian-local-gpt Plugin Setup")

----------------------------------------------------------------------------------------------------------------------

Make sure you have already installed the Obsidian app on your device.

### Install the Obsidian-local-gpt Plugin[​](#install-the-obsidian-local-gpt-plugin "Direct link to Install the Obsidian-local-gpt Plugin")

*   Open Obsidian settings, navigate to "Community plugins", and search for `obsidian-local-gpt`.
*   Install the plugin by clicking “Install”.

![](/assets/images/obsidian-enable-f3f05cad21f516b0d9002e773e5116bb.png)

Then click “Enable”.

### **Configure the Plugin**[​](#configure-the-plugin "Direct link to configure-the-plugin")

1.  Go to the plugin settings.
2.  Select "AI Provider" as "OpenAI compatible server".
3.  Set the server URL. Use [https://llama8b.gaia.domains/](https://llama8b.gaia.domains/)
     if you are using a public Gaia node. Or, use [http://localhost:8080/](http://localhost:8080/)
     if you are running a local Gaia node.
4.  Configure API key to Gaia.

![](/assets/images/obsidian-configure-9692f400bc4a5ae8288cbd471af0bc78.png)

Make sure to click the refresh button and choose the **llama** model if you’re using the public Gaia node url and **Phi-3-mini-4k-instruct** if you’re using the local Gaia node.

![](/assets/images/obsidian-model-226eda81908ccfefab621a99f057ee8a.png)

### Configure Obsidian Hotkey[​](#configure-obsidian-hotkey "Direct link to Configure Obsidian Hotkey")

1.  Open Obsidian Settings.
2.  Go to Hotkeys.
3.  Filter "Local" and you should see "Local GPT: Show context menu".
4.  Click on `+` icon and press hotkey (e.g. `⌘ + M`).

![](/assets/images/obsidian-hotkey-0bf43c29ebdc7ed95a10b1a7bba6bb58.png)

As long as you have set the hotkey, while writing or editing a note, select the text you want to interact with, and press the hotkey you have set to use this LLM powered plugin!

Use Cases[​](#use-cases "Direct link to Use Cases")

----------------------------------------------------

### **Text Continuation**[​](#text-continuation "Direct link to text-continuation")

*   Select a text segment, right-click, and choose "Continue writing". The model will generate the continuation (displayed in the screenshot in grey).

![](/assets/images/obsidian-text-continuation-0147058b4cc7761f3061dfe8b4ba96c2.png)

### **Summarization**[​](#summarization "Direct link to summarization")

*   Select a longer text segment, right-click, and choose "Summarize text". The model provides a summary as below of the CNAI report content I selected.

![](/assets/images/obsidian-summarization-9e61cc6f77a1dfa00ca064cb1b61421a.png)

Here is a concise summary of the key points:

*   Cloud Native (CN) refers to well-balanced systems built using microservices, promoting modular design and reusability.
*   Kubernetes has become the de facto cloud operating system, offering scalability, resilience, and DevOps best practices.
*   Every Cloud Service Provider offers Kubernetes as a service, facilitating access to infrastructure and support services for various workloads, including AI/ML.
*   The Cloud Native Computing Foundation defines Cloud Native as empowering organizations to build scalable applications in modern environments using containers, microservices, and declarative APIs.
*   Cloud Native Artificial Intelligence (CNAI) is an evolving extension of Cloud Native, focusing on building and deploying AI applications and workloads using Cloud Native principles.

### **Spelling and Grammar Check**[​](#spelling-and-grammar-check "Direct link to spelling-and-grammar-check")

*   Select text, right-click, and choose "Fix spelling and grammar". The model will correct errors.

![](/assets/images/obsidian-grammar-d32655e11368ec52c30f1a1ed52a58b7.png)

### **Extract Action Items**[​](#extract-action-items "Direct link to extract-action-items")

*   For notes with multiple tasks, select the text and click "Find action items". The model will list the tasks.

![](/assets/images/obsidian-extract-ea27bc2ab9b125a59440f4567894bbef.png) The Generated content are displayed below your own text:

`Here are the action items extracted from the document in Markdown checkbox format:`

*   `[ ] Add feedback on WasmEdge Q3 Roadmap Discussion`
*   `[ ] Provide input on Update on WasmEdge Community Governance`
*   `[ ] Discuss any additional topics at the meeting (add to [https://docs.google.com/document/d/1iFlVl7R97Lze4RDykzElJGDjjWYDlkI8Rhf8g4dQ5Rk/edit#](https://docs.google.com/document/d/1iFlVl7R97Lze4RDykzElJGDjjWYDlkI8Rhf8g4dQ5Rk/edit))`

`Let me know if you'd like me to help with anything else!`

### **General Assistance**[​](#general-assistance "Direct link to general-assistance")

*   Select any text and click "General help" to get contextually relevant information from the model.

I entered and selected some information on KubeCon + CloudNativeCon + Open Source Summit + AI\_dev China 2024. Because llama3 has not been trained with info on this conference, so the output is not very helpful:

`The information you're looking for is not present in this context.`

`If you need to know the format and dates of KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024, I suggest searching for official announcements or websites related to these events.`

Try it now![​](#try-it-now "Direct link to Try it now!")

---------------------------------------------------------

Ready to supercharge your note-taking with AI? Get started with the Obsidian-local-gpt plugin and Gaia today:

1.  Set up the Obsidian-local-gpt plugin in your Obsidian app.
2.  Explore the various AI-powered features to enhance your productivity.

Start your journey towards smarter, more efficient note-taking now!

*   [Prerequisites](#prerequisites)
    
*   [Obsidian-local-gpt Plugin Setup](#obsidian-local-gpt-plugin-setup)
    *   [Install the Obsidian-local-gpt Plugin](#install-the-obsidian-local-gpt-plugin)
        
    *   [**Configure the Plugin**](#configure-the-plugin)
        
    *   [Configure Obsidian Hotkey](#configure-obsidian-hotkey)
        
*   [Use Cases](#use-cases)
    *   [**Text Continuation**](#text-continuation)
        
    *   [**Summarization**](#summarization)
        
    *   [**Spelling and Grammar Check**](#spelling-and-grammar-check)
        
    *   [**Extract Action Items**](#extract-action-items)
        
    *   [**General Assistance**](#general-assistance)
        
*   [Try it now!](#try-it-now)

---

# AI coding assistant: Continue | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

[Continue](https://github.com/continuedev/continue)
 is the leading open-source AI code assistant.

It is a copilot-like plugin for VSCode and JetBrains to provide custom autocomplete and chat experiences inside those IDEs. You can easily configure it to use Gaia nodes as LLM backends. In fact, you can choose different Gaia nodes for

*   The autocomplete model for coding tasks.
*   The chat model for understanding and discussing code.
*   The embedding model to provide chat context based on local files.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [Run your own node](/getting-started/quick-start)
    
*   [Use a public node](/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama8b.gaia.domains/v1/](https://llama8b.gaia.domains/v1/) | llama |
| Embedding | [https://llama8b.gaia.domains/v1/](https://llama8b.gaia.domains/v1/) | nomic |
| Autocompletion | [https://codestral.us.gaianet.network/v1/](https://codestral.us.gaianet.network/v1/) | codestral |

> It is important to note that Continue requires the API endpoint to include a `/` at the end.

Install Continue[​](#install-continue "Direct link to Install Continue")

-------------------------------------------------------------------------

[Load this link](https://marketplace.visualstudio.com/items?itemName=Continue.continue)
 to install the Continue IDE plugin.  
It will open up your VSCode when you click on the **Install** button on the web page. When you are asked to configure Continue, just click on **Skip** and finish the installation without selecting a local model.

Configure Continue[​](#configure-continue "Direct link to Configure Continue")

-------------------------------------------------------------------------------

Click on the gear icon on the toolbar to load the `config.json` file for the Continue plugin. The file is located in your own home directory `$HOME/.continue/config.json`. You can now change the `config.json` file as follows. It asks the Continue plugin to use different public Gaia nodes and models for chat, code autocomplete and embeddings.

    {  "models": [    {      "model": "llama",      "title": "LlamaEdge",      "apiBase": "https://llama8b.gaia.domains/v1/",      "provider": "openai"    }  ],  "tabAutocompleteModel": {      "title": "Autocomplete",      "apiBase": "https://codestral.us.gaianet.network/v1/",      "model": "codestral",      "provider": "openai"  },  "embeddingsProvider": {    "provider": "openai",    "model": "nomic-embed",    "apiBase": "https://llama8b.gaia.domains/v1/"  },  "customCommands": [    {      "name": "test",      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",      "description": "Write unit tests for highlighted code"    }  ],  "allowAnonymousTelemetry": true}

Save the `config.json` file and you are done!

Use the plugin[​](#use-the-plugin "Direct link to Use the plugin")

-------------------------------------------------------------------

The following screenshot shows how you can chat with an error message inside the IDE.

![](/assets/images/continue-01-e2a5bd512068829ed1debacc682accc7.png)

*   [Prerequisites](#prerequisites)
    
*   [Install Continue](#install-continue)
    
*   [Configure Continue](#configure-continue)
    
*   [Use the plugin](#use-the-plugin)

---

# CodeGPT | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

CodeGPT is a pair-programming partner for developers. It offers AI chat assistance, auto-completion, code explanation, error-checking, and much more. You can find the CodeGPT extension in VScode and Jetbrains. You can easily configure it to use Gaia nodes as LLM backends.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use the public CodeStral nodes to power the CodeGPT plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://codestral.us.gaianet.network/v1/v1/](https://codestral.us.gaianet.network/v1/v1/) | codestral |

> For some reason, CodeGPT requires the API endpoint to include an extra `v1/` at the end.

Install CodeGPT[​](#install-codegpt "Direct link to Install CodeGPT")

----------------------------------------------------------------------

Download [the CodeGPT for VScode](https://marketplace.visualstudio.com/items?itemName=DanielSanMedium.dscodegpt&ssr=false#overview)
 and [Jetbrains](https://plugins.jetbrains.com/plugin/21056-codegpt)
.

Once you install it successfully, you can find the plugin on the right sidebar. You don't need to sign up for an account here.

![](/assets/images/codegpt-01-1f1e441d81450929755cc238163b43ec.png)

Configure CodeGPT[​](#configure-codegpt "Direct link to Configure CodeGPT")

----------------------------------------------------------------------------

Click the CODEGPT on the right sidebar and enter the settings page for CodeGPT.

1.  Go through the **Select Your AI** and choose the **Custom** PROVIDER. Copy and paste `codestral` into the MODEL field.

![](/assets/images/codegpt-02-6a086ea00d37fddaf000948e4bd1c229.png)

3.  Click the Connect button to configure the model base url and API key. Again, note the extra `v1\` at the end of the URL.

| Attribute | Value |
| --- | --- |
| API endponit URL | [https://codestral.us.gaianet.network/v1/v1/](https://codestral.us.gaianet.network/v1/v1/) |
| API Key | gaia |

![](/assets/images/codegpt-03-5c71bbb294022836ff2fd41aa89e73c1.png)

Save the above settings.

Use the plugin[​](#use-the-plugin "Direct link to Use the plugin")

-------------------------------------------------------------------

You can summon the coding assistant using slash commands defined in the plugin. You can ask the coding assistant to fix bugs, explain codes, write documentation, refactor the docs, and create unit test cases for the specific code.

![](/assets/images/codegpt-04-333560c9eba11ed3702d5d52d1271458.png)

Video Guide[​](#video-guide "Direct link to Video Guide")

----------------------------------------------------------

*   [Prerequisites](#prerequisites)
    
*   [Install CodeGPT](#install-codegpt)
    
*   [Configure CodeGPT](#configure-codegpt)
    
*   [Use the plugin](#use-the-plugin)
    
*   [Video Guide](#video-guide)

---

# LlamaCoder | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

LlamaCoder is an open-source tool designed to generate small apps with a single prompt. It leverages LLM to help you quickly create and enhance React applications.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use a public Llama3 node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, we will need to get the source code of the forked LlamaTutor

    git clone https://github.com/second-state/llamacoder.gitcd llamacoder

Next, configure the `.env` file.

    cp .example.env .env

You will need to configure three parameters here.

*   LLAMAEDGE\_BASE\_URL: URL for the LLM API base URL.
*   LLAMAEDGE\_MODEL\_NAME: Name of the model to be used.
*   LLAMAEDGE\_API\_KEY: API key for accessing the LLM services.

For example, you can use the following `.env` setting.

    LLAMAEDGE_BASE_URL=https://llama8b.gaia.domains/v1LLAMAEDGE_MODEL_NAME=llamaLLAMAEDGE_API_KEY=GaiaNet

Then, we will need to install the required dependencies.

    npm install

Next, let's run the application locally.

    npm run dev

Finally, open [http://localhost:3000](http://localhost:3000)
 in your browser and start to build your React app.

![](/assets/images/llamacoder-ca7bef8ba04ce6f5ad1ed95898144cde.png)

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# Translation Agent + Gaia | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

The Translation Agent originally built by Prof. Andrew Ng, designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use any Gaia node as the LLM backend.

> For commands on starting and running this agent, refer to [GitHub - Second State/translation-agent](https://github.com/second-state/translation-agent/blob/use_llamaedge/step-by-step-use-LocalAI.md)
> .

You can run the Translation Agent on top of a public Gaia Node as a backend and then translate the content in your target language. If you’d like to learn more about the Translation Agent and how open source LLMs perform, check out the article [Agentic translation on Gaia](https://docs.gaianet.ai/tutorial/translator-agent)
.

Prepare the environment[​](#prepare-the-environment "Direct link to Prepare the environment")

----------------------------------------------------------------------------------------------

Here, we will use the public Gaia node with gemma-2-27b model. `https://gemma.us.gaianet.network/`.

> As an alternative, you can also start a Gaia node locally on your device. Refer to [this guide](https://github.com/GaiaNet-AI/node-configs/tree/main/gemma-2-27b-it)
> .

To get started, clone the Translation Agent that supports open source LLMs.

    git clone https://github.com/second-state/translation-agent.git    cd translation-agentgit checkout use_llamaedge

Set environment variables and install necessary Python packages if needed. Replace the OPENAI\_BASE\_URL with `https://gemma.us.gaianet.network/`

    export OPENAI_BASE_URL="https://gemma.us.gaianet.network/v1"export PYTHONPATH=${PWD}/srcexport OPENAI_API_KEY="GAIANET"pip install python-dotenvpip install openai tiktoken icecream langchain_text_splitters

Prepare your translation task[​](#prepare-your-translation-task "Direct link to Prepare your translation task")

----------------------------------------------------------------------------------------------------------------

Find the `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here because we named our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text-Intro-of-Forbidden-City)
 in Chinese `forbiddencity.txt` since it is an introduction on this Chinese royal palace, then note down its document path, `sample-texts/forbiddencity.txt`. This will be the `relative path` in our `example_script.py` file.

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the relative path to the above. Change the model name to the one you are using; here, we're using the `gemma` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import os  import translation_agent as ta      if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/forbiddencity.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="gemma",    )        print(f"Translation:\n\n{translation}")

Translate[​](#translate "Direct link to Translate")

----------------------------------------------------

Run the python translation script as follows.

    cd examples    python example_script.py

Wait a few minutes and [the English translation](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-gemma-2-27b)
 will appear on your terminal screen.

*   [Prepare the environment](#prepare-the-environment)
    
*   [Prepare your translation task](#prepare-your-translation-task)
    
*   [Translate](#translate)

---

# Agent Zero | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

[Agent Zero](https://github.com/frdel/agent-zero)
 is a general purpose AI agent application. You can simply ask it to accomplish tasks on the command line. It is designed to be dynamic, organically growing, and learning as users use it. It leverages your computer as a tool to accomplish your tasks.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node to provide LLM services to the agent app. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use the public [Llama-3.1-8B node](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct)
 to power the Agent Zero.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama8b.gaia.domains/v1/](https://llama8b.gaia.domains/v1/) | llama |
| Embedding | [https://llama8b.gaia.domains/v1/](https://llama8b.gaia.domains/v1/) | nomic-embed |

**You will also need to make sure your Docker engine is running.** Because the Agent Zero framework will leverage Docker to execute the generated code.

> You can start a local LLM service using [Gaia](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct)
>  or [LlamaEdge](https://llamaedge.com/docs/user-guide/quick-start-command)
>  or [Moxin](https://github.com/moxin-org/moxin)
> , and then use `http://localhost:8080/v1/` as the LLM API service endpoint URL.

Configure the agent[​](#configure-the-agent "Direct link to Configure the agent")

----------------------------------------------------------------------------------

First, we will need to get the source code of a Gaia-compatible version of Agent Zero.

    git clone https://github.com/JYC0413/agent-zero-gaia.gitcd agent-zero-gaia

Then, let's install the required dependencies.

    pip install -r requirements.txt

Next, let's configure the gaia node and other parameters.

    cp example.env .env

You will need to configure the following items:

*   `CHAT_MODEL_BASE_URL`: URL for the LLM API base URL. E.g., `https://llama8b.gaia.domains/v1/`
*   `CHAT_MODEL_NAME`: Name of the chat model to be used. E.g., `llama`
*   `CHAT_API_KEY`: An API key to access the LLM services. You can enter several random characters here. E.g., `GAIA`
*   `EMBEDDING_MODEL_BASE_URL`: URL for the embedding model API base URL. E.g., `https://llama8b.gaia.domains/v1/`
*   `EMBEDDING_MODEL_NAME`: Name of the embedding model name. E.g., `nomic-embed`
*   `EMBEDDING_API_KEY`: An API key to access the embedding services. You can enter several random characters here. E.g., `GAIA`

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

Finally, let's run the Agent Zero application backed by the [Llama 3.1 8b](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct)
 Gaia node.

    python main.py

You can interact with Agent Zero via the command line. You will see the Agent Zero framework will call the related tools and execute some code to complete your task.

### Example 1[​](#example-1 "Direct link to Example 1")

Get the time in berlin and seattle

![](/assets/images/agent-zero-01-5977b5b913065e4d018f6154a23ee527.png)

![](/assets/images/agent-zero-02-7043b5f92118e22f7fc418c131c35965.png)

Save the above result in a file using node.js

![](/assets/images/agent-zero-03-b5ebd8903dbb3ebdc2a9b01b9b8d522e.png)

### Example 2[​](#example-2 "Direct link to Example 2")

Install [mc](https://midnight-commander.org/)
. do not use `-y` flag

![](/assets/images/agent-zero-04-169d90a36a516babfd2025f500ce5cc9.png)

### Example 3[​](#example-3 "Direct link to Example 3")

Run `top`, show the system load

![](/assets/images/agent-zero-05-8f884d2fdad5e381880fb9fde3c1c14b.png)

### Example 4[​](#example-4 "Direct link to Example 4")

Memorize my openai api key - 563434643464

![](/assets/images/agent-zero-06-865e4e1235f0eceec9395c8d85a01db9.png)

### Example 5[​](#example-5 "Direct link to Example 5")

Extract an mp3 audio track from a mp4 video file using ffmpeg. You will need to put an mp4 file (e.g., `video.mp4`) into the `work_dir` and tell the agent about it.

![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAPQAAAEUCAIAAACTfoyfAAAgAElEQVR4Ae2dzW8cx5XADUF3LZwbBcQ50LlFQLzQcndEkRyJMulhIlnikEwsRcowgiyK4oiWIpFWbFIRydBKolgKRx+UJUE2zLEcS/ywvTK2B1hsgL0YC3ixcW4+ZP+FLLLnXdRHv35d/THdMz3TUzVPIISq6qpXVa9+/fiq2a/ruT0D+ZCf5dLDPQP5nmuPQ372DORFNUVO5ui1tY+uDR8YhoSokDnw+txHqxMHhvcM5DPnVit+/yzr47mjrAL9NEEDqw/ev3FrNbyjG7dWVx+8H15HuTp45Pjqg/c3tj6Dn9UH7w8eOa5Ua1z2uXDRycI9vPDxyrlhBW7r9owyBl6hFriHRgoffvQJqBISH370ydBIQemFsqCBj5+sg642tj6D8j0DeVz+8ZN1fKlpaTyGoLTvYIyCe89A/six15Wl+vjJ+pFjr/tOngqFBhSNYbVgmAhux5EAbwQSvpbbzyup1OOWvHZy6snGlliVJxtbr52cwqtFaa8GyC0JdLvDfG7LqlQqa7dXhfM9cZs5G41zS2DZTp69uL712frWZyfPXoRCSgRp4NLbi083tobGxqGCMA2QHRobf7qxdentRSiJmOj7wY+u37wjpF2/eafvBz+K2DCkWi5/Ipc/EVIBLjXKLZm4zbBeOedsCsXeUTHJmXOrCfrcMKs9A/lzl+bOXZrDJZQO0kD/q8cef7JeuvugJzcm6mC4e3JjpbsPHn+y3v/qsSAJIeXdgyNvL/z67YVfdw+OhFSLeCmXP/How8ePPnwche9Gwa2MdeK2ZVnsCUnmwMyKZQHQ9LREUVRa2cKZC+ubn5buPhD2G+AeGhsv3X2wvvlp4cyFtMYG/Qqyxdii8B0JbpAelPB9FCgqM4fbstYWXFu64YWPBd+Ns9xBQ6XyIA0Uzlx4/Mn6042tm7fuCYBu3rr3dGPr8SfrcckWzcP/DxpGUDkmOyLf1eFeLj2M8hM0JirXSAP9h45eentx9cH7f3iy8YcnG6sP3r/09mL/oaNxpxCOtbgaV2YN9avAXYNEakIaaBENENzOs8sWWRIaRlIaILgJbmM1QHAbu7RJ2T995RDcBLexGiC4jV1afS1uUiN/7iX6RxowVAPPdfdm6Yc0YKQGCG66t43VAMFt7NIaaYxjTYrgJriN1UBqcC+v3IOf+WvvvvbTk7FuSqpMGqiqgZaAW1D+09Nnqw63/gqZnrG5srUy2RddVGayZJWvDPf0ZUaurFnluZEYbaP3QjUT10ACcOd+eOjlV3LKyF5+JZf74SGlEGfBbFdN4Fb1pwnu+nWoi4QE4L5+4/fvPXyE+X75ldx7Dx9dv/H7EC1UZRoqhAip4VI9cNfQHTVJUQMJwC1QBr6VbNDcgN2qiSAJtZUT3LXpTcdWCcDd3ZsFoF8dHnnv4SMAPUQjVZmGCljIRMlamx/r7s1mes6uWNJ1dnxiXijC6S2rNNHDnGObZlbfKl85sncUfG7uQ1tWycfXF/IhMh/53EysInOYd4THSelW0EAycAPf65ufRiG7uzcL7FZNYDUNz5cFi5nJ0lq5LEAXxAscRQljerIk+BYgeljv4xX8d4f4zunuzU6U2F1hbygduEEmHiGlW0cDicEt+L72m+vY+Q6ZZ1WmoQIWwm0tw2uiVJ4bObvCmBubKzNGwX6L+rZxBSsrH3HI8pJEHwuHtCrK9bTEgTvWIxcQTommaSBJuGMNGtitmsBiOZpurCevrAmzaiMo6ts1/eG2yuWQh3oEN9a5vmnN4O7uzQ7Pl9fK0jkRaeyFu9wSAb37wbZj0dlDa/8H3tgtEfV93RKy3C3OvX5wi12gAEv5qwreBYJDDDQji27vRPkGFGripcpMluyNaXluHv8Rh9wSbf5cnxrcmCRKkwYaoQGCWxs71IjlN1smwS0fmcMjbfaJwzhvnpjNh9azI7jJchurAYLb2KXV2ugmMniCm+A2VgMEt7FLm4jx01oIwU1wG6sBgtvYpdXa6CYyeM3gxi+iUORlIgQYLCRNuHM/PPS7GzcjvkUo1gDDLdLNirzcNbW4ozBE0ZM6/aJLE24IcYjOtxfuoJJkDVKmh+DWCWux+mnCDSEOEeMbag5xqB90grt+HTZfQspwx+U7yE57y5NVJcGdrD6bIy19uLt7s68Oj6xvfnrtN9erztkLcVBJkCiO6bbFpe2LS9tnxrtYQNpQ58yiLJkahYCdXVN24cx4l4R7vEM0XCjuEvK90oL6pfLmayB9uGN53kEoe8t9VSlYFEyLCpmerkKxc1CEEo92LCx25Fn8LyPbXY2VCKb5zcA2l5J4vstkcma3wb3h2zsVNlkDKcMdi+z6fe7MaMfCrEQZK3pwfIc0yQJuTzUPxxzuUWnIRVv4VYAlUzpFDaQJd1yyGwG38EmQf8Itd2S4fW+VFJeTusYaSBPuRJ5zex0SUYIn6XYkpPOQ6dlVGO/CtnxwfAd2S4SPIauhR4HcAwG3xHFF8kV2Y+B+KZ2uBtKEu4aZB6HsLcfCAW68fbQ5Zr6y3F8WO2a4W+JXzXnODXDjaotL28nhxjpvhbRmcLeCymgMumiA4NbvD2+6sJX6OAlugttYDRDcxi5t6oYz9QEQ3AS3sRoguI1d2tQNZ+oDILgJbmM1QHAbu7SpG87UB0BwE9zGakAzuPFfIimGMnXT2OIDSBPuRN4taU4MZYuvIg3PVwNpwt38twJ9VUCFpmogTbjjxpjV/8qrqatI8/LVQMpwx+Ub+9zhad/ZUmFbaSB9uJsZQykDauxQSBmjICPEnKBJeJObfatkVEZYTo32ZezQGxyB1la46DXZ9OGO5XmHW2t81XcZRHCkJxSSvdItXu9mr2i7IylFrI3AGjWkuAQNHiCmDHcssuv3uf1DId2xvRDq666MgxWctO8tRIUtooE04Y5LNsHdItDoMow04U7kOTd2RXAaLwCEmbmNcVdhVoRCcrfE/hSJO5KSVWC+iiuGkiy3Bj5Jd282TbgxfxHTGN/wNBYYBe6pov3BHQijDAAag457oXSraUAzuBuhPnCyGyGcZKaoAYI7S3CnyF9Duya4CW49HOgabgOC29ilrYEGw5oQ3AS3sRp47tBQln5IA0ZqgOCme9tYDRDcxi6tkcY41qQIboLbWA1oBvfW0V3ws/bj7188/E+xbmWq3FYaSBPuoz86tLJycyyfi65xIBsSc0f+MXrzGmoePN6xcLlzLNdXQ9uam4R3ejC36/xiR6GOIR3MDU/OnDqW6zt4+Nj0zLmTh/sO5nInp2cmf9zUadasn4gN04R7LJ97+OjRw0ePovMNTFdNRJx/1WrhnFVtXluF8E4J7ohaTRPuQ0PZuHxXZRoqRJx/1WrhnFVtXluF8E4ThBuGp4vljjXOlOGOyzewWzUBy1ZnIpyzOoUHNQ/vlOCO6D6lD/ehoeyJn7BzKN9993rQYkN5VaahAjQ5NJTlNOw4c1q+1Hr5dJe4ysvlmSELyIvF5YtL28HnxuUgxKej452X+RmW54/3HTxeV6cH852X7YFBmg9D+ty+Qxo77RzO5uuau31u7nxLn3t4cmZmlv8TANmWkpXPTB8bkW56YJ1L5376+vTM9LjcRx388SnRCrTkI5BtAKTAGb4TsFdHLefDtmvywYBY30T6cMfyTIDdqgk8W0HAwvQuBjrDZceZfJ8oBEaZseQYifLzx+XWqjC9TcDNy1lDfrd0nbm8DepAX7IjvgEVWKNOGY6xOw2F23dIcA/AqLyJILiBLQal2HFy6J3yXO7kBEOcKSGoDgL62IS6SRVwI4GMYNfNIPsNKo+x8U0Z7lhkHxrKVmUaKuAV9UDA4XY/BjmYk7wqLgFkwQbDqZNwY0Bf7o52ned3kfOrI88MOfwewDeJWm5Xw6RCmvfCbxX71wIeEr+6bcHuGsaGE0Fww6972776P0UZGT8nrDu+AVDb4Un5BEY+k3F37aJTMe1Ov+gO4YqSraAClhmUThPuuGQ3Hm4/6IEzOxGkSgxxUFqFmN1RoZ2GW+7gIYl7NQjxmuHmjw6loeVp7NI4jxFHxs9Nj+dGxs/NTAwr6lLo9IObP5r0gTv2I8s04U7kOTeYaiWBdeo2qDZP7GnxNrC+gJ2wfMLlkIgIN4PXB1ekMO3zpNndkZ/ljtup7USxnfdpdkym7dtAwvGOxJAO5jvPcJ8KfhdhVYh0ENzAIuMSuSWOVUbMBdVhN/bhY9PTpyanGY5K7yrc3OF2uSXCsw8sdxl+RbiSTRNuZShRsgrBIVkszc2chNs2rn4bSvvXPbN8px1HgjsGsj5QHtyRD9w1dAq7w8vTHWJzyacjby3fIRWm7UnxbQYeoUgHwT05cQr7G3y0LpgEmqLO9MSpab8bQHRxbEJuQJXeFbh5F+rGEQ3S3j7yjkS5cIqUfarSi8hqBrfvHKiwBTVwbMLZJqY1PILb2NeG0kKKGWP2V33mjqc4hkNDWYK7driFdw5PKhaXtvv6KukucPN7PzbBfAlw05s/AOiR4K4dblAiJVpTAwQ3wW2sBihA2NjwWMNC2WuYDsFNcBurAYLb2KWtwdQZ1kRvuAsTU97vGhu2QjSdmjWgMdyFianllXsEd81rb3xDXeEWZMeFu85vXkb5eLFvHfiGsvE8tdQEtYQbyCa4WwqmVhuMlnArSlS+Qq9cxdm0LDceg5Kuc0iKNMpiDWgJt0KzksXTU9J1kuTrcni6iHeoSJ1DUnqnLNaAxnDv29ev/AjK8fSUtE2SeuQkO/VmSJ43ubi0XZxD2d2bHRx3ghHzPX0SbvsYS3Fyn6cLDre7DhcuT/fDMof3fm+KR1uySM3ZzkHZhf3CqnOACTu1Z2qUDfvqzLdPzW6DgzDZwYK8oTIMyup3Jo5YM8VUK9mQdRVw+x052VUoMrZc51Ay3F3nTXK4t6HTKOVxULhH3zoANySgiX2/wbFSbnD5AHyGbQOdL26DWxFkUkJooD0tNyOJn1EmLKLMYptq22kWjCgONPOcaSYPQ1NIwq4Lh5IfmGbfJxJ9l0xnDIoZBu4hYQ9bej5cmuv2UwbT5lmN4VYMNmRDVtRDiQRL+CTytGwbRJskVkcg7guu0p1vHcVgi2HYMsPh5veG+xhY4S/NjHexUwXtEwaVYVBWb7cEO9xAtng4qCwtZ8uhBIBwjpxEnqtTONRZGBXegn0PuA7vq8ly+8hEcDP5brdEOuJOHTE1NqPZjil+jqYyWcqCBtrCcitw+x05yegRYQczxQ5wtfNFe2/HDaSvVQZVioRvHWy5FZmwbY2woZTuk+goX9xGW0lF+Uq2LeBW5mxGNl90bLwZM0p8FgS3lm8F4l8FiTNhjEAt4W4d7XMnRLouwqtpwoM54dg0oaPW0XNtIyG4tbTctS12u7UiuAluYzVAcBu7tO1mp73zJbgJbmM1QHAbu7ReS9ZuJXrDTTGU7cZrrPlqDDfFUMZa6TasrCvcEGkWK0BYeXEq7nrjP63HbUv1m68BLeEGssVrUvitKd8Xp0CtBDeooh0SWsKtLAzBrSiEskIDWsKt0KxkQ5aWLHeIcsy7pDHc+H1ukRaUhyySDXeSMZRYpve1VQh2DBkVXWqQBjSGWzHYkA3RlAAx2RhKVaY74CBkMHSp0RrQGO46LHeSMZS25bZloqCeRi8eyQ/XgMZwg6lWEiETVkG0YxP569Hy3X/lVWnbMIsoNeebJLzcCV2DF1CVIN+QwdClRmtAY7ix5cZ8e1XGeXVATDaG0nPDMIdesJ7p2VUY7/KOh0qaowGN4cZA47RXcQrcycZQKnDj7/uAc+8dEpU0QQNtAXcT9EhdtKAGCG56K9BYDWgJdwsaCRpSC2qA4DbWbrUgbU0eEsFNcBurAYLb2KVtsplswe4IboLbWA0Q3MYubQua0iYPSW+4KYayybjo1Z3GcFMMpV6oNX+0usINkWaxYiiD9JsZubJmledGXN8IzvSMzZWtlUlXYZAEKm9BDWgJN5AdN4YyaAEI7iDNaF2uJdyKxvFbU+EBwkpDbxZba5z21qSS1teAlnArNCvZepSOgcbpemRS27Q0oDHc+H3uiDGU/ESysyuWValULKs8N1+yyleGe/q4W1I6vXdSXGJXy1eO7B0Fn5tXsKzSWSyhUqmszY+Jlcv0SLFQaN8bTjm5702mXGO4FYMN2RANCgQFZAI+DPcEO+PU2URCOjNZYncC325yCZCWlQMK2VXLKk2I4y2ZEJkOGSFdSlADGsNdg+VmmHJTLTQIWWG5/eEuuaDMTJYq7n9r82P+heg+4fbeuW0SXD8SFaIBjeEGU60kQmYLNIs6kA2B2yqX8VNCaIJ78S8kuPlJtlhRTU5rDDe23JhvrwY5u8yXqNEtYU/B5QNvLKG7NztRYp5GQCF3S7ib3t2bHZ4vk1viXZqGlmgMNwYap736AriZe8BJ9d1QCud4eL7ss6Hk+0VBJ0ioVCqwR/QWSpe9JN0YItu7Lo0uaQu4g5To604EVY5bDvvRuA2pflIaaC+4OXD24wtujOFZXlIKBTkEN6girYSWcNejLOw/NI5sejxSzxol1bbt4E5KcSSn9TVAcFOwgrEaILiNXdrWt6yNHiHBTXAbqwGC29ilbbRdbH35esNNMZStT1iKI9QYboqhTJEbLbrWFW6INEskhjL6UokPFi8ubT97vHNmEb5I35Hn77VGl0M1m6ABLeEGspOKoYyu6MHxHeLb9eib37umFgnuVty6aAm3wiJ+a6rOGEpFsjebL8rTReASP1aY4Ca4E3rBV6FZyQJ2jUgQ3I3QaoNkamm5Bc34fe6IMZT2ER/qOZS43HuWJJz+kS9uW1zavri0fWGxYzjXOcO9EWy5eVrWoRMoG4RsdLEaw60YbMiGTF5ADLCyk8ckoF2F2W1OufssSajW3ZsFyw2HngHcPMG2mPytKSYQjjgLGRJdapwGNIa7DsttnxkJR/XZCaFo5bg9266zVmFwj3YIuw7/k/FuHLhRJGsMN5hqJREybYwptq9qufugVH5VmuRwuIVLEzIAutRMDWgMN7bcmG+v+tBjO+5+FHeJOuy5HnJLwIsQrjPYXWzIw+DmzgwIyRfpEUrKj1A0hhsDjdNV4Q46hxK45Ebd2XSCLx7ulrBWQ50zi3JDiaV5h0QlTdBAW8ANelTcDyinhJEaILhT/tVpJFUtMikt4a5Zd2S5a1adjg3bC24dV4jGXLMGCG5yS4zVAMFt7NLWbPCMaUhwE9zGaoDgNnZpjTHANU9Eb7gphrLmhW+HhhrDTTGU7QBoPXPUFW6INGtCDGWjn47Dqy/1LCS19WpAS7iB7ObEUBLcXm60KNESbkWz8NbU3LV3GxFD2Wi4lelQNikNaAk30CwSby5cy7/2k77+A9292b797P9kfwjuZPXZNGnaw/3W8u8ODA5deOvq0o273b3ZpRt3f/7WQt/+l7EGbTqdt1jhfVTfl1SVUEi7OY8f4y+1oq87qC+4QtuFxR2F8Q4IX4DyxaXt8Ka4GCQfg3z5e3B8B4Rpim+h4Ibw8i0vZPJFZUUgnns7p7WH++TZN05PX/zVzdV9A6/kDh3ePzD4q5urp6cv4kUVdDpk4NDJYuegOCfSKWT3AMYF4ObBlChKMqCtuHNkp7NMvmQxILwS4IYEDF6QDYOBaE5Rju4xOSpoSInu3qz2cB/72evnf/HLC29dPVo4tbxy77XCqQtvXT3/i1/i1QU6RaGSVYwljrtx1S/KaGIsObwtiMqEhlcC0xJZ/iEr2bVPwBuLO/bcLQS3jy+qN9zdvdme7P6rv105e+HNE6fOLK/cO3HqzNkLb1797UpPdj9QqNAMWU6VNNIOYW6eINRyYXaH+H6axI77J8KmBrXFcIN/AqOCBDSXknm0MvNqhvpAArrEy1lImwSaT4fgNhHuex9t3ri/1j+Ym35zfnnl3vSb8/2DuRv31+59tOnQI3Dxhk4ijlE8JXNLbNdiV2G8S7kZ5KXqbXm8puOWOF96EOGVnGlOKrtPmM+dGeosjLq+DOHjljgCCW4foGHRtXdL4lnuotx+Oc43h17uyYodgjAcCilqAtwQW8k+yrP3Hwqz9vd3/Nu6N5Se8Eov3CJGU24o7VtR8O23yyS4jYZ7eeUe+NyvcZ/7aASfG9/cDU0rTkVQX4pbElSNyuNqQG+fe3nl3s8mp+FpySsHD+8beGX59/5PS+DxX1wdRa/Pbbx8qKd4FCFCmEfEnY2QOnSpBg1oCbcyz779L+Pn3BfeuhrwnFt+aEppnmyWm2HbXRnvChcunqKAmxRema7G1YAJcMedM9VvEw0Q3FU2JW3CgZHTJLgJbmM1QHAbu7RGGuNYkyK4CW5jNaA33BRDGcuStVtljeGmGMp2gzXufHWFGyLNDIihFGvG/+gT75Pe+L0A78LDn/e9l9qkREu4gWwzYigJ7gbdbFrCregCos60jqFM3HIrWmrDrJZwA80mxVAS3InfftrDbUAMpVhUDDdPyxdU8JsnuHwGvWgOr+mKwDMp0H5NvLs3q0QMwbu7/m/SmhKaqT3cBsRQKnALgoPiJqGcAypPu2Tv1tqBOWD/4E1aSDiXAg7aFF0bE5qpPdzwPrd2MZRghmVEGQOOx+OgGB+VYPebsfJpiV9wp23LuUCOMqZfedEcnrrwIZkTA6E33PEicXgElwuXCHGQUL9xMZSq5faBm0ejucuDBuaYZ+SWOJXbKTRTe7j1jaEEClW4vT6DEzcZIbiTMc1vBhvutg3N1B7uGNHvLRZDGQQ3t7LO94NcG0r+q2ZxaXtYcGdOhbttQzP1htuMGEqF8qSy3n1kUpJ1kaM93AbEUDaIFQrN1BJuhQZ9YyiViSSVpdBMoUkT4E6KCZJjmAYIbmNf1TeM1BqmQ3AT3MZqgOA2dmlrMHWGNUkH7tndL3394s6/fedb//ftv6OfpDTwt+986+sXd17c/ZJhjNY8nRTgvrT7+0ktJ8nx1cDPiW9+dEwKcH/94k7fJaHCpDTw9Ys7a7Z2JjVMAe7/feH5pFaR5Phq4K8vPG8SozXPJQW48XrUNu56YigbGjYLr5LiXthfCu2PbeN0bXMPaVW/YkOE63hJS7gVRStRZ8pVeNuzGZ8w9ryVKl5aggiDfNF1lJR3qPWURIcbXuD27S78qm+T1izUEm6FZiXrVXTTVgssNx4DBhqncZ1E0gS3okaN4d63r1/5EZQrM4SAF/am6GznAHsjFOJTdhRGO2cWWbQiOyLMPnMMDC1uC4Uh8kUX7Gw++13qfNEJhTw55aTZCTj8vW15aAn/krd9E7L3XcXn6P3q8KOePGGOmZ5dCtzetmLkuHxhtvPw+A54qzZf3HZ16oUprhCYizJfjbIaw60YbMh6tW9DA6ekAtw2QxxrFDsIFXDMlXNiE3QhQAGHJ1+0BdpwB7klvKEqXIwTUAuow9HnTjx49mIYGG7ftmjYMvhSlIjfJ/BrB6sLmuiY0BhuxWzv29fva7kVnxtsqnv58cl3Mg2GXBhX78m/TLLbyYYs9BIIt/1bAgtXqPIdgHvYXYVZJwLNBbeffABU7cgdZKlchVbaJTSGG0y1kvCuAV4twM5NiT/cVY+qAZpFp5CFXkLg9grH4/TeObILBiI2+YFwe+WDZtSOCG5QTZ0JbGBqEyVoxpYb8+2VidcSsKsON19y5HIwX0URzoVAXCM6eDKSW+L4OfJkSn50IPSIhfObxNdZ4nDzoWLF+rdloxL1fdwSCG7A6lLmq1e2LSw3fJXGZ0Mpz2PH5hClGQ1yFwjMKQsMzgP7dsJ4h9wIVoObGWaPcC9VfnXQ8NjNYFvxoU4Mt798G26skKNT9j4BHUUrPuITYvsVJbRmtl3gbk3tJzsqBe5khesojeA255VXglu5A7WEW5lD07LCkYXnG+LpeNN6r9oRwa2oiOAmy22OBghuY9eSLLcK91Bxo8k/eA2a3LXZ3ZFilfV9Tsk3IUtr0CAlk2IVxRLczf7FpSxAglmCW1Gm3nCfWWJo/vr+pvPz3qYyw/bJEtzKWmsM95klhrUKNy9RJtkmWYJbWWhd4RZk+8D9XiQ3I3f588fWF0uX1xV15N55Zj39/MSUWq5Ua80swa2si5ZwA9leuK9Fc0taFu7c1Gd3LOvxrS1lnaJkCW5FS1rCrczBcbjvb0aEW5EA2dQt94lbX1QqFYIbVqSehJZwY5ohvXBnszC3cfhCJLckSGXpws1+nzz9fOkDstx1LSIsriFwX729+er5zQOlL7Mb32Q3vjlQ+nLonHxskpvaWnrq4HLi1hfW08+PM5/72RvctxaeQMX+Bz43LgdTigstWwJok1/9Yumdzx9bVqVSufPOeu6dZ0KwkCAGc+cd5nuI8jvvSP/evrT+BsGd0B8WDYG7MLdxoPRl37O/9N//qv/+V33P/sL4tnUE9hgA4j43g1vACoS98YEl4JaY8h2n08rtEDOxbr6FNCmBY2198NlQcQN1x+40aIUlANOQgPFHTJDPrSjKELgPn9/Irn/Tf/8rMb3++19l17+BqQKpDmS25QbuRWXIgsW1DTrzg+GqrMx/J8CNwSBm9MuHML5puE8UCVgywQ0LV2fCELgPvuHy0vY/+i8M91BxQxBz4tYX0j2IADf4J6BijCBHeWvpqet5oi/QGHpfuG8vby49lV4K3Eve3mEYQQmy3IpmDIFbbCv53DYP3P2Pvn/+y8Div+Kp8r3aszs2i44J556GsL6CPOSWWGCV3/jA8WEc/9t+KM6lMcojwi18laHiBtsAuB0buA/x4COmCW5FUabB/fLv/r3v2X8PLP+bMk8JLveAsRPM0vaez7K+WLrl/BGHI+vd9jl7QeAyLtx3PpC7TJCAR0tuCdZGPWnj4C592f/gP+vRSEPbKm5Jsn2R5Vb0aRrcA9f+OHDtj8okWydLcDdzLbSEO0RB+z/88/4P/xxSId1LBHcz9W8a3M3UXav1RW6JsiIEt+sZoqIdvbIEt7JeFJjKxvQAAAKXSURBVP1uTrwwhlsJlW3PLMFNcJujAeUeJrjNWVqy3C0Kd2Fiqrs3iz/WurxyTxkrZcM1QHAr+mkJyy1OJyO4lbWJmyW4FY2lDzecu5cI3JmRK2tWeW5E/ZC2Mu3oWRCY6RmbK1srk4lJjj6GiDUJbkVRKcMNZAsnpH63BFhU5llzFgQS3DXrMK2GKcOtTLt+uEFgVRZxBZwGCUoiSh1oEqsytKozQZZbUWDKcCs0K1llrLGyVfHCFXA6qJcodaBtrMrQqs4Ewa0osCXgxqfbiLSgXBmrks30nF3hkYiWVZ6bL1nlK8M9fdyLKJ3eOykuVSoVVC7fX12Z7IO2lUrlX9ZWVu2IRqt85cjeUe5bM+Esm7+yZpUm2LGRwueWnVYqFeF/ix4n+HE5gb3bQ61UKmvzY2Iiw/NlEZpgcfnK7GrIEtyK0loCbsVgQ1YZK84KOiVeHDsEMWaRnz3ZMzZXYuizk2ImSwImG1a7gr1ZFOUAHLCrloMctoVlPTLhdtot/OyKvceFcqiJJ1VnmuBWFNgScNdguRmj3FSL+UAWoAGMYMKKpcQVgtJBvLJy++EJ9BhUOTNZguAxkVibHxM3J/udk9yDHYIb1lokWgJuMNVKQhkrzgLNohCygBrAZzNnCX/Av4JNKqZWSvYzxrgaCLQ78vzecN+HrlmI3zkJIU5wY91292ZTgPuvLzyPl4HSiWvgry88ryxze2ZTgPtP392Z+HKSQKyBP313Z3vSrMw6Bbgv7f4+XglKJ66BC7v/Xlnm9symAHd3b/bS7pe+frHjby98K/F1bWeB//Od5//03Z0Xd7/Unih7Z50O3N5x1FDCd3Ly0TU8PK5BDjUxVQMaw23qktC8ktIAwW1OsEJSTBgj5/8BuSLzmOs50NAAAAAASUVORK5CYII=)

![](/assets/images/agent-zero-08-f040087da5fa37caa0fa94399e22e464.png)

![](/assets/images/agent-zero-09-f745e0628a6fafe58231ca5987e2bad3.png)

*   [Prerequisites](#prerequisites)
    
*   [Configure the agent](#configure-the-agent)
    
*   [Run the agent](#run-the-agent)
    *   [Example 1](#example-1)
        
    *   [Example 2](#example-2)
        
    *   [Example 3](#example-3)
        
    *   [Example 4](#example-4)
        
    *   [Example 5](#example-5)

---

# A planning agent | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

The [gpt planner](https://github.com/mshumer/gpt-prompt-engineer/blob/main/gpt_planner.ipynb)
 is a Python application that demonstrate the planning capabilities of LLMs. When you run it, it will ask the LLM to generate multiple action plans for a goal or a query. It will then ask the LLM to compare and select the best plan, and then rewrite it to answer the user query.

Since the program uses the official OpenAI Python library, we can [easily change it to use a Gaia node](/intro)
.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/getting-started/quick-start)
    
*   [use a public node](/nodes)
    

In this tutorial, we will use a public node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama8b.gaia.domains/v1](https://llama8b.gaia.domains/v1) |
| Model Name | llama |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, [load the nodebook in colab](https://colab.research.google.com/github/mshumer/gpt-prompt-engineer/blob/main/gpt_planner.ipynb)
.

Edit the code to create an OpenAI client. We will pass in the `base_url` here.

    client = openai.OpenAI(base_url="https://llama8b.gaia.domains/v1", api_key=OPENAI_API_KEY)

Next, replace all the `gpt-4o-mini` model name with the `llama` model name in the code. Here is an example.

    response = client.chat.completions.create(    model="llama",    messages=[        {"role": "system", "content": "You are a helpful assistant. Use the given plan to create a detailed and high-quality response to the user's query."},        {"role": "user", "content": f"User Query: {user_query}\n\nPlan: {best_plan}\n\nGenerate a detailed response based on this plan."}    ],    temperature=0.5,    max_tokens=2000)

Change the query to your own.

    user_query = "How do I debug a TLS connection timeout?"

Finally, run the notebook to see the results!

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# Knowledge base from a plain text file | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a plain text file. The snapshot file can then be [loaded by a Gaia node as its knowledge base](/getting-started/customize#select-a-knowledge-base)
.

The text file is segmented into multiple chunks by blank lines. [See an example](https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt)
. Each chunk is turned into a vector, and when retrieved, added to the prompt context for the LLM.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

Install the WasmEdge Runtime, the cross-platform LLM runtime.

    curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Download an embedding model.

    curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base.

Start a vector database[​](#start-a-vector-database "Direct link to Start a vector database")

----------------------------------------------------------------------------------------------

By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](/getting-started/quick-start)
.

note

Or, you can start a Qdrant server using Docker. The following command starts it in the background.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Create the vector collection snapshot[​](#create-the-vector-collection-snapshot "Direct link to Create the vector collection snapshot")

----------------------------------------------------------------------------------------------------------------------------------------

Delete the default collection if it exists.

    curl -X DELETE 'http://localhost:6333/collections/default'

Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model.

    curl -X PUT 'http://localhost:6333/collections/default' \  -H 'Content-Type: application/json' \  --data-raw '{    "vectors": {      "size": 768,      "distance": "Cosine",      "on_disk": true    }  }'

Download a program to chunk a document and create embeddings.

    curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/paragraph_embed/paragraph_embed.wasm

It chunks the document based on empty lines. So, you MUST prepare your source document this way -- segment the document into sections of around 200 words with empty lines. You can check out the [Rust source code here](https://github.com/GaiaNet-AI/embedding-tools/tree/main/paragraph_embed)
 and modify it if you need to use a different chunking strategy.

> The `paragraph_embed.wasm` program would NOT break up code listings even if there are empty lines with in the listing.

Next, you can run the program by passing a collection name, vector dimension, and the source document. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from [paris\_chunks.txt](https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt)
 and saves them into the default collection.

    curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txtwasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \  paragraph_embed.wasm embedding default 768 paris_chunks.txt -c 8192

### Options[​](#options "Direct link to Options")

You can pass the following options to the program.

*   Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length.
*   Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection.
*   Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512.

Example: the above example but to append the London guide to the end of an existing collection starting from index 42.

    wasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \   paragraph_embed.wasm embedding default 768 london.txt -c 8192 -s 42

Create a vector snapshot[​](#create-a-vector-snapshot "Direct link to Create a vector snapshot")

-------------------------------------------------------------------------------------------------

You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node.

    curl -X POST 'http://localhost:6333/collections/default/snapshots'

We also recommend you to compress the snapshot file.

    tar czvf my.snapshot.tar.gz my.snapshot

Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](/getting-started/customize#select-a-knowledge-base)
.

Video Guide[​](#video-guide "Direct link to Video Guide")

----------------------------------------------------------

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Start](/getting-started/quick-start)
     a new Gaia node
*   [Customize](/getting-started/customize)
     the Gaia node

*   [Prerequisites](#prerequisites)
    
*   [Start a vector database](#start-a-vector-database)
    
*   [Create the vector collection snapshot](#create-the-vector-collection-snapshot)
    *   [Options](#options)
        
*   [Create a vector snapshot](#create-a-vector-snapshot)
    
*   [Video Guide](#video-guide)
    
*   [Next steps](#next-steps)

---

# LlamaEdgeBook | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

LlamaEdgeBook, forked from GroqBook, is an open-source tool that scaffolds the creation of books from a one-line prompt using open-source LLMs. You can configure the LlamaEdgeBook framework using any Gaia node as the backend LLM API.

Steps[​](#steps "Direct link to Steps")

----------------------------------------

First, get the source code of the LlamaEdgeBook. Open your terminal and enter the following command line.

    git clone https://github.com/second-state/LlamaEdgeBookcd LlamaEdgeBook

Ensure you have Python 3.11 or later installed. Then, install the necessary packages:

    pip install -r requirements.txt

Next, let's configure the Gaia node as the LLM backend.

    export OPENAI_BASE_URL="https://llama8b.gaia.domains/v1"export OPENAI_MODEL_NAME="llama" export OPENAI_API_KEY="GAIANET" 

**Hint:** if you don't know the model name of the Gaia node, you can retrieve the model information using:

    curl -X POST https://0x57b00e4f3d040e28dc8aabdbe201212e5fb60ebc.us.gaianet.network/v1/models

Then, use the following command line to run the app.

    streamlit run main.py

When the app runs successfully, the terminal will output the following information.

    Collecting usage statistics. To deactivate, set browser.gatherUsageStats to false.  You can now view your Streamlit app in your browser.  Local URL: http://localhost:8501  Network URL: http://10.128.0.8:8501  External URL: http://35.222.115.181:8501

Finally, you can open the `http://localhost:8501` in your browser to generate a book.

![](/assets/images/book-01-8612df6e0c495f2d68979b76a4cb7ca4.png)

The LlamaEdgeBook will first generate an outline based on your prompt, and then create the chapter content based on the outline.

![](/assets/images/book-02-3c158830817920dbdb920985c0dd6ce5.png)

You can also download the book after the generation is complete.

![](/assets/images/book-03-1a63c3c4eb72eaf948d529875ee479a3.png)

*   [Steps](#steps)

---

# Knowledge base from source / summary pairs | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot for optimal retrieval of long-form text documents. The approach is to create two columns of text in a CSV file.

*   The first column is the long-form source text from the knowledge document, such as a book chapter or a markdown section.
*   The long-form source text is difficult to search. The second column is a "search-friendly" summary of the source text. It could contain a list of questions that can be answered by the first column source text.

We will create a vector snapshot where each vector is computed from the summary text (second column), but the retrieved source text for that vector is from the first column. The snapshot file can then be [loaded by a Gaia node as its knowledge base](/getting-started/customize#select-a-knowledge-base)
.

> We have a simple Python script to build properly formatted CSV files from a set of articles or chapters. [See how it works](https://github.com/GaiaNet-AI/embedding-tools/tree/main/csv_embed#create-a-csv-file)
> .

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

Install the WasmEdge Runtime, the cross-platform LLM runtime.

    curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Download an embedding model.

    curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base.

Start a vector database[​](#start-a-vector-database "Direct link to Start a vector database")

----------------------------------------------------------------------------------------------

By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](/getting-started/quick-start)
.

note

Or, you can start a Qdrant server using Docker. The following command starts it in the background.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Create the vector collection snapshot[​](#create-the-vector-collection-snapshot "Direct link to Create the vector collection snapshot")

----------------------------------------------------------------------------------------------------------------------------------------

Delete the default collection if it exists.

    curl -X DELETE 'http://localhost:6333/collections/default'

Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model.

    curl -X PUT 'http://localhost:6333/collections/default' \  -H 'Content-Type: application/json' \  --data-raw '{    "vectors": {      "size": 768,      "distance": "Cosine",      "on_disk": true    }  }'

Download a program to create embeddings from the CSV file.

    curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/csv_embed/csv_embed.wasm

You can check out the [Rust source code](https://github.com/GaiaNet-AI/embedding-tools/tree/main/csv_embed)
 here and modify it if you need to use a different CSV layout.

Next, you can run the program by passing a collection name, vector dimension, and the CSV document. The `--ctx_size` option matches the embedding model's context window size, which in this case is 8192 tokens allowing it to process long sections of text. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from `paris.csv` and saves them into the default collection.

    curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris.csvwasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \  csv_embed.wasm embedding default 768 paris.csv --ctx_size 8192

### Options[​](#options "Direct link to Options")

You can pass the following options to the program.

*   Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512.
*   Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length.
*   Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection.

Example: the above example but to append the London guide to the end of an existing collection starting from index 42.

    wasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \   csv_embed.wasm embedding default 768 london.csv -c 8192 -l 1 -s 42

Create a vector snapshot[​](#create-a-vector-snapshot "Direct link to Create a vector snapshot")

-------------------------------------------------------------------------------------------------

You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node.

    curl -X POST 'http://localhost:6333/collections/default/snapshots'

We also recommend you to compress the snapshot file.

    tar czvf my.snapshot.tar.gz my.snapshot

Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](/getting-started/customize#select-a-knowledge-base)
.

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Start](/getting-started/quick-start)
     a new Gaia node
*   [Customize](/getting-started/customize)
     the Gaia node

*   [Prerequisites](#prerequisites)
    
*   [Start a vector database](#start-a-vector-database)
    
*   [Create the vector collection snapshot](#create-the-vector-collection-snapshot)
    *   [Options](#options)
        
*   [Create a vector snapshot](#create-a-vector-snapshot)
    
*   [Next steps](#next-steps)

---

# Knowledge base from a markdown file | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a markdown file. The snapshot file can then be [loaded by a Gaia node as its knowledge base](/getting-started/customize#select-a-knowledge-base)
.

The markdown file is segmented into multiple sections by headings. [See an example](https://huggingface.co/datasets/gaianet/paris/raw/main/paris.md)
. Each section is turned into a vector, and when retrieved, added to the prompt context for the LLM.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

Install the WasmEdge Runtime, the cross-platform LLM runtime.

    curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Download an embedding model.

    curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base.

Start a vector database[​](#start-a-vector-database "Direct link to Start a vector database")

----------------------------------------------------------------------------------------------

By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](/getting-started/quick-start)
.

note

Or, you can start a Qdrant server using Docker. The following command starts it in the background.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Create the vector collection snapshot[​](#create-the-vector-collection-snapshot "Direct link to Create the vector collection snapshot")

----------------------------------------------------------------------------------------------------------------------------------------

Delete the default collection if it exists.

    curl -X DELETE 'http://localhost:6333/collections/default'

Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model.

    curl -X PUT 'http://localhost:6333/collections/default' \  -H 'Content-Type: application/json' \  --data-raw '{    "vectors": {      "size": 768,      "distance": "Cosine",      "on_disk": true    }  }'

Download a program to segment the markdown document and create embeddings.

    curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/markdown_embed/markdown_embed.wasm

It chunks the document based on markdown sections. You can check out the [Rust source code](https://github.com/GaiaNet-AI/embedding-tools/tree/main/markdown_embed)
 here and modify it if you need to use a different chunking strategy.

Next, you can run the program by passing a collection name, vector dimension, and the source document. You can pass in the desired markdown heading level for chunking using the `--heading_level` option. The `--ctx_size` option matches the embedding model's context window size, which in this case is 8192 tokens allowing it to process long sections of text. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from `paris.md` and saves them into the default collection.

    curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris.mdwasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \  markdown_embed.wasm embedding default 768 paris.md --heading_level 1 --ctx_size 8192

### Options[​](#options "Direct link to Options")

You can pass the following options to the program.

*   Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512.
*   Using `-l` or `--heading_level` to specify the markdown heading level for each vector. This defaults to 1.
*   Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length.
*   Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection.

Example: the above example but to append the London guide to the end of an existing collection starting from index 42.

    wasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \   markdown_embed.wasm embedding default 768 london.md -c 8192 -l 1 -s 42

Create a vector snapshot[​](#create-a-vector-snapshot "Direct link to Create a vector snapshot")

-------------------------------------------------------------------------------------------------

You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node.

    curl -X POST 'http://localhost:6333/collections/default/snapshots'

We also recommend you to compress the snapshot file.

    tar czvf my.snapshot.tar.gz my.snapshot

Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](/getting-started/customize#select-a-knowledge-base)
.

Video Guide[​](#video-guide "Direct link to Video Guide")

----------------------------------------------------------

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Start](/getting-started/quick-start)
     a new Gaia node
*   [Customize](/getting-started/customize)
     the Gaia node

*   [Prerequisites](#prerequisites)
    
*   [Start a vector database](#start-a-vector-database)
    
*   [Create the vector collection snapshot](#create-the-vector-collection-snapshot)
    *   [Options](#options)
        
*   [Create a vector snapshot](#create-a-vector-snapshot)
    
*   [Video Guide](#video-guide)
    
*   [Next steps](#next-steps)

---

# Knowledge base from a PDF file | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a PDF file. First, we will parse the unstructured PDF file to a structured markdown file. Then, we will follow the steps from [Knowledge base from a markdown file](/knowledge-bases/how-to/markdown)
 to create embedding for your PDF files.

Tools to convert a PDF file to a markdown file[​](#tools-to-convert-a-pdf-file-to-a-markdown-file "Direct link to Tools to convert a PDF file to a markdown file")

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

### Tool #1: LlamaParse[​](#tool-1-llamaparse "Direct link to Tool #1: LlamaParse")

LlamaParse is a tool to parse files for optimal RAG. You will need a LlamaCloud key from [https://cloud.llamaindex.ai](https://cloud.llamaindex.ai)
.

First, install the dependencies. we are assuming that you already have Node.JS 20+ installed.

    git clone https://github.com/alabulei1/llamaparse-integration.gitcd llamaparse-integrationnpm install llamaindexnpm install dotenv

Then, edit the `.env` file to set up the PDF file path and LlamaCloud Key. In this case, you don't need to care about the LLM-related settings.

After that, run the following command line to parse your pdf into a markdown file.

    npx tsx transMd.ts

The output markdown file will be located in this folder named `output.md` by default. You can change the path in the `.env` file.

### Tool #2: GPTPDF[​](#tool-2-gptpdf "Direct link to Tool #2: GPTPDF")

GPTPDF is an open-source tool using GPT-4o to parse PDF into markdown. You will need an OpenAI key here.

First, install the gptpdf software.

    pip install gptpdf

Then, enter the Python environment.

    python

Next, use the following command to parse your pdf.

    from gptpdf import parse_pdfapi_key = 'Your OpenAI API Key'content, image_paths = parse_pdf(Your_Pdf_Path, api_key=api_key)print(content)

The output markdown files called `output.md` will be located in your root directory.

Create embeddings from the markdown files[​](#create-embeddings-from-the-markdown-files "Direct link to Create embeddings from the markdown files")

----------------------------------------------------------------------------------------------------------------------------------------------------

Please follow the tutorial [Knowledge base from a markdown file](/knowledge-bases/how-to/markdown)
 to convert your markdown file to a snapshot of embeddings that can be imported into a GaiaNet node.

*   [Tools to convert a PDF file to a markdown file](#tools-to-convert-a-pdf-file-to-a-markdown-file)
    *   [Tool #1: LlamaParse](#tool-1-llamaparse)
        
    *   [Tool #2: GPTPDF](#tool-2-gptpdf)
        
*   [Create embeddings from the markdown files](#create-embeddings-from-the-markdown-files)

---

# Knowledge base from a URL | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a Web URL. First, we will parse the URL to a structured markdown file. Then, we will follow the steps from [Knowledge base from a markdown file](/knowledge-bases/how-to/markdown)
 to create embedding for your URL.

Parse the URL content to a markdown file[​](#parse-the-url-content-to-a-markdown-file "Direct link to Parse the URL content to a markdown file")

-------------------------------------------------------------------------------------------------------------------------------------------------

Firecrawl can crawl and convert any website into LLM-ready markdown or structured data. It also supports crawling a URL and all accessible subpages.

> To use Firecrawl, you need to sign up on [Firecrawl](https://firecrawl.dev/)
>  and get an API key.

First, install the dependencies. We are assuming that you already have Node.JS 20+ installed.

    git clone https://github.com/JYC0413/firecrawl-integration.gitcd firecrawl-integrationnpm install

Then, export the API key in the terminal.

    export FIRECRAWL_KEY="your_api_key_here"

next, we can use the following command line to run the service.

    node crawlWebToMd.js

After the application is running successfully, you will see the prompt appear on the Terminal.

![](/assets/images/firecrawl-01-b1a611217130d603f863d886375f865a.png)

You can type your URL in the terminal right now. Here we have two choices.

*   Multiple pages: input your link with `/` at the end, the program will crawl and convert the page and its subpages to one single markdown file. This way will cost lots of API token usage.
*   One single page: input your link without `/` at the end. the program will crawl and convert the current page to one single markdown file.

The output markdown file will be located in this folder named `output.md`.

Create embeddings from the markdown files[​](#create-embeddings-from-the-markdown-files "Direct link to Create embeddings from the markdown files")

----------------------------------------------------------------------------------------------------------------------------------------------------

Please follow the tutorial [Knowledge base from a markdown file](/knowledge-bases/how-to/markdown)
 to convert your markdown file to a snapshot of embeddings that can be imported into a GaiaNet node.

*   [Parse the URL content to a markdown file](#parse-the-url-content-to-a-markdown-file)
    
*   [Create embeddings from the markdown files](#create-embeddings-from-the-markdown-files)

---

# Install CUDA on Linux | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

If you are using an Nvidia-enabled VM instance from a public cloud, you should probably use the VM image provided by the cloud. It typically has the correct versions of Nvidia driver and CUDA toolkit already installed. Read on if you need to install Nvidia driver and CUDA toolkit on your own machine.

Ubuntu 22.04[​](#ubuntu-2204 "Direct link to Ubuntu 22.04")

------------------------------------------------------------

### 1 Install the Nvidia driver.[​](#1-install-the-nvidia-driver "Direct link to 1 Install the Nvidia driver.")

Rebuild the grub configuration:

    sudo apt-get install -y gcc make linux-headers-$(uname -r)cat << EOF | sudo tee --append /etc/modprobe.d/blacklist.confblacklist vga16fbblacklist nouveaublacklist rivafbblacklist nvidiafbblacklist rivatvEOFsudo sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="rdblacklist=nouveau"/' /etc/default/grubsudo update-grub

Download and install the Nvidia driver

    wget https://storage.googleapis.com/nvidia-drivers-us-public/GRID/vGPU16.1/NVIDIA-Linux-x86_64-535.104.05-grid.runsudo sh NVIDIA-Linux-x86_64-535.104.05-grid.run

Confirm the driver is installed successfully

    nvidia-smi -q | head==============NVSMI LOG==============Timestamp                                 : Fri Oct 27 21:54:05 2023Driver Version                            : 535.104.05CUDA Version                              : 12.2Attached GPUs                             : 1GPU 00000000:00:1E.0    Product Name                          : NVIDIA A10G

Disable GSP and reboot.

    sudo touch /etc/modprobe.d/nvidia.confecho "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.confsudo reboot

### 2 Install the CUDA toolkit.[​](#2-install-the-cuda-toolkit "Direct link to 2 Install the CUDA toolkit.")

    wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.runsudo sh cuda_12.2.2_535.104.05_linux.run --silent --override --toolkit --samples --toolkitpath=/usr/local/cuda-12 --samplespath=/usr/local/cuda --no-opengl-libs

Confirm that CUDA is installed.

    /usr/local/cuda/bin/nvcc --versionnvcc: NVIDIA (R) Cuda compiler driverCopyright (c) 2005-2023 NVIDIA CorporationBuilt on Tue_Aug_15_22:02:13_PDT_2023Cuda compilation tools, release 12.2, V12.2.140Build cuda_12.2.r12.2/compiler.33191640_0

After that, use the following two commands to set up the environment path. You should probably add these two lines to your `~/.bashrc` and `~/.bash_profile` (or `~/.zshrc` and `~/.profile`) files so that new terminals and future logins will still be able to find these CUDA library files.

    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}export PATH=/usr/local/cuda/bin:${PATH}

More resources[​](#more-resources "Direct link to More resources")

-------------------------------------------------------------------

Here are more scripts that could help you in case you are stuck.

*   The [Nvidia official install guide](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/)
     for Linux.
*   [Yam Peleg's popular script](https://x.com/yampeleg/status/1751823896800583924)
     for Ubuntu 22.04
*   [Make CUDA available in Docker containers](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
    

*   [Ubuntu 22.04](#ubuntu-2204)
    *   [1 Install the Nvidia driver.](#1-install-the-nvidia-driver)
        
    *   [2 Install the CUDA toolkit.](#2-install-the-cuda-toolkit)
        
*   [More resources](#more-resources)

---

# Use my GaiaNet node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

When you [start a GaiaNet node](/1.0.0/node-guide/quick-start)
, or you find a node on the web, you could use it as a web-based chatbot UI and an OpenAI compatible web service. Just load the node's public URL in the browser to open its dashboard. Let's say the URL is as follows.

    https://0x1234...xyz.gaianet.network/

> Please refer to the [agent apps](/1.0.0/user-guide/apps/intro)
>  section to see how to use the GaiaNet node API in your favorite agent frameworks or apps.

Web-based chatbot[​](#web-based-chatbot "Direct link to Web-based chatbot")

----------------------------------------------------------------------------

On the GaiaNet node dashboard, you will see a "Chat with this node" button.

![](/assets/images/chat_button-8557659182506ceed736a17b449dd9d8.png)

OpenAI API replacement[​](#openai-api-replacement "Direct link to OpenAI API replacement")

-------------------------------------------------------------------------------------------

The GaiaNet node is a drop-in replacement for OpenAI API in [agent and LLM apps](/1.0.0/user-guide/apps/intro)
. On the GaiaNet node dashboard, you will see a table that shows how to replace OpenAI parameters in those apps.

![](/assets/images/openai_api_options-6610435c06758b955dcd1f73bcfd1d52.png)

*   [Web-based chatbot](#web-based-chatbot)
    
*   [OpenAI API replacement](#openai-api-replacement)

---

# Working with Coinbase AgentKit | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

You can use a Gaia node to power the [Coinbase AgentKit](https://github.com/coinbase/cdp-agentkit)
. The Gaia node must run an LLM that is optimized for [tool calling](/1.0.0/tutorial/tool-call)
. Or, you could simply use our public node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llamatool.us.gaianet.network/v1](https://llamatool.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Quickstart[​](#quickstart "Direct link to Quickstart")

-------------------------------------------------------

First, you need a [Coinbase Developer Platform account](https://www.coinbase.com/developer-platform)
 and then [create an API key](https://docs.cdp.coinbase.com/advanced-trade/docs/auth/#creating-api-keys)
.

Next, check out the AgentKit example code.

    git clone https://github.com/coinbase/cdp-agentkitcd cdp-agentkit/cdp-langchain/examples/chatbot

Set the environment variables for your API key.

    export CDP_API_KEY_NAME='organizations/.../apiKeys/...'export CDP_API_KEY_PRIVATE_KEY='-----BEGIN EC...END EC PRIVATE KEY-----\n'

Edit the `chatbot.py` file to configure the agent to use the Gaia node above.

    llm = ChatOpenAI(model="llama", api_key="GAIA", base_url="https://llamatool.us.gaianet.network/v1")

Finally, run the agent using Python.

    python chatbot.py

You can see a [video demo here](https://x.com/juntao/status/1858634152599224828)
.

A Telegram bot for AgentKit[​](#a-telegram-bot-for-agentkit "Direct link to A Telegram bot for AgentKit")

----------------------------------------------------------------------------------------------------------

We have also built a Telegram bot that allows you to interact with your own wallet through text and voice messages. You will need to bring your own Coinbase credentials and wallet for your bot to operate on. Go to chat the with bot on Telegram.

[https://t.me/agentkit\_bot](https://t.me/agentkit_bot)

It asks you to use slash commands to set your wallet credentials for your bot. It is like this.

    /name organizations/.../apiKeys/.../pk -----BEGIN EC...END EC PRIVATE KEY-----\n/wallet {"wallet_id": "...

Then, you can just tell the Telegram bot what you want to do on-chain, and the agent will do it for you.

*   [Quickstart](#quickstart)
    
*   [A Telegram bot for AgentKit](#a-telegram-bot-for-agentkit)

---

# Working with eliza | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

eliza is a simple, fast, and lightweight AI agent framework. Recently, eliza has integrated Gaia as one of the [model service provider](https://github.com/elizaOS/eliza/pull/762)
. This means you can now use Gaia as the LLM service backend for the Eliza framework.

### Build a Trump agent with eliza and Gaia[​](#build-a-trump-agent-with-eliza-and-gaia "Direct link to Build a Trump agent with eliza and Gaia")

This guide demonstrates how to create an agent with Trump-like characteristics using [the Get Started guide from eliza](https://elizaos.github.io/eliza/docs/quickstart/)
.

#### Set up the environment[​](#set-up-the-environment "Direct link to Set up the environment")

> Note: Ensure your Node.js version is above 0.23.0 before proceeding.

Firstly, clone the Eliza repository:

    git clone https://github.com/elizaos/eliza.gitcd elizagit checkout v0.1.7-alpha.1

Next, install the required dependencies.

    pnpm install

Then, build the local libraries.

    pnpm build

#### Choose Gaia as the model service provider[​](#choose-gaia-as-the-model-service-provider "Direct link to Choose Gaia as the model service provider")

After that, we will need to configure the environment and use a Gaia node as model service provider.

    cp .env.example .env

Then, edit the `.env` file to include Gaia-related configuration values:

    # Gaianet ConfigurationGAIANET_MODEL=llama3bGAIANET_SERVER_URL=https://llama3b.gaia.domains/v1SMALL_GAIANET_MODEL=            # Default: llama3bSMALL_GAIANET_SERVER_URL=       # Default: https://llama3b.gaia.domains/v1MEDIUM_GAIANET_MODEL=           # Default: llamaMEDIUM_GAIANET_SERVER_URL=      # Default: https://llama8b.gaia.domains/v1LARGE_GAIANET_MODEL=            # Default: qwen72bLARGE_GAIANET_SERVER_URL=       # Default: https://qwen72b.gaia.domains/v1GAIANET_EMBEDDING_MODEL=nomic-embedUSE_GAIANET_EMBEDDING=TRUE          # Set to TRUE for GAIANET/768, leave blank for local

By using this configuration, the system will utilize the Llama 3b Gaia domain as the LLM backend. You can replace `GAIANET_SERVER_URL` with a URL for your custom node or domain.

#### Create the Trump agent[​](#create-the-trump-agent "Direct link to Create the Trump agent")

The default character templates are located in the `characters` folder.

Update the `modelProvider` for the desired character. For Gaia, the provider name is `gaianet`.

        "name": "trump",    "clients": [],    "modelProvider": "gaianet",    "settings": {        "secrets": {},        "voice": {            "model": "en_US-male-medium"        }    },    "plugins": [],

Then, we can use the following command line to start running the agent.

    pnpm run dev --character="characters/trump.character.json"

After the service runs successfully, we can launch the client UI to interact with the agent:

    pnpm start:client

Finally, open `http://localhost:5174/` on your browser to start chatting with the agent.

![image](https://github.com/user-attachments/assets/48474b35-1d40-4334-8ed0-c97f373c2b0b)

### Advanced use case[​](#advanced-use-case "Direct link to Advanced use case")

For more inspiration, refer to[Nader Fabit's example on building a Twitter AI bot](https://x.com/dabit3/status/1863772029565981144)
.

*   [Build a Trump agent with eliza and Gaia](#build-a-trump-agent-with-eliza-and-gaia)
    
*   [Advanced use case](#advanced-use-case)

---

# Agentic translation on GaiaNet | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Prof. Andrew Ng's [agentic translation](https://www.linkedin.com/posts/andrewyng_github-andrewyngtranslation-agent-activity-7206347897938866176-5tDJ/)
 is a great demonstration on how to coordinate multiple LLM "agents" to work on a single task. It allows multiple smaller LLMs (like Llama-3 or Gemma-2) to work together and produce better results than a single large LLM (like ChatGPT).

[GaiaNet](https://www.gaianet.ai/)
, with 2000+ nodes running all kinds of finetuned LLms and knowledge bases, provides a huge opportunity for agentic apps to choose and use their own LLM backends.

Introduction to the LLM Translation Agent[​](#introduction-to-the-llm-translation-agent "Direct link to Introduction to the LLM Translation Agent")

----------------------------------------------------------------------------------------------------------------------------------------------------

This LLM Translation Agent is designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use your own fine-tuned models or any LLMs on Hugging Face like Meta's Llama 3.

> For detailed commands on starting and running this agent, please visit [GitHub - Second State/translation-agent](https://github.com/second-state/translation-agent/blob/use_llamaedge/step-by-step-use-LocalAI.md)
> .

To get started, clone the Translation Agent.

    git clone https://github.com/second-state/translation-agent.git    cd translation-agentgit checkout use_llamaedge

Next, we will install a local GaiaNet node, which provides the backend API services required by the agent. You can, of course, use [GaiaNet nodes from the community](https://docs.gaianet.ai/user-guide/nodes)
 if you do not want to start your own.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash

You will also need the following configurations and prerequisites to run the agent app. If you are using a public GaiaNet node instead of your local node, replace the `http://localhost:8080` with `https://node_id.us.gaianet.network`.

    export OPENAI_BASE_URL="http://localhost:8080/v1"export PYTHONPATH=${PWD}/srcexport OPENAI_API_KEY="GAIANET"pip install python-dotenvpip install openai tiktoken icecream langchain_text_splitters

Demo 1: Running Translation Agents with Llama-3-8B[​](#demo-1-running-translation-agents-with-llama-3-8b "Direct link to Demo 1: Running Translation Agents with Llama-3-8B")

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

First, let's run the translation agent with Meta AI's popular Llama-3 model. We select the smallest Llama-3 model (the 8b model) for this demo. The translation task is from Chinese to English. Our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text)
 is in Chinese, a brief intro to the ancient Chinese royal palace, the Forbidden City.

### Step 1.1: Run a Llama-3-8B GaiaNet node[​](#step-11-run-a-llama-3-8b-gaianet-node "Direct link to Step 1.1: Run a Llama-3-8B GaiaNet node")

Configure and download the model. Since the size of the model is 5.73 GB. It can take a while to download.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json

Next, use the following command to start the GaiaNet node.

    gaianet start

### Step 1.2 Run the Translation Agent on top of Llama-3-8B[​](#step-12-run-the-translation-agent-on-top-of-llama-3-8b "Direct link to Step 1.2 Run the Translation Agent on top of Llama-3-8B")

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `Meta-Llama-3-8B-Instruct-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import osimport translation_agent as ta        if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/forbiddencity.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="Meta-Llama-3-8B-Instruct-Q5_K_M",    )        print(f"Translation:\n\n{translation}")

Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here, because we named our source text `forbiddencity.txt`, the relative path to the document would be `sample-texts/forbiddencity.txt`.

Run the below commands to have your text file translated into English.

    cd examplespython example_script.py

Wait for several minutes and you will have [a fully translated version](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-Llama-3-8B)
 appear on your terminal screen.

Demo 2: Running Translation Agents with gemma-2-27b[​](#demo-2-running-translation-agents-with-gemma-2-27b "Direct link to Demo 2: Running Translation Agents with gemma-2-27b")

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The benefit of running the Translation Agent with GaiaNet is the ability for users to choose and embed different LLMs for different agentic tasks. To demonstrate this point, we will now change the translation agent LLM from Llama-3-8b to Google's gemma-2-27b, which is of similar size but scores higher on many language-related benchmarks.

The translation task is the same as before. Our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text)
 is in Chinese, a brief intro to the ancient Chinese royal palace, the Forbidden City. The translation target is English.

### Step 2.1 Run a gemma-2-27b GaiaNet node[​](#step-21-run-a-gemma-2-27b-gaianet-node "Direct link to Step 2.1 Run a gemma-2-27b GaiaNet node")

Configure and download the model. Since the size of the model is 6.40G, it could take a while to download.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/gemma-2-27b-it/config.json

Next, use the following command to start the GaiaNet node.

    gaianet start

### Step 2.2 Run the Translation Agent to run on top of gemma-2-27b[​](#step-22-run-the-translation-agent-to-run-on-top-of-gemma-2-27b "Direct link to Step 2.2 Run the Translation Agent to run on top of gemma-2-27b")

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `gemma-2-27b-it-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import os  import translation_agent as ta      if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/forbiddencity.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="gemma-2-27b-it-Q5_K_M",    )        print(f"Translation:\n\n{translation}")

Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here,because we named our source text `forbiddencity.txt`, the relative path to the document would be `sample-texts/forbiddencity.txt`.

Run the below commands to have your text file translated into English.

    cd examples    python example_script.py

You can find the translated result in English [here](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-gemma-2-27b)
.

Demo 3: Running Translation Agents with Phi-3-Medium long context model[​](#demo-3-running-translation-agents-with-phi-3-medium-long-context-model "Direct link to Demo 3: Running Translation Agents with Phi-3-Medium long context model")

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The Llama-3 and Gemma-2 models are great LLMs, but they have relatively small context windows. The agent requires all text to fit into the LLM context window, and that limits the size of articles they can translate. To fix this problem, we could select an open source LLM with a large context window. For this demo, we choose Microsoft's Phi-3-medium-128k model, which has a massive 128k (over 100 thousand words or the length of several books) context window.

We run [a lengthy Chinese article on Forbidden City's collaboration with the Varsaille Palace](https://hackmd.io/vuFYZTVsQZyKmkeQ3ThZQw?view#Source-text)
 through our Translation Agent powered by a Phi-3-medium-128k model we start locally.

### Step 3.1: Run a Phi-3-medium-128k GaiaNet node[​](#step-31-run-a-phi-3-medium-128k-gaianet-node "Direct link to Step 3.1: Run a Phi-3-medium-128k GaiaNet node")

Configure and download the model.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/phi-3-medium-instruct-128k/config_full.json

Next, use the following command to start the GaiaNet node with a 128k context window.

    gaianet start

### Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k[​](#step-32-clone-and-run-the-translation-agent-on-top-of-phi-3-medium-128k "Direct link to Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k")

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `Phi-3-medium-128k-instruct-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import os  import translation_agent as ta      if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/long_article.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="Phi-3-medium-128k-instruct-Q5_K_M",    )        print(f"Translation:\n\n{translation}")

Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here, because we named our source text `long_article.txt`, the relative path to the document would be `sample-texts/long_article.txt`.

    cd examplespython example_script.py

[The translated results were impressive,](https://hackmd.io/vuFYZTVsQZyKmkeQ3ThZQw?view#Source-text)
 with the translation capturing the nuances and context of the original text with high fidelity.

Evaluation of Translation Quality[​](#evaluation-of-translation-quality "Direct link to Evaluation of Translation Quality")

----------------------------------------------------------------------------------------------------------------------------

The three models, Llama-3-8B, gemma-2-27b, and Phi-3-medium, have exhibited varying levels of performance in translating complex historical and cultural content from Chinese to English.

Llama-3-8B provides a translation that effectively captures the factual content but shows occasional stiffness in language, possibly indicating a direct translation approach that doesn't fully adapt idiomatic expressions. It does not keep section title and the format of the original text and left certain part untranslated.

In contrast, The translation by gemma-2-27b is quite accurate and retains the original meaning of the short intro article of Forbidden city. gemma-2-27b's translation exhibits a smooth and natural English flow, suggesting a sophisticated understanding of both the source language and the target language’s grammatical structures. The choice of words and sentence structures in gemma-2-27b's output demonstrates a high degree of linguistic finesse, suggesting it might be well-suited for translating formal and historically nuanced texts.

The Phi-3-medium-128k model can translate book-length text from Chinese to English. It demonstrates robust capabilities in handling large volumes of complex content, suggesting advanced memory handling and contextual awareness. The quality of translation remains consistent even with increased text length, indicating Phi's utility in projects requiring extensive, detailed translations. But you can see it makes certain mistakes like mistaken "Wenhua Hall" as "also known as Forbidden City" in the first paragraph.

Overall, each model has its strengths, with gemma-2-27b standing out for linguistic finesse and Phi-3-medium-128k for handling lengthy texts.

Conclusion[​](#conclusion "Direct link to Conclusion")

-------------------------------------------------------

[GaiaNet](https://github.com/GaiaNet-AI)
 provides an easy way to select and use different open-source LLMs in your agentic applications to fully take advantage of their finetuned capabilities for specific tasks.

Once you have a local GaiaNet node up and running, you could share it with others and make $$$ by joining the [GaiaNet network](https://www.gaianet.ai/)
!

*   [Introduction to the LLM Translation Agent](#introduction-to-the-llm-translation-agent)
    
*   [Demo 1: Running Translation Agents with Llama-3-8B](#demo-1-running-translation-agents-with-llama-3-8b)
    *   [Step 1.1: Run a Llama-3-8B GaiaNet node](#step-11-run-a-llama-3-8b-gaianet-node)
        
    *   [Step 1.2 Run the Translation Agent on top of Llama-3-8B](#step-12-run-the-translation-agent-on-top-of-llama-3-8b)
        
*   [Demo 2: Running Translation Agents with gemma-2-27b](#demo-2-running-translation-agents-with-gemma-2-27b)
    *   [Step 2.1 Run a gemma-2-27b GaiaNet node](#step-21-run-a-gemma-2-27b-gaianet-node)
        
    *   [Step 2.2 Run the Translation Agent to run on top of gemma-2-27b](#step-22-run-the-translation-agent-to-run-on-top-of-gemma-2-27b)
        
*   [Demo 3: Running Translation Agents with Phi-3-Medium long context model](#demo-3-running-translation-agents-with-phi-3-medium-long-context-model)
    *   [Step 3.1: Run a Phi-3-medium-128k GaiaNet node](#step-31-run-a-phi-3-medium-128k-gaianet-node)
        
    *   [Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k](#step-32-clone-and-run-the-translation-agent-on-top-of-phi-3-medium-128k)
        
*   [Evaluation of Translation Quality](#evaluation-of-translation-quality)
    
*   [Conclusion](#conclusion)

---

# API Reference | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Introduction[​](#introduction "Direct link to Introduction")

-------------------------------------------------------------

Each GaiaNet node is an OpenAI compatible API server. You can build your application based on the GaiaNet node API. You can also replace OpenAI API configuration with the GaiaNet node API in other AI agent frameworks.

The base URL to send all API requests is `https://node_id.gaianet.network/v1`.

Endpoints[​](#endpoints "Direct link to Endpoints")

----------------------------------------------------

### Chat[​](#chat "Direct link to Chat")

The `chat/completions` endpoint returns an LLM response based on the system prompt and user query.

#### Non-streaming[​](#non-streaming "Direct link to Non-streaming")

By default, the API responds with a full answer in the HTTP response.

**Request**

    curl -X POST https://node_id.gaianet.network/v1/chat/completions \  -H 'accept:application/json' \  -H 'Content-Type: application/json' \  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the capital of France?"}], "model": "model_name"}'

**Response:**

    {"id":"chatcmpl-bcfeebe0-5372-42c0-ac92-0615213e1c97","object":"chat.completion","created":1716380086,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","choices":[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish_reason":"stop"}],"usage":{"prompt_tokens":61,"completion_tokens":4,"total_tokens":65}}%  

#### streaming[​](#streaming "Direct link to streaming")

Add `"stream":true` in your request to make the API send back partial responses as the LLM generates its answer.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/chat/completions \  -H 'accept:application/json' \  -H 'Content-Type: application/json' \  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the capital of France?"}], "model": "model_name", "stream":true}'

**Response:**

    data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":"I"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" am"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" a"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}...data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" an"},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" AI"},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"}data: [DONE]

#### Request body[​](#request-body "Direct link to Request body")

| Field | Type | Required | Description | Default | Example |
| --- | --- | --- | --- | --- | --- |
| messages | List | Required | A list of messages for the conversation.  <br>1 . System message (depends on the large language mode you use)  <br>\* `content` of the system messages is required  <br>\* `"role":"system"` is required  <br>2\. User message (required)  <br>\* `content` is required.  <br>\* `"role":"user"` is required | N/A | "messages": \["role": "system","content": "You are a helpful assistant."},{"role": "user",  <br>"content": "Hello!"}\] |
| model | String | Required | The chat model you used | N/A | Llama-3-8B-262k-Q5\_K\_M |
| top\_p | Number | Optional | An alternative to sampling with temperature. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | 1   | Number between 0 and 1. |
| temperature | Number | Optional | Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | 1   | Number between 0 and 2. |
| presence\_penalty | Number | Optional | Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. | 0   | Number between -2.0 and 2.0. |
| stream | boolean | Optional | Make the answer streaming output | FALSE | "stream":true |
| frequency\_penalty | Number | Optional | Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood of repeating the same line verbatim. | 0   | Number between -2.0 and 2.0. |

#### Response body[​](#response-body "Direct link to Response body")

| Field | Type | Streaming or non-streaming | Description | Default | Example |
| --- | --- | --- | --- | --- | --- |
| id  | string | Both | A unique identifier for the chat completion. | Generated randomly | chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4 |
| object | string | Both | The object type | `chat.completion.chunk` in the streaming mode.  <br>`chat.completion` in the non-streaming mode. | `chat.completion.chunk` in the streaming mode.  <br>`chat.completion` in the non-streaming mode. |
| choices | array | Both | A list of chat completion choices. |     | "choices":\[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish\_reason":"stop"}\] |
| created | integer | Both | The Unix timestamp (in seconds) of when the chat completion was created. | N/A | 1716380086 |
| model | string | Both | The model used for the chat completion. | Depends on the model you use. | Llama-3-8B-Instruct-Q5\_K\_M |
| usage | object | Both | Usage statistics for the completion request, including completion\_tokens, prompt\_tokens, and total\_tokens. | N/A | "usage":{"prompt\_tokens":61,"completion\_tokens":4,"total\_tokens":65} |

### Embedding[​](#embedding "Direct link to Embedding")

The `embeddings` endpoint computes embeddings for user queries or file chunks.

**Request**

    curl -X POST https://node_id.gaianet.network/v1/embeddings \    -H 'accept:application/json' \    -H 'Content-Type: application/json' \    -d '{"model": "nomic-embed-text-v1.5.f16", "input":["Paris, city and capital of France, ..., for Paris has retained its importance as a centre for education and intellectual pursuits.", "Paris’s site at a crossroads ..., drawing to itself much of the talent and vitality of the provinces."]}'

**Response:**

    {    "object": "list",    "data": [        {            "index": 0,            "object": "embedding",            "embedding": [                0.1428378969,                -0.0447309874,                0.007660218049,                ...                -0.0128974719,                -0.03543198109,                0.03974733502,                0.00946635101,                -0.01531364303            ]        },        {            "index": 1,            "object": "embedding",            "embedding": [                0.0697753951,                -0.0001159032545,                0.02073983476,                ...                0.03565846011,                -0.04550019652,                0.02691745944,                0.02498772368,                -0.003226313973            ]        }    ],    "model": "nomic-embed-text-v1.5.f16",    "usage": {        "prompt_tokens": 491,        "completion_tokens": 0,        "total_tokens": 491    }}

### Retrieve[​](#retrieve "Direct link to Retrieve")

The `retrieve` endpoint can retrieve text from the node's vector collection based on the user's query.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/retrieve \    -H 'accept:application/json' \    -H 'Content-Type: application/json' \    -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the location of Paris?"}], "model":"nomic-embed-text-v1.5.f16"}'

**Response:**

    {    "points": [        {            "source": "\"Paris is located in northern central France, in a north-bending arc of the river Seine whose crest includes two islands, the Île Saint-Louis and the larger Île de la Cité, which form the oldest part of the city. The river's mouth on the English Channel is about 233 mi downstream from the city. The city is spread widely on both banks of the river. Overall, the city is relatively flat, and the lowest point is 35 m above sea level. Paris has several prominent hills, the highest of which is Montmartre at 130 m.\\n\"",            "score": 0.74011195        },        {            "source": "\"The Paris region is the most active water transport area in France, with most of the cargo handled by Ports of Paris in facilities located around Paris. The rivers Loire, Rhine, Rhône, Me\\n\"",            "score": 0.63990676        },        {            "source": "\"Paris\\nCountry\\tFrance\\nRegion\\nÎle-de-France\\r\\nDepartment\\nParis\\nIntercommunality\\nMétropole du Grand Paris\\nSubdivisions\\n20 arrondissements\\nGovernment\\n • Mayor (2020–2026)\\tAnne Hidalgo (PS)\\r\\nArea\\n1\\t105.4 km2 (40.7 sq mi)\\n • Urban\\n (2020)\\t2,853.5 km2 (1,101.7 sq mi)\\n • Metro\\n (2020)\\t18,940.7 km2 (7,313.0 sq mi)\\nPopulation\\n (2023)\\n2,102,650\\n • Rank\\t9th in Europe\\n1st in France\\r\\n • Density\\t20,000/km2 (52,000/sq mi)\\n • Urban\\n (2019)\\n10,858,852\\n • Urban density\\t3,800/km2 (9,900/sq mi)\\n • Metro\\n (Jan. 2017)\\n13,024,518\\n • Metro density\\t690/km2 (1,800/sq mi)\\nDemonym(s)\\nParisian(s) (en) Parisien(s) (masc.), Parisienne(s) (fem.) (fr), Parigot(s) (masc.), \\\"Parigote(s)\\\" (fem.) (fr, colloquial)\\nTime zone\\nUTC+01:00 (CET)\\r\\n • Summer (DST)\\nUTC+02:00 (CEST)\\r\\nINSEE/Postal code\\t75056 /75001-75020, 75116\\r\\nElevation\\t28–131 m (92–430 ft)\\n(avg. 78 m or 256 ft)\\nWebsite\\twww.paris.fr\\r\\n1 French Land Register data, which excludes lakes, ponds, glaciers > 1 km2 (0.386 sq mi or 247 acres) and river estuaries.\\n\"",            "score": 0.62259054        },        {            "source": "\" in Paris\\n\"",            "score": 0.6152092        },        {            "source": "\"The Parisii, a sub-tribe of the Celtic Senones, inhabited the Paris area from around the middle of the 3rd century BC. One of the area's major north–south trade routes crossed the Seine on the île de la Cité, which gradually became an important trading centre. The Parisii traded with many river towns (some as far away as the Iberian Peninsula) and minted their own coins.\\n\"",            "score": 0.5720232        }    ],    "limit": 5,    "score_threshold": 0.4}

### Get the model[​](#get-the-model "Direct link to Get the model")

The `models` endpoint provides the chat and embedding models available on the node.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/models

**Response:**

    {"object":"list","data":[{"id":"Llama-3-8B-Instruct-262k-Q5_K_M","created":1716383261,"object":"model","owned_by":"Not specified"},{"id":"nomic-embed-text-v1.5.f16","created":1716383261,"object":"model","owned_by":"Not specified"}]}%   

### Get the node info[​](#get-the-node-info "Direct link to Get the node info")

The `info` endpoint provides detailed information about the node.

**Request:**

    curl -X POST https://node_id.gaianet.network/v1/info

**Response:**

    {    "version": "0.5.0",    "plugin_version": "b2694 (commit 0d56246f)",    "port": "8080",    "models": [        {            "name": "Llama-2-7b-chat-hf-Q5_K_M",            "type": "chat",            "prompt_template": "Llama2Chat",            "n_predict": 1024,            "n_gpu_layers": 100,            "ctx_size": 4096,            "batch_size": 512,            "temperature": 1.0,            "top_p": 1.0,            "repeat_penalty": 1.1,            "presence_penalty": 0.0,            "frequency_penalty": 0.0        },        {            "name": "all-MiniLM-L6-v2-ggml-model-f16",            "type": "embedding",            "prompt_template": "Llama2Chat",            "n_predict": 1024,            "n_gpu_layers": 100,            "ctx_size": 384,            "batch_size": 512,            "temperature": 1.0,            "top_p": 1.0,            "repeat_penalty": 1.1,            "presence_penalty": 0.0,            "frequency_penalty": 0.0        }    ],    "qdrant_config": {        "url": "http://localhost:6333",        "collection_name": "default",        "limit": 5,        "score_threshold": 0.4    }} 

Status Codes[​](#status-codes "Direct link to Status Codes")

-------------------------------------------------------------

| HTTP response code | Description | Reason | Solutions |
| --- | --- | --- | --- |
| 404 | Not found | The endpoint URL is invalid | Please check the endpoint URL |
| 500 | Internal Server Error | Model is not found. | Please check out the model name. |
| 400 | Bad request |     |     |

*   [Introduction](#introduction)
    
*   [Endpoints](#endpoints)
    *   [Chat](#chat)
        
    *   [Embedding](#embedding)
        
    *   [Retrieve](#retrieve)
        
    *   [Get the model](#get-the-model)
        
    *   [Get the node info](#get-the-node-info)
        
*   [Status Codes](#status-codes)

---

# Start a node on AWS using AMI images | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

We have created a series of public AMIs for you to start Gaia nodes in AWS with just a few clicks.

Now we have three AMI images available in the Asia Pacific (Osaka) and all the US regions including N. Virginia, Ohio, N. California, and Oregon.

| AMI Images Name | Architecture | Regions |
| --- | --- | --- |
| GaiaNet\_ubuntu22.04\_amd64\_cuda12 | GPU | N. Virginia, Ohio, N. California, Oregon, and Osaka |
| GaiaNet\_ubuntu22.04\_amd64 | x86 CPU machines | N. Virginia, Ohio, N. California, Oregon, and Osaka |
| GaiaNet\_ubuntu22.04\_arm64 | ARM CPU machines | N. Virginia, Ohio, N. California, Oregon, and Osaka |

Running an Nvidia GPU-enabled AWS instance[​](#running-an-nvidia-gpu-enabled-aws-instance "Direct link to Running an Nvidia GPU-enabled AWS instance")

-------------------------------------------------------------------------------------------------------------------------------------------------------

Load the [AWS console](https://aws.amazon.com/console/)
 and sign into your account. Go to EC2 | instances and click on the "Launch instance" button.

In the "Application and OS Images" section, search the AMI catalog and select the image named `GaiaNet_ubuntu22.04_amd64_cuda12`.

![](/assets/images/aws_ami-ac1a273dd1e4ba668951e1780804f6dc.png)

In the "Instance type" section, select any of the `g4dn` types. Those are EC2 VMs with Nvidia T4 GPUs.

![](/assets/images/aws_instance_type-b9d961478e8808a27fe30437e154f496.png)

In the "Network settings", make sure that you allow SSH connections.

![](/assets/images/aws_network-c87d461d165d2d2814df19d1af71e734.png)

Click on the "Launch instance" button and wait for the instance to start up. Once the instance is ready, SSH into its public IP address. Once you are in the VM, run the following two commands.

    gaianet initgaianet start

The node is ready when it shows `The Gaia node is started at: https://...` on the console. You can go to that URL from your browser to interact with the Gaia node.

You can [customize your Gaia node](/getting-started/customize)
 with your own choice of LLMs and knowledge base snapshots.

Running a CPU-only AWS instance[​](#running-a-cpu-only-aws-instance "Direct link to Running a CPU-only AWS instance")

----------------------------------------------------------------------------------------------------------------------

Load the [AWS console](https://aws.amazon.com/console/)
 and sign into your account. Go to EC2 | instances and click on the "Launch instance" button.

In the "Application and OS Images" section, search the AMI catalog and select the image named

*   `GaiaNet_ubuntu22.04_amd64` for x86 CPU machines
*   `GaiaNet_ubuntu22.04_arm64` for ARM CPU machines

In the "Instance type" section, select an instance with at least 8GB of RAM. For example, we recommend `t2.large` or `t2.xlarge` instances.

In the "Network settings", make sure that you allow SSH connections.

Click on the "Launch instance" button and wait for instance to start up. Once the instance is ready, SSH into its public IP address. Once you are in the VM, run the following two commands.

    gaianet initgaianet start

The node is ready when it shows `The Gaia node is started at: https://...` on the console. You can go to that URL from your browser to interact with the Gaia node.

You can [customize your Gaia node](/getting-started/customize)
 with your own choice of LLMs and knowledge base snapshots.

*   [Running an Nvidia GPU-enabled AWS instance](#running-an-nvidia-gpu-enabled-aws-instance)
    
*   [Running a CPU-only AWS instance](#running-a-cpu-only-aws-instance)

---

# Calling external tools | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Tool calling is one of the truly "LLM native" interaction modes that has never existed before. It gives the "thinking" LLMs the ability to "act" -- both in acquiring new knowledge and in performing real world actions. It is a crucial part of any agentic application.

Open source LLMs are increasingly good at using tools. The Llama 3 models have now made it possible to have reliable tool calling performance on 8b class of LLMs running on your own laptop!

In this tutorial, we will show you a simple Python program that allows a local LLM to run code and manipulate data on the local computer!

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    . You will need to start a Gaia node for the [Llama-3-Groq model](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool)
     or the [Mistral-7B-v0.3 Instruct model](https://github.com/GaiaNet-AI/node-configs/tree/main/mistral-0.3-7b-instruct-tool-call)
     . You can then use the node's API URL endpoint and model name in your tool call apps.
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use a public Llama3 node with the function call support.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llamatool.us.gaianet.network/v1](https://llamatool.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the demo agent[​](#run-the-demo-agent "Direct link to Run the demo agent")

-------------------------------------------------------------------------------

The [agent app](https://github.com/second-state/llm_todo)
 is written in Python. It demonstrates how the LLM could use tools to operate a SQL database. In this case, it starts and operates an in-memory SQLite database. The database stores a list of todo items.

Download the code and install the Python dependencies as follows.

    git clone https://github.com/second-state/llm_todocd llm_todopip install -r requirements.txt

Set the environment variables for the API server and model name we just set up.

    export OPENAI_MODEL_NAME="llama"export OPENAI_BASE_URL= "https://llamatool.us.gaianet.network/v1"

Run the `main.py` application and bring up the command line chat interface.

    python main.py

Use the agent[​](#use-the-agent "Direct link to Use the agent")

----------------------------------------------------------------

Now, you can ask the LLM to perform tasks. For example, you can say

    User: Help me to write down it I'm going to have a meeting with the marketing team.

The LLM understands that you need to insert a record into the database and returns a tool call response in JSON.

    Assistant:<tool_call>{"id": 0, "name": "create_task", "arguments": {"task": "have a meeting with the marketing team"}}</tool_call>

The agent app (i.e., `main.py`) executes the tool call `create_task` in the JSON response, and sends back the results as role `Tool`. You do not need to do anything here as it happens automatically in `main.py`. The SQLite database is updated when the agent app executes the tool call.

    Tool:[{'result': 'ok'}]

The LLM receives the execution result and then answers you.

    Assistant:I've added "have a meeting with the marketing team" to your task list. Is there anything else you'd like to do?

You can continue the conversation.

To learn more about how tool calling works, see [this article](https://github.com/LlamaEdge/LlamaEdge/blob/main/api-server/ToolUse.md)
.

Make it robust[​](#make-it-robust "Direct link to Make it robust")

-------------------------------------------------------------------

One of the major challenges for LLM applications is the frequent unreliability of their responses. For example:

_If the LLM generates an incorrect tool call that fails to address the user’s query,_

you can refine and optimize the descriptions for each tool call function. The LLM chooses its tools based on these descriptions, so it's vital to craft them in a way that aligns with typical user queries.

If the LLM hallucinates and produces tool calls with non-existent function names or incorrect parameters,

_the agent app should identify this issue and prompt the LLM to create a new response._

Tool calling is a fundamental feature in the evolving field of agentic LLM applications. We’re eager to see the innovative ideas you bring forward!

*   [Prerequisites](#prerequisites)
    
*   [Run the demo agent](#run-the-demo-agent)
    
*   [Use the agent](#use-the-agent)
    
*   [Make it robust](#make-it-robust)

---

# System requirements | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

You can install the GaiaNet on a wide variety of devices and operating systems with or without GPUs. The node installing and operating instructions work on devices ranging from Raspberry Pi, MacBooks, Linux servers, Windows Desktop, to cloud-based Nvidia H100 clusters. For institutional operators, we recommend EITHER of the following for a GaiaNet node.

*   Mac desktop or server computers (i.e., iMac, Mini, Studio or Pro) with Apple Silicon (M1 to M4), and at least 16GB of RAM (32GB or more recommended).
*   Ubuntu Linux 22.04 server with NVIDIA CUDA 12 SDK installed. At least 8GB of VRAM on the GPU is required (24GB or more recommended). On AWS and Azure, that means GPU instances with at least the Nvidia T4 GPU installed.

> Check out our [tutorial](/1.0.0/node-guide/tasks/cuda)
>  on how to install the NVIDIA driver and the CUDA toolkit on a Ubuntu 22.04 machine.

If you are hosting the node in your home or office, it needs access to the public Internet to join the GaiaNet network.

Supported on[​](#supported-on "Direct link to Supported on")

-------------------------------------------------------------

GaiaNet node software is designed to be cross-platform, allowing it to run on various CPU and GPU architectures. The GaiaNet installer automatically detects the presence of NVIDIA CUDA drivers and leverages the power of GPU accelerators on the device. More hardware support is on the way.

### GPU[​](#gpu "Direct link to GPU")

The GaiaNet node can run on all types of NVIDIA GPU products from H100 to NVIDIA Jetson series of hardware. It also runs on all Apple Silicon M-series GPUs.

### CPU[​](#cpu "Direct link to CPU")

*   Arm-64 based on CPU chips
*   X86 based on CPU chips
*   Apple M1 chips
*   Apple M1 Pro chips
*   Apple M1 Max chips
*   Apple M1 Ultra chips
*   Apple M2 chips
*   Apple M2 Pro chips
*   Apple M2 Max chips
*   Apple M2 Ultra chips
*   Apple M3 chips
*   Apple M3 Pro chips
*   Apple M3 Max chips
*   Apple M3 Ultra chips

### Oses[​](#oses "Direct link to Oses")

*   macOS
*   Linux-like OS

*   [Supported on](#supported-on)
    *   [GPU](#gpu)
        
    *   [CPU](#cpu)
        
    *   [Oses](#oses)

---

# GaiaNet CLI options | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

After installing the GaiaNet software, you can use the `gaianet` CLI to manage the node. The following are the CLI options.

help[​](#help "Direct link to help")

-------------------------------------

You can use `gaianet --help` to check all the available CLI options.

    gaianet --help## OutputUsage: gaianet {config|init|run|stop|OPTIONS}Subcommands:  config             Update the configuration.  init               Initialize the GaiaNet node.  run|start          Start the GaiaNet node.  stop               Stop the GaiaNet node.Options:  --help             Show this help message

version[​](#version "Direct link to version")

----------------------------------------------

You can use `gaianet --version` to check your GaiaNet version.

    gaianet --version

init[​](#init "Direct link to init")

-------------------------------------

The `gaianet init` command initializes the node according to the `$HOME/gaianet/config.json` file. You can use some of our [pre-set configurations](https://github.com/GaiaNet-AI/node-configs)
.

*   `gaianet init` will init the default node. It's an RAG application with Gaianet knowledge.
*   `gaianet init --config mua` will init a node with the MUA project knowledge.
*   `gaianet init --base <path>` will init a node in an alternative directory.

You can also use `gaianet init url_your_config_json` to init your customized settings for the node. You can customize your node using the Gaianet node link. If you're familiar with the Gaianet config.json, you can create your own manually. See an example [here](https://github.com/GaiaNet-AI/gaianet-node/blob/main/config.json)
.

    gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/pure-llama-3-8b/config.json

start[​](#start "Direct link to start")

----------------------------------------

The `gaianet start` is to start running the node.

*   Use `gaianet start` to start the node according to the `$HOME/gaianet/config.json` file.
*   Use `gaianet start --base $HOME/gaianet-2.alt` to start the node according to the `$HOME/gaianet-2/config.json` file.
*   Use `gaianet start --local-only` to start the node for local use according to the `$HOME/gaianet/config.json` file.

stop[​](#stop "Direct link to stop")

-------------------------------------

The `gaianet stop` is to stop the running node.

*   Use `gaianet stop` to stop running the node.
*   Use `gaianet stop --force` to force stop the GaiaNet node.
*   Use `gaianet stop --base $HOME/gaianet-2.alt` to stop the node according to the `$HOME/gaianet-2/config.json` file.

config[​](#config "Direct link to config")

-------------------------------------------

The `gaianet config` can update the key fields defined in the `config.json` file.

*   `gaianet config --help` will list all the available arguments
*   `gaianet config --chat-url <url>` will change the download link of the chat model.
*   `gaianet config --prompt-template <template>` will change the prompt\_template of the chat model.
*   `gaianet config --chat-ctx-size <size>` will change the context size of the chat model. The default value is 4096.
*   `gaianet config --embedding-url <url>` will change the download link of the embedding model.
*   `gaianet config --embedding-ctx-size <size>` will change the context size of the embedding model. The value here is associated with the embedding model you choose.
*   `gaianet config --port <port>` will change the port of the Gaianet node API server.
*   `gaianet config --system-prompt "<prompt>"` will change the system prompt.
*   `gaianet config --rag-prompt "<prompt>"` will change the rag prompt.
*   `gaianet config --reverse-prompt "<prompt>"` will change the reverse prompt.
*   `gainet config --base <path>` will modify the `<path>/config.json` parameters.

After you use `gaianet conifg` to change some parameters, please

1.  use `gaianet init` to make your settings take effect.
2.  use `gaianet start` to start your new node.

If you use `gaianet config --base $HOME/gaianet-2.alt` to update some settings, please

1.  use `gaianet init --base $HOME/gaianet-2.alt` to make your settings take effect.
2.  use `gaianet start --base $HOME/gaianet-2.alt` to start your new node.

The `gaianet config` supports multiple parameters in one command. The example below will change the download link and prompt template of the chat model at the same time.

    gaianet config --chat-url https://huggingface.co/gaianet/gemma-1.1-2b-it-GGUF/resolve/main/gemma-1.1-2b-it-Q5_K_M.gguf --prompt-template gemma-chat

The output is the following.

    [+] Updating the url of chat model ...    * Old url: https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf    * New url: https://huggingface.co/gaianet/gemma-1.1-2b-it-GGUF/resolve/main/gemma-1.1-2b-it-Q5_K_M.gguf[+] Updating the prompt template of chat model ...    * Old template: llama-3-chat    * New template: gemma-chat[+] COMPLETED! The config.json is updated successfully.

base[​](#base "Direct link to base")

-------------------------------------

The `--base` option is global. You can combine it with other subcommands to specify a base directory for the GaiaNet node other than the `$HOME/gaianet`.

*   [help](#help)
    
*   [version](#version)
    
*   [init](#init)
    
*   [start](#start)
    
*   [stop](#stop)
    
*   [config](#config)
    
*   [base](#base)

---

# Start a node with Docker | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 2.0.0

On this page

You can run all the commands in this document without any change on any machine with the latest Docker and at least 8GB of RAM available to the container. By default, the container uses the CPU to perform computations, which could be slow for large LLMs. For GPUs,

*   Mac: Everything here works on [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/)
    . However, the Apple GPU cores will not be available inside Docker containers until [WebGPU is supported by Docker](https://github.com/LlamaEdge/LlamaEdge/blob/main/docker/webgpu.md)
     later in 2024.
*   Windows and Linux with Nvidia GPU: You will need to install [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installation)
     for Docker. In the instructions below, replace the `latest` tag with `cuda12` or `cuda11` to use take advantage of the GPU, and add the `--device nvidia.com/gpu=all` flag. If you need to build the images yourself, replace `Dockerfile` with `Dockerfile.cuda12` or `Dockerfile.cuda11`.

Find [Gaia Docker images](https://hub.docker.com/?namespace=gaianet)
 you can run!

Quick start[​](#quick-start "Direct link to Quick start")

----------------------------------------------------------

Start a Docker container for the Gaia node. It will print running logs from the Gaia node in this terminal.

    docker run --name gaianet \  -p 8080:8080 \  -v $(pwd)/qdrant_storage:/root/gaianet/qdrant/storage:z \  gaianet/phi-3-mini-instruct-4k_paris:latest

The node is ready when it shows `The Gaia node is started at: https://...` on the console. You can go to that URL from your browser to interact with the Gaia node.

The docker image contains the LLM and embedding models required by the node. However, the vector collection snapshot (i.e., knowledge base) is downloaded and imported at the time when the node starts up. That is because the knowledge based could be updated frequently. The `qdrant_storage` directory on the host machine stores the vector database content.

Alternatively, the command to run the Gaia on your Nvidia CUDA 12 machine is as follows.

    docker run --name gaianet \  -p 8080:8080 --device nvidia.com/gpu=all \  -v $(pwd)/qdrant_storage:/root/gaianet/qdrant/storage:z \  gaianet/phi-3-mini-instruct-4k_paris:cuda12

Stop and re-start[​](#stop-and-re-start "Direct link to Stop and re-start")

----------------------------------------------------------------------------

You can stop and re-start the node as follows. Every time you re-start, it will re-initailize the vector collection (knowledge base).

    docker stop gaianetdocker start gaianet

NOTE: When you restart the node, the log messages will no longer be printed to the console. You will need to wait for a few minutes before the restarted node comes back online. You can still see the logs by logging into the container as follows.

    docker exec -it gaianet /bin/bashtail -f /root/gaianet/log/start-llamaedge.log

You can also delete the node if you no longer needs it.

    docker stop gaianetdocker rm gaianet

Make changes to the node[​](#make-changes-to-the-node "Direct link to Make changes to the node")

-------------------------------------------------------------------------------------------------

You can update the configuration parameters of the node, such as context size for the models, by executing the `config` command on the `gaianet` program inside the container. For example, the following command changes the chat LLM's context size to 8192 tokens.

    docker exec -it gaianet /root/gaianet/bin/gaianet config --chat-ctx-size 8192

Then, restart the node for the new configuration to take effect. You will need to wait for a few minutes for the server to start again, or you can monitor the log files inside the container as discussed above.

    docker stop gaianetdocker start gaianet

Change the node ID[​](#change-the-node-id "Direct link to Change the node ID")

-------------------------------------------------------------------------------

You can update the node ID (Ethereum address) associated with the node. Start the node and copy the `nodeid.json` file, as well as the keystore file defined in `nodeid.json` into the container.

    docker cp /local/path/to/nodeid.json gaianet:/root/gaianet/nodeid.jsondocker cp /local/path/to/1234-abcd-key-store gaianet:/root/gaianet/1234-abcd-key-store

THen, restart the node for the new address and keystore to take effect.

    docker stop gaianetdocker start gaianet

Build a node image locally[​](#build-a-node-image-locally "Direct link to Build a node image locally")

-------------------------------------------------------------------------------------------------------

Each Gaia is defined by a `config.json` file. It defines the node's required LLM and embedding models, model parameters, prompts, and vector snapshots (e.g., knowledge base). The following command builds a Docker image with two platforms for a node based on the specified `config.json` file.

    docker buildx build . --platform linux/arm64,linux/amd64 \  --tag gaianet/phi-3-mini-instruct-4k_paris:latest -f Dockerfile \  --build-arg CONFIG_URL=https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/config.json

> The `Dockerfile` is available [here](https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/docker/Dockerfile)
> . Feel free to change it to Nvidia [CUDA versions](https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/docker/Dockerfile.cuda12)
>  if your Docker is enabled with the [Nvidia container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
> .

You can publish your node for other people to use it.

    docker push gaianet/phi-3-mini-instruct-4k_paris:latest

*   [Quick start](#quick-start)
    
*   [Stop and re-start](#stop-and-re-start)
    
*   [Make changes to the node](#make-changes-to-the-node)
    
*   [Change the node ID](#change-the-node-id)
    
*   [Build a node image locally](#build-a-node-image-locally)

---

# Install and uninstall | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The GaiaNet node utilizes version control from [its source GitHub repo](https://github.com/GaiaNet-AI/gaianet-node)
. You can check out the GaiaNet node versions from [the release page](https://github.com/GaiaNet-AI/gaianet-node/releases)
.

Install[​](#install "Direct link to Install")

----------------------------------------------

You can install the WasmEdge Runtime on any generic Linux and MacOS platform.

### Install the latest version of GaiaNet node[​](#install-the-latest-version-of-gaianet-node "Direct link to Install the latest version of GaiaNet node")

To install the most recent version of GaiaNet node, run the following command line.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash

The GaiaNet node will be installed in your `$HOME/gaianet` folder by default.

> If you want to install gaianet in a different directory, you can use `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --base <Full_Path>` to specify where you want to install GaiaNet. Once you use `--base` to define a different directory, you should always add `--base <Full_Path>` to init and start your node. Here is an example:
> 
>     # Assume that you're in the root directorymkdir testcurl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s --  --base $HOME/testgaianet init --base $HOME/testgaianet start --base $HOME/testgaianet stop --base $HOME/test

### Install the specific version of GaiaNet Node[​](#install-the-specific-version-of-gaianet-node "Direct link to Install the specific version of GaiaNet Node")

If you want to install a particular GaiaNet node version, change the version number in the following command line.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/download/0.1.3/install.sh' | bash

Check out the release log [here](https://github.com/GaiaNet-AI/gaianet-node/releases)
.

Update the current Gaianet node[​](#update-the-current-gaianet-node "Direct link to Update the current Gaianet node")

----------------------------------------------------------------------------------------------------------------------

Simply run the following command to upgrade your node.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --upgrade

The `upgrade` option will keep your node id.

Uninstall[​](#uninstall "Direct link to Uninstall")

----------------------------------------------------

To uninstall or clear the environment, run the following command.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/uninstall.sh' | bash

**Important reminder: This command will remove all the GaiaNet-related files, including the `nodeid.json`. It's your responsibility to keep your nodeid.json safe. If you want to run the same node after reinstalling, please save the `nodeid.json` file and `frpc.toml` file carefully.**

What's installed[​](#whats-installed "Direct link to What's installed")

------------------------------------------------------------------------

If you install the GaiaNet node in the `$HOME/gaianet` directory by default, you will have the following directories and files after installation:

*   The `$HOME/gaianet/bin` directory contains the GaiaNet CLI tool, frpc binary and Qdrant Vector database binary.
*   The `$HOME/gaianet/` directory contains the `llamaedge-api-server.wasm` and `rag-api-server.wasm` for the LLM inference, dashboard (chatbot ui), nodeid.json for the registering your node, and gaianet-domain binary.
*   The `$HOME/.wasmedge/bin` directory contains the WasmEdge Runtime CLI executable files, which serve as the LLM runtime.

CLI options for the installer[​](#cli-options-for-the-installer "Direct link to CLI options for the installer")

----------------------------------------------------------------------------------------------------------------

You can use the following command line to check out all the available CLI options

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --help

The output should be as follows. You can use the following options to customize your installation.

    Usage:  ./install.sh [Options]Options:  --config <Url>     Specify a url to the config file  --base <Path>      Specify a path to the gaianet base directory  --reinstall        Install and download all required deps  --tmpdir <Path>    Specify a path to the temporary directory [default: /tmp]  --ggmlcuda [11/12] Install a specific CUDA enabled GGML plugin version [Possible values: 11, 12].  --enable-vector:   Install vector log aggregator  --version          Print version  --help             Print usage

*   [Install](#install)
    *   [Install the latest version of GaiaNet node](#install-the-latest-version-of-gaianet-node)
        
    *   [Install the specific version of GaiaNet Node](#install-the-specific-version-of-gaianet-node)
        
*   [Update the current Gaianet node](#update-the-current-gaianet-node)
    
*   [Uninstall](#uninstall)
    
*   [What's installed](#whats-installed)
    
*   [CLI options for the installer](#cli-options-for-the-installer)

---

# Troubleshooting | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The system cannot find CUDA libraries[​](#the-system-cannot-find-cuda-libraries "Direct link to The system cannot find CUDA libraries")

----------------------------------------------------------------------------------------------------------------------------------------

Sometimes, the CUDA toolkit is installed in a non-standard location. The error message here is often not able to find `libcu*12`. For example, you might have CUDA installed with your Python setup. The following command would install CUDA into Python's environment.

    sudo apt install python3-pip -ypip3 install --upgrade fschat accelerate autoawq vllm

The easiest way to fix is simply to link those non-standard CUDA libraries to the standard location, like this.

    ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cublas/lib/libcublas.so.12 /usr/lib/libcublas.so.12ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cuda_runtime/lib/libcudart.so.12 /usr/lib/libcudart.so.12ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cublas/lib/libcublasLt.so.12 /usr/lib/libcublasLt.so.12

Failed to recover from collection snapshot on Windows WSL[​](#failed-to-recover-from-collection-snapshot-on-windows-wsl "Direct link to Failed to recover from collection snapshot on Windows WSL")

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

On Windows WSL, you could see this error while running `gaianet init`.

       * Import the Qdrant collection snapshot ...      The process may take a few minutes. Please wait ...    * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Tokio task join error: task 1242 panicked"},"time":0.697784244}

When you look into the `~/gaianet/log/init-qdrant.log` file, you could see this line of error

    2024-05-20T07:24:52.900895Z ERROR qdrant::startup: Panic occurred in file /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cgroups-rs-0.3.4/src/memory.rs at line 587: called `Result::unwrap()` on an `Err` value: Error { kind: ReadFailed("/sys/fs/cgroup/memory.high"), cause: Some(Os { code: 2, kind: NotFound, message: "No such file or directory" }) }  

The solution is to disable the `autoMemoryReclaim` feature in WSL. Step to turn on/off this feature:

1.  Edit `C:\Users<Your user name>.wslconfig`
2.  Remove or comment out `autoMemoryReclaim` in `[experimental]` section.

![](/assets/images/disable_autoMemoryReclaim_wsl-af7a70038bb2a972c29c0e1b2bc44cf8.png)

Thanks to [RoggeOhta](https://github.com/RoggeOhta)
 for discovering this. You can learn more about it [here](https://github.com/GaiaNet-AI/gaianet-node/issues/46)
.

Failed to start the node with an error message `Port 8080 is in use. Exit ...`[​](#failed-to-start-the-node-with-an-error-message-port-8080-is-in-use-exit- "Direct link to failed-to-start-the-node-with-an-error-message-port-8080-is-in-use-exit-")

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

You may see the following error when you run `gaianet start`.

    gaianet start[+] Checking the config.json file ...You already have a private key.[+] Starting LlamaEdge API Server ...    Port 8080 is in use. Exit ...

The solution is to run `gaianet stop` first to kill all processes, and then run `gaianet start` to start the node.

Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory[​](#load-library-failed-libgompso1-cannot-open-shared-object-file-no-such-file-or-directory "Direct link to Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory")

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

On Windows WSL, you may see this error when running `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash`

    * Generate node IS[2024-07-02 17:50:55.175] [error] loading failed: invalid path, Code: 0x20[2024-07-02 17:50:55.175] [error]   Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory[2024-07-02 17:50:55.176] [error] loading failed: invalid path, Code: 0x20[2024-07-02 17:50:55.176] [error]   Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory

The error is caused by the lack of `libgomp.so.1`, a library that should be automatically installed on Ubuntu by default.

To solve this, you must install the `libgomp.so.1` library.

    sudo apt-get updatesudo apt-get install libgomp1

If you're using CentOS, you can use

    yum install libgomp

This issue was fixed in `version 0.2.2`.

Failed to remove the default collection[​](#failed-to-remove-the-default-collection "Direct link to Failed to remove the default collection")

----------------------------------------------------------------------------------------------------------------------------------------------

    Failed to remove the default collection. {"status":{"error":"Service internal error: No such file or directory (os error 2)"},"time":0.050924542}

It typically indicates that the Qdrant instance was not shut down properly before you try to init it again with a new snapshot. The solution is to stop the GaiaNet node first.

    gaianet stop

Alternatively, you could manually kill the processes from the terminal or in the OS's Activity Monitor.

    sudo pkill -9 qdrantsudo pkill -9 wasmedgesudo pkill -9 frpc

Then you can run `gaianet init` and then `gaianet start` again.

File I/O error[​](#file-io-error "Direct link to File I/O error")

------------------------------------------------------------------

        * Import the Qdrant collection snapshot ...      The process may take a few minutes. Please wait ...    * [Error] Failed to recover from the collection snapshot. An error occurred processing field `snapshot`: File I/O error: Operation not permitted (os error 1) 

It typically indicates that the Qdrant instance was not shut down properly before you try to init it again with a new snapshot. The solution is to stop the GaiaNet node first.

    gaianet stop

Alternatively, you could manually kill the processes from the terminal or in the OS's Activity Monitor.

    sudo pkill -9 qdrantsudo pkill -9 wasmedgesudo pkill -9 frpc

Then you can run `gaianet init` and then `gaianet start` again.

The "Failed to open the file" Error[​](#the-failed-to-open-the-file-error "Direct link to The "Failed to open the file" Error")

--------------------------------------------------------------------------------------------------------------------------------

    Warning: Failed to open the file Warning: https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolveWarning: /main/consensus/consensus.snapshot: No such file or directorycurl: (23) Failure writing output to destination

The reason for this type of error is a mis-configured `config.json` file. The solution is to delete the comments in `config.json` and re-run the `gaianet init` command.

The "Too many open files" Error on macOS[​](#the-too-many-open-files-error-on-macos "Direct link to The "Too many open files" Error on macOS")

-----------------------------------------------------------------------------------------------------------------------------------------------

When running `gaianet init` to initialize a new node on macOS, you may encounter an error related to snapshot recovery if your snapshot contains a large amount of text. The error message may be the following:

     * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Too many open files (os error 24)"},"time":1.574064833}    * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Too many open files (os error 24)"},"time":1.574064833}

This issue is caused by the default file descriptor (FD) limit on macOS, which is set to a relatively low value of 256.

To resolve this issue, you can increase the default FD limit on your system. To do so, run the following command:

    ulimit -n 10000

This will temporarily set the FD limit to 10,000. Next, use `gaianet init` and `gaianet start` commands in the SAME terminal.

Permission denied when use the installer script to install WasmEdge[​](#permission-denied-when-use-the-installer-script-to-install-wasmedge "Direct link to Permission denied when use the installer script to install WasmEdge")

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

When running `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash` to install GaiaNet node software, you may meet the permission denied error especially installing the WasmEdge runtime.

![](/assets/images/troubleshooting-01-38a832c2c9b16055602256ce4af65948.png)

This error is caused by the lack of `/tmp` write permission. You can use `--tmpdir` to specify where you want to install the WasmEdge runtime. Please note, you will need to have written permission to the `<YOUR_PATH>` folder.

    curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --tmpdir YOUR_PATH

This problem is fixed in `version 0.2.3`.

*   [The system cannot find CUDA libraries](#the-system-cannot-find-cuda-libraries)
    
*   [Failed to recover from collection snapshot on Windows WSL](#failed-to-recover-from-collection-snapshot-on-windows-wsl)
    
*   [Failed to start the node with an error message `Port 8080 is in use. Exit ...`](#failed-to-start-the-node-with-an-error-message-port-8080-is-in-use-exit-)
    
*   [Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory](#load-library-failed-libgompso1-cannot-open-shared-object-file-no-such-file-or-directory)
    
*   [Failed to remove the default collection](#failed-to-remove-the-default-collection)
    
*   [File I/O error](#file-io-error)
    
*   [The "Failed to open the file" Error](#the-failed-to-open-the-file-error)
    
*   [The "Too many open files" Error on macOS](#the-too-many-open-files-error-on-macos)
    
*   [Permission denied when use the installer script to install WasmEdge](#permission-denied-when-use-the-installer-script-to-install-wasmedge)

---

# How do I ... | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

[📄️ Start a node with Docker\
----------------------------\
\
You can run all the commands in this document without any change on any machine with the latest Docker and at least 8GB of RAM available to the container.](/1.0.0/node-guide/tasks/docker)

[📄️ Start a node on AWS using AMI images\
----------------------------------------\
\
We have created a series of public AMIs for you to start GaiaNet nodes in AWS with just a few clicks.](/1.0.0/node-guide/tasks/aws)

[📄️ Install CUDA on Linux\
-------------------------\
\
If you are using an Nvidia-enabled VM instance from a public cloud, you should probably use the VM image provided by the cloud. It typically has the correct versions of Nvidia driver and CUDA toolkit already installed.](/1.0.0/node-guide/tasks/cuda)

[📄️ Install multiple nodes on a single machine\
----------------------------------------------\
\
The default GaiaNet installer installs the node into the $HOME/gaianet base directory.](/1.0.0/node-guide/tasks/multiple)

[📄️ Run a local-only node\
-------------------------\
\
By default, the GaiaNet node registers itself with a GaiaNet domain and is accesible from the public.](/1.0.0/node-guide/tasks/local)

[📄️ Protect the server process\
------------------------------\
\
Sometimes, the OS could kill the wasmedge process on the GaiaNet node if it consumes too much resources. For production](/1.0.0/node-guide/tasks/protect)

---

# Protect the server process | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Sometimes, the OS could kill the `wasmedge` process on the GaiaNet node if it consumes too much resources. For production servers, you should protect the server process.

Use Supervise[​](#use-supervise "Direct link to Use Supervise")

----------------------------------------------------------------

The `supervise` tool can help us monitor the `wasmedge` process, and automatically restart the process in case the process dies or is killed. The `gaianet` CLI will make use of `supervise` automatically if it is installed on the system.

For macOS users, you can install `supervise` via the daemontools tool by running the following command.

    brew install daemontools

For Linux users, please refer to [the installation guide](https://cr.yp.to/daemontools/install.html)
 to install the `daemontools` tool.

Reduce the nice value[​](#reduce-the-nice-value "Direct link to Reduce the nice value")

----------------------------------------------------------------------------------------

If the `supervise` tool is too heavy handed, we could also increase the priority of the `wasmedge` process. The OS will try NOT to kill high priority processes until it absolutely has to. We do that by reducing the `nice` value of the `wasmedge` process.

    sudo renice -n -19 $(cat $HOME/gaianet/llamaedge.pid)

Good luck!

*   [Use Supervise](#use-supervise)
    
*   [Reduce the nice value](#reduce-the-nice-value)

---

# Dify + GaiaNet | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

You can configure the Dify framework using any GaiaNet node as the backend LLM API. That allows you to use your own or community GaiaNet nodes in any application built on Dify. It supports

*   The hosted [Dify.ai](https://dify.ai/)
     service.
*   Products and services with embedded Dify framework, such as the [Terminus](https://www.jointerminus.com/)
     project.
*   Any product that is built on the open source [Dify framework](https://github.com/langgenius/dify)
    .

Steps[​](#steps "Direct link to Steps")

----------------------------------------

First, log into Dify's web portal and select `Settings | Model Provider`. From the list, you can add an OpenAI-API-compatible provider.

Add an LLM model with the modela name and API endpoint listed on your GaiaNet node's web dashboard. Or, you can just add [a popular GaiaNet node](/1.0.0/user-guide/nodes)
. Leave the API Key field empty.

![Configure a GaiaNet Llama3 8b model in Dify](/assets/images/dify_chat-5ed526b07c37f2ec1eac1628ba724b77.png)

Most Dify applications also require an embedding model to search text in the vector space. Add an mebedding model with the modela name and API endpoint listed on your GaiaNet node's web dashboard. Or, you can just add [a popular GaiaNet node](/1.0.0/user-guide/nodes)
. Leave the API Key field empty.

![Configure a GaiaNet embedding model in Dify](/assets/images/dify_embedding-4e1036868a26c71ea5da20fb1eb8b819.png)

That's it. You can now see that the new models are available at the top panel of Dify for every chatbot or agent. Just select your GaiaNet models for chat or embedding, and the Dify app will automagically use it!

![Select a GaiaNet node as backend model in Dify](/assets/images/dify_select-ca2df00e17b8049965bd0543bff1c3be.png)

![Chat with the GaiaNet Llama3 8b model in Dify](/assets/images/dify_chatbot_ui-eabc342c591397b0f2c4746bfd64e09a.png)

*   [Steps](#steps)

---

# Anything LLM | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Anything LLM is the all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities. You can configure Anything LLM using the GaiaNet node as the LLM backend.

It's recommended to start a node without any snapshots, like [this one](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-8b-instruct)
.

Steps[​](#steps "Direct link to Steps")

----------------------------------------

First, we will need to add GaiaNet Node as the LLM chat model and embedding model.

*   Go to Settings
*   Choose Local AI as the LLM provider in the LLM inference section. Then copy and paste the GaiaNet node API base URL in the LocalAI Base URL. The chat model and embedding model along with your node will be loaded automatically. Choose the chat model here and input the context length. Remember to click on Save Changes to make the changes take effect.

![](/assets/images/anything-llm-01-f6c30d16fad64e2e6d8d2c0735914a25.png)

*   Then go to the Embedding Preference section and choose LocalAI as the embedding provider. Then copy and paste the GaiaNet node API base URL in the LocalAI Base URL. The chat model and embedding model along with your node will be loaded automatically. Choose the embedding model here and input the max embedding chunk length. Don't forget to click on Save Changes to make the changes take effect.

The above operations make the GaiaNet node as the LLM backend.

Second, let's set up the data source. You can upload a file or use the data connectors provided by Anything LLM.

![](/assets/images/anything-llm-02-3764f4ddb2aeef48cbd11eead1a2aa57.png)

When you move a data source to the workspace, Anything LLM will call the GaiaNet node's embedding API to chunk and compute embeddings for your documentation, which may take some minutes.

That's it. When the embedding is done, go back to your workspace and ask a question.

![](/assets/images/anything-llm-03-3630c1f82a45461237e6a1fcf825a8d3.png)

> You can check out the `start-llamaedge.log` to check what happens.

*   [Steps](#steps)

---

# llama.cpp | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The popular llama.cpp tool comes with a `finetune` utility. It works well on CPUs! This fine-tune guide is reproduced with permission from Tony Yuan's [Finetune an open-source LLM for the chemistry subject](https://github.com/YuanTony/chemistry-assistant/tree/main/fine-tune-model)
 project.

Build the fine-tune utility from llama.cpp[​](#build-the-fine-tune-utility-from-llamacpp "Direct link to Build the fine-tune utility from llama.cpp")

------------------------------------------------------------------------------------------------------------------------------------------------------

The `finetune` utility in llama.cpp can work with quantized GGUF files on CPUs, and hence dramatically reducing the hardware requirements and expenses for fine-tuning LLMs.

Check out and download the llama.cpp source code.

    git clone https://github.com/ggerganov/llama.cppcd llama.cpp

Build the llama.cpp binary.

    mkdir buildcd buildcmake ..cmake --build . --config Release

If you have NVIDIA GPU and CUDA toolkit installed, you should build llama.cpp with CUDA support.

    mkdir buildcd buildcmake .. -DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcccmake --build . --config Release

Get the base model[​](#get-the-base-model "Direct link to Get the base model")

-------------------------------------------------------------------------------

We are going to use Meta's Llama2 chat 13B model as the base model. Note that we are using a Q5 quantized GGUF model file directly to save computing resources. You can use any of the Llama2 compatible GGUF models on Hugging Face.

    cd .. # change to the llama.cpp directorycd models/curl -LO https://huggingface.co/gaianet/Llama-2-13B-Chat-GGUF/resolve/main/llama-2-13b-chat.Q5_K_M.gguf

Create a question and answer set for fine-tuning[​](#create-a-question-and-answer-set-for-fine-tuning "Direct link to Create a question and answer set for fine-tuning")

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Next we came up with 1700+ pairs of QAs for the chemistry subject. It is like the following in a [CSV file](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.csv)
.

| Question | Answer |
| --- | --- |
| What is unique about hydrogen? | It's the most abundant element in the universe, making up over 75% of all matter. |
| What is the main component of Jupiter? | Hydrogen is the main component of Jupiter and the other gas giant planets. |
| Can hydrogen be used as fuel? | Yes, hydrogen is used as rocket fuel. It can also power fuel cells to generate electricity. |
| What is mercury's atomic number? | The atomic number of mercury is 80 |
| What is Mercury? | Mercury is a silver colored metal that is liquid at room temperature. It has an atomic number of 80 on the periodic table. It is toxic to humans. |

> We used GPT-4 to help me come up many of these QAs.

Then, we wrote a [Python script](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/convert.py)
 to convert each row in the CSV file into a sample QA in the Llama2 chat template format. Notice that each QA pair starts with `<SFT>` as an indicator for the fine-tune program to start a sample. The result [train.txt](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.txt)
 file can now be used in fine-tuning.

Put the [train.txt](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.txt)
 file in the `llama.cpp/models` directory with the GGUF base model.

Finetune![​](#finetune "Direct link to Finetune!")

---------------------------------------------------

Use the following command to start the fine-tuning process on your CPUs. I am putting it in the background so that it can run continuously now. It could take several days or even a couple of weeks depending on how many CPUs you have.

    nohup ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --lora-out lora.bin --train-data train.txt --sample-start '<SFT>' --adam-iter 1024 &

You can check the process every few hours in the `nohup.out` file. It will report the `loss` for each iteration. You can stop the process when the `loss` goes consistently under `0.1`.

**Note 1** If you have multiple CPUs (or CPU cores), you can speed up the fine-tuning process by adding a `-t` parameter to the above command to use more threads. For example, if you have 60 CPU cores, you could do `-t 60` to use all of them.

**Note 2** If your fine-tuning process is interrupted, you can restart it from `checkpoint-250.gguf`. The next file it outputs is `checkpoint-260.gguf`.

    nohup ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in checkpoint-250.gguf --lora-out lora.bin --train-data train.txt --sample-start '<SFT>' --adam-iter 1024 &

Merge[​](#merge "Direct link to Merge")

----------------------------------------

The fine-tuning process updates several layers of the LLM's neural network. Those updated layers are saved in a file called `lora.bin` and you can now merge them back to the base LLM to create the new fine-tuned LLM.

    ../build/bin/export-lora --model-base llama-2-13b-chat.Q5_K_M.gguf --lora lora.bin --model-out chemistry-assistant-13b-q5_k_m.gguf

The result is this file.

    curl -LO https://huggingface.co/juntaoyuan/chemistry-assistant-13b/resolve/main/chemistry-assistant-13b-q5_k_m.gguf

**Note 3** If you want to use a checkpoint to generate a `lora.bin` file, use the following command. This is needed when you believe the final `lora.bin` is an overfit.

    ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in checkpoint-250.gguf --only-write-lora --lora-out lora.bin

*   [Build the fine-tune utility from llama.cpp](#build-the-fine-tune-utility-from-llamacpp)
    
*   [Get the base model](#get-the-base-model)
    
*   [Create a question and answer set for fine-tuning](#create-a-question-and-answer-set-for-fine-tuning)
    
*   [Finetune!](#finetune)
    
*   [Merge](#merge)

---

# Open WebUI | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

You can configure the Open WebUI framework, a self-hosted WebUI, using any GaiaNet node as the backend LLM API. That allows you to use your own or community GaiaNet nodes in any application built on Open WebUI.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) | llama |
| Embedding | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) | nomic |

Start the Open WebUI on your machine[​](#start-the-open-webui-on-your-machine "Direct link to Start the Open WebUI on your machine")

-------------------------------------------------------------------------------------------------------------------------------------

After successfully starting the GaiaNet node, you can use `docker run` to start the Open WebUI.

    docker run -d -p 3000:8080 \  -v open-webui:/app/backend/data \  -e OPENAI_API_BASE_URL="https://llama.us.gaianet.network/v1" \  -e OPENAI_API_KEYS="gaianet" \  --name open-webui \  --restart always \  ghcr.io/open-webui/open-webui:main

Then, open `http://localhost:3000` in your browser and you will see the Open WebUI page.

You can also configure your own node when the webUI is started.

*   Click on your profile on the top right corner and choose **Setting**.
*   Then choose Connections. In the OpenAI API field, type your node base URL and enter several random characters.
*   Click on Save to make the change take effective.

![](/assets/images/openwebui-02-931678d7734f60414bc258b7859c4928.png)

Use Open WebUI as a Chatbot UI[​](#use-open-webui-as-a-chatbot-ui "Direct link to Use Open WebUI as a Chatbot UI")

-------------------------------------------------------------------------------------------------------------------

Simply choose the chat model under **Select a model** and then you can send messages to the GaiaNet node.

![](/assets/images/openwebui-01-1a26ed6523b22a60f60f40dd656cb407.png)

Use Open WebUI as a client-side RAG tool[​](#use-open-webui-as-a-client-side-rag-tool "Direct link to Use Open WebUI as a client-side RAG tool")

-------------------------------------------------------------------------------------------------------------------------------------------------

Open WebUI also offers a way to implement RAG application. Since the Gaianet nodes have OpenAI-compatible embedding APIs, you can also use this feature. However, to use this feature, it's recommend to start a node without any snapshots, like [this one](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-8b-instruct)
.

**Step 1:** Use GaiaNet node as the embedding API

*   Click on **Workspace** on left top and choose **Documents** tab. This is where you manage the uploaded documents.
*   Click on **Document Settings** to configure the embedding setting.
*   In the **General Settings**, choose OpenAI as the Embedding Model Engine. Enter the node API base URL and several random characters. Then, enter the embedding model name in the Embedding Model field. Click Save to apply the changes.

![](/assets/images/openwebui-04-1b648b9e231f053f6f761981134cdf81.png)

**Step 2:** Use GaiaNet node as the embedding API

Click on **+** to upload your documentations.

**Step 3:** Chat

Then go back to the chat page. Before you send a message, type **#** to choose the document you want to as the context.

![](/assets/images/openwebui-05-ba004a234279045cee6af7fa304126c8.png)

That's it.

*   [Prerequisites](#prerequisites)
    
*   [Start the Open WebUI on your machine](#start-the-open-webui-on-your-machine)
    
*   [Use Open WebUI as a Chatbot UI](#use-open-webui-as-a-chatbot-ui)
    
*   [Use Open WebUI as a client-side RAG tool](#use-open-webui-as-a-client-side-rag-tool)

---

# FlowiseAI RAG chat | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use Gaia nodes as LLM service providers.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) | llama |
| Embedding | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) | nomic |

Start a FlowiseAI server[​](#start-a-flowiseai-server "Direct link to Start a FlowiseAI server")

-------------------------------------------------------------------------------------------------

Follow [the FlowiseAI guide](https://docs.flowiseai.com/getting-started)
 to install Flowise locally

    npm install -g flowisenpx flowise start

After running successfully, you can open [http://localhost:3000](http://localhost:3000)
 to check out the Flowise AI tool.

Build a documents QnA chatbot[​](#build-a-documents-qna-chatbot "Direct link to Build a documents QnA chatbot")

----------------------------------------------------------------------------------------------------------------

FlowiseAI allows you to visually set up all the workflow components for an AI agent. If you're new to FlowiseAI, it's recommended to use a template quick start. In fact, there are lots of templates around OpenAI in the Flowise marketplace. All we need to do is to replace the ChatOpenAI component with the ChatLocalAI component.

Let's take the **Flowise Docs QnA** as an example. You can build a QnA chatbot based on your documents. In this example, we would like to chat with a set of documents in a GitHub repo. The default template was built with OpenAI and we will now change it to use an open-source LLM on a Gaia node.

### Get the **Flowise Docs QnA** template[​](#get-the-flowise-docs-qna-template "Direct link to get-the-flowise-docs-qna-template")

![](/assets/images/flowise-01-38ce1dc6e85edf44ce942f3c032751bc.png)

Click on Marketplaces on the left tab to browse all the templates. The template **Flowise Docs QnA** we will use is the first one.

![](/assets/images/flowise-02-5e4c3abe45028dea057cadc04cae301b.png)

Then, click on Use this template button on the left top corner to open the visual editor.

### Connect the chat model API[​](#connect-the-chat-model-api "Direct link to Connect the chat model API")

You will need to delete the ChatOpenAI component and click the + button to search ChatLocalAI, and then drag the ChatLocalAI to the screen.

![](/assets/images/flowise-03-00ebd2de39ebd00533f63659e11b1e48.png)

Then, you will need to input

*   the Gaia node base URL `https://llama.us.gaianet.network/v1`
*   the model name `llama`

Next, connect the ChatLocalAI component with the field `Chat model` in the **Conversational Retrieval QA Chain** component.

### Connect the embedding model API[​](#connect-the-embedding-model-api "Direct link to Connect the embedding model API")

The default template uses the OpenAI Embeddings component to create embeddings for your documents. We need to replace the **OpenAI Embeddings** component with the **LocalAI Embeddings** component.

*   Use the Gaia node base URL `https://llama.us.gaianet.network/v1` in the Base Path field.
*   Input the model name `nomic-embed-text-v1.5.f16` in the Model Name field.

Next, connect the **LocalAI Embeddings** component with the field `embedding` in the **In-Memory Vector Store** component.

### Set up your documents[​](#set-up-your-documents "Direct link to Set up your documents")

Then, let's go through the GitHub component to connect the chat application to our documents on GitHub. You will need to put your docs GitHub link into the **Repo Link** field. For example, you can put GaiaNet's docs link: `https://github.com/GaiaNet-AI/docs/tree/main/docs`.

Give it a try[​](#give-it-a-try "Direct link to Give it a try")

----------------------------------------------------------------

You can send a question like "How to install a GaiaNet node" after saving the current chatflow.

![](/assets/images/flowise-04-7ee98ecd7039fbf54776e2300c1e9ca9.png)

You will get the answer based on the GaiaNet docs, which are more accurate.

More examples[​](#more-examples "Direct link to More examples")

----------------------------------------------------------------

There are lots of examples on the Flowise marketplace. To build a Flowise agent based on GaiaNet, simply replace the **Chat OpenAI** and **OpenAI Embeddings** component with the GaiaNet base URL.

*   [Prerequisites](#prerequisites)
    
*   [Start a FlowiseAI server](#start-a-flowiseai-server)
    
*   [Build a documents QnA chatbot](#build-a-documents-qna-chatbot)
    *   [Get the **Flowise Docs QnA** template](#get-the-flowise-docs-qna-template)
        
    *   [Connect the chat model API](#connect-the-chat-model-api)
        
    *   [Connect the embedding model API](#connect-the-embedding-model-api)
        
    *   [Set up your documents](#set-up-your-documents)
        
*   [Give it a try](#give-it-a-try)
    
*   [More examples](#more-examples)

---

# Cursor AI IDE | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

[Cursor](https://www.cursor.com/)
 is an AI-powered code editor / IDE. Using LLMs to generate and review code, Cursor is an alternative to the very popular GitHub Copilot. You can use Cursor with your own Gaia node as the LLM backend. There are two big reasons for that

*   Your Gaia node could be supplemented by a knowledge base that is specific to your proprietary code repository, programming language choices, and coding guidelines / styles.
*   Your Gaia node could ensure that your code stays private within your organization.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node to provide LLM API services. You can

*   [Run your own node](/1.0.0/node-guide/quick-start)
    
*   [Use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use public [Qwen 2.5 Coder](https://github.com/QwenLM/Qwen2.5-Coder)
 nodes to power Cursor.

| Model type | API base URL | Model name |
| --- | --- | --- |
| General coding assistant | `https://coder.gaia.domains/v1` | coder |
| Coding assistant with Rust knowledge | `https://rustcoder.gaia.domains/v1` | rustcoder |
| Rust expert (slower but more accurate) | `https://rustexpert.gaia.domains/v1` | rustexpert |

> A limitation of Cursor is that it does not support local LLM services. A Gaia node comes with a default networking tunnel that turns your local LLM service into a HTTPS service accessible from the Internet. That allows Cursor to use your own private LLM for coding. Start your own [Qwen Coder](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct)
>  or [Qwen Coder with Rust](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct_rustlang)
>  nodes today!

Configure Cursor[​](#configure-cursor "Direct link to Configure Cursor")

-------------------------------------------------------------------------

First, download and install [Cursor](https://www.cursor.com/)
. Click on the **Settings** button on the top right. Then, click on **Models** to configure the backend LLM service.

Second, add a model named `coder` and turn off all the other models like `gpt-4o`.

Third, go to the OpenAI API Key section,

*   Click on **Override OpenAI Base URL**. Type `https://coder.gaia.domains/v1` here.
*   For the OpenAI API key, you can use any random chars such as `GAIA`. Click on **Verfiy** to test if the connection is correct.

![](/assets/images/cursor-01-fad7a1003a06d1009e698e9df24fde0c.png)

Use Cursor[​](#use-cursor "Direct link to Use Cursor")

-------------------------------------------------------

You can use

*   **command + K** to edit the highlighted code
*   **command + L** to open the chat room and ask questions about the code.

![](/assets/images/cursor-02-28b91f7d5e5c8be2cf75d3b8d3333184.png)

*   [Prerequisites](#prerequisites)
    
*   [Configure Cursor](#configure-cursor)
    
*   [Use Cursor](#use-cursor)

---

# Stockbot | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Stockbot is a lightning fast AI Chatbot that responds with live interactive stock charts, financials, news, screeners, and more. You can configure a Gaia node as the LLM backend.

> Please note, the Stockbot leverages function call to call the external API. You will need to use the model which supports function call, like [llama-3.1-8b](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool)
> .

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use a public Llama3 node with the function call support.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llamatool.us.gaianet.network/v1](https://llamatool.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, we will need to get the source code of the forked Stockbot.

    git clone https://github.com/JYC0413/stockbot-on-groq.gitcd stockbot-on-groq

Next, configure the `.env` file.

    cp .env.example .env.local

You will need to configure four parameters here.

*   LLAMAEDGE\_BASE\_URL: URL for the LLM API base URL.
*   LLAMAEDGE\_MODEL\_NAME: Name of the model to be used.
*   LLAMAEDGE\_API\_KEY: API key for accessing the LLM services.

Then, we will need to install the required dependencies.

    npm install

Next, let's run the application locally.

    npm run dev

Finally, you can open [http://localhost:3000](http://localhost:3000)
 and ask the stock related questions.

![](/assets/images/stockbot-01-fa5ede68e47bee6abc9b86c9d0d1441c.png)

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# Zed | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

[Zed](https://zed.dev/)
 is a next-generation code editor designed for high-performance collaboration with humans and AI, and it is written in Rust. You can use Zed with your own Gaia node as the LLM backend. There are two big reasons for that

*   Your Gaia node could be supplemented by a knowledge base that is specific to your proprietary code repository, programming language choices, and coding guidelines/styles.
*   Your Gaia node could ensure that your code stays private within your organization.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node to provide LLM services to Zed. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use public [Qwen 2.5 Coder](https://github.com/QwenLM/Qwen2.5-Coder)
 nodes to power Cursor.

| Model type | API base URL | Model name |
| --- | --- | --- |
| General coding assistant | `https://coder.gaia.domains/v1` | coder |
| Coding assistant with Rust knowledge | `https://rustcoder.gaia.domains/v1` | rustcoder |
| Rust expert (slower but more accurate) | `https://rustexpert.gaia.domains/v1` | rustexpert |

> A limitation of Cursor is that it does not support local LLM services. A Gaia node comes with a default networking tunnel that turns your local LLM service into a HTTPS service accessible from the Internet. That allows Cursor to use your own private LLM for coding. Start your own [Qwen Coder](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct)
>  or [Qwen Coder with Rust](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct_rustlang)
>  nodes today!

Configure Zed[​](#configure-zed "Direct link to Configure Zed")

----------------------------------------------------------------

First, download and install [Zed](https://zed.dev/)
. Click on your profile on the top right and choose **Setting**. Then a new tab called `settings.json` will be opened. You can configure your Zed by editing this file.

![](/assets/images/zed-01-cf0cb473f9d2facf71d318043fca0179.png)

Below is the `settings.json` we used. You can copy and paste sections `language_models` and `assistant` to your own. They configure Zed to use an OpenAI-compatible API provider and then specify the API endpoint URL and model name for that provider.

    {  "features": {    "inline_completion_provider": "none"  },  "language_models": {    "openai": {      "version": "1",      "api_url": "https://rustcoder.gaia.domains/v1",      "low_speed_timeout_in_seconds": 60,      "available_models": [        {          "name": "yicoder9b",          "max_tokens": 8096        }      ]    }  },  "assistant": {    "provider": "openai",    "default_model": {      "provider": "openai",      "model": "yicoder9b"    },    "version": "2"  },  "ui_font_size": 16,  "buffer_font_size": 16,  "theme": {    "mode": "system",    "light": "One Light",    "dark": "One Dark"  }}

Next we will configure the API key to access this Gaia node.

Go back to the folder you opened. Click on the Star icon at the bottom to turn on the Assistant panel.

![](/assets/images/zed-02-0120fdb612302c83426a76807ee18115.png)

Click on **Open configuration** to set up the API Key.

![](/assets/images/zed-03-71751fef612aad8f6d0b69312847f495.png)

Since we are using a free public Gaia node, you can use any API key in the OpenAI section at the bottom of the screen. For example, you can enter `GAIA`.

Now, we have everything ready.

Use Zed
=======

You can

*   Edit the highlighted code by selecting the code and clicking on the **Inline Assistant** button.

![](/assets/images/zed-04-693b6322554fe5bb3d0c1972ab22c709.png)

*   Open the Assistant panel by clicking on the **Assistant** icon at the bottom to turn on the Assistant panel.

![](/assets/images/zed-05-6785a1b71e9a38dee8d0d505d9296d2b.png)

*   [Prerequisites](#prerequisites)
    
*   [Configure Zed](#configure-zed)

---

# FlowiseAI tool call | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use a Gaia node that supports [LLM tool calling](https://github.com/LlamaEdge/LlamaEdge/blob/main/api-server/ToolUse.md)
.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. In this tutorial, you will need to [set up a public node with tool call support](https://github.com/GaiaNet-AI/node-configs/blob/main/mistral-0.3-7b-instruct-tool-call/README.md)
.

Start a FlowiseAI server[​](#start-a-flowiseai-server "Direct link to Start a FlowiseAI server")

-------------------------------------------------------------------------------------------------

Follow [the FlowiseAI guide](https://docs.flowiseai.com/getting-started)
 to install Flowise locally

    npm install -g flowisenpx flowise start

After running successfully, you can open `http://localhost:3000` to check out the Flowise AI tool.

Build a chatbot for realtime IP lookup[​](#build-a-chatbot-for-realtime-ip-lookup "Direct link to Build a chatbot for realtime IP lookup")

-------------------------------------------------------------------------------------------------------------------------------------------

Step 1: Create a new **Chatflow** from the UI.

![](/assets/images/flowise-tool-01-54bc5131cf142cebbfc44675f276c962.png)

Step 2: On the **Chatflow** canvas, add a node called **ChatLocalAI**.

![](/assets/images/flowise-tool-02-82e1dcf92b2aceeccff81ac1e7e891de.png)

Step 3: Configure the **ChatLocalAI** widget to use the Gaia node with tool call support you have created.

*   Base path: `https://YOUR-NODE-ID.us.gaianet.network/v1`
*   Model name: e.g., `Mistral-7B-Instruct-v0.3.Q5_K_M`

Step 4: Add a node called **Custom Tool**

Create a function named `get_ip_address_geo_location`. The function requires a `string` parameter called `ip`.

The **Tool description** field is the "prompt" that tells the LLM when to use this function. In this example, if the LLM detects that the user is asking about the city or country of an IP address, it will return a tool call response asking FlowiseAI to perform this function call first.

![](/assets/images/flowise-tool-03-93283784efe5a1f1343443da51c35c88.png)

Now you can add JavaScript code for this function. It looks up the location of the input `ip` parameter.

    const fetch = require("node-fetch")const url = "http://ipwho.is/"+$iptry {  const response = await fetch(url)  const result = await response.text()  console.log(result)  return result} catch(error) {  console.error(error)}

![](/assets/images/flowise-tool-04-9217ad7610930f56bc859d250c1538ec.png)

Step 5: Add a node called **Buffer Memory** to the canvas.

Step 6: Add a node called **Tool Agent**.

Step 7: Connect the nodes.

Connect the **Custom Tool** and **Buffer Memory** nodes to the appropriate connectors on the **Tool Agent** node. Connect the **ChatLocalAI** node to the **Custom Tool**.

![](/assets/images/flowise-tool-05-8cf21165d8466b203f3f521ba9e55b61.png)

Step 8: Save the **Chatflow**.

Give it a try[​](#give-it-a-try "Direct link to Give it a try")

----------------------------------------------------------------

From the FlowiseAI UI, you can open a chat window to chat with the **ChatLocalAI** you just created. Let's ask a question:

    What's the location of this address 35.222.115.181

The LLM understands that the request is to find a location for an IP address, and sees that we have a function called `get_ip_address_geo_location` in tools, which has a description that matches this task. So, it responses with a JSON message to call this function with the IP address it extracts from the user query.

This tool calling JSON message is NOT displayed to the user in the chatbot. Instead, the FlowiseAI **Custom Tool** node captures it and executes the JavaScript code associated with this tool call. The result of the tool call is then sent back to the LLM together with the original query, which is why we need the **Buffer Memory** node BTW, and the LLM formulates a human readable response to the original question.

![](/assets/images/flowise-tool-06-3691b227be09ee5bd6e7c6badf78c2ff.png)

*   [Prerequisites](#prerequisites)
    
*   [Start a FlowiseAI server](#start-a-flowiseai-server)
    
*   [Build a chatbot for realtime IP lookup](#build-a-chatbot-for-realtime-ip-lookup)
    
*   [Give it a try](#give-it-a-try)

---

# LlamaCloud | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. LlamaParse can support different kinds of files, like pdf, doc, .ppt, and other formats.

You can configure LlamaParse to use the GaiaNet node as the LLM backend, hence you can create a RAG application based on your PDF files locally.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://gemma.us.gaianet.network/v1](https://gemma.us.gaianet.network/v1) | gemma |
| Embedding | [https://gemma.us.gaianet.network/v1](https://gemma.us.gaianet.network/v1) | nomic-embed |

Steps[​](#steps "Direct link to Steps")

----------------------------------------

We will use an open-sourced GitHub repo, called `llamaparse-integration`, to make LlamaPase easy to use. The `llamaparse-integration` application supports

*   Multiple file formats, like `.pdf` and `.doc`,
*   Multiple files

We will need to get the source code in your terminal first.

    git clone https://github.com/alabulei1/llamaparse-integration.gitcd llamaparse-integration

Next, install the required mode packages.

    npm install llamaindexnpm install dotenv

Start a Qdant instance. The Qdrant instance is to store the embeddings.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Then, we will need to set up the LLM model settings. We can configure the model setting in the `.env` file.

    OPENAI_BASE_URL=https://gemma.us.gaianet.network/v1/OPENAI_API_KEY=gaianetLLAMAEDGE_CHAT_MODEL=gemmaLLAMAEDGE_EMBEDDING_MODEL=nomicLLAMA_CLOUD_API_KEY=Your_Own_KEYFILE_PATH=FILE_DIR=./pdf_dirCOLLECTION_NAME=defaultQDRANT_URL=http://127.0.0.1:6333SAVE_MARKDOWN_PATH=output.md

Here are some notes about the `.env` setting:

*   You can get the LlamaCloud key from [https://cloud.llamaindex.ai](https://cloud.llamaindex.ai)
    
*   You may need to make changes according to your model setting and file path.
*   If you put your file name in the `FILE_PATH=`, the program will build a RAG application with this single pdf file.
*   If the `FILE_PATH=` is empty, the program will build a RAG application with the files under the `FILE_DIR=./pdf_dir`. You can include multiple files in the folder.

Next, we can run the program to build an RAG application based on the PDF file

    npx tsx pdfRender.ts

After it runs successfully, you can send a query via the command line.

![](/assets/images/llamaparse-01-07933ece762716a4ea93c06f31abb0d5.png)

*   [Prerequisites](#prerequisites)
    
*   [Steps](#steps)

---

# LobeChat | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

You can configure [LobeChat](https://lobehub.com/)
 to use a GaiaNet node as its backend LLM API. It provides a richer and more customizable UI than the default GaiaNet chatbot UI.

Steps[​](#steps "Direct link to Steps")

----------------------------------------

**Step 1: Set up the GaiaNet node API base url as the OpenAI provider**

Go to the [Language Model Setting page](https://chat-preview.lobehub.com/settings/modal?agent=&session=inbox&tab=llm)
 and choose OpenAI.

1.  Enter a random string in the OpenAI API Key field. It does not matter what you enter here since we are going to ignore it on the backend.
2.  Enter the GaiaNet node API base URL in the API Proxy Address field. For example, you can use `https://llama.gaianet.network/v1` here.
3.  Enable Use Client-Side Fetching Mode
4.  Click on the Get Model List text and it will automatically fetch LLMs available on the GaiaNet node. Choose the chat model `llama` here.
5.  Optional: click on the Check button to check the connection status.

![](/assets/images/lobechat-gaianet-01-bf6655ad5eac47bd6eb47de87c3a6759.png)

**Step 2: Start chatting via the LobeChat UI**

Next, let's go back to the chat page. Before starting chatting, choose the model you just chose in the previous step around **Just Chat**.

Now you can chat with the GaiaNet node via LobeChat.

![](/assets/images/lobechat-gaianet-02-f15f6809e36ea088a2a9c2abd89adc2b.png)

*   [Steps](#steps)

---

# CodeGPT | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

CodeGPT is a pair-programming partner for developers. It offers AI chat assistance, auto-completion, code explanation, error-checking, and much more. You can find the CodeGPT extension in VScode and Jetbrains. You can easily configure it to use Gaia nodes as LLM backends.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use the public CodeStral nodes to power the CodeGPT plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://codestral.us.gaianet.network/v1/v1/](https://codestral.us.gaianet.network/v1/v1/) | codestral |

> For some reason, CodeGPT requires the API endpoint to include an extra `v1/` at the end.

Install CodeGPT[​](#install-codegpt "Direct link to Install CodeGPT")

----------------------------------------------------------------------

Download [the CodeGPT for VScode](https://marketplace.visualstudio.com/items?itemName=DanielSanMedium.dscodegpt&ssr=false#overview)
 and [Jetbrains](https://plugins.jetbrains.com/plugin/21056-codegpt)
.

Once you install it successfully, you can find the plugin on the right sidebar. You don't need to sign up for an account here.

![](/assets/images/codegpt-01-1f1e441d81450929755cc238163b43ec.png)

Configure CodeGPT[​](#configure-codegpt "Direct link to Configure CodeGPT")

----------------------------------------------------------------------------

Click the CODEGPT on the right sidebar and enter the settings page for CodeGPT.

1.  Go through the **Select Your AI** and choose the **Custom** PROVIDER. Copy and paste `codestral` into the MODEL field.

![](/assets/images/codegpt-02-6a086ea00d37fddaf000948e4bd1c229.png)

3.  Click the Connect button to configure the model base url and API key. Again, note the extra `v1\` at the end of the URL.

| Attribute | Value |
| --- | --- |
| API endponit URL | [https://codestral.us.gaianet.network/v1/v1/](https://codestral.us.gaianet.network/v1/v1/) |
| API Key | gaia |

![](/assets/images/codegpt-03-5c71bbb294022836ff2fd41aa89e73c1.png)

Save the above settings.

Use the plugin[​](#use-the-plugin "Direct link to Use the plugin")

-------------------------------------------------------------------

You can summon the coding assistant using slash commands defined in the plugin. You can ask the coding assistant to fix bugs, explain codes, write documentation, refactor the docs, and create unit test cases for the specific code.

![](/assets/images/codegpt-04-333560c9eba11ed3702d5d52d1271458.png)

*   [Prerequisites](#prerequisites)
    
*   [Install CodeGPT](#install-codegpt)
    
*   [Configure CodeGPT](#configure-codegpt)
    
*   [Use the plugin](#use-the-plugin)

---

# Obsidian | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

Obsidian is a note-taking application that enables users to create, link, and visualize ideas directly on their devices. With Obsidian, you can seamlessly sync notes across devices, publish your work, and collaborate with others. The app is highly customizable, allowing users to enhance functionality through a wide range of plugins and themes. Its unique features include a graph view to visualize connections between notes, making it ideal for managing complex information and fostering creativity. Obsidian also emphasizes data privacy by storing notes locally.

**Obsidian-local-gpt is a plugin that** allows users to run a local large language model within Obsidian note-taking application. This plugin enables various AI-powered features directly in Obsidian, such as text generation, summarization, spelling and grammar checks, and task extraction.

A key feature of this plugin is that it supports a large number of open source LLMs. You can choose an LLM that is finetuned for your specific task — eg if you take a lot of coding notes, you could choose a Codestral or CodeLlama or DeepSeek LLM. Furthermore, if you choose to run the LLM locally on your own computer, the plugin would support private and offline use of the LLM features. For more details, you can visit the [obsidian-local-gpt GitHub page](https://github.com/pfrankov/obsidian-local-gpt)
.

This guide explains how to set up and use the plugin with a GaiaNet node as an alternative to OpenAI or Ollama.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [Run your own node](/1.0.0/node-guide/quick-start)
    
*   [Use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use a public node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) |
| Model Name | llama |

Obsidian-local-gpt Plugin Setup[​](#obsidian-local-gpt-plugin-setup "Direct link to Obsidian-local-gpt Plugin Setup")

----------------------------------------------------------------------------------------------------------------------

Make sure you have already installed the Obsidian app on your device.

### Install the Obsidian-local-gpt Plugin[​](#install-the-obsidian-local-gpt-plugin "Direct link to Install the Obsidian-local-gpt Plugin")

*   Open Obsidian settings, navigate to "Community plugins", and search for `obsidian-local-gpt`.
*   Install the plugin by clicking “Install”.

![](/assets/images/obsidian-enable-f3f05cad21f516b0d9002e773e5116bb.png)

Then click “Enable”.

### **Configure the Plugin**[​](#configure-the-plugin "Direct link to configure-the-plugin")

1.  Go to the plugin settings.
2.  Select "AI Provider" as "OpenAI compatible server".
3.  Set the server URL. Use [https://llama.us.gaianet.network/](https://llama.us.gaianet.network/)
     if you are using a public GaiaNet node. Or, use [http://localhost:8080/](http://localhost:8080/)
     if you are running a local GaiaNet node.
4.  Configure API key to GaiaNet.

![](/assets/images/obsidian-configure-9692f400bc4a5ae8288cbd471af0bc78.png)

Make sure to click the refresh button and choose the **llama** model if you’re using the public GaiaNet node url and **Phi-3-mini-4k-instruct** if you’re using the local GaiaNet node.

![](/assets/images/obsidian-model-226eda81908ccfefab621a99f057ee8a.png)

### Configure Obsidian Hotkey[​](#configure-obsidian-hotkey "Direct link to Configure Obsidian Hotkey")

1.  Open Obsidian Settings.
2.  Go to Hotkeys.
3.  Filter "Local" and you should see "Local GPT: Show context menu".
4.  Click on `+` icon and press hotkey (e.g. `⌘ + M`).

![](/assets/images/obsidian-hotkey-0bf43c29ebdc7ed95a10b1a7bba6bb58.png)

As long as you have set the hotkey, while writing or editing a note, select the text you want to interact with, and press the hotkey you have set to use this LLM powered plugin!

Use Cases[​](#use-cases "Direct link to Use Cases")

----------------------------------------------------

### **Text Continuation**[​](#text-continuation "Direct link to text-continuation")

*   Select a text segment, right-click, and choose "Continue writing". The model will generate the continuation (displayed in the screenshot in grey).

![](/assets/images/obsidian-text-continuation-0147058b4cc7761f3061dfe8b4ba96c2.png)

### **Summarization**[​](#summarization "Direct link to summarization")

*   Select a longer text segment, right-click, and choose "Summarize text". The model provides a summary as below of the CNAI report content I selected.

![](/assets/images/obsidian-summarization-9e61cc6f77a1dfa00ca064cb1b61421a.png)

Here is a concise summary of the key points:

*   Cloud Native (CN) refers to well-balanced systems built using microservices, promoting modular design and reusability.
*   Kubernetes has become the de facto cloud operating system, offering scalability, resilience, and DevOps best practices.
*   Every Cloud Service Provider offers Kubernetes as a service, facilitating access to infrastructure and support services for various workloads, including AI/ML.
*   The Cloud Native Computing Foundation defines Cloud Native as empowering organizations to build scalable applications in modern environments using containers, microservices, and declarative APIs.
*   Cloud Native Artificial Intelligence (CNAI) is an evolving extension of Cloud Native, focusing on building and deploying AI applications and workloads using Cloud Native principles.

### **Spelling and Grammar Check**[​](#spelling-and-grammar-check "Direct link to spelling-and-grammar-check")

*   Select text, right-click, and choose "Fix spelling and grammar". The model will correct errors.

![](/assets/images/obsidian-grammar-d32655e11368ec52c30f1a1ed52a58b7.png)

### **Extract Action Items**[​](#extract-action-items "Direct link to extract-action-items")

*   For notes with multiple tasks, select the text and click "Find action items". The model will list the tasks.

![](/assets/images/obsidian-extract-ea27bc2ab9b125a59440f4567894bbef.png) The Generated content are displayed below your own text:

`Here are the action items extracted from the document in Markdown checkbox format:`

*   `[ ] Add feedback on WasmEdge Q3 Roadmap Discussion`
*   `[ ] Provide input on Update on WasmEdge Community Governance`
*   `[ ] Discuss any additional topics at the meeting (add to [https://docs.google.com/document/d/1iFlVl7R97Lze4RDykzElJGDjjWYDlkI8Rhf8g4dQ5Rk/edit#](https://docs.google.com/document/d/1iFlVl7R97Lze4RDykzElJGDjjWYDlkI8Rhf8g4dQ5Rk/edit))`

`Let me know if you'd like me to help with anything else!`

### **General Assistance**[​](#general-assistance "Direct link to general-assistance")

*   Select any text and click "General help" to get contextually relevant information from the model.

I entered and selected some information on KubeCon + CloudNativeCon + Open Source Summit + AI\_dev China 2024. Because llama3 has not been trained with info on this conference, so the output is not very helpful:

`The information you're looking for is not present in this context.`

`If you need to know the format and dates of KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024, I suggest searching for official announcements or websites related to these events.`

Try it now![​](#try-it-now "Direct link to Try it now!")

---------------------------------------------------------

Ready to supercharge your note-taking with AI? Get started with the Obsidian-local-gpt plugin and GaiaNet today:

1.  Set up the Obsidian-local-gpt plugin in your Obsidian app.
2.  Explore the various AI-powered features to enhance your productivity.

Start your journey towards smarter, more efficient note-taking now!

*   [Prerequisites](#prerequisites)
    
*   [Obsidian-local-gpt Plugin Setup](#obsidian-local-gpt-plugin-setup)
    *   [Install the Obsidian-local-gpt Plugin](#install-the-obsidian-local-gpt-plugin)
        
    *   [**Configure the Plugin**](#configure-the-plugin)
        
    *   [Configure Obsidian Hotkey](#configure-obsidian-hotkey)
        
*   [Use Cases](#use-cases)
    *   [**Text Continuation**](#text-continuation)
        
    *   [**Summarization**](#summarization)
        
    *   [**Spelling and Grammar Check**](#spelling-and-grammar-check)
        
    *   [**Extract Action Items**](#extract-action-items)
        
    *   [**General Assistance**](#general-assistance)
        
*   [Try it now!](#try-it-now)

---

# AI coding assistant: Continue | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

[Continue](https://github.com/continuedev/continue)
 is the leading open-source AI code assistant. It is a copilot-like plugin for VSCode and JetBrains to provide custom autocomplete and chat experiences inside those IDEs. You can easily configure it to use Gaia nodes as LLM backends. In fact, you can choose different Gaia nodes for

*   The autocomplete model for coding tasks.
*   The chat model for understanding and discussing code.
*   The embedding model to provide chat context based on local files.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [Run your own node](/1.0.0/node-guide/quick-start)
    
*   [Use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use public nodes to power the Continue plugin.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://gemma.us.gaianet.network/v1/](https://gemma.us.gaianet.network/v1/) | gemma |
| Embedding | [https://gemma.us.gaianet.network/v1/](https://gemma.us.gaianet.network/v1/) | nomic |
| Autocompletion | [https://codestral.us.gaianet.network/v1/](https://codestral.us.gaianet.network/v1/) | codestral |

> It is important to note that Continue requires the API endpoint to include a `/` at the end.

Install Continue[​](#install-continue "Direct link to Install Continue")

-------------------------------------------------------------------------

[Load this link](https://marketplace.visualstudio.com/items?itemName=Continue.continue)
 to install the Continue IDE plugin.  
It will open up your VSCode when you click on the **Install** button on the web page. When you are asked to configure Continue, just click on **Skip** and finish the installation without selecting a local model.

Configure Continue[​](#configure-continue "Direct link to Configure Continue")

-------------------------------------------------------------------------------

Click on the gear icon on the toolbar to load the `config.json` file for the Continue plugin. The file is located in your own home directory `$HOME/.continue/config.json`. You can now change the `config.json` file as follows. It asks the Continue plugin to use different public Gaia nodes and models for chat, code autocomplete and embeddings.

    {  "models": [    {      "model": "gemma",      "title": "LlamaEdge",      "apiBase": "https://gemma.us.gaianet.network/v1/",      "provider": "openai"    }  ],  "tabAutocompleteModel": {      "title": "Autocomplete",      "apiBase": "https://codestral.us.gaianet.network/v1/",      "model": "codestral",      "provider": "openai"  },  "embeddingsProvider": {    "provider": "openai",    "model": "nomic-embed",    "apiBase": "https://gemma.us.gaianet.network/v1/"  },  "customCommands": [    {      "name": "test",      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",      "description": "Write unit tests for highlighted code"    }  ],  "allowAnonymousTelemetry": true}

Save the `config.json` file and you are done!

Use the plugin[​](#use-the-plugin "Direct link to Use the plugin")

-------------------------------------------------------------------

The following screenshot shows how you can chat with an error message inside the IDE.

![](/assets/images/continue-01-e2a5bd512068829ed1debacc682accc7.png)

*   [Prerequisites](#prerequisites)
    
*   [Install Continue](#install-continue)
    
*   [Configure Continue](#configure-continue)
    
*   [Use the plugin](#use-the-plugin)

---

# Translation Agent  + GaiaNet | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The Translation Agent originally built by Prof. Andrew Ng, designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use any GaiaNet node as the LLM backend.

> For commands on starting and running this agent, refer to [GitHub - Second State/translation-agent](https://github.com/second-state/translation-agent/blob/use_llamaedge/step-by-step-use-LocalAI.md)
> .

You can run the Translation Agent on top of a public GaiaNet Node as a backend and then translate the content in your target language. If you’d like to learn more about the Translation Agent and how open source LLMs perform, check out the article [Agentic translation on GaiaNet](https://docs.gaianet.ai/tutorial/translator-agent)
.

Prepare the environment[​](#prepare-the-environment "Direct link to Prepare the environment")

----------------------------------------------------------------------------------------------

Here, we will use the public GaiaNet node with gemma-2-27b model. `https://gemma.us.gaianet.network/`.

> As an alternative, you can also start a GaiaNet node locally on your device. Refer to [this guide](https://github.com/GaiaNet-AI/node-configs/tree/main/gemma-2-27b-it)
> .

To get started, clone the Translation Agent that supports open source LLMs.

    git clone https://github.com/second-state/translation-agent.git    cd translation-agentgit checkout use_llamaedge

Set environment variables and install necessary Python packages if needed. Replace the OPENAI\_BASE\_URL with `https://gemma.us.gaianet.network/`

    export OPENAI_BASE_URL="https://gemma.us.gaianet.network/v"export PYTHONPATH=${PWD}/srcexport OPENAI_API_KEY="GAIANET"pip install python-dotenvpip install openai tiktoken icecream langchain_text_splitters

Prepare your translation task[​](#prepare-your-translation-task "Direct link to Prepare your translation task")

----------------------------------------------------------------------------------------------------------------

Find the `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here because we named our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text-Intro-of-Forbidden-City)
 in Chinese `forbiddencity.txt` since it is an introduction on this Chinese royal palace, then note down its document path, `sample-texts/forbiddencity.txt`. This will be the `relative path` in our `example_script.py` file.

Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the relative path to the above. Change the model name to the one you are using; here, we're using the `gemma` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language).

    import os  import translation_agent as ta      if __name__ == "__main__":    source_lang, target_lang, country = "Chinese", "English", "Britain"        relative_path = "sample-texts/forbiddencity.txt"    script_dir = os.path.dirname(os.path.abspath(__file__))        full_path = os.path.join(script_dir, relative_path)        with open(full_path, encoding="utf-8") as file:        source_text = file.read()        print(f"Source text:\n\n{source_text}\n------------\n")        translation = ta.translate(            source_lang=source_lang,            target_lang=target_lang,            source_text=source_text,            country=country,            model="gemma",    )        print(f"Translation:\n\n{translation}")

Translate[​](#translate "Direct link to Translate")

----------------------------------------------------

Run the python translation script as follows.

    cd examples    python example_script.py

Wait a few minutes and [the English translation](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-gemma-2-27b)
 will appear on your terminal screen.

*   [Prepare the environment](#prepare-the-environment)
    
*   [Prepare your translation task](#prepare-your-translation-task)
    
*   [Translate](#translate)

---

# Agent Zero | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

[Agent Zero](https://github.com/frdel/agent-zero)
 is a general purpose AI agent application. You can simply ask it to accomplish tasks on the command line. It is designed to be dynamic, organically growing, and learning as users use it. It leverages your computer as a tool to accomplish your tasks.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node to provide LLM services to the agent app. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use the public [Llama-3.1-8B node](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct)
 to power the Agent Zero.

| Model type | API base URL | Model name |
| --- | --- | --- |
| Chat | [https://llama.us.gaianet.network/v1/](https://llama.us.gaianet.network/v1/) | llama |
| Embedding | [https://llama.us.gaianet.network/v1/](https://llama.us.gaianet.network/v1/) | nomic-embed |

**You will also need to make sure your Docker engine is running.** Because the Agent Zero framework will leverage Docker to execute the generated code.

> You can start a local LLM service using [Gaia](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct)
>  or [LlamaEdge](https://llamaedge.com/docs/user-guide/quick-start-command)
>  or [Moxin](https://github.com/moxin-org/moxin)
> , and then use `http://localhost:8080/v1/` as the LLM API service endpoint URL.

Configure the agent[​](#configure-the-agent "Direct link to Configure the agent")

----------------------------------------------------------------------------------

First, we will need to get the source code of a Gaia-compatible version of Agent Zero.

    git clone https://github.com/JYC0413/agent-zero-gaia.gitcd agent-zero-gaia

Then, let's install the required dependencies.

    pip install -r requirements.txt

Next, let's configure the gaia node and other parameters.

    cp example.env .env

You will need to configure the following items:

*   `CHAT_MODEL_BASE_URL`: URL for the LLM API base URL. E.g., `https://llama.us.gaianet.network/v1/`
*   `CHAT_MODEL_NAME`: Name of the chat model to be used. E.g., `llama`
*   `CHAT_API_KEY`: An API key to access the LLM services. You can enter several random characters here. E.g., `GAIA`
*   `EMBEDDING_MODEL_BASE_URL`: URL for the embedding model API base URL. E.g., `https://llama.us.gaianet.network/v1/`
*   `EMBEDDING_MODEL_NAME`: Name of the embedding model name. E.g., `nomic-embed`
*   `EMBEDDING_API_KEY`: An API key to access the embedding services. You can enter several random characters here. E.g., `GAIA`

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

Finally, let's run the Agent Zero application backed by the [Llama 3.1 8b](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct)
 Gaia node.

    python main.py

You can interact with Agent Zero via the command line. You will see the Agent Zero framework will call the related tools and execute some code to complete your task.

### Example 1[​](#example-1 "Direct link to Example 1")

Get the time in berlin and seattle

![](/assets/images/agent-zero-01-5977b5b913065e4d018f6154a23ee527.png)

![](/assets/images/agent-zero-02-7043b5f92118e22f7fc418c131c35965.png)

Save the above result in a file using node.js

![](/assets/images/agent-zero-03-b5ebd8903dbb3ebdc2a9b01b9b8d522e.png)

### Example 2[​](#example-2 "Direct link to Example 2")

Install [mc](https://midnight-commander.org/)
. do not use `-y` flag

![](/assets/images/agent-zero-04-169d90a36a516babfd2025f500ce5cc9.png)

### Example 3[​](#example-3 "Direct link to Example 3")

Run `top`, show the system load

![](/assets/images/agent-zero-05-8f884d2fdad5e381880fb9fde3c1c14b.png)

### Example 4[​](#example-4 "Direct link to Example 4")

Memorize my openai api key - 563434643464

![](/assets/images/agent-zero-06-865e4e1235f0eceec9395c8d85a01db9.png)

### Example 5[​](#example-5 "Direct link to Example 5")

Extract an mp3 audio track from a mp4 video file using ffmpeg. You will need to put an mp4 file (e.g., `video.mp4`) into the `work_dir` and tell the agent about it.

![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAPQAAAEUCAIAAACTfoyfAAAgAElEQVR4Ae2dzW8cx5XADUF3LZwbBcQ50LlFQLzQcndEkRyJMulhIlnikEwsRcowgiyK4oiWIpFWbFIRydBKolgKRx+UJUE2zLEcS/ywvTK2B1hsgL0YC3ixcW4+ZP+FLLLnXdRHv35d/THdMz3TUzVPIISq6qpXVa9+/fiq2a/ruT0D+ZCf5dLDPQP5nmuPQ372DORFNUVO5ui1tY+uDR8YhoSokDnw+txHqxMHhvcM5DPnVit+/yzr47mjrAL9NEEDqw/ev3FrNbyjG7dWVx+8H15HuTp45Pjqg/c3tj6Dn9UH7w8eOa5Ua1z2uXDRycI9vPDxyrlhBW7r9owyBl6hFriHRgoffvQJqBISH370ydBIQemFsqCBj5+sg642tj6D8j0DeVz+8ZN1fKlpaTyGoLTvYIyCe89A/six15Wl+vjJ+pFjr/tOngqFBhSNYbVgmAhux5EAbwQSvpbbzyup1OOWvHZy6snGlliVJxtbr52cwqtFaa8GyC0JdLvDfG7LqlQqa7dXhfM9cZs5G41zS2DZTp69uL712frWZyfPXoRCSgRp4NLbi083tobGxqGCMA2QHRobf7qxdentRSiJmOj7wY+u37wjpF2/eafvBz+K2DCkWi5/Ipc/EVIBLjXKLZm4zbBeOedsCsXeUTHJmXOrCfrcMKs9A/lzl+bOXZrDJZQO0kD/q8cef7JeuvugJzcm6mC4e3JjpbsPHn+y3v/qsSAJIeXdgyNvL/z67YVfdw+OhFSLeCmXP/How8ePPnwche9Gwa2MdeK2ZVnsCUnmwMyKZQHQ9LREUVRa2cKZC+ubn5buPhD2G+AeGhsv3X2wvvlp4cyFtMYG/Qqyxdii8B0JbpAelPB9FCgqM4fbstYWXFu64YWPBd+Ns9xBQ6XyIA0Uzlx4/Mn6042tm7fuCYBu3rr3dGPr8SfrcckWzcP/DxpGUDkmOyLf1eFeLj2M8hM0JirXSAP9h45eentx9cH7f3iy8YcnG6sP3r/09mL/oaNxpxCOtbgaV2YN9avAXYNEakIaaBENENzOs8sWWRIaRlIaILgJbmM1QHAbu7RJ2T995RDcBLexGiC4jV1afS1uUiN/7iX6RxowVAPPdfdm6Yc0YKQGCG66t43VAMFt7NIaaYxjTYrgJriN1UBqcC+v3IOf+WvvvvbTk7FuSqpMGqiqgZaAW1D+09Nnqw63/gqZnrG5srUy2RddVGayZJWvDPf0ZUaurFnluZEYbaP3QjUT10ACcOd+eOjlV3LKyF5+JZf74SGlEGfBbFdN4Fb1pwnu+nWoi4QE4L5+4/fvPXyE+X75ldx7Dx9dv/H7EC1UZRoqhAip4VI9cNfQHTVJUQMJwC1QBr6VbNDcgN2qiSAJtZUT3LXpTcdWCcDd3ZsFoF8dHnnv4SMAPUQjVZmGCljIRMlamx/r7s1mes6uWNJ1dnxiXijC6S2rNNHDnGObZlbfKl85sncUfG7uQ1tWycfXF/IhMh/53EysInOYd4THSelW0EAycAPf65ufRiG7uzcL7FZNYDUNz5cFi5nJ0lq5LEAXxAscRQljerIk+BYgeljv4xX8d4f4zunuzU6U2F1hbygduEEmHiGlW0cDicEt+L72m+vY+Q6ZZ1WmoQIWwm0tw2uiVJ4bObvCmBubKzNGwX6L+rZxBSsrH3HI8pJEHwuHtCrK9bTEgTvWIxcQTommaSBJuGMNGtitmsBiOZpurCevrAmzaiMo6ts1/eG2yuWQh3oEN9a5vmnN4O7uzQ7Pl9fK0jkRaeyFu9wSAb37wbZj0dlDa/8H3tgtEfV93RKy3C3OvX5wi12gAEv5qwreBYJDDDQji27vRPkGFGripcpMluyNaXluHv8Rh9wSbf5cnxrcmCRKkwYaoQGCWxs71IjlN1smwS0fmcMjbfaJwzhvnpjNh9azI7jJchurAYLb2KXV2ugmMniCm+A2VgMEt7FLm4jx01oIwU1wG6sBgtvYpdXa6CYyeM3gxi+iUORlIgQYLCRNuHM/PPS7GzcjvkUo1gDDLdLNirzcNbW4ozBE0ZM6/aJLE24IcYjOtxfuoJJkDVKmh+DWCWux+mnCDSEOEeMbag5xqB90grt+HTZfQspwx+U7yE57y5NVJcGdrD6bIy19uLt7s68Oj6xvfnrtN9erztkLcVBJkCiO6bbFpe2LS9tnxrtYQNpQ58yiLJkahYCdXVN24cx4l4R7vEM0XCjuEvK90oL6pfLmayB9uGN53kEoe8t9VSlYFEyLCpmerkKxc1CEEo92LCx25Fn8LyPbXY2VCKb5zcA2l5J4vstkcma3wb3h2zsVNlkDKcMdi+z6fe7MaMfCrEQZK3pwfIc0yQJuTzUPxxzuUWnIRVv4VYAlUzpFDaQJd1yyGwG38EmQf8Itd2S4fW+VFJeTusYaSBPuRJ5zex0SUYIn6XYkpPOQ6dlVGO/CtnxwfAd2S4SPIauhR4HcAwG3xHFF8kV2Y+B+KZ2uBtKEu4aZB6HsLcfCAW68fbQ5Zr6y3F8WO2a4W+JXzXnODXDjaotL28nhxjpvhbRmcLeCymgMumiA4NbvD2+6sJX6OAlugttYDRDcxi5t6oYz9QEQ3AS3sRoguI1d2tQNZ+oDILgJbmM1QHAbu7SpG87UB0BwE9zGakAzuPFfIimGMnXT2OIDSBPuRN4taU4MZYuvIg3PVwNpwt38twJ9VUCFpmogTbjjxpjV/8qrqatI8/LVQMpwx+Ub+9zhad/ZUmFbaSB9uJsZQykDauxQSBmjICPEnKBJeJObfatkVEZYTo32ZezQGxyB1la46DXZ9OGO5XmHW2t81XcZRHCkJxSSvdItXu9mr2i7IylFrI3AGjWkuAQNHiCmDHcssuv3uf1DId2xvRDq666MgxWctO8tRIUtooE04Y5LNsHdItDoMow04U7kOTd2RXAaLwCEmbmNcVdhVoRCcrfE/hSJO5KSVWC+iiuGkiy3Bj5Jd282TbgxfxHTGN/wNBYYBe6pov3BHQijDAAag457oXSraUAzuBuhPnCyGyGcZKaoAYI7S3CnyF9Duya4CW49HOgabgOC29ilrYEGw5oQ3AS3sRp47tBQln5IA0ZqgOCme9tYDRDcxi6tkcY41qQIboLbWA1oBvfW0V3ws/bj7188/E+xbmWq3FYaSBPuoz86tLJycyyfi65xIBsSc0f+MXrzGmoePN6xcLlzLNdXQ9uam4R3ejC36/xiR6GOIR3MDU/OnDqW6zt4+Nj0zLmTh/sO5nInp2cmf9zUadasn4gN04R7LJ97+OjRw0ePovMNTFdNRJx/1WrhnFVtXluF8E4J7ohaTRPuQ0PZuHxXZRoqRJx/1WrhnFVtXluF8E4ThBuGp4vljjXOlOGOyzewWzUBy1ZnIpyzOoUHNQ/vlOCO6D6lD/ehoeyJn7BzKN9993rQYkN5VaahAjQ5NJTlNOw4c1q+1Hr5dJe4ysvlmSELyIvF5YtL28HnxuUgxKej452X+RmW54/3HTxeV6cH852X7YFBmg9D+ty+Qxo77RzO5uuau31u7nxLn3t4cmZmlv8TANmWkpXPTB8bkW56YJ1L5376+vTM9LjcRx388SnRCrTkI5BtAKTAGb4TsFdHLefDtmvywYBY30T6cMfyTIDdqgk8W0HAwvQuBjrDZceZfJ8oBEaZseQYifLzx+XWqjC9TcDNy1lDfrd0nbm8DepAX7IjvgEVWKNOGY6xOw2F23dIcA/AqLyJILiBLQal2HFy6J3yXO7kBEOcKSGoDgL62IS6SRVwI4GMYNfNIPsNKo+x8U0Z7lhkHxrKVmUaKuAV9UDA4XY/BjmYk7wqLgFkwQbDqZNwY0Bf7o52ned3kfOrI88MOfwewDeJWm5Xw6RCmvfCbxX71wIeEr+6bcHuGsaGE0Fww6972776P0UZGT8nrDu+AVDb4Un5BEY+k3F37aJTMe1Ov+gO4YqSraAClhmUThPuuGQ3Hm4/6IEzOxGkSgxxUFqFmN1RoZ2GW+7gIYl7NQjxmuHmjw6loeVp7NI4jxFHxs9Nj+dGxs/NTAwr6lLo9IObP5r0gTv2I8s04U7kOTeYaiWBdeo2qDZP7GnxNrC+gJ2wfMLlkIgIN4PXB1ekMO3zpNndkZ/ljtup7USxnfdpdkym7dtAwvGOxJAO5jvPcJ8KfhdhVYh0ENzAIuMSuSWOVUbMBdVhN/bhY9PTpyanGY5K7yrc3OF2uSXCsw8sdxl+RbiSTRNuZShRsgrBIVkszc2chNs2rn4bSvvXPbN8px1HgjsGsj5QHtyRD9w1dAq7w8vTHWJzyacjby3fIRWm7UnxbQYeoUgHwT05cQr7G3y0LpgEmqLO9MSpab8bQHRxbEJuQJXeFbh5F+rGEQ3S3j7yjkS5cIqUfarSi8hqBrfvHKiwBTVwbMLZJqY1PILb2NeG0kKKGWP2V33mjqc4hkNDWYK7driFdw5PKhaXtvv6KukucPN7PzbBfAlw05s/AOiR4K4dblAiJVpTAwQ3wW2sBihA2NjwWMNC2WuYDsFNcBurAYLb2KWtwdQZ1kRvuAsTU97vGhu2QjSdmjWgMdyFianllXsEd81rb3xDXeEWZMeFu85vXkb5eLFvHfiGsvE8tdQEtYQbyCa4WwqmVhuMlnArSlS+Qq9cxdm0LDceg5Kuc0iKNMpiDWgJt0KzksXTU9J1kuTrcni6iHeoSJ1DUnqnLNaAxnDv29ev/AjK8fSUtE2SeuQkO/VmSJ43ubi0XZxD2d2bHRx3ghHzPX0SbvsYS3Fyn6cLDre7DhcuT/fDMof3fm+KR1uySM3ZzkHZhf3CqnOACTu1Z2qUDfvqzLdPzW6DgzDZwYK8oTIMyup3Jo5YM8VUK9mQdRVw+x052VUoMrZc51Ay3F3nTXK4t6HTKOVxULhH3zoANySgiX2/wbFSbnD5AHyGbQOdL26DWxFkUkJooD0tNyOJn1EmLKLMYptq22kWjCgONPOcaSYPQ1NIwq4Lh5IfmGbfJxJ9l0xnDIoZBu4hYQ9bej5cmuv2UwbT5lmN4VYMNmRDVtRDiQRL+CTytGwbRJskVkcg7guu0p1vHcVgi2HYMsPh5veG+xhY4S/NjHexUwXtEwaVYVBWb7cEO9xAtng4qCwtZ8uhBIBwjpxEnqtTONRZGBXegn0PuA7vq8ly+8hEcDP5brdEOuJOHTE1NqPZjil+jqYyWcqCBtrCcitw+x05yegRYQczxQ5wtfNFe2/HDaSvVQZVioRvHWy5FZmwbY2woZTuk+goX9xGW0lF+Uq2LeBW5mxGNl90bLwZM0p8FgS3lm8F4l8FiTNhjEAt4W4d7XMnRLouwqtpwoM54dg0oaPW0XNtIyG4tbTctS12u7UiuAluYzVAcBu7tO1mp73zJbgJbmM1QHAbu7ReS9ZuJXrDTTGU7cZrrPlqDDfFUMZa6TasrCvcEGkWK0BYeXEq7nrjP63HbUv1m68BLeEGssVrUvitKd8Xp0CtBDeooh0SWsKtLAzBrSiEskIDWsKt0KxkQ5aWLHeIcsy7pDHc+H1ukRaUhyySDXeSMZRYpve1VQh2DBkVXWqQBjSGWzHYkA3RlAAx2RhKVaY74CBkMHSp0RrQGO46LHeSMZS25bZloqCeRi8eyQ/XgMZwg6lWEiETVkG0YxP569Hy3X/lVWnbMIsoNeebJLzcCV2DF1CVIN+QwdClRmtAY7ix5cZ8e1XGeXVATDaG0nPDMIdesJ7p2VUY7/KOh0qaowGN4cZA47RXcQrcycZQKnDj7/uAc+8dEpU0QQNtAXcT9EhdtKAGCG56K9BYDWgJdwsaCRpSC2qA4DbWbrUgbU0eEsFNcBurAYLb2KVtsplswe4IboLbWA0Q3MYubQua0iYPSW+4KYayybjo1Z3GcFMMpV6oNX+0usINkWaxYiiD9JsZubJmledGXN8IzvSMzZWtlUlXYZAEKm9BDWgJN5AdN4YyaAEI7iDNaF2uJdyKxvFbU+EBwkpDbxZba5z21qSS1teAlnArNCvZepSOgcbpemRS27Q0oDHc+H3uiDGU/ESysyuWValULKs8N1+yyleGe/q4W1I6vXdSXGJXy1eO7B0Fn5tXsKzSWSyhUqmszY+Jlcv0SLFQaN8bTjm5702mXGO4FYMN2RANCgQFZAI+DPcEO+PU2URCOjNZYncC325yCZCWlQMK2VXLKk2I4y2ZEJkOGSFdSlADGsNdg+VmmHJTLTQIWWG5/eEuuaDMTJYq7n9r82P+heg+4fbeuW0SXD8SFaIBjeEGU60kQmYLNIs6kA2B2yqX8VNCaIJ78S8kuPlJtlhRTU5rDDe23JhvrwY5u8yXqNEtYU/B5QNvLKG7NztRYp5GQCF3S7ib3t2bHZ4vk1viXZqGlmgMNwYap736AriZe8BJ9d1QCud4eL7ss6Hk+0VBJ0ioVCqwR/QWSpe9JN0YItu7Lo0uaQu4g5To604EVY5bDvvRuA2pflIaaC+4OXD24wtujOFZXlIKBTkEN6girYSWcNejLOw/NI5sejxSzxol1bbt4E5KcSSn9TVAcFOwgrEaILiNXdrWt6yNHiHBTXAbqwGC29ilbbRdbH35esNNMZStT1iKI9QYboqhTJEbLbrWFW6INEskhjL6UokPFi8ubT97vHNmEb5I35Hn77VGl0M1m6ABLeEGspOKoYyu6MHxHeLb9eib37umFgnuVty6aAm3wiJ+a6rOGEpFsjebL8rTReASP1aY4Ca4E3rBV6FZyQJ2jUgQ3I3QaoNkamm5Bc34fe6IMZT2ER/qOZS43HuWJJz+kS9uW1zavri0fWGxYzjXOcO9EWy5eVrWoRMoG4RsdLEaw60YbMiGTF5ADLCyk8ckoF2F2W1OufssSajW3ZsFyw2HngHcPMG2mPytKSYQjjgLGRJdapwGNIa7DsttnxkJR/XZCaFo5bg9266zVmFwj3YIuw7/k/FuHLhRJGsMN5hqJREybYwptq9qufugVH5VmuRwuIVLEzIAutRMDWgMN7bcmG+v+tBjO+5+FHeJOuy5HnJLwIsQrjPYXWzIw+DmzgwIyRfpEUrKj1A0hhsDjdNV4Q46hxK45Ebd2XSCLx7ulrBWQ50zi3JDiaV5h0QlTdBAW8ANelTcDyinhJEaILhT/tVpJFUtMikt4a5Zd2S5a1adjg3bC24dV4jGXLMGCG5yS4zVAMFt7NLWbPCMaUhwE9zGaoDgNnZpjTHANU9Eb7gphrLmhW+HhhrDTTGU7QBoPXPUFW6INGtCDGWjn47Dqy/1LCS19WpAS7iB7ObEUBLcXm60KNESbkWz8NbU3LV3GxFD2Wi4lelQNikNaAk30CwSby5cy7/2k77+A9292b797P9kfwjuZPXZNGnaw/3W8u8ODA5deOvq0o273b3ZpRt3f/7WQt/+l7EGbTqdt1jhfVTfl1SVUEi7OY8f4y+1oq87qC+4QtuFxR2F8Q4IX4DyxaXt8Ka4GCQfg3z5e3B8B4Rpim+h4Ibw8i0vZPJFZUUgnns7p7WH++TZN05PX/zVzdV9A6/kDh3ePzD4q5urp6cv4kUVdDpk4NDJYuegOCfSKWT3AMYF4ObBlChKMqCtuHNkp7NMvmQxILwS4IYEDF6QDYOBaE5Rju4xOSpoSInu3qz2cB/72evnf/HLC29dPVo4tbxy77XCqQtvXT3/i1/i1QU6RaGSVYwljrtx1S/KaGIsObwtiMqEhlcC0xJZ/iEr2bVPwBuLO/bcLQS3jy+qN9zdvdme7P6rv105e+HNE6fOLK/cO3HqzNkLb1797UpPdj9QqNAMWU6VNNIOYW6eINRyYXaH+H6axI77J8KmBrXFcIN/AqOCBDSXknm0MvNqhvpAArrEy1lImwSaT4fgNhHuex9t3ri/1j+Ym35zfnnl3vSb8/2DuRv31+59tOnQI3Dxhk4ijlE8JXNLbNdiV2G8S7kZ5KXqbXm8puOWOF96EOGVnGlOKrtPmM+dGeosjLq+DOHjljgCCW4foGHRtXdL4lnuotx+Oc43h17uyYodgjAcCilqAtwQW8k+yrP3Hwqz9vd3/Nu6N5Se8Eov3CJGU24o7VtR8O23yyS4jYZ7eeUe+NyvcZ/7aASfG9/cDU0rTkVQX4pbElSNyuNqQG+fe3nl3s8mp+FpySsHD+8beGX59/5PS+DxX1wdRa/Pbbx8qKd4FCFCmEfEnY2QOnSpBg1oCbcyz779L+Pn3BfeuhrwnFt+aEppnmyWm2HbXRnvChcunqKAmxRema7G1YAJcMedM9VvEw0Q3FU2JW3CgZHTJLgJbmM1QHAbu7RGGuNYkyK4CW5jNaA33BRDGcuStVtljeGmGMp2gzXufHWFGyLNDIihFGvG/+gT75Pe+L0A78LDn/e9l9qkREu4gWwzYigJ7gbdbFrCregCos60jqFM3HIrWmrDrJZwA80mxVAS3InfftrDbUAMpVhUDDdPyxdU8JsnuHwGvWgOr+mKwDMp0H5NvLs3q0QMwbu7/m/SmhKaqT3cBsRQKnALgoPiJqGcAypPu2Tv1tqBOWD/4E1aSDiXAg7aFF0bE5qpPdzwPrd2MZRghmVEGQOOx+OgGB+VYPebsfJpiV9wp23LuUCOMqZfedEcnrrwIZkTA6E33PEicXgElwuXCHGQUL9xMZSq5faBm0ejucuDBuaYZ+SWOJXbKTRTe7j1jaEEClW4vT6DEzcZIbiTMc1vBhvutg3N1B7uGNHvLRZDGQQ3t7LO94NcG0r+q2ZxaXtYcGdOhbttQzP1htuMGEqF8qSy3n1kUpJ1kaM93AbEUDaIFQrN1BJuhQZ9YyiViSSVpdBMoUkT4E6KCZJjmAYIbmNf1TeM1BqmQ3AT3MZqgOA2dmlrMHWGNUkH7tndL3394s6/fedb//ftv6OfpDTwt+986+sXd17c/ZJhjNY8nRTgvrT7+0ktJ8nx1cDPiW9+dEwKcH/94k7fJaHCpDTw9Ys7a7Z2JjVMAe7/feH5pFaR5Phq4K8vPG8SozXPJQW48XrUNu56YigbGjYLr5LiXthfCu2PbeN0bXMPaVW/YkOE63hJS7gVRStRZ8pVeNuzGZ8w9ryVKl5aggiDfNF1lJR3qPWURIcbXuD27S78qm+T1izUEm6FZiXrVXTTVgssNx4DBhqncZ1E0gS3okaN4d63r1/5EZQrM4SAF/am6GznAHsjFOJTdhRGO2cWWbQiOyLMPnMMDC1uC4Uh8kUX7Gw++13qfNEJhTw55aTZCTj8vW15aAn/krd9E7L3XcXn6P3q8KOePGGOmZ5dCtzetmLkuHxhtvPw+A54qzZf3HZ16oUprhCYizJfjbIaw60YbMh6tW9DA6ekAtw2QxxrFDsIFXDMlXNiE3QhQAGHJ1+0BdpwB7klvKEqXIwTUAuow9HnTjx49mIYGG7ftmjYMvhSlIjfJ/BrB6sLmuiY0BhuxWzv29fva7kVnxtsqnv58cl3Mg2GXBhX78m/TLLbyYYs9BIIt/1bAgtXqPIdgHvYXYVZJwLNBbeffABU7cgdZKlchVbaJTSGG0y1kvCuAV4twM5NiT/cVY+qAZpFp5CFXkLg9grH4/TeObILBiI2+YFwe+WDZtSOCG5QTZ0JbGBqEyVoxpYb8+2VidcSsKsON19y5HIwX0URzoVAXCM6eDKSW+L4OfJkSn50IPSIhfObxNdZ4nDzoWLF+rdloxL1fdwSCG7A6lLmq1e2LSw3fJXGZ0Mpz2PH5hClGQ1yFwjMKQsMzgP7dsJ4h9wIVoObGWaPcC9VfnXQ8NjNYFvxoU4Mt798G26skKNT9j4BHUUrPuITYvsVJbRmtl3gbk3tJzsqBe5khesojeA255VXglu5A7WEW5lD07LCkYXnG+LpeNN6r9oRwa2oiOAmy22OBghuY9eSLLcK91Bxo8k/eA2a3LXZ3ZFilfV9Tsk3IUtr0CAlk2IVxRLczf7FpSxAglmCW1Gm3nCfWWJo/vr+pvPz3qYyw/bJEtzKWmsM95klhrUKNy9RJtkmWYJbWWhd4RZk+8D9XiQ3I3f588fWF0uX1xV15N55Zj39/MSUWq5Ua80swa2si5ZwA9leuK9Fc0taFu7c1Gd3LOvxrS1lnaJkCW5FS1rCrczBcbjvb0aEW5EA2dQt94lbX1QqFYIbVqSehJZwY5ohvXBnszC3cfhCJLckSGXpws1+nzz9fOkDstx1LSIsriFwX729+er5zQOlL7Mb32Q3vjlQ+nLonHxskpvaWnrq4HLi1hfW08+PM5/72RvctxaeQMX+Bz43LgdTigstWwJok1/9Yumdzx9bVqVSufPOeu6dZ0KwkCAGc+cd5nuI8jvvSP/evrT+BsGd0B8WDYG7MLdxoPRl37O/9N//qv/+V33P/sL4tnUE9hgA4j43g1vACoS98YEl4JaY8h2n08rtEDOxbr6FNCmBY2198NlQcQN1x+40aIUlANOQgPFHTJDPrSjKELgPn9/Irn/Tf/8rMb3++19l17+BqQKpDmS25QbuRWXIgsW1DTrzg+GqrMx/J8CNwSBm9MuHML5puE8UCVgywQ0LV2fCELgPvuHy0vY/+i8M91BxQxBz4tYX0j2IADf4J6BijCBHeWvpqet5oi/QGHpfuG8vby49lV4K3Eve3mEYQQmy3IpmDIFbbCv53DYP3P2Pvn/+y8Div+Kp8r3aszs2i44J556GsL6CPOSWWGCV3/jA8WEc/9t+KM6lMcojwi18laHiBtsAuB0buA/x4COmCW5FUabB/fLv/r3v2X8PLP+bMk8JLveAsRPM0vaez7K+WLrl/BGHI+vd9jl7QeAyLtx3PpC7TJCAR0tuCdZGPWnj4C592f/gP+vRSEPbKm5Jsn2R5Vb0aRrcA9f+OHDtj8okWydLcDdzLbSEO0RB+z/88/4P/xxSId1LBHcz9W8a3M3UXav1RW6JsiIEt+sZoqIdvbIEt7JeFJjKxvQAAAKXSURBVP1uTrwwhlsJlW3PLMFNcJujAeUeJrjNWVqy3C0Kd2Fiqrs3iz/WurxyTxkrZcM1QHAr+mkJyy1OJyO4lbWJmyW4FY2lDzecu5cI3JmRK2tWeW5E/ZC2Mu3oWRCY6RmbK1srk4lJjj6GiDUJbkVRKcMNZAsnpH63BFhU5llzFgQS3DXrMK2GKcOtTLt+uEFgVRZxBZwGCUoiSh1oEqsytKozQZZbUWDKcCs0K1llrLGyVfHCFXA6qJcodaBtrMrQqs4Ewa0osCXgxqfbiLSgXBmrks30nF3hkYiWVZ6bL1nlK8M9fdyLKJ3eOykuVSoVVC7fX12Z7IO2lUrlX9ZWVu2IRqt85cjeUe5bM+Esm7+yZpUm2LGRwueWnVYqFeF/ix4n+HE5gb3bQ61UKmvzY2Iiw/NlEZpgcfnK7GrIEtyK0loCbsVgQ1YZK84KOiVeHDsEMWaRnz3ZMzZXYuizk2ImSwImG1a7gr1ZFOUAHLCrloMctoVlPTLhdtot/OyKvceFcqiJJ1VnmuBWFNgScNdguRmj3FSL+UAWoAGMYMKKpcQVgtJBvLJy++EJ9BhUOTNZguAxkVibHxM3J/udk9yDHYIb1lokWgJuMNVKQhkrzgLNohCygBrAZzNnCX/Av4JNKqZWSvYzxrgaCLQ78vzecN+HrlmI3zkJIU5wY91292ZTgPuvLzyPl4HSiWvgry88ryxze2ZTgPtP392Z+HKSQKyBP313Z3vSrMw6Bbgv7f4+XglKJ66BC7v/Xlnm9symAHd3b/bS7pe+frHjby98K/F1bWeB//Od5//03Z0Xd7/Unih7Z50O3N5x1FDCd3Ly0TU8PK5BDjUxVQMaw23qktC8ktIAwW1OsEJSTBgj5/8BuSLzmOs50NAAAAAASUVORK5CYII=)

![](/assets/images/agent-zero-08-f040087da5fa37caa0fa94399e22e464.png)

![](/assets/images/agent-zero-09-f745e0628a6fafe58231ca5987e2bad3.png)

*   [Prerequisites](#prerequisites)
    
*   [Configure the agent](#configure-the-agent)
    
*   [Run the agent](#run-the-agent)
    *   [Example 1](#example-1)
        
    *   [Example 2](#example-2)
        
    *   [Example 3](#example-3)
        
    *   [Example 4](#example-4)
        
    *   [Example 5](#example-5)

---

# LlamaCoder | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

LlamaCoder is an open-source tool designed to generate small apps with a single prompt. It leverages LLM to help you quickly create and enhance React applications.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use a public Llama3 node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, we will need to get the source code of the forked LlamaTutor

    git clone https://github.com/second-state/llamacoder.gitcd llamacoder

Next, configure the `.env` file.

    cp .example.env .env

You will need to configure three parameters here.

*   LLAMAEDGE\_BASE\_URL: URL for the LLM API base URL.
*   LLAMAEDGE\_MODEL\_NAME: Name of the model to be used.
*   LLAMAEDGE\_API\_KEY: API key for accessing the LLM services.

For example, you can use the following `.env` setting.

    LLAMAEDGE_BASE_URL=https://llama.us.gaianet.network/v1LLAMAEDGE_MODEL_NAME=llamaLLAMAEDGE_API_KEY=GaiaNet

Then, we will need to install the required dependencies.

    npm install

Next, let's run the application locally.

    npm run dev

Finally, open [http://localhost:3000](http://localhost:3000)
 in your browser and start to build your React app.

![](/assets/images/llamacoder-ca7bef8ba04ce6f5ad1ed95898144cde.png)

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# A planning agent | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The [gpt planner](https://github.com/mshumer/gpt-prompt-engineer/blob/main/gpt_planner.ipynb)
 is a Python application that demonstrate the planning capabilities of LLMs. When you run it, it will ask the LLM to generate multiple action plans for a goal or a query. It will then ask the LLM to compare and select the best plan, and then rewrite it to answer the user query.

Since the program uses the official OpenAI Python library, we can [easily change it to use a Gaia node](/1.0.0/intro)
.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [run your own node](/1.0.0/node-guide/quick-start)
    
*   [use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use a public node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) |
| Model Name | llama |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, [load the nodebook in colab](https://colab.research.google.com/github/mshumer/gpt-prompt-engineer/blob/main/gpt_planner.ipynb)
.

Edit the code to create an OpenAI client. We will pass in the `base_url` here.

    client = openai.OpenAI(base_url="https://llama.us.gaianet.network/v1", api_key=OPENAI_API_KEY)

Next, replace all the `gpt-4o-mini` model name with the `llama` model name in the code. Here is an example.

    response = client.chat.completions.create(    model="llama",    messages=[        {"role": "system", "content": "You are a helpful assistant. Use the given plan to create a detailed and high-quality response to the user's query."},        {"role": "user", "content": f"User Query: {user_query}\n\nPlan: {best_plan}\n\nGenerate a detailed response based on this plan."}    ],    temperature=0.5,    max_tokens=2000)

Change the query to your own.

    user_query = "How do I debug a TLS connection timeout?"

Finally, run the notebook to see the results!

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# LlamaEdgeBook | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

LlamaEdgeBook, forked from GroqBook, is an open-source tool that scaffolds the creation of books from a one-line prompt using open-source LLMs. You can configure the LlamaEdgeBook framework using any GaiaNet node as the backend LLM API.

Steps[​](#steps "Direct link to Steps")

----------------------------------------

First, get the source code of the LlamaEdgeBook. Open your terminal and enter the following command line.

    git clone https://github.com/second-state/LlamaEdgeBookcd LlamaEdgeBook

Ensure you have Python 3.11 or later installed. Then, install the necessary packages:

    pip install -r requirements.txt

Next, let's configure the GaiaNet node as the LLM backend.

    export OPENAI_BASE_URL="https://gemma.us.gaianet.network/v1"export OPENAI_MODEL_NAME="gemma" export OPENAI_API_KEY="GAIANET" 

**Hint:** if you don't know the model name of the GaiaNet node, you can retrieve the model information using:

    curl -X POST https://0x57b00e4f3d040e28dc8aabdbe201212e5fb60ebc.us.gaianet.network/v1/models

Then, use the following command line to run the app.

    streamlit run main.py

When the app runs successfully, the terminal will output the following information.

    Collecting usage statistics. To deactivate, set browser.gatherUsageStats to false.  You can now view your Streamlit app in your browser.  Local URL: http://localhost:8501  Network URL: http://10.128.0.8:8501  External URL: http://35.222.115.181:8501

Finally, you can open the `http://localhost:8501` in your browser to generate a book.

![](/assets/images/book-01-8612df6e0c495f2d68979b76a4cb7ca4.png)

The LlamaEdgeBook will first generate an outline based on your prompt, and then create the chapter content based on the outline.

![](/assets/images/book-02-3c158830817920dbdb920985c0dd6ce5.png)

You can also download the book after the generation is complete.

![](/assets/images/book-03-1a63c3c4eb72eaf948d529875ee479a3.png)

*   [Steps](#steps)

---

# Gaia nodes with long-term knowledge | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The LLM app requires both long-term and short-term memories. Long-term memory includes factual knowledge, historical facts, background stories etc. They are best added to the context as complete chapters instead of small chunks of text to maintain the internal consistency of the knowledge.

[RAG](https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/)
 is an important technique to inject contextual knowledge into an LLM application. It improves accuracy and reduces the hallucination of LLMs. An effective RAG application combines real-time and user-specific short-term memory (chunks) with stable long-term memory (chapters) in the prompt context.

Since the application's long-term memory is stable (even immutable), we package it in a vector database tightly coupled with the LLM. The client app assembles the short-term memory in the prompt and is supplemented with the long-term memory on the LLM server. We call the approach "server-side RAG".

> The long context length supported by modern LLMs are especially well-suited for long-term knowledge that are best represented by chapters of text.

A Gaia node is an OpenAI compatible LLM service that is grounded by long-term knowledge on the server side. The client application can simply chat with it or provide realtime / short-term memory since the LLM is already aware of the domain or background.

> For example, if you ask ChatGPT the question What is Layer 2, the answer is that Layer 2 is a concept from the computer network. However, if you ask a blockchain person, they answer that Layer 2 is a way to scale the original Ethereum network. That's the difference between a generic LLM and knowledge-supplemented LLMs.

We will cover the external knowledge preparation and how a knowledge-supplemented LLM completes a conversation. If you have learned how a RAG application works, go to [Build a RAG application with Gaia](/1.0.0/creator-guide/knowledge/web-tool)
 to start building one.

1.  Create embeddings for your own knowledge as the long-term memory.
2.  Lifecycle of a user query on a knowledge-supplemented LLM.

For this solution, we will use

*   a chat model like Llama-3-8B for generating responses to the user.
*   a text embedding model like [nomic-embed-text](https://huggingface.co/second-state/Nomic-embed-text-v1.5-Embedding-GGUF)
     for creating and retrieving embeddings.
*   a Vector DB like Qdrant for storing embeddings.

Workflow for creating knowledge embeddings[​](#workflow-for-creating-knowledge-embeddings "Direct link to Workflow for creating knowledge embeddings")

-------------------------------------------------------------------------------------------------------------------------------------------------------

The first step is to create embeddings for our knowledge base and store the embeddings in a vector DB.

![create-embedding](https://github.com/GaiaNet-AI/docs/assets/45785633/2ff40178-64f4-4e2e-bbd9-f12ce35186b7)

First of all, we split the long text into sections (i.e, chunks). All LLMs have a maximum context length. The model can't read the context if the text is too long. The most used rule for a Gaia node is to put the content in one chapter together. Remember, insert a blank line between two chunks. You can also use other algorithms to chunk your text.

After chunking the document, we can convert these chunks into embeddings leveraging the embedding model. The embedding model is trained to create embeddings based on text and search for similar embeddings. We will use the latter function in the process of user query.

Additionally, we will need a vector DB to store the embeddings so that we can retrieve these embeddings quickly at any time.

On a Gaia node, we will get a database snapshot with the embeddings to use at last. Check out how to create your embeddings using [Gaia web tool](/1.0.0/creator-guide/knowledge/web-tool)
, [from a plain text file](/1.0.0/creator-guide/knowledge/text)
, and [from a markdown file](/1.0.0/creator-guide/knowledge/markdown)
.

Lifecycle of a user query on a knowledge-supplemented LLM[​](#lifecycle-of-a-user-query-on-a-knowledge-supplemented-llm "Direct link to Lifecycle of a user query on a knowledge-supplemented LLM")

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Next, let's learn the lifecycle of a user query on a knowledge-supplemented LLM. We will take [a Gaia Node with Gaia knowledge](https://knowledge.gaianet.network/chatbot-ui/index.html)
 as an example.

![user-query-rag](https://github.com/GaiaNet-AI/docs/assets/45785633/c64b85ea-65f0-43d2-8ab3-78889d21c248)

### Ask a question[​](#ask-a-question "Direct link to Ask a question")

when you send a question in human language to the node, the embedding model will first convert your question to embedding.

### Retrieve similar embeddings[​](#retrieve-similar-embeddings "Direct link to Retrieve similar embeddings")

Then, the embedding model will search all the embeddings stored in the Qdrant vector DB and retrieve the embeddings that are similar to the question embeddings.

### Response to the user query[​](#response-to-the-user-query "Direct link to Response to the user query")

The embedding node will return the retrieved embeddings to the chat model. The chat model will use the retrieved embeddings plus your input questions as context to answer your queries finally.

*   [Workflow for creating knowledge embeddings](#workflow-for-creating-knowledge-embeddings)
    
*   [Lifecycle of a user query on a knowledge-supplemented LLM](#lifecycle-of-a-user-query-on-a-knowledge-supplemented-llm)
    *   [Ask a question](#ask-a-question)
        
    *   [Retrieve similar embeddings](#retrieve-similar-embeddings)
        
    *   [Response to the user query](#response-to-the-user-query)

---

# LlamaTutor | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

The [LlamaTutor](https://github.com/Nutlope/llamatutor)
 is a TypeScript application using Llama 3.1 to act as an open-source AI personal tutor. When you run it, it will ask the LLM to search on the web based on your inquiry topic and then generate content for you to learn.

The program didn't use the official OpenAI library, we can use the forked one that supports [using a Gaia node](/1.0.0/intro)
 as the LLM backend.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

You will need a Gaia node ready to provide LLM services through a public URL. You can

*   [Run your own node](/1.0.0/node-guide/quick-start)
    
*   [Use a public node](/1.0.0/user-guide/nodes)
    

In this tutorial, we will use a public Llama3 node.

| Attribute | Value |
| --- | --- |
| API endpoint URL | [https://llama.us.gaianet.network/v1](https://llama.us.gaianet.network/v1) |
| Model Name | llama |
| API KEY | gaia |

Run the agent[​](#run-the-agent "Direct link to Run the agent")

----------------------------------------------------------------

First, we will need to get the source code of the forked LlamaTutor

    git clone https://github.com/JYC0413/llamatutor.gitcd llamatutor

Next, configure the `.env` file.

    cp .example.env .env

You will need to configure four parameters here.

*   SERPER\_API\_KEY: The [serper API key](https://serper.dev/)
     for searching content online. You can also use BING\_API\_KEY here.
*   LLAMAEDGE\_BASE\_URL: URL for the LLM API base URL.
*   LLAMAEDGE\_MODEL\_NAME: Name of the model to be used.
*   LLAMAEDGE\_API\_KEY: API key for accessing the LLM services.

Then, we will need to install the required dependencies.

    npm install

Next, let's run the application locally.

    npm run dev

Finally, open [http://localhost:3000](http://localhost:3000)
 in your browser and start to learn.

![](/assets/images/llamatutor-01-f14f9e39e0ece9f4ae961504fe4d5eda.png)

*   [Prerequisites](#prerequisites)
    
*   [Run the agent](#run-the-agent)

---

# Fine-tune LLMs | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

You could fine-tune an open-source LLM to

*   Teach it to follow conversations.
*   Teach it to respect and follow instructions.
*   Make it refuse to answer certain questions.
*   Give it a specific "speaking" style.
*   Make it response in certain formats (e.g., JSON).
*   Give it focus on a specific domain area.
*   Teach it certain knowledge.

To do that, you need to create a set of question and answer pairs to show the model the prompt and the expected response. Then, you can use a fine-tuning tool to perform the training and make the model respond the expected answer for each question.

---

# Knowledge base from a plain text file | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a plain text file. The snapshot file can then be [loaded by a Gaia node as its knowledge base](/1.0.0/node-guide/customize#select-a-knowledge-base)
.

The text file is segmented into multiple chunks by blank lines. [See an example](https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt)
. Each chunk is turned into a vector, and when retrieved, added to the prompt context for the LLM.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

Install the WasmEdge Runtime, the cross-platform LLM runtime.

    curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Download an embedding model.

    curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base.

Start a vector database[​](#start-a-vector-database "Direct link to Start a vector database")

----------------------------------------------------------------------------------------------

By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](/1.0.0/node-guide/quick-start)
.

note

Or, you can start a Qdrant server using Docker. The following command starts it in the background.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Create the vector collection snapshot[​](#create-the-vector-collection-snapshot "Direct link to Create the vector collection snapshot")

----------------------------------------------------------------------------------------------------------------------------------------

Delete the default collection if it exists.

    curl -X DELETE 'http://localhost:6333/collections/default'

Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model.

    curl -X PUT 'http://localhost:6333/collections/default' \  -H 'Content-Type: application/json' \  --data-raw '{    "vectors": {      "size": 768,      "distance": "Cosine",      "on_disk": true    }  }'

Download a program to chunk a document and create embeddings.

    curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/paragraph_embed/paragraph_embed.wasm

It chunks the document based on empty lines. So, you MUST prepare your source document this way -- segment the document into sections of around 200 words with empty lines. You can check out the [Rust source code here](https://github.com/GaiaNet-AI/embedding-tools/tree/main/paragraph_embed)
 and modify it if you need to use a different chunking strategy.

> The `paragraph_embed.wasm` program would NOT break up code listings even if there are empty lines with in the listing.

Next, you can run the program by passing a collection name, vector dimension, and the source document. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from [paris\_chunks.txt](https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt)
 and saves them into the default collection.

    curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txtwasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \  paragraph_embed.wasm embedding default 768 paris_chunks.txt -c 8192

### Options[​](#options "Direct link to Options")

You can pass the following options to the program.

*   Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length.
*   Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection.
*   Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512.

Example: the above example but to append the London guide to the end of an existing collection starting from index 42.

    wasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \   paragraph_embed.wasm embedding default 768 london.txt -c 8192 -s 42

Create a vector snapshot[​](#create-a-vector-snapshot "Direct link to Create a vector snapshot")

-------------------------------------------------------------------------------------------------

You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node.

    curl -X POST 'http://localhost:6333/collections/default/snapshots'

We also recommend you to compress the snapshot file.

    tar czvf my.snapshot.tar.gz my.snapshot

Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](/1.0.0/node-guide/customize#select-a-knowledge-base)
.

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Start](/1.0.0/node-guide/quick-start)
     a new Gaia node
*   [Customize](/1.0.0/node-guide/customize)
     the Gaia node

Have fun!

*   [Prerequisites](#prerequisites)
    
*   [Start a vector database](#start-a-vector-database)
    
*   [Create the vector collection snapshot](#create-the-vector-collection-snapshot)
    *   [Options](#options)
        
*   [Create a vector snapshot](#create-a-vector-snapshot)
    
*   [Next steps](#next-steps)

---

# Knowledge base from source / summary pairs | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot for optimal retrieval of long-form text documents. The approach is to create two columns of text in a CSV file.

*   The first column is the long-form source text from the knowledge document, such as a book chapter or a markdown section.
*   The long-form source text is difficult to search. The second column is a "search-friendly" summary of the source text. It could contain a list of questions that can be answered by the first column source text.

We will create a vector snapshot where each vector is computed from the summary text (second column), but the retrieved source text for that vector is from the first column. The snapshot file can then be [loaded by a Gaia node as its knowledge base](/1.0.0/node-guide/customize#select-a-knowledge-base)
.

> We have a simple Python script to build properly formatted CSV files from a set of articles or chapters. [See how it works](https://github.com/GaiaNet-AI/embedding-tools/tree/main/csv_embed#create-a-csv-file)
> .

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

Install the WasmEdge Runtime, the cross-platform LLM runtime.

    curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Download an embedding model.

    curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base.

Start a vector database[​](#start-a-vector-database "Direct link to Start a vector database")

----------------------------------------------------------------------------------------------

By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](/1.0.0/node-guide/quick-start)
.

note

Or, you can start a Qdrant server using Docker. The following command starts it in the background.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Create the vector collection snapshot[​](#create-the-vector-collection-snapshot "Direct link to Create the vector collection snapshot")

----------------------------------------------------------------------------------------------------------------------------------------

Delete the default collection if it exists.

    curl -X DELETE 'http://localhost:6333/collections/default'

Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model.

    curl -X PUT 'http://localhost:6333/collections/default' \  -H 'Content-Type: application/json' \  --data-raw '{    "vectors": {      "size": 768,      "distance": "Cosine",      "on_disk": true    }  }'

Download a program to create embeddings from the CSV file.

    curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/csv_embed/csv_embed.wasm

You can check out the [Rust source code](https://github.com/GaiaNet-AI/embedding-tools/tree/main/csv_embed)
 here and modify it if you need to use a different CSV layout.

Next, you can run the program by passing a collection name, vector dimension, and the CSV document. The `--ctx_size` option matches the embedding model's context window size, which in this case is 8192 tokens allowing it to process long sections of text. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from `paris.csv` and saves them into the default collection.

    curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris.csvwasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \  csv_embed.wasm embedding default 768 paris.csv --ctx_size 8192

### Options[​](#options "Direct link to Options")

You can pass the following options to the program.

*   Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512.
*   Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length.
*   Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection.

Example: the above example but to append the London guide to the end of an existing collection starting from index 42.

    wasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \   csv_embed.wasm embedding default 768 london.csv -c 8192 -l 1 -s 42

Create a vector snapshot[​](#create-a-vector-snapshot "Direct link to Create a vector snapshot")

-------------------------------------------------------------------------------------------------

You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node.

    curl -X POST 'http://localhost:6333/collections/default/snapshots'

We also recommend you to compress the snapshot file.

    tar czvf my.snapshot.tar.gz my.snapshot

Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](/1.0.0/node-guide/customize#select-a-knowledge-base)
.

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Start](/1.0.0/node-guide/quick-start)
     a new Gaia node
*   [Customize](/1.0.0/node-guide/customize)
     the Gaia node

Have fun!

*   [Prerequisites](#prerequisites)
    
*   [Start a vector database](#start-a-vector-database)
    
*   [Create the vector collection snapshot](#create-the-vector-collection-snapshot)
    *   [Options](#options)
        
*   [Create a vector snapshot](#create-a-vector-snapshot)
    
*   [Next steps](#next-steps)

---

# Build a knowledge base using Gaia web tool | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

GaiaNet has developed a tool for making vector collection snapshot files, so everyone can easily create their own knowledge base.

Access it here: [https://tools.gaianet.xyz/](https://tools.gaianet.xyz/)

Segment your text file[​](#segment-your-text-file "Direct link to Segment your text file")

-------------------------------------------------------------------------------------------

First, copy unformatted text into a txt file. Then follow the two rules to chunk your content, ie putting similar content together.

*   Each title and related content are a chunk. There is no blank lines in one chunk.
*   Use a blank line to recognize different chunks.

After that, save it as a txt file.

For example, below is your source.

![The input knowledge in a text file](/assets/images/web_tool_input-1b1323226ee6bfd7974431e25296b961.png)

After formatted, it will look like the following.

    What is a blockchain?A blockchain is a distributed, cryptographically-secure database structure that allows network participants to establish a trusted and immutable record of transactional data without the need for intermediaries. A blockchain can execute a variety of functions beyond transaction settlement, such as smart contracts. Smart contracts are digital agreements that are embedded in code and can have limitless formats and conditions. Blockchains have proven themselves as superior solutions for securely coordinating data, but they are capable of much more, including tokenization, incentive design, attack-resistance, and reducing counterparty risk. The very first blockchain was the Bitcoin blockchain, which was itself a culmination of over a century of advancements in cryptography and database technology.What is blockchain software?Blockchain software is like any other software. The first of its kind was Bitcoin, which was released as open source software, making it available to anyone to use or change. There are a wide variety of efforts across the blockchain ecosystem to improve upon Bitcoin's original software. Ethereum has its own open source blockchain software. Some blockchain software is proprietary and not available to the public.

Generate the snapshot file[​](#generate-the-snapshot-file "Direct link to Generate the snapshot file")

-------------------------------------------------------------------------------------------------------

1.  Visit this URL: [https://tools.gaianet.xyz/](https://tools.gaianet.xyz/)
    , upload the above prepared txt file.
2.  Edit your `dbname` . _**Note: Do not include spaces or special characters in the dbname**_.
3.  Choose Embedding model, we suggest use `nomic-embed-text-v1.5.f16`.
4.  Click the "Make RAG" button and wait.

When finished, the chatbot will display GaiaNet Node config info. It is a JSON format as follows.

    {  "embedding": "https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf",  "embedding_ctx_size": 768,  "snapshot": "https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolve/main/test/test.snapshot"}

Update the node config[​](#update-the-node-config "Direct link to Update the node config")

-------------------------------------------------------------------------------------------

Run the following

    gaianet config \  --snapshot https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolve/main/test/test.snapshot \  --embedding-url https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf \  --embedding-ctx-size 768

and then

    gaianet initgaianet start

Have fun!

*   [Segment your text file](#segment-your-text-file)
    
*   [Generate the snapshot file](#generate-the-snapshot-file)
    
*   [Update the node config](#update-the-node-config)

---

# Knowledge base from a URL | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a Web URL. First, we will parse the URL to a structured markdown file. Then, we will follow the steps from [Knowledge base from a markdown file](/1.0.0/creator-guide/knowledge/markdown)
 to create embedding for your URL.

Parse the URL content to a markdown file[​](#parse-the-url-content-to-a-markdown-file "Direct link to Parse the URL content to a markdown file")

-------------------------------------------------------------------------------------------------------------------------------------------------

Firecrawl can crawl and convert any website into LLM-ready markdown or structured data. It also supports crawling a URL and all accessible subpages.

> To use Firecrawl, you need to sign up on [Firecrawl](https://firecrawl.dev/)
>  and get an API key.

First, install the dependencies. We are assuming that you already have Node.JS 20+ installed.

    git clone https://github.com/JYC0413/firecrawl-integration.gitcd firecrawl-integrationnpm install

Then, export the API key in the terminal.

    export FIRECRAWL_KEY="your_api_key_here"

next, we can use the following command line to run the service.

    node crawlWebToMd.js

After the application is running successfully, you will see the prompt appear on the Terminal.

![](/assets/images/firecrawl-01-b1a611217130d603f863d886375f865a.png)

You can type your URL in the terminal right now. Here we have two choices.

*   Multiple pages: input your link with `/` at the end, the program will crawl and convert the page and its subpages to one single markdown file. This way will cost lots of API token usage.
*   One single page: input your link without `/` at the end. the program will crawl and convert the current page to one single markdown file.

The output markdown file will be located in this folder named `output.md`.

Create embeddings from the markdown files[​](#create-embeddings-from-the-markdown-files "Direct link to Create embeddings from the markdown files")

----------------------------------------------------------------------------------------------------------------------------------------------------

Please follow the tutorial [Knowledge base from a markdown file](/1.0.0/creator-guide/knowledge/markdown)
 to convert your markdown file to a snapshot of embeddings that can be imported into a GaiaNet node.

*   [Parse the URL content to a markdown file](#parse-the-url-content-to-a-markdown-file)
    
*   [Create embeddings from the markdown files](#create-embeddings-from-the-markdown-files)

---

# Knowledge base from a PDF file | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a PDF file. First, we will parse the unstructured PDF file to a structured markdown file. Then, we will follow the steps from [Knowledge base from a markdown file](/1.0.0/creator-guide/knowledge/markdown)
 to create embedding for your PDF files.

Tools to convert a PDF file to a markdown file[​](#tools-to-convert-a-pdf-file-to-a-markdown-file "Direct link to Tools to convert a PDF file to a markdown file")

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

### Tool #1: LlamaParse[​](#tool-1-llamaparse "Direct link to Tool #1: LlamaParse")

LlamaParse is a tool to parse files for optimal RAG. You will need a LlamaCloud key from [https://cloud.llamaindex.ai](https://cloud.llamaindex.ai)
.

First, install the dependencies. we are assuming that you already have Node.JS 20+ installed.

    git clone https://github.com/alabulei1/llamaparse-integration.gitcd llamaparse-integrationnpm install llamaindexnpm install dotenv

Then, edit the `.env` file to set up the PDF file path and LlamaCloud Key. In this case, you don't need to care about the LLM-related settings.

After that, run the following command line to parse your pdf into a markdown file.

    npx tsx transMd.ts

The output markdown file will be located in this folder named `output.md` by default. You can change the path in the `.env` file.

### Tool #2: GPTPDF[​](#tool-2-gptpdf "Direct link to Tool #2: GPTPDF")

GPTPDF is an open-source tool using GPT-4o to parse PDF into markdown. You will need an OpenAI key here.

First, install the gptpdf software.

    pip install gptpdf

Then, enter the Python environment.

    python

Next, use the following command to parse your pdf.

    from gptpdf import parse_pdfapi_key = 'Your OpenAI API Key'content, image_paths = parse_pdf(Your_Pdf_Path, api_key=api_key)print(content)

The output markdown files called `output.md` will be located in your root directory.

Create embeddings from the markdown files[​](#create-embeddings-from-the-markdown-files "Direct link to Create embeddings from the markdown files")

----------------------------------------------------------------------------------------------------------------------------------------------------

Please follow the tutorial [Knowledge base from a markdown file](/1.0.0/creator-guide/knowledge/markdown)
 to convert your markdown file to a snapshot of embeddings that can be imported into a GaiaNet node.

*   [Tools to convert a PDF file to a markdown file](#tools-to-convert-a-pdf-file-to-a-markdown-file)
    *   [Tool #1: LlamaParse](#tool-1-llamaparse)
        
    *   [Tool #2: GPTPDF](#tool-2-gptpdf)
        
*   [Create embeddings from the markdown files](#create-embeddings-from-the-markdown-files)

---

# Knowledge base from a markdown file | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

In this section, we will discuss how to create a vector collection snapshot from a markdown file. The snapshot file can then be [loaded by a Gaia node as its knowledge base](/1.0.0/node-guide/customize#select-a-knowledge-base)
.

The markdown file is segmented into multiple sections by headings. [See an example](https://huggingface.co/datasets/gaianet/paris/raw/main/paris.md)
. Each section is turned into a vector, and when retrieved, added to the prompt context for the LLM.

Prerequisites[​](#prerequisites "Direct link to Prerequisites")

----------------------------------------------------------------

Install the WasmEdge Runtime, the cross-platform LLM runtime.

    curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Download an embedding model.

    curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base.

Start a vector database[​](#start-a-vector-database "Direct link to Start a vector database")

----------------------------------------------------------------------------------------------

By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](/1.0.0/node-guide/quick-start)
.

note

Or, you can start a Qdrant server using Docker. The following command starts it in the background.

    mkdir qdrant_storagemkdir qdrant_snapshotsnohup docker run -d -p 6333:6333 -p 6334:6334 \    -v $(pwd)/qdrant_storage:/qdrant/storage:z \    -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \    qdrant/qdrant

Create the vector collection snapshot[​](#create-the-vector-collection-snapshot "Direct link to Create the vector collection snapshot")

----------------------------------------------------------------------------------------------------------------------------------------

Delete the default collection if it exists.

    curl -X DELETE 'http://localhost:6333/collections/default'

Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model.

    curl -X PUT 'http://localhost:6333/collections/default' \  -H 'Content-Type: application/json' \  --data-raw '{    "vectors": {      "size": 768,      "distance": "Cosine",      "on_disk": true    }  }'

Download a program to segment the markdown document and create embeddings.

    curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/markdown_embed/markdown_embed.wasm

It chunks the document based on markdown sections. You can check out the [Rust source code](https://github.com/GaiaNet-AI/embedding-tools/tree/main/markdown_embed)
 here and modify it if you need to use a different chunking strategy.

Next, you can run the program by passing a collection name, vector dimension, and the source document. You can pass in the desired markdown heading level for chunking using the `--heading_level` option. The `--ctx_size` option matches the embedding model's context window size, which in this case is 8192 tokens allowing it to process long sections of text. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from `paris.md` and saves them into the default collection.

    curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris.mdwasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \  markdown_embed.wasm embedding default 768 paris.md --heading_level 1 --ctx_size 8192

### Options[​](#options "Direct link to Options")

You can pass the following options to the program.

*   Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512.
*   Using `-l` or `--heading_level` to specify the markdown heading level for each vector. This defaults to 1.
*   Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length.
*   Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection.

Example: the above example but to append the London guide to the end of an existing collection starting from index 42.

    wasmedge --dir .:. \  --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \   markdown_embed.wasm embedding default 768 london.md -c 8192 -l 1 -s 42

Create a vector snapshot[​](#create-a-vector-snapshot "Direct link to Create a vector snapshot")

-------------------------------------------------------------------------------------------------

You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node.

    curl -X POST 'http://localhost:6333/collections/default/snapshots'

We also recommend you to compress the snapshot file.

    tar czvf my.snapshot.tar.gz my.snapshot

Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](/1.0.0/node-guide/customize#select-a-knowledge-base)
.

Next steps[​](#next-steps "Direct link to Next steps")

-------------------------------------------------------

*   [Start](/1.0.0/node-guide/quick-start)
     a new Gaia node
*   [Customize](/1.0.0/node-guide/customize)
     the Gaia node

Have fun!

*   [Prerequisites](#prerequisites)
    
*   [Start a vector database](#start-a-vector-database)
    
*   [Create the vector collection snapshot](#create-the-vector-collection-snapshot)
    *   [Options](#options)
        
*   [Create a vector snapshot](#create-a-vector-snapshot)
    
*   [Next steps](#next-steps)

---

# Install multiple nodes on a single machine | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

The [default GaiaNet installer](/1.0.0/node-guide/quick-start)
 installs the node into the `$HOME/gaianet` base directory. You could install multiple nodes on the same machine. Each node has its own "base directory". To do that, you just need to use the `--base` option.

Let's say that the base directory for your second node is `$HOME/node-2`. You can install GaiaNet node software using the following command.

    curl -sSfL 'https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/install.sh' | bash -s -- --base $HOME/node-2

After that, you can specify the `gaianet` CLI to operate on `node-2` by appending the `--base` option as well.

    gaianet init --base $HOME/node-2gaianet start --base $HOME/node-2

---

# Install CUDA on Linux | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

If you are using an Nvidia-enabled VM instance from a public cloud, you should probably use the VM image provided by the cloud. It typically has the correct versions of Nvidia driver and CUDA toolkit already installed. Read on if you need to install Nvidia driver and CUDA toolkit on your own machine.

Ubuntu 22.04[​](#ubuntu-2204 "Direct link to Ubuntu 22.04")

------------------------------------------------------------

### 1 Install the Nvidia driver.[​](#1-install-the-nvidia-driver "Direct link to 1 Install the Nvidia driver.")

Rebuild the grub configuration:

    sudo apt-get install -y gcc make linux-headers-$(uname -r)cat << EOF | sudo tee --append /etc/modprobe.d/blacklist.confblacklist vga16fbblacklist nouveaublacklist rivafbblacklist nvidiafbblacklist rivatvEOFsudo sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="rdblacklist=nouveau"/' /etc/default/grubsudo update-grub

Download and install the Nvidia driver

    wget https://storage.googleapis.com/nvidia-drivers-us-public/GRID/vGPU16.1/NVIDIA-Linux-x86_64-535.104.05-grid.runsudo sh NVIDIA-Linux-x86_64-535.104.05-grid.run

Confirm the driver is installed successfully

    nvidia-smi -q | head==============NVSMI LOG==============Timestamp                                 : Fri Oct 27 21:54:05 2023Driver Version                            : 535.104.05CUDA Version                              : 12.2Attached GPUs                             : 1GPU 00000000:00:1E.0    Product Name                          : NVIDIA A10G

Disable GSP and reboot.

    sudo touch /etc/modprobe.d/nvidia.confecho "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.confsudo reboot

### 2 Install the CUDA toolkit.[​](#2-install-the-cuda-toolkit "Direct link to 2 Install the CUDA toolkit.")

    wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.runsudo sh cuda_12.2.2_535.104.05_linux.run --silent --override --toolkit --samples --toolkitpath=/usr/local/cuda-12 --samplespath=/usr/local/cuda --no-opengl-libs

Confirm that CUDA is installed.

    /usr/local/cuda/bin/nvcc --versionnvcc: NVIDIA (R) Cuda compiler driverCopyright (c) 2005-2023 NVIDIA CorporationBuilt on Tue_Aug_15_22:02:13_PDT_2023Cuda compilation tools, release 12.2, V12.2.140Build cuda_12.2.r12.2/compiler.33191640_0

After that, use the following two commands to set up the environment path. You should probably add these two lines to your `~/.bashrc` and `~/.bash_profile` (or `~/.zshrc` and `~/.profile`) files so that new terminals and future logins will still be able to find these CUDA library files.

    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}export PATH=/usr/local/cuda/bin:${PATH}

More resources[​](#more-resources "Direct link to More resources")

-------------------------------------------------------------------

Here are more scripts that could help you in case you are stuck.

*   The [Nvidia official install guide](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/)
     for Linux.
*   [Yam Peleg's popular script](https://x.com/yampeleg/status/1751823896800583924)
     for Ubuntu 22.04
*   [Make CUDA available in Docker containers](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
    

Good luck!

*   [Ubuntu 22.04](#ubuntu-2204)
    *   [1 Install the Nvidia driver.](#1-install-the-nvidia-driver)
        
    *   [2 Install the CUDA toolkit.](#2-install-the-cuda-toolkit)
        
*   [More resources](#more-resources)

---

# Start a node with Docker | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

You can run all the commands in this document without any change on any machine with the latest Docker and at least 8GB of RAM available to the container. By default, the container uses the CPU to perform computations, which could be slow for large LLMs. For GPUs,

*   Mac: Everything here works on [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/)
    . However, the Apple GPU cores will not be available inside Docker containers until [WebGPU is supported by Docker](https://github.com/LlamaEdge/LlamaEdge/blob/main/docker/webgpu.md)
     later in 2024.
*   Windows and Linux with Nvidia GPU: You will need to install [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installation)
     for Docker. In the instructions below, replace the `latest` tag with `cuda12` or `cuda11` to use take advantage of the GPU, and add the `--device nvidia.com/gpu=all` flag. If you need to build the images yourself, replace `Dockerfile` with `Dockerfile.cuda12` or `Dockerfile.cuda11`.

Find [GaiaNet Docker images](https://hub.docker.com/?namespace=gaianet)
 you can run!

Quick start[​](#quick-start "Direct link to Quick start")

----------------------------------------------------------

Start a Docker container for the GaiaNet node. It will print running logs from the GaiaNet node in this terminal.

    docker run --name gaianet \  -p 8080:8080 \  -v $(pwd)/qdrant_storage:/root/gaianet/qdrant/storage:z \  gaianet/phi-3-mini-instruct-4k_paris:latest

The node is ready when it shows `The GaiaNet node is started at: https://...` on the console. You can go to that URL from your browser to interact with the GaiaNet node.

The docker image contains the LLM and embedding models required by the node. However, the vector collection snapshot (i.e., knowledge base) is downloaded and imported at the time when the node starts up. That is because the knowledge based could be updated frequently. The `qdrant_storage` directory on the host machine stores the vector database content.

Alternatively, the command to run the GaiaNet on your Nvidia CUDA 12 machine is as follows.

    docker run --name gaianet \  -p 8080:8080 --device nvidia.com/gpu=all \  -v $(pwd)/qdrant_storage:/root/gaianet/qdrant/storage:z \  gaianet/phi-3-mini-instruct-4k_paris:cuda12

Stop and re-start[​](#stop-and-re-start "Direct link to Stop and re-start")

----------------------------------------------------------------------------

You can stop and re-start the node as follows. Every time you re-start, it will re-initailize the vector collection (knowledge base).

    docker stop gaianetdocker start gaianet

NOTE: When you restart the node, the log messages will no longer be printed to the console. You will need to wait for a few minutes before the restarted node comes back online. You can still see the logs by logging into the container as follows.

    docker exec -it gaianet /bin/bashtail -f /root/gaianet/log/start-llamaedge.log

You can also delete the node if you no longer needs it.

    docker stop gaianetdocker rm gaianet

Make changes to the node[​](#make-changes-to-the-node "Direct link to Make changes to the node")

-------------------------------------------------------------------------------------------------

You can update the configuration parameters of the node, such as context size for the models, by executing the `config` command on the `gaianet` program inside the container. For example, the following command changes the chat LLM's context size to 8192 tokens.

    docker exec -it gaianet /root/gaianet/bin/gaianet config --chat-ctx-size 8192

Then, restart the node for the new configuration to take effect. You will need to wait for a few minutes for the server to start again, or you can monitor the log files inside the container as discussed above.

    docker stop gaianetdocker start gaianet

Change the node ID[​](#change-the-node-id "Direct link to Change the node ID")

-------------------------------------------------------------------------------

You can update the node ID (Ethereum address) associated with the node. Start the node and copy the `nodeid.json` file, as well as the keystore file defined in `nodeid.json` into the container.

    docker cp /local/path/to/nodeid.json gaianet:/root/gaianet/nodeid.jsondocker cp /local/path/to/1234-abcd-key-store gaianet:/root/gaianet/1234-abcd-key-store

THen, restart the node for the new address and keystore to take effect.

    docker stop gaianetdocker start gaianet

Build a node image locally[​](#build-a-node-image-locally "Direct link to Build a node image locally")

-------------------------------------------------------------------------------------------------------

Each GaiaNet is defined by a `config.json` file. It defines the node's required LLM and embedding models, model parameters, prompts, and vector snapshots (e.g., knowledge base). The following command builds a Docker image with two platforms for a node based on the specified `config.json` file.

    docker buildx build . --platform linux/arm64,linux/amd64 \  --tag gaianet/phi-3-mini-instruct-4k_paris:latest -f Dockerfile \  --build-arg CONFIG_URL=https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/config.json

> The `Dockerfile` is available [here](https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/docker/Dockerfile)
> . Feel free to change it to Nvidia [CUDA versions](https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/docker/Dockerfile.cuda12)
>  if your Docker is enabled with the [Nvidia container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
> .

You can publish your node for other people to use it.

    docker push gaianet/phi-3-mini-instruct-4k_paris:latest

Good luck!

*   [Quick start](#quick-start)
    
*   [Stop and re-start](#stop-and-re-start)
    
*   [Make changes to the node](#make-changes-to-the-node)
    
*   [Change the node ID](#change-the-node-id)
    
*   [Build a node image locally](#build-a-node-image-locally)

---

# Start a node on AWS using AMI images | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

On this page

We have created a series of public AMIs for you to start GaiaNet nodes in AWS with just a few clicks.

Now we have three AMI images available in the Asia Pacific (Osaka) and all the US regions including N. Virginia, Ohio, N. California, and Oregon.

| AMI Images Name | Architecture | Regions |
| --- | --- | --- |
| GaiaNet\_ubuntu22.04\_amd64\_cuda12 | GPU | N. Virginia, Ohio, N. California, Oregon, and Osaka |
| GaiaNet\_ubuntu22.04\_amd64 | x86 CPU machines | N. Virginia, Ohio, N. California, Oregon, and Osaka |
| GaiaNet\_ubuntu22.04\_arm64 | ARM CPU machines | N. Virginia, Ohio, N. California, Oregon, and Osaka |

Running an Nvidia GPU-enabled AWS instance[​](#running-an-nvidia-gpu-enabled-aws-instance "Direct link to Running an Nvidia GPU-enabled AWS instance")

-------------------------------------------------------------------------------------------------------------------------------------------------------

Load the [AWS console](https://aws.amazon.com/console/)
 and sign into your account. Go to EC2 | instances and click on the "Launch instance" button.

In the "Application and OS Images" section, search the AMI catalog and select the image named `GaiaNet_ubuntu22.04_amd64_cuda12`.

![](/assets/images/aws_ami-ac1a273dd1e4ba668951e1780804f6dc.png)

In the "Instance type" section, select any of the `g4dn` types. Those are EC2 VMs with Nvidia T4 GPUs.

![](/assets/images/aws_instance_type-b9d961478e8808a27fe30437e154f496.png)

In the "Network settings", make sure that you allow SSH connections.

![](/assets/images/aws_network-c87d461d165d2d2814df19d1af71e734.png)

Click on the "Launch instance" button and wait for the instance to start up. Once the instance is ready, SSH into its public IP address. Once you are in the VM, run the following two commands.

    gaianet initgaianet start

The node is ready when it shows `The GaiaNet node is started at: https://...` on the console. You can go to that URL from your browser to interact with the GaiaNet node.

You can [customize your GaiaNet node](/1.0.0/node-guide/customize)
 with your own choice of LLMs and knowledge base snapshots.

Running a CPU-only AWS instance[​](#running-a-cpu-only-aws-instance "Direct link to Running a CPU-only AWS instance")

----------------------------------------------------------------------------------------------------------------------

Load the [AWS console](https://aws.amazon.com/console/)
 and sign into your account. Go to EC2 | instances and click on the "Launch instance" button.

In the "Application and OS Images" section, search the AMI catalog and select the image named

*   `GaiaNet_ubuntu22.04_amd64` for x86 CPU machines
*   `GaiaNet_ubuntu22.04_arm64` for ARM CPU machines

In the "Instance type" section, select an instance with at least 8GB of RAM. For example, we recommend `t2.large` or `t2.xlarge` instances.

In the "Network settings", make sure that you allow SSH connections.

Click on the "Launch instance" button and wait for instance to start up. Once the instance is ready, SSH into its public IP address. Once you are in the VM, run the following two commands.

    gaianet initgaianet start

The node is ready when it shows `The GaiaNet node is started at: https://...` on the console. You can go to that URL from your browser to interact with the GaiaNet node.

You can [customize your GaiaNet node](/1.0.0/node-guide/customize)
 with your own choice of LLMs and knowledge base snapshots.

Good luck!

*   [Running an Nvidia GPU-enabled AWS instance](#running-an-nvidia-gpu-enabled-aws-instance)
    
*   [Running a CPU-only AWS instance](#running-a-cpu-only-aws-instance)

---

# Run a local-only node | Gaia

[Skip to main content](#__docusaurus_skipToContent_fallback)

Version: 1.0.0

By default, the GaiaNet node registers itself with a GaiaNet domain and is accesible from the public. For many users, it could also be important to start a local server for testing. To do that, you just need to pass the `--local-only` option.

    gaianet start --local-only

---