# Table of Contents - [Introduction | 10 Days Realtime LLM Bootcamp](#introduction-10-days-realtime-llm-bootcamp) - [Getting Started | 10 Days Realtime LLM Bootcamp](#getting-started-10-days-realtime-llm-bootcamp) - [Course Syllabus | 10 Days Realtime LLM Bootcamp](#course-syllabus-10-days-realtime-llm-bootcamp) - [Course Structure | 10 Days Realtime LLM Bootcamp](#course-structure-10-days-realtime-llm-bootcamp) - [First Exercise (Ungraded) | 10 Days Realtime LLM Bootcamp](#first-exercise-ungraded-10-days-realtime-llm-bootcamp) - [Prerequisites | 10 Days Realtime LLM Bootcamp](#prerequisites-10-days-realtime-llm-bootcamp) - [Greetings from your Instructors | 10 Days Realtime LLM Bootcamp](#greetings-from-your-instructors-10-days-realtime-llm-bootcamp) - [Basics of LLM | 10 Days Realtime LLM Bootcamp](#basics-of-llm-10-days-realtime-llm-bootcamp) - [What is Generative AI? | 10 Days Realtime LLM Bootcamp](#what-is-generative-ai-10-days-realtime-llm-bootcamp) - [What is a Large Language Model? | 10 Days Realtime LLM Bootcamp](#what-is-a-large-language-model-10-days-realtime-llm-bootcamp) - [Word Vectors Simplified | 10 Days Realtime LLM Bootcamp](#word-vectors-simplified-10-days-realtime-llm-bootcamp) - [Advantages and Applications of Large Language Models | 10 Days Realtime LLM Bootcamp](#advantages-and-applications-of-large-language-models-10-days-realtime-llm-bootcamp) - [Bonus Resource: Multimodal LLMs | 10 Days Realtime LLM Bootcamp](#bonus-resource-multimodal-llms-10-days-realtime-llm-bootcamp) - [What is a Word Vector | 10 Days Realtime LLM Bootcamp](#what-is-a-word-vector-10-days-realtime-llm-bootcamp) - [Word Vector Relationships | 10 Days Realtime LLM Bootcamp](#word-vector-relationships-10-days-realtime-llm-bootcamp) - [Role of Context in LLMs | 10 Days Realtime LLM Bootcamp](#role-of-context-in-llms-10-days-realtime-llm-bootcamp) - [Let's Track Our Progress | 10 Days Realtime LLM Bootcamp](#let-s-track-our-progress-10-days-realtime-llm-bootcamp) - [Prompt Engineering | 10 Days Realtime LLM Bootcamp](#prompt-engineering-10-days-realtime-llm-bootcamp) - [Prompt Engineering and In-context Learning | 10 Days Realtime LLM Bootcamp](#prompt-engineering-and-in-context-learning-10-days-realtime-llm-bootcamp) - [Neural Networks and Transformers (Bonus Module) | 10 Days Realtime LLM Bootcamp](#neural-networks-and-transformers-bonus-module-10-days-realtime-llm-bootcamp) - [Transforming Vectors into LLM Responses | 10 Days Realtime LLM Bootcamp](#transforming-vectors-into-llm-responses-10-days-realtime-llm-bootcamp) - [Token Limits in Prompts | 10 Days Realtime LLM Bootcamp](#token-limits-in-prompts-10-days-realtime-llm-bootcamp) - [What is Prompt Engineering | 10 Days Realtime LLM Bootcamp](#what-is-prompt-engineering-10-days-realtime-llm-bootcamp) - [Tasks in the Excercise | 10 Days Realtime LLM Bootcamp](#tasks-in-the-excercise-10-days-realtime-llm-bootcamp) - [Story for the Excercise: The eSports Enigma | 10 Days Realtime LLM Bootcamp](#story-for-the-excercise-the-esports-enigma-10-days-realtime-llm-bootcamp) - [Best Practices to Follow in Prompt Engineering | 10 Days Realtime LLM Bootcamp](#best-practices-to-follow-in-prompt-engineering-10-days-realtime-llm-bootcamp) - [Prompt Engineering Excercise | 10 Days Realtime LLM Bootcamp](#prompt-engineering-excercise-10-days-realtime-llm-bootcamp) - [Attention and Transformers (Bonus Module) | 10 Days Realtime LLM Bootcamp](#attention-and-transformers-bonus-module-10-days-realtime-llm-bootcamp) - [Multi-Head Attention and Further Reads (Bonus Module) | 10 Days Realtime LLM Bootcamp](#multi-head-attention-and-further-reads-bonus-module-10-days-realtime-llm-bootcamp) - [Hands-on Development | 10 Days Realtime LLM Bootcamp](#hands-on-development-10-days-realtime-llm-bootcamp) - [Live Interactions with Jan Chorowski and Adrian Kosowski | Bonus Resource | 10 Days Realtime LLM Bootcamp](#live-interactions-with-jan-chorowski-and-adrian-kosowski-bonus-resource-10-days-realtime-llm-bootcamp) - [Prerequisites | 10 Days Realtime LLM Bootcamp](#prerequisites-10-days-realtime-llm-bootcamp) - [Final Project + Giveaways | 10 Days Realtime LLM Bootcamp](#final-project-giveaways-10-days-realtime-llm-bootcamp) - [Retrieval Augmented Generation and LLM Architecture | 10 Days Realtime LLM Bootcamp](#retrieval-augmented-generation-and-llm-architecture-10-days-realtime-llm-bootcamp) - [Dropbox Retrieval App in 15 Minutes | 10 Days Realtime LLM Bootcamp](#dropbox-retrieval-app-in-15-minutes-10-days-realtime-llm-bootcamp) - [Final Submission | 10 Days Realtime LLM Bootcamp](#final-submission-10-days-realtime-llm-bootcamp) - [In-Context Learning | 10 Days Realtime LLM Bootcamp](#in-context-learning-10-days-realtime-llm-bootcamp) - [High level LLM Architecture Components for In-context Learning | 10 Days Realtime LLM Bootcamp](#high-level-llm-architecture-components-for-in-context-learning-10-days-realtime-llm-bootcamp) - [Understanding Docker | 10 Days Realtime LLM Bootcamp](#understanding-docker-10-days-realtime-llm-bootcamp) - [Primer to RAG Functioning and LLM Architecture: Pre-trained and Fine-tuned LLMs | 10 Days Realtime LLM Bootcamp](#primer-to-rag-functioning-and-llm-architecture-pre-trained-and-fine-tuned-llms-10-days-realtime-llm-bootcamp) - [How the Project Works | 10 Days Realtime LLM Bootcamp](#how-the-project-works-10-days-realtime-llm-bootcamp) - [Tracks for Submission | 10 Days Realtime LLM Bootcamp](#tracks-for-submission-10-days-realtime-llm-bootcamp) - [Prizes and Giveaways | 10 Days Realtime LLM Bootcamp](#prizes-and-giveaways-10-days-realtime-llm-bootcamp) - [Diving Deeper: LLM Architecture Components | 10 Days Realtime LLM Bootcamp](#diving-deeper-llm-architecture-components-10-days-realtime-llm-bootcamp) - [Amazon Discounts App | 10 Days Realtime LLM Bootcamp](#amazon-discounts-app-10-days-realtime-llm-bootcamp) - [LLM Architecture Diagram and Various Steps | 10 Days Realtime LLM Bootcamp](#llm-architecture-diagram-and-various-steps-10-days-realtime-llm-bootcamp) - [What is Retrieval Augmented Generation (RAG)? | 10 Days Realtime LLM Bootcamp](#what-is-retrieval-augmented-generation-rag-10-days-realtime-llm-bootcamp) - [Building the app without Dockerization | 10 Days Realtime LLM Bootcamp](#building-the-app-without-dockerization-10-days-realtime-llm-bootcamp) - [RAG versus Fine-Tuning and Prompt Engineering | 10 Days Realtime LLM Bootcamp](#rag-versus-fine-tuning-and-prompt-engineering-10-days-realtime-llm-bootcamp) - [Step-by-Step Process | 10 Days Realtime LLM Bootcamp](#step-by-step-process-10-days-realtime-llm-bootcamp) - [Track your Progress | 10 Days Realtime LLM Bootcamp](#track-your-progress-10-days-realtime-llm-bootcamp) - [How to Run the Examples | 10 Days Realtime LLM Bootcamp](#how-to-run-the-examples-10-days-realtime-llm-bootcamp) - [Key Benefits of RAG for Enterprise-Grade LLM Applications | 10 Days Realtime LLM Bootcamp](#key-benefits-of-rag-for-enterprise-grade-llm-applications-10-days-realtime-llm-bootcamp) - [Versatility and Efficiency in Retrieval-Augmented Generation (RAG) | 10 Days Realtime LLM Bootcamp](#versatility-and-efficiency-in-retrieval-augmented-generation-rag-10-days-realtime-llm-bootcamp) - [Using kNN and LSH to Enhance Similarity Search in Vector Embeddings (Bonus Module) | 10 Days Realtime LLM Bootcamp](#using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module-10-days-realtime-llm-bootcamp) - [Similarity Search in Vectors (Bonus Module) | 10 Days Realtime LLM Bootcamp](#similarity-search-in-vectors-bonus-module-10-days-realtime-llm-bootcamp) - [Using Docker to Build the App | 10 Days Realtime LLM Bootcamp](#using-docker-to-build-the-app-10-days-realtime-llm-bootcamp) --- # Introduction | 10 Days Realtime LLM Bootcamp Welcome to this exciting journey into the world of Large Language Models (LLMs)! **Please Note:** * The same coursework is also available at the Institute Technical Council (ITC) IIT Bombay Website: [**https://itc.gymkhana.iitb.ac.in/AIC/realtime-llm-bootcamp/**](https://itc.gymkhana.iitb.ac.in/AIC/realtime-llm-bootcamp/) . The link is open to everyone. * However, if you are not on the IIT Bombay campus and encounter any difficulties accessing the provided link, you can alternatively continue your learning here, on this link itself. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FCyrvhCJmKIVeWAbvurIV%252FA%252010%2520DAY%2520BOOTCAMP%2520ON%2520%2860%2520x%252024%2520in%29%2520%281600%2520x%2520400%2520px%29%2520%28Presentation%29%2520%282%29-min.png%3Falt%3Dmedia%26token%3Da9bd085a-48b9-4b1e-b47c-aaacf3d83ee3&width=768&dpr=4&quality=100&sign=a64ee265&sv=2) [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp#about-the-bootcamp) About the bootcamp ------------------------------------------------------------------------------------------------------------------------------ In November 2023, the Web & Coding Club at IIT Bombay and the AI Community at IIT Bombay, supported by the Institute Technical Council, joined hands with Pathway to launch this free, cohort-based course. This initiative, focusing on Generative AI, LLMs, and Real-time Data Processing, addressed the growing need to understand, leverage, and build Generative AI solutions. **Key highlights:** * The course was shaped by contributions from global leaders in academia and deep learning, boasting past affiliations with prominent organizations like Google Brain, Mila-Quebec AI Institute, and Microsoft Research. * The bootcamp attracted over 1400 learners, including over 1000 students from IIT Bombay. Participants included students and professionals from HEC Paris, IIT Delhi, Indiana University, Qualcomm, and Novartis. * Despite the tight 10-day schedule, the bootcamp inspired active involvement. Various student developers went ahead to create interesting enterprise-centric apps. For example: * A project management assistant integrated with Slack, capable of delivering real-time insights from Trello Dashboards. * An LLM-powered application providing current insights into digital marketing trends influenced by ad spending data. [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp#making-the-coursework-available) Making the coursework available -------------------------------------------------------------------------------------------------------------------------------------------------------- We're opening up our archived coursework for free, driven by our belief in the power of open education and Generative AI. This is an invitation for students and professionals to use these resources as a springboard for creating solutions that make a real difference. This resource is particularly beneficial for those starting or looking to deeper into the world of Large Language Models (LLMs), Generative AI, Retrieval Augmented Generation (RAG), and related areas, providing a thorough educational guide. Happy learning! 😄 [NextGetting Started](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started) Last updated 1 year ago --- # Getting Started | 10 Days Realtime LLM Bootcamp This course is meticulously crafted for beginners grounded in programming, especially with a forte in Python. Prior expertise in LLMs or generative AI is not necessary, as our curriculum ensures a progressive learning experience. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#kick-off-session-recording-optional) Kick-off Session Recording (Optional) Suppose you didn't attend the kick-off session of the bootcamp live. In that case, you can watch the 30-minute recorded interaction below between Mudit Srivastava from Pathway and Navyansh Mahla from the AI Community at IIT Bombay. However, reading the course introduction is highly recommended. If you prefer, you can read the course introduction and then return to the kick-off session's video if you have any questions or need further clarification. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#course-progression) Course Progression * **Starting with the Basics**: Our journey commences by laying the foundational bedrock of Large Language Models and Generative AI. * **Intermediate Exploration**: As we navigate deeper waters, we touch upon essential components like vector indices, word vectors, and the intricate art of prompt engineering. * **Advanced Insights**: The final stretch of our curriculum focuses on in-depth topics such as RAG, LLM Architecture, LLM Pipelines, and hands-on development exercises. **Note:** Consider your existing knowledge when tackling this coursework. If you're starting from scratch, this coursework could be a bit overwhelming to be completed in 10 days, especially with the hands-on project and bonus resources. These timelines were implemented back when the cohort was active, taking into account the academic semester exams schedule and an upcoming R&D expo at IIT Bombay. However, if you're taking a self-paced approach as an independent learner, take the time to assess your current understanding, explore the curriculum, and create a practical schedule that aligns with your existing knowledge and capacity. Happy learning! ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#the-power-of-real-time-data-and-large-language-models) The power of Real-time Data and Large Language Models A pivotal theme of this course is the amalgamation of real-time data with Large Language Models. Harnessing this combination can lead to transformative solutions addressing pressing societal and business challenges. While you will pretty much be able to build custom LLM Apps for static data sources as well, our chosen open-source framework effortlessly supports both real-time ("Streaming") and static ("Batch") data with a slight change in Python code. In our digital age, the fusion of immediate data with LLMs is transformative. It accelerates processes, from financial transactions to healthcare responses. Think about it: a financial transaction that once took days can now be executed in mere milliseconds, a testament to the immense power of real-time data. By integrating real-time data streams with LLMs, we can develop applications that not only respond promptly but also have the potential to drive meaningful impacts for humanity. And that's pretty much one of the key goals of this course. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#learning-beyond-the-curriculum) Learning Beyond the Curriculum To enrich your learning experience, we've included an insightful session with deep learning reference figure [Jan Chorowski](https://pathway.com/our-story/#jan-chorowski) . This ensures a comprehensive learning journey, equipping you with a spectrum of knowledge, irrespective of your starting point. You can access it eventually as you progress within the coursework. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#your-role-as-a-learner) Your Role as a Learner While we lay the groundwork and provide the tools, the epicenter of learning and exploration resides with you. We set the stage, but the performance—the application, the innovation, the breakthroughs—depends on your initiative and drive. As we chart this transformative journey, are you poised to harness the untapped potentials of LLMs combined with real-time data for the greater good? Let's embark on this venture together! 🚀 ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#get-to-know-pathway-our-collaborators-for-this-course) Get to Know Pathway – our collaborators for this course Pathway is the world's fastest data processing engine ([benchmarks](https://pathway.com/blog/streaming-benchmarks-pathway-fastest-engine-on-the-market) ), supporting unified workflows for batch, streaming data, and LLM applications. It is the single, fastest integrated data processing layer for real-time intelligence. With Pathway, you can * Mix-and-match: batch, streaming, API calls, including LLMs. * Effortless transition from batch to real-time data processing - just like setting a flag in your Spark code. * Reduce the cost of any computations – it is powered by a highly efficient and scalable Rust engine. * Enable use cases enterprises crave, and make advanced data transformations fast and easy to implement. Discover on GitHub: Pathway's Data Processing Framework [![Logo](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Fgithub.com%2Ffluidicon.png&width=20&dpr=4&quality=100&sign=34473101&sv=2)GitHub - pathwaycom/pathway: Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.GitHub](https://github.com/pathwaycom/pathway) Discover on GitHub: Pathway's RAG Framework for Large Language Models (LLMs) [![Logo](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Fgithub.com%2Ffluidicon.png&width=20&dpr=4&quality=100&sign=34473101&sv=2)GitHub - pathwaycom/llm-app: Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.GitHub](https://github.com/pathwaycom/llm-app) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started#why-is-pathway-so-fast) Why is Pathway so Fast The Pathway engine is built in Rust. 🩀. Rust is built for speed, parallel computation and low-level control over hardware resources. This allows them to execute maximum optimization for performance and speed. The team at Pathway also realize our love for Python 🐍. This is why the Pathway engine is made in a way that when you write your data processing code in Python, Pathway will automatically compile it into a Rust dataflow. In other words, when using Pathway, you don’t need to know anything about Rust to enjoy its enormous performance benefits! For now, this is a simple enough starting point (that said, feel free to find more details in Pathway's [ArXiv Paper](https://arxiv.org/abs/2307.13116) – your first bonus resource 🙂). [PreviousIntroduction](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp) [NextCourse Syllabus](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-syllabus) Last updated 1 year ago --- # Course Syllabus | 10 Days Realtime LLM Bootcamp By the end of this course, you will: * Be proficient in developing LLM-based applications for production applications from day 0. * Have a clear understanding of LLM architecture and pipeline. * Be able to perform prompt engineering to best use generative AI tools such as ChatGPT. * Create an open-source project on a real-time stream of data or static data. [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-syllabus#what-well-be-learning-to-get-there) What we'll be learning to get there: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Module 1 – Basics of LLMs Topics * What is generative AI and how it's different * Understanding LLMs * Advantages and Common Industry Applications of LLMs * Bonus section: Google Gemini and Multimodal LLMs Module 2 – Word Vectors Topics * What are word vectors and word-vector relationships * Role of context in LLMs * Transforming vectors in LLM responses * Bonus Resource: Overview of Transformers Architecture and Vision Transformers Module 3 – Prompt Engineering Topics * Introduction and in-context learning * Best practices to follow: Few Shot Prompting and more * Token Limits * Prompt Engineering Peer Reviewed Exercise Module 4 – RAG and LLM Architecture Topics * Introduction to RAG * LLM Architecture Used by Enterprises * Architecture Diagram and LLM Pipeline * RAG vs Fine-Tuning and Prompt Engineering * Key Benefits of RAG for Realtime Applications * Simialrity Search for Efficient Information Retrieval * Bonus Resource: Use of LSH + kNN and Incremental Indexing Module 5 – Hands-on Project Topics * Installing Dependencies and Pre-requisites * Building a Dropbox RAG App using open-source * Building Realtime Discounted Products Fetcher for Amazon Users * Problem Statements for Projects * Project Submission ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-syllabus#undefined) [PreviousGetting Started](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/getting-started) [NextCourse Structure](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure) Last updated 1 year ago --- # Course Structure | 10 Days Realtime LLM Bootcamp ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure#hybrid-approach) Hybrid Approach This coursework was meticulously designed using a hybrid approach. While the core of the content was delivered through recorded sessions, we also integrated interactive live sessions. Since the bootcamp cohort is now concluded, we've added the session recordings within the coursework for you to watch them at your own pace. 😄 ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FqXspSPFceMDi58hBFPHf%252Fhappy-dance-seinfeld-friends-7b1taoq2en5qy1tz%2520%281%29.gif%3Falt%3Dmedia%26token%3D093f02c4-a3ca-4c7a-b891-ecb2362d2e12&width=768&dpr=4&quality=100&sign=c668a83a&sv=2) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure#interactive-sessions) Interactive Sessions * **Kick-off Session**: * You can find its recording [here](https://youtu.be/AjmeopQuuGI) if you couldn't watch the session live. * This session recording will provide an overview of the course essentials. * **Fireside Chat with Jan Chorowski | Exploring the Frontiers of Large Language Models**: * You can find the session recording after the RAG Module as a Bonus Resource. * In the session, we will delve into topics like: * The potentials and boundaries of LLMs. * Overcoming challenges in deploying LLM applications. * Strategies for building real-time LLM apps. * The intriguing concept of "Learning to Forget" in Large Language Models. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure#addressing-doubts) Addressing Doubts * **Self-Reliance**: Our philosophy revolves around nurturing independent learners. While our guidance is continuous, we urge you to harness the expansive online knowledge, exploring answers through search engines and existing literature. * **Communication Channels**: Ensure you're part of [Pathway's Discord](https://discord.gg/pathway) to ask doubts directly to the framework's creators. This being said, please use Google Search, ChatGPT, GitHub Copilot, or other tools to find the answer yourself. If you're unable to do so, as a best practice, unresolved queries can be raised via GitHub issues, and then their link can be shared on Discord. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure#bonus-resources-and-bonus-modules) Bonus Resources & Bonus Modules * From time to time, you'll find Bonus Resources alongside your coursework. * These resources are **optional** and not essential for quizzes or project completion. They're designed to deepen your understanding, complementing the core material succinctly crafted for those eager to develop their first real-time LLM application. Feel free to explore these additional materials at your leisure for a broader and more detailed learning experience. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure#quick-checks) Quick Checks * **WhatsApp Community**: If you've shared your contact during registration, you should've been added to a WhatsApp group managed by AI Community IIT Bombay and WnCC IIT Bombay in collaboration with Pathway. If not, please get in touch with [wncc@iitb.ac.in](mailto:wncc@iitb.ac.in) . * If you happen to come across this coursework after the conclusion of the bootcamp cohort, there's no need to be concerned if you haven't been added to the WhatsApp community. * We wholeheartedly encourage you to proceed with the valuable insights gained from this course and actively participate in the [Pathway Discord community](https://discord.gg/pathway) . Should any questions arise, please don't hesitate to seek assistance within the community; your inquiries are always welcome. [PreviousCourse Syllabus](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-syllabus) [NextPrerequisites](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/prerequisites) Last updated 1 year ago --- # First Exercise (Ungraded) | 10 Days Realtime LLM Bootcamp To make this course interactive and engaging, let's get started with a small optional exercise. * **Star the GitHub Repo**: * We invite you to visit the [Pathway repository](https://github.com/pathwaycom/pathway) and the [LLM App GitHub repository](https://github.com/pathwaycom/llm-app) and give it a star if you like the work. Doing so will show your support and keep you updated as the project evolves. * **Join** [**Pathway’s Discord**](https://discord.gg/SzSHw8Hd) **community:** * Join your peers and mentors as we embark on this short yet exciting learning journey. We encourage you to introduce yourself in the #introductions channel, fostering a more personalized community experience. With the presence of some of the world's leading AI and Data researchers, you're in good company. * Additionally, as a best practice, we advocate for active participation in open-source communities like that of Pathway. Engaging proactively not only accelerates your learning but also opens doors to valuable connections and collaborations. * When introducing yourself, consider sharing any prior engagements in Artificial Intelligence, Data Engineering, or Academic Research. For those just beginning, feel free to discuss your existing background and how you aim to harness the potential of real-time LLMs. 😊 Ready to kickstart your LLM journey? The adventure awaits! 🚀 [PreviousGreetings from your Instructors](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/greetings-from-your-instructors) [NextBasics of LLM](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm) Last updated 1 year ago --- # Prerequisites | 10 Days Realtime LLM Bootcamp This bootcamp is tailored to accommodate participants with minimal prerequisites, ensuring it's approachable for individuals from various backgrounds. For those in business functions or educational domains outside of programming, the course's theoretical aspects are still largely comprehensible. This understanding will aid in making more informed decisions and better use of these technologies. However, having a grasp of the listed prerequisites will greatly facilitate your journey through the entire curriculum, including bonus resources and the hands-on development module. **1 – Python Foundations**: If you're new to Python or need a refresher, here are some free Python tutorials to get you started. * [https://docs.python.org/3/tutorial/index.html](https://docs.python.org/3/tutorial/index.html) * [https://www.py4e.com/](https://www.py4e.com/) **2 – Docker Fundamentals:** Docker or containerization resolves the "it works on my machine" issue. Hence, it's great not just for this bootcamp but also for enabling you to explore open-source frameworks in general. Here are a couple of resources you can consider: * [Docker Tutorial on FreeCodeCamp](https://youtu.be/fqMOX6JJhGo) * [How to Dockerize your Python Applications](https://youtu.be/0UG2x2iWerk) **3 – Open Source Familiarity:** While previous experience with open-source projects can be beneficial, it isn't essential. It's worth noting that many routine challenges, such as publishing a local project on GitHub or using widely-accepted APIs, can often be resolved through a simple Google search or by consulting the relevant documentation. While these challenges won't present themselves at every turn, independently navigating them will equip you with the skills to devise impactful solutions in the future. [PreviousCourse Structure](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/course-structure) [NextGreetings from your Instructors](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/greetings-from-your-instructors) Last updated 1 year ago --- # Greetings from your Instructors | 10 Days Realtime LLM Bootcamp Hello learners! Before we plunge into the captivating realm of Large Language Models, enjoy a special video greeting from one of your instructors, Adrian Kosowski ([Google Scholar Profile](https://scholar.google.com/citations?user=om8De_0AAAAJ&hl=pl) ). Beyond his role as Chief Product Officer at Pathway and a father of two, Adrian is a seasoned researcher and entrepreneur. He earned his PhD in Computer Science at just 20 years old and has over 15 years of research experience. Adrian has co-authored 100+ research publications and co-founded Spoj.com, through which he impacted the lives of millions of programmers around the globe. In this introductory video, you’ll get to meet Adrian and know: * Why Pathway is supporting this initiative and a brief introduction of Pathway * What you'll learn throughout this course * Immense importance of understanding LLMs in today's world * What you can expect to gain at the end of this course ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/greetings-from-your-instructors#other-instructors-and-course-contributors) Other Instructors and Course Contributors A heartfelt thanks to those who made this course richer with their expertise: * **Adrian Kosowski**, CPO at Pathway | PhD at 20 | Prev - Professor at École Polytechnique and Co-founder of SPOJ | 100+ research publications * **Anup Surendran**, Head of Product Marketing & Growth at Pathway | Prev - VP at QuestionPro | Advisor, Texas A&M University * **Jan Chorowski**, CTO at Pathway | PhD in Neural Networks | Worked with Godfathers of AI | Former Researcher at Google Brain, MILA AI | 10K+ citations * **Sergey Kulik**, Lead Software Research Engineer and Solutions Architect at Pathway | IOI Gold Medalist | Prev - Head of Service at Yandex * **Mudit Srivastava**, Growth Manager at Pathway | Prev - Growth Head at AI Planet * **Bobur Umurzokov**, Developer Advocate at Pathway | Ex-Lead at Microsoft Tallinn * **Berke Can Rizai,** LLM Research Engineer at Pathway | Ex-Data Scientist at Getir * **Olivier Ruas,** R&D Engineer - Algorithms and Data Processing Magician at Pathway | Did PhD Focused on KNNs | Postdoc at Peking University We especially thank **Mike Chambers**, Developer Advocate at Amazon Web Services for allowing us to cite some of his finest educational resources from the BuildOnAWS YouTube channel. Along with that we've also cited some wonderful open-resources published by IBM Technology, Microsoft, and others. We are highly thankful to them too. Let's embark on this captivating voyage into the world of Large Language Models. Ready to dive in? 🌟 [PreviousPrerequisites](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/prerequisites) [NextFirst Exercise (Ungraded)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/first-exercise-ungraded) Last updated 1 year ago --- # Basics of LLM | 10 Days Realtime LLM Bootcamp Welcome to the realm of Large Language Models, often known as LLMs. As we stand at the threshold of this transformative domain of artificial intelligence, it's essential to understand its foundation. What are these models, and why are they heralded as a monumental shift in machine learning? This module promises to unravel the mystery behind LLMs, clarifying their significance, advantages, and the myriad applications they empower. You'll gain a holistic understanding of LLMs through expert insights, interactive content, and self-assessment tools, laying a robust foundation for the deeper explorations ahead. Let's embark on this enlightening journey, shall we? [PreviousFirst Exercise (Ungraded)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/introduction/first-exercise-ungraded) [NextWhat is Generative AI?](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/what-is-generative-ai) Last updated 1 year ago --- # What is Generative AI? | 10 Days Realtime LLM Bootcamp Before delving into the depths of Large Language Models, grasping the broader concept of Generative AI is imperative. We present a comprehensive video by Gwendolyn Stripling from Google to aid this understanding. * This video will provide a clear overview of Generative AI and its significance. * Insights into how Generative AI is shaping modern applications. * By the end of this video, you'll have a robust foundational understanding of Generative AI, paving the way for our exploration of LLMs. _(Credits: Google Cloud on YouTube)_ Eager to discover the magic of Generative AI? Dive in! 🌌 [PreviousBasics of LLM](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm) [NextWhat is a Large Language Model?](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/what-is-a-large-language-model) Last updated 1 year ago --- # What is a Large Language Model? | 10 Days Realtime LLM Bootcamp As we embark on this educational expedition, it's crucial to start with the fundamental question—What exactly are Large Language Models (LLMs)? To answer this and set the stage for everything that will follow, we have a special video by our friend, Mike Chambers from Amazon Web Services. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/what-is-a-large-language-model#what-youll-learn-from-the-video) What You'll Learn from the Video * An easy-to-understand explanation of what Large Language Models are * The diverse range of applications that LLMs empower * This video serves as an essential foundation and a smooth start for your learning journey. _(Credits: Build on AWS)_ Ready to dive into the captivating world of LLMs? Let's get started! 🌟 [PreviousWhat is Generative AI?](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/what-is-generative-ai) [NextAdvantages and Applications of Large Language Models](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/advantages-and-applications-of-large-language-models) Last updated 1 year ago --- # Word Vectors Simplified | 10 Days Realtime LLM Bootcamp Delve into the intricate world of word vectors, the foundational pillars of Large Language Models (LLMs). This section offers a deep dive into the nuances of how words are represented, how context shapes meaning, and the transformative mechanisms that empower LLMs to interpret and generate human-like text. [PreviousBonus Resource: Multimodal LLMs](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/bonus-resource-multimodal-llms) [NextWhat is a Word Vector](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/what-is-a-word-vector) Last updated 1 year ago --- # Advantages and Applications of Large Language Models | 10 Days Realtime LLM Bootcamp Building on what Mike outlined in the previous video, Large Language Models (LLMs) aren't just another iteration of neural networks; they represent a significant leap forward. Before we delve into their myriad applications, let's first unpack what sets LLMs apart from traditional neural networks. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FM8fhDwDNyNquOcwyWp7y%252Fminions-celebration%2520%281%29.gif%3Falt%3Dmedia%26token%3Dc08b9b06-acc4-48b4-aa9b-369fd2219c0b&width=768&dpr=4&quality=100&sign=2e3c9976&sv=2) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/advantages-and-applications-of-large-language-models#key-advantages-over-traditional-neural-networks) Key Advantages Over Traditional Neural Networks * Scale of Data: LLMs are trained on enormous datasets, capturing the breadth and depth of human knowledge. This allows them to understand context better, making their outputs more nuanced and accurate. * Transfer Learning: The general-purpose nature of LLMs allows them to adapt to a wide array of tasks without needing to be retrained from scratch, saving both time and computational resources. It’s like learning on the fly with the help of pre-trained data. And with the help of certain libraries, you can do even more. You know how you don't need to learn how to catch a ball every time you switch from cricket to baseball? LLMs can do the same. Once they know one thing, they can use that knowledge for other tasks without starting from scratch. * Contextual Understanding: Unlike simpler models that focus on individual words or sentences, LLMs can grasp the context within a paragraph or document. This leads to more coherent and contextually relevant outputs. * Multi-Tasking: Traditional neural networks are usually specialized for a single task. In contrast, a single LLM can perform multiple NLP tasks like translation, summarisation, and question-answering, among others. Given these powerful capabilities of Large Language Models, how can you contribute towards making them useful for the community? ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/advantages-and-applications-of-large-language-models#few-components-around-developing-meaningful-llm-applications) Few Components around Developing Meaningful LLM Applications **1\. Domains of LLM Applications:** * **Industry Perspective:** If you're not part of a dedicated LLM research team, knowing the common domains where LLMs are applied is crucial. It gives you insight into the priorities of the industry, and the areas where LLMs can drive substantial value. This being said, there's no hard and fast rule around this and you can always choose to go beyond. * **Examples:** Customer service (chatbots), healthcare (drug discovery and diagnostics), creative writing (content creation), and the financial sector (fraud detection, summarization of financial meetings). Feel free to Google along these lines and you'll find a plethora of resources around every single domain. Quick recommendation: During ideation or even execution, focusing on one problem area at a time is often a good strategy. * **Bonus Resource:** Check out [this list](https://www.ycombinator.com/companies/industry/generative-ai) of the Top 100 Y-Combinator-backed generative AI startups. This resource is an excellent way to discover a comprehensive list of startup-led innovations. However, it is limited to startups within the Y-Combinator portfolio, and these startups will continue to evolve over time. Therefore, feel free to conduct a Google search or use a web-access enabled large language model (LLM) to find more recent innovations. 😄 **2\. Creating Novel Solutions:** * **Real-Time LLM Applications:** Building a "real-time" system fundamentally revolves around processing streaming data—handling new information as it arrives, and incrementally indexing it efficiently for LLMs. Think of this as a continuous learning process for LLMs similar to the way we humans learn. As we delve deeper into the course, we'll explore the nuances of incremental indexing via bonus resources, but for now, picture it as a system that constantly evolves and adapts. * **Combining Real-Time Data Processing with LLMs:** This integration forms a powerful value chain, which you'll learn to master by the end of this bootcamp. This synergy is pivotal in developing impactful solutions. **3\. Evolving Scope of LLMs:** * **Multimodality in LLMs:** Continuous advancements in LLM capabilities, such as those in Google Deepmind's Gemini project, are expanding LLM interactions beyond text to include video, audio, and images. This opens a realm of possibilities for more dynamic and integrated AI applications. * **Expanding Domains of LLM Research:** Research is progressing in areas like reducing hallucinations, enhancing automated decision-making levels, and ensuring safer LLM applications. Innovations are also being made in processing larger data inputs more efficiently, exploring new model architectures beyond transformers, improving real-time data indexing, and enhancing the user experience in LLM applications. These core components of impact are hinged on a few existing areas of advantages of LLms. What are those? ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/advantages-and-applications-of-large-language-models#key-advantages-of-available-foundational-llms-over-traditional-neural-networks) Key Advantages of Available Foundational LLMs over Traditional Neural Networks * **Scale of Data:** Training on extensive datasets enhances LLMs' context understanding, leading to more nuanced outputs. * **Transfer Learning:** Similar to learning different sports, LLMs apply knowledge across tasks without starting anew. * **Contextual Understanding:** They perceive larger text contexts, not just isolated words or sentences. * **Multi-Tasking Capability:** Capable of handling diverse NLP tasks, unlike specialized traditional networks. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/advantages-and-applications-of-large-language-models#bonus-resources) Bonus Resources For a deeper dive into the expansive world of LLM applications, feel free to explore these bonus resources: * [Nvidia article](https://blogs.nvidia.com/blog/2023/01/26/what-are-large-language-models-used-for/) as a starting point. * Then head over to this blog about using [LLM Applications in production](https://huyenchip.com/2023/04/11/llm-engineering.html) . * If you’re curious about the potential limitations of LLMs as well, don’t worry we’ve got that covered towards the end of this course. While the bonus resources across this course are provided to ignite your curiosity, for now, you simply need to grasp the basics of Large Language Models (LLMs) and their varied applications. This will prime you for a deeper understanding of the upcoming modules and help you fully appreciate their transformative potential. Let's continue! 🌐 [PreviousWhat is a Large Language Model?](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/what-is-a-large-language-model) [NextBonus Resource: Multimodal LLMs](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/bonus-resource-multimodal-llms) Last updated 1 year ago --- # Bonus Resource: Multimodal LLMs | 10 Days Realtime LLM Bootcamp At this point, you're likely familiar with large language models (LLMs) and generative AI. Now, let's delve into the exciting world of Google Gemini via this short explainer by Mudit Srivastava. 😄 Google Deepmind introduced Gemini in December 2023, showcasing its potential to revolutionize the capabilities of LLMs. This launch, however, didn't come without its [share of critique](https://www.cnbc.com/2023/12/08/google-faces-controversy-over-edited-gemini-ai-demo-video.html) , especially regarding Google's approach to refining Gemini's initial release for a more polished presentation. This practice, quite prevalent among developers and innovators, often stirs up a vital discussion on ethics. But setting that aside, one thing is clear: Google's video offers an intriguing peek into the possibilities of multimodal LLMs. It's an exciting hint at what the future holds for enthusiasts like us in the world of generative AI. Let's dive into the video and see what's in store! DeepMind's release comprised three models: Gemini Pro, which matches the abilities of GPT-3.5, and the more advanced Gemini Ultra, surpassing GPT-4. The Nano versions, optimized for mobile use, add an extra layer of innovation. * Interested in their applications for Android? [Explore here](https://android-developers.googleblog.com/2023/12/leverage-generative-ai-in-your-android-apps.html) . * And for those of you who are developers, the Gemini API is now accessible on Kaggle. Explore [it here](https://www.kaggle.com/models/google/gemini-api) . But have you ever wondered what sets multimodal LLMs apart? Unlike the conventional text-only models, these models are unique in their ability to process diverse data types. Let's dive into this intriguing world! ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/bonus-resource-multimodal-llms#what-are-multimodal-models) What are Multimodal Models? Envision an AI that perceives the world not only through text but also through visuals, audio, and beyond. This is the essence of multimodal models. A notable illustration is found in Google DeepMind's research on Google Gemini. In this example, Gemini showcases its capability for inverse graphics, where it deduces the underlying code that could have produced specific plots. This process involves reconstructing the visual elements into code and applying necessary mathematical transformations to generate the corresponding code accurately. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FVsxQFnNZ2hEFMUANfNe7%252FUntitled%2520design-8.png%3Falt%3Dmedia%26token%3Db96d394a-c11e-4010-a365-809971f5520b&width=768&dpr=4&quality=100&sign=7e72faee&sv=2) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/bonus-resource-multimodal-llms#exploring-various-data-modalities) Exploring various data modalities This section draws upon the valuable insights from an informative [write-up on MLLMs by Chip Huyen](https://huyenchip.com/2023/10/10/multimodal.html) . In Multimodal Large Language Models (MLLMs), we explore the fascinating world where different data types are translated and interchanged, opening up a realm of possibilities. Let's take a closer look: * **Audio as Visuals:** Imagine audio waves transformed into visual spectrums like mel spectrograms. This conversion offers a new perspective, making audio data visually interpretable. * **Speech into Text:** When we transcribe speech, we're capturing words but also losing out on nuances like the speaker's tone, volume, and pauses. It's a trade-off between capturing the literal and missing the emotional cues. * **Images in Textual Form:** Here's where it gets interesting. An image, in essence, can be broken down into a vector format and then represented as a sequence of text tokens. It's like translating a visual story into a textual narrative. * **Videos – Beyond Moving Images:** While it's common to see videos as sequences of images, this overlooks the rich layer of audio that accompanies them. Remember, in platforms like TikTok, sound is not just an add-on; it's a vital part of the experience for most users. * **Text as Images:** Something as simple as photographing text turns it into an image. This is a straightforward but effective way of changing data modalities. * **Data Tables to Visual Charts:** Converting tabular data into charts or graphs transforms dry numbers into engaging visuals, enhancing understanding and insight. Beyond these, think about the potential of other data types. The possibilities would be endless if we could effectively teach models to learn from bitstrings, the foundational elements of digital data. Imagine a model that could seamlessly learn from any data type! What about data types like graphs, 3D assets, or even sensory data like smell and touch (haptics)? While we haven't delved deeply into these areas yet, the future of MLLMs in these uncharted territories is both exciting and promising! ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/bonus-resource-multimodal-llms#bonus-resource-recommended-if-youre-already-aware-of-the-encoder-decoder-architecture) Bonus Resource: Recommended if you're already aware of the Encoder-Decoder Architecture To delve deeper into the workings of Google Gemini, it's essential to understand its architecture, rooted in the encoder-decoder model. Though not elaborated in detail in their publications so far, Gemini's design appears to draw from DeepMind's Flamingo, which features separate text and vision encoders. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FldNN5PDOPvnQWzX2tp1r%252Fmultimodal%2520llm.png%3Falt%3Dmedia%26token%3Dc608a348-a578-434b-b323-d524219777c4&width=768&dpr=4&quality=100&sign=b00a3b94&sv=2) Source: Gemini Team, Google (2023). Gemini: A Family of Highly Capable Multimodal Models Following our [module on Transformers](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses) in this bootcamp, we've also included a Bonus Resource – a live session on Pathway led by Dr Vijay Srinivas Agneeswaran (Sr. Director, ML Research at Microsoft). This session is ideal for those interested in delving deeper into the role of computer vision in LLMs and exploring Microsoft's advancements in vision transformers. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/bonus-resource-multimodal-llms#papers-for-further-reading) Papers for Further Reading * [Wu, S., Fei, H., Qu, L., Ji, W., & Chua, T.-S. (2023). NExT-GPT: Any-to-Any Multimodal LLM. NExT++, School of Computing, National University of Singapore](https://arxiv.org/abs/2309.05519) * [Gemini Team. (2023). Gemini: A Family of Highly Capable Multimodal Models. Google](https://assets.bwbx.io/documents/users/iqjWHBFdfxIU/r7G7RrtT6rnM/v0) * [Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., & Chen, E. (2023). A Survey on Multimodal Large Language Models. USTC & Tencent YouTu Lab](https://arxiv.org/abs/2306.13549) [PreviousAdvantages and Applications of Large Language Models](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/basics-of-llm/advantages-and-applications-of-large-language-models) [NextWord Vectors Simplified](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified) Last updated 1 year ago --- # What is a Word Vector | 10 Days Realtime LLM Bootcamp Before diving into the intricacies of Large Language Models, it's crucial to understand the building blocks: Word Vectors. Imagine language as a multi-dimensional space; each word takes up a specific point within that space. That's what word vectors do; they represent text in numerical form to make it comprehensible for LLMs. In the accompanying video, Anup Surendran lays down the foundation for understanding LLMs by answering key questions: * What do LLMs stand for? * What exactly are Word Vectors? [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/what-is-a-word-vector#undefined) ----------------------------------------------------------------------------------------------------------------------------------------------- [PreviousWord Vectors Simplified](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified) [NextWord Vector Relationships](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/word-vector-relationships) Last updated 1 year ago --- # Word Vector Relationships | 10 Days Realtime LLM Bootcamp Navigating the landscape of text representation, and grasping how words relate to each other in vector form is essential. In the upcoming video, Anup Surendran dives into the history of word vectors and takes a closer look at Google's groundbreaking Word2Vec project. Why are vector relationships so critical, and what biases do they bring? Let's find out! Here, Anup traces the evolution of word vectors, emphasizing the milestone that is Google's Word2Vec project. One of the standout features of Word2Vec is vector arithmetic, allowing us to reason about words mathematically. For example, the vector equation "King - Man + Woman = Queen" showcases this property brilliantly. The video also explores the role of word vector relationships in similarity search—a key capability in large language models. However, Anup discusses a very important component, i.e., the inherent biases in these major technological developments. Understanding these relationships and their implications deepens our grasp of Large Language Models and equips us to use them more responsibly. 🌐 ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/word-vector-relationships#a-practical-insight) **💡** A practical insight You’ll often hear the “vector embeddings” and “word vectors” being used interchangeably in the context of LLMs. These vector embeddings are then stored in vector indexes, specialized data structures engineered to ensure rapid and relevant data access using these embeddings. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/word-vector-relationships#how-to-choose-the-right-vector-embeddings-model) How to Choose the Right Vector Embeddings Model Selecting the appropriate model for generating embeddings is an intriguing topic on its own. It's essential to recognize that this domain has no one-size-fits-all solution. A glance at this [MTEB Leaderboard on Hugging Face](https://huggingface.co/spaces/mteb/leaderboard) reveals a variety of embedding models, each tailored for specific applications. Currently, OpenAI's `**text-embedding-ada-002**` stands out as the go-to model for producing efficient vector embeddings from diverse data, whether structured or unstructured. We'll delve deeper into its utilization in our tutorials by the end of this course. [PreviousWhat is a Word Vector](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/what-is-a-word-vector) [NextRole of Context in LLMs](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/role-of-context-in-llms) Last updated 1 year ago --- # Role of Context in LLMs | 10 Days Realtime LLM Bootcamp Let's dive a bit deeper into the world of word vectors and explore how context comes into play. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Flh7-us.googleusercontent.com%2FEvfCUbfK_6pq4fESccxg02JZ21d00I_foFoXMKsOTeqIf-_SLkTLs65U6yCR8YG1JICjH-NOSAxN02f1Scl7Xq3g3M0f5dYIbbqecmTh7Wc4GZMCdTp84aflWV_T4fJbE648B_s_c0k_6blYUc2RZNc&width=768&dpr=4&quality=100&sign=50a648b2&sv=2) Imagine you're trying to understand the word "apple." Without context, it could be a fruit or a tech company. But what if I say, "I ate an apple"? Now it's clear, right? Context helps us make sense of words, and it's no different for large language models. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/role-of-context-in-llms#technical-explanation-made-simple) Technical Explanation Made Simple Large language models like GPT-4 or Llama use various techniques to understand the context surrounding each word. For instance, GPT-4 leverages a popular and efficient technique called the "attention mechanism," which helps the model focus on different parts of the text to understand it better. However, older models might use other strategies like Recurrent Neural Networks (RNNs) or Long Short-Term Memory Networks (LSTMs) to capture context differently. Whether it's attention mechanisms or RNNs, the goal is to give the model a better **understanding of how words relate to each other**. This understanding is crucial for tasks like language translation, text summarisation, and question answering. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/role-of-context-in-llms#now-you-know-how-context-matters) Now you know how Context Matters. Context is not just a technical requirement but a functional necessity. By understanding the context, these models can perform tasks ranging from simple ones like spelling correction to complex ones like reading comprehension. So, the next time you see a language model perform a task incredibly well, remember that it's not just about the individual words but also the context in which they are used. [PreviousWord Vector Relationships](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/word-vector-relationships) [NextTransforming Vectors into LLM Responses](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses) Last updated 1 year ago --- # Let's Track Our Progress | 10 Days Realtime LLM Bootcamp Up next is a 10-question graded quiz. It's designed to test your understanding of LLMs—from their advantages to applications. Take this chance to self-assess and solidify your grasp of the topics covered. Ready? 📝 * Here's the link to the graded quiz: [https://www.flexiquiz.com/SC/N/vector-quiz](https://www.flexiquiz.com/SC/N/vector-quiz) (Note: While attempting the quiz, please enter the email ID you used to register for the bootcamp.) [PreviousMulti-Head Attention and Further Reads (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module) [NextPrompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering) Last updated 1 year ago --- # Prompt Engineering | 10 Days Realtime LLM Bootcamp Venture into the realm of prompt engineering, the strategic technique behind optimizing interactions with Large Language Models (LLMs). This module unravels the science and art of crafting precise context to garner desired outcomes from LLMs. By diving deep into the principles of in-context learning and the nuances of prompt design, you'll gain insights into the intricate dance between human queries and machine-generated responses. As you journey through this section, you'll discover the foundational concepts that underpin effective communication with LLMs. [PreviousLet's Track Our Progress](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/lets-track-our-progress) [NextWhat is Prompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/what-is-prompt-engineering) Last updated 1 year ago --- # Prompt Engineering and In-context Learning | 10 Days Realtime LLM Bootcamp Ready to dive deeper? Our next resource is an easy-to-understand video by Mike Chambers from AWS that serves as your foundation in prompt engineering. He'll introduce you to the various techniques involved, further illuminating the strategies for extracting more value from LLMs. Consider it your stepping stone to becoming proficient in this crucial skill. _(Credits: Build on AWS)_ [PreviousWhat is Prompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/what-is-prompt-engineering) [NextBest Practices to Follow in Prompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering) Last updated 1 year ago --- # Neural Networks and Transformers (Bonus Module) | 10 Days Realtime LLM Bootcamp Now that you have an overview let's dive deeper into this bonus module. This Bonus Module will be easier to understand if you're familiar with Neural Networks, Backpropagation, Sequence-to-sequence learning, and libraries such as NumPy. In machine learning, Transformers are akin to Optimus Prime in the Transformers movie series. Just as Optimus Prime can transform from a truck into a powerful leader, these models transform simple inputs into complex, insightful outputs, mastering tasks from language translation to code generation. Transformers are central to revolutionary projects like AlphaFold 2 and NLP giants like GPT-4 and Llama. To truly grasp machine learning's potential, understanding Transformers is essential. Today, we delve slightly into the core of these AI 'Optimus Primes'. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FMt7ITS1noGveLVQQSv1Z%252Fimage.png%3Falt%3Dmedia%26token%3D78985c96-d518-4a6a-a54d-4645c9ebdc07&width=768&dpr=4&quality=100&sign=285511c9&sv=2) "The transformers in LLMs aren't about me, but they have their own flair!" said no Optimus Prime ever. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/neural-networks-and-transformers-bonus-module#first-things-first-neural-networks-and-rnns) First things first: Neural Networks and RNNs Before we get into transformers, let's first understand the background. Let's start by getting a quick understanding of neural networks. Imagine them as the brains inside computers, designed to make sense of all sorts of information, whether a picture, a piece of music, or a sentence. **Quick Look at Neural Networks** * **What They Are:** Neural networks are like virtual brains in computers. They learn from examples and get good at specific tasks. * **How They Function:** They're made up of layers of 'neurons' that work together to understand and interpret data. **Different Types for Different Tasks** * **Convolutional Neural Networks (CNNs):** * **Role:** CNNs are like the eyes of the AI world, great for understanding images. * **How They Work:** They break down images into smaller pieces and learn to recognize patterns, sort of like how we piece together a puzzle. * **Uses:** They're behind the magic of facial recognition and reading handwritten notes. * **Recurrent Neural Networks (RNNs):** * **Role:** RNNs are like the AI's language experts. * **How They Work:** They read sentences word by word, remembering each word as they go, much like reading a book. * **Challenges:** RNNs can get overwhelmed with really long texts, like long paragraphs, and sometimes forget the beginning by the time they reach the end. They also can be challenging to train, with problems like forgetting earlier information (vanishing gradients) or learning too much all at once (exploding gradients). * **Uses:** They help in translating languages or powering chatbots. Neural networks, like different tools in a toolkit, are specialized for specific kinds of jobs. CNNs excel with visuals, while RNNs handle language. But, as we'll see next, transformers brought new abilities to the AI world, especially in dealing with language more smartly. [PreviousTransforming Vectors into LLM Responses](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses) [NextAttention and Transformers (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module) Last updated 1 year ago --- # Transforming Vectors into LLM Responses | 10 Days Realtime LLM Bootcamp Alright, you've got a handle on word vectors and the role of context. Before we delve into the comprehensive pipelines that make Large Language Models (LLMs) function, it's crucial to understand the roles of tokenisers and detokenisers. Think of a tokenizer as a "sentence chopper." It breaks down a sentence into smaller parts, like words, characters, subwords, or symbols, which the model can understand. This generally depends on the type and the size of the model. Detokenizers do the reverse; they take the LLM's output and stitch it back into sentences we can understand. This process is a foundational step for LLMs to translate human queries into actionable tasks. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses#up-next-explainer-by-mike-chambers-from-aws) Up Next: Explainer by Mike Chambers from AWS To give you a more concrete understanding, our next resource is an insightful video by Mike Chambers. This video demystifies what happens when you send a ‘prompt’ (or an input text) to an LLM. Though the internal mathematics may be intricate, the overall goal is straightforward: word prediction. The video will guide you through how your prompts are processed to generate coherent text responses. This will set the stage for our subsequent discussions on Prompt Engineering and LLM pipelines, offering a cohesive picture of how these models operate. _(Credits: Build on AWS)_ Here you see how a Large Language Model’s job is to predict the next word based on the context. Now that you understand the role of "context," you might want to grasp some concepts to appreciate how these models work at a granular level. Given the timelines of this course, while foundational for a better understanding of LLMs, these concepts are tagged as Bonus Resources. * **Attention in Large Language Models:** Imagine being in a room where multiple conversations are happening. Your ability to focus on one conversation over the others is similar to how Attention works in neural networks. It allows the model to 'focus' on relevant parts of the input for tasks. * **Encoder-Decoder Architecture:** An encoder translates the input (e.g., a sentence) into a fixed-size context vector. The decoder takes this context vector to generate an output sequence (e.g., a translated sentence). When the attention mechanism is in action, it guides the Decoder to focus on certain parts of the Encoder’s output, enhancing the translation or text generation task. The concept of Attention complements the Encoder-Decoder architecture, making it more effective and efficient. This architecture is a building block for LLMs such as GPT-3.5. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses#bonus-links) Bonus Links If you're interested in delving further into the details, you may find the following links on embeddings, attention mechanisms, and encoder-decoder architecture beneficial. A foundational understanding of neural networks, backpropagation, the softmax function, and cross-entropy will enhance your comprehension of these resources. These topics are not the primary focus of this course, so they're provided as bonus links. * **Understanding Transformers:** Check the Bonus Module Right Ahead. * [Attention mechanism: Overview](https://youtu.be/fjJOgb-E41w?t=18) | Short Video by Google Cloud * [Word2Vec and Word Embeddings](https://youtu.be/viZrOnJclY0) | Video by StatQuest * [Seq2Seq Encoder-Decoder Neural Networks](https://youtu.be/L8HKweZIOmg) | Video by StatQuest * [Attention is all you need](https://arxiv.org/abs/1706.03762) | Read the Paper on ArXiv * [Attention is all you need](https://youtu.be/XfpMkf4rD6E?t=1211) | Watch the seminar by Stanford Online [PreviousRole of Context in LLMs](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/role-of-context-in-llms) [NextNeural Networks and Transformers (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/neural-networks-and-transformers-bonus-module) Last updated 1 year ago --- # Token Limits in Prompts | 10 Days Realtime LLM Bootcamp By now, you know LLMs are the AI powerhouses trained on heaps of data, and prompts enable you to make the most out of them. However, it’s important to learn that different LLMs have specific token limits that define their performance. Ideally, when creating your prompt, you need to ensure you’re not crossing these token limits. Let’s understand this concept quickly. * Token Limits: These dictate how many tokens an LLM can handle in one go. * Estimated Word Counts: This refers to the approximate number of words that can fit within a model’s token limit. It helps you gauge how much content you can generate or process. You'll notice an error if you try copy-pasting a lengthy Wikipedia article (for example, that of Google). ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Flh7-us.googleusercontent.com%2FRQJ9ihv8p-nSve8fMd6Rrwf1vLFW5F8dvEm3Qw2itfONS4JKx9eqwSo_WyA8BGptZWgN1b0ZfhJceTSHBA1K434uT27WNAlV-6ZYlVLrWaeu6ZNg8BFw78lHIzk4WP_92tmMTcfabsLDhdq6ld5kYsg&width=768&dpr=4&quality=100&sign=3938de11&sv=2) Think of token and word counts as your LLM's capacity. While tokens define the technical limit, estimated word counts translate this into a more human-understandable measure. Why It Matters: Knowing the estimated word count helps you manage your input prompts and outputs more efficiently. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/token-limits-in-prompts#comparative-analysis-token-and-estimated-word-counts-in-a-few-leading-llms) Comparative Analysis: Token and Estimated Word Counts in a Few Leading LLMs ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Flh7-us.googleusercontent.com%2F-0SGuT0T4JTuCH6OYwR025-wlzSd-63bpWWMmFgxuRkbifpP4BBuByciK1YGIgnWXm3TUnBUZCTdfWLJl_i72LoT_2ZaUNBQHfF5tEKm1Y3_nRX7bs0zEca6TWc4IiZw5LNGpVOnwe3jUvWcCjuq97o&width=768&dpr=4&quality=100&sign=e37d3f98&sv=2) While the foundational knowledge provided is adequate for course progression, further exploration of tokens is available in the documentation linked below. * [Tokens and Efficient Prompt Design | Open AI](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them) * [LLM AI Tokens | Microsoft](https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/tokens) [PreviousBest Practices to Follow in Prompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering) [NextPrompt Engineering Excercise](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-excercise) Last updated 1 year ago --- # What is Prompt Engineering | 10 Days Realtime LLM Bootcamp Have you ever wondered how to get more accurate and tailored results from a Large Language Model? Enter the world of in-context learning. In simple terms, in-context learning allows a model to understand and adapt based on the information you feed it. The more context you provide, the more refined the output. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/what-is-prompt-engineering#unpacking-prompt-engineering) Unpacking Prompt Engineering So, where does prompt engineering fit in? Think of prompt engineering as the art of crafting these pieces of context in a way that guides the model toward the desired outcome. It’s essentially in-context learning itself and your direct channel of communication with the model. You can present your problem statements in two broad ways—either with minimal context, known as zero-shot or one-shot prompts, or with additional guiding context, called few-shot prompts. However, for now, you can park the jargons and realize that each prompting approach has its strengths and limitations, but the aim is the same: to pull the most precise responses out of the LLM. [PreviousPrompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering) [NextPrompt Engineering and In-context Learning](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-and-in-context-learning) Last updated 1 year ago --- # Tasks in the Excercise | 10 Days Realtime LLM Bootcamp **Subtask 1: Plot Extension** Design a prompt to generate a continuation of the mystery story that uncovers another layer of the hacking plot. Perhaps ValĂ©rie finds out there's a mole in her organization, or maybe Sophie encounters another similar case. Craft a prompt that nudges the story into revealing this new dimension. Evaluate how seamlessly the new narrative fits with the original story. **Subtask 2: Few-Shot Prompting for Character Analysis** Utilize a few-shot prompt that instructs ChatGPT to analyze the main characters' emotional states at crucial points in the story. For instance, ask the model to perform sentiment analysis on ValĂ©rie when she discovers the hacking, and Sophie when she finally solves the case. Note the efficacy of few-shot prompting in eliciting nuanced character analysis. **Subtask 3: Limitation Recognition due to Outdated Data** Pose a question about the use of AI and machine learning algorithms in contemporary eSports as depicted in the story, asking for the latest advancements as of 2023. Evaluate ChatGPT's response for potential inaccuracies or outdated information given its last training data is from April 2023 (this date is certainly expected to change from time-to-time). Discuss how this limitation affects the believability of the plot and the technological aspects described in the story. For each subtask, provide the prompt you used, summarize the response, and give a thorough analysis of how well ChatGPT performed in terms of context, plot coherency, and technological accuracy. [PreviousStory for the Excercise: The eSports Enigma](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-excercise/story-for-the-excercise-the-esports-enigma) [NextRetrieval Augmented Generation and LLM Architecture](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture) Last updated 1 year ago --- # Story for the Excercise: The eSports Enigma | 10 Days Realtime LLM Bootcamp In Lyon's bustling technology district, ValĂ©rie Leroux was a sorceress of software, a conjurer of code. She had created an analytics program that could break down and analyze real-time strategy for the hyper-competitive game "WarScape Online." Yet, a dark cloud hovered over her digital utopia; someone had infiltrated the sacred realms of her software, sowing discord in its algorithms and threatening to upheave the imminent World Championship. ValĂ©rie knew she needed specialized help. That's where Detective Sophie Martin came into the picture—a school friend turned cybersecurity sleuth, who could trace digital footprints like a bloodhound. Sophie's interest piqued immediately; this was no ordinary hack. The integrity of eSports, a burgeoning industry with more at stake than just pixels and points, hung in the balance. Sophie arrived at ValĂ©rie's whimsical office, where tech gadgets met old-world elegance. "If I understand correctly," Sophie said, half-amused, "your software is essentially the Professor McGonagall of eSports—stern but invaluable?" "Very much so," ValĂ©rie responded. "Except someone has confounded McGonagall into giving rather questionable advice." \--- The World Championship transformed the local stadium into a cathedral of digital warfare. Fans donned vibrant costumes, clutching merchandise like talismans. ValĂ©rie and Sophie occupied a discreet corner, scrutinizing the spectators, players, and support staff. One person caught their attention—Jean-Pierre, a competitor in the analytics sector. He was glued to his phone, a sentinel of secrets, scheming in silence. As the matches unfolded, ValĂ©rie's software hiccuped, whispering faulty strategies into the ears of players. Sophie and ValĂ©rie exchanged worried glances; their nemesis had struck again. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252Fa14HKp6H7ueaQKrCkIPA%252Fimage.png%3Falt%3Dmedia%26token%3Dd33f0fe1-75ea-4f09-801c-f7908c20e0bc&width=768&dpr=4&quality=100&sign=c00d8a75&sv=2) \--- In a secluded chamber, Sophie leaned in, "We're hunting a phantom, someone who knows how to erase their traces. What can we find that they've left behind?" Armed with her laptop, ValĂ©rie navigated the labyrinthine logs of her software. The hacker had disguised their moves meticulously, but a sliver of evidence gleamed—a distinctive encryption pattern that Jean-Pierre had championed in professional talks. It was circumstantial but pointed. Sophie contacted her sources. Jean-Pierre had a solid alibi—he'd been hosting a webinar during the time of the original hack. However, upon deeper probing, Sophie uncovered something fascinating. Jean-Pierre had recently acquired software capable of initiating a delayed, remote hack. He could set up the intrusion ahead of time and have an alibi for when it occurred. Confronted with this evidence, Jean-Pierre's resistance shattered. Financial desperation had led him into the murky waters of sabotage. Though he tried to justify his actions, the weight of ethical betrayal was inescapable. \--- As they parted ways after the case, Sophie reflected, "Every technology is a Pandora's Box, isn't it? Filled with wonders and horrors alike." ValĂ©rie agreed, solemnly. "Yes, but the key is recognizing which is which and acting accordingly." Sophie drove back to her precinct, and ValĂ©rie to her fortress of dreams, both forever bound by the curious adventure that had led them through the veils separating right and wrong, reality and virtuality. [PreviousPrompt Engineering Excercise](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-excercise) [NextTasks in the Excercise](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-excercise/tasks-in-the-excercise) Last updated 1 year ago --- # Best Practices to Follow in Prompt Engineering | 10 Days Realtime LLM Bootcamp Mastering the art of designing a prompt comes with practice, and it can significantly improve your interactions with Large Language Models (LLMs). It's crucial to note that the best practices discussed here are primarily geared towards generating language-based outputs. For more specialized tasks, such as generating code, images, or other types of non-textual data, it's advisable to consult the specific guidelines and documentation related to those tasks. Let's delve into some best practices that could act as your guiding principles. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering#basic-prompts-the-starting-point) Basic Prompts: The Starting Point * **Be Concise** Avoid verbosity for succinct and effective prompts. ❌ "What do you think could be a good name for a flower shop that specializes in selling bouquets of dried flowers?" ✅ "Suggest a name for a flower shop that sells bouquets of dried flowers." * **Be Specific** Narrow your instructions to get the most accurate response. ❌ "Tell me about Earth" ✅ "Generate a list of ways that makes Earth unique compared to other planets." * **Prompt Structuring** Ask One Task at a Time: Avoid combining multiple tasks in one prompt. ❌ "What's the best method of boiling water and why is the sky blue?" ✅ "What's the best method of boiling water?" * **Detailing**: Specify context, outcome, format, length, etc. * **Example-Driven**: Utilize examples to guide the output. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering#zero-shot-vs.-few-shot-prompts) Zero-Shot vs. Few-Shot Prompts When you give more examples to the model, it gets better at understanding what you're asking. This helps it give answers that are more on-point or accurate. **Zero-Shot Prompting:** * You ask the model to do something without giving any examples. * Example: "Is a goldfish a pet or not a pet?" Output: "Pet" **One-Shot Prompting:** * You give the model one example to help it understand your question. * Example: "For instance, a dog is a pet. Now, is a goldfish a pet or not a pet?" Output: "Pet" **Few-Shot Prompting:** * You give the model several examples to ensure it understands your question. * Example: "A dog is a pet." "A lion is not a pet." Now, "Is a goldfish a pet or not a pet?" Output: "Pet" In this example, all prompting types resulted in the same answer: "Pet". However, with few-shot prompting, you can be more confident that the model truly understands what you mean by "pet" since it has more examples to learn from. Usually, giving more examples (few shots) helps the model give better answers, especially for more complicated questions. Thumb rule, Zero-shot, one-shot, and few-shot prompting have distinct advantages and challenges. Zero-shot is more open-ended while few-shot is more controlled. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering#elements-of-a-prompt-know-the-ingredients) Elements of a Prompt: Know the Ingredients * Instruction: What task do you want the model to perform? * Context: Additional information that can steer the model. * Input Data: The question or data of interest. * Output Indicator: Desired format or type of the output. You don't always need all these elements; it depends on your needs. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering#general-tips-the-dos-and-donts) General Tips: The Do's and Don'ts * Start Simple: Initial iterations should be straightforward, and you can build complexity as you refine your prompts. * Avoid Redundancy: Use concise, non-redundant language. * Be Specific: Vague instructions often yield vague results. * Avoid Negative Instructions: Instead of saying what not to do, focus on what the model should do. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/best-practices-to-follow-in-prompt-engineering#bonus-resources) Bonus Resources: Curious to learn more? Once you’ve completed this course, you might want to check these resources that will help you dive deeper into the nuances of prompt engineering: * [LearnPrompting's Comprehensive Guide](https://learnprompting.org/docs/intro) * [Official Docs by OpenAI](https://platform.openai.com/docs/guides/gpt-best-practices) * [Concise Article by OpenAI](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api) Take your time to experiment and iterate, as mastery comes with practice and refinement. And remember, this is a living, evolving field; staying updated with best practices is key to success. [PreviousPrompt Engineering and In-context Learning](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-and-in-context-learning) [NextToken Limits in Prompts](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/token-limits-in-prompts) Last updated 1 year ago --- # Prompt Engineering Excercise | 10 Days Realtime LLM Bootcamp Up next is an interesting story for you. This exercise aims to explore the capabilities and limitations of prompt engineering in the context of the mystery story set in the eSports arena. By focusing on critical aspects like plot extension, few-shot prompting, and model limitations, you will assess how well you can steer the narrative or extract specific insights. [PreviousToken Limits in Prompts](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/token-limits-in-prompts) [NextStory for the Excercise: The eSports Enigma](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-excercise/story-for-the-excercise-the-esports-enigma) Last updated 1 year ago --- # Attention and Transformers (Bonus Module) | 10 Days Realtime LLM Bootcamp Having understood the sequential nature of RNNs and their limitations, it's time to introduce the game-changer in the world of language processing: Transformers. * **Breaking the Sequential Barrier:** Unlike RNNs that process words one after another, transformers can look at an entire sentence, or even a paragraph, all at once. This ability to process data in parallel is a major leap forward, significantly speeding up how the model learns and understands language. * **Efficiency in Learning:** Since transformers don't have to read in sequence, they can be trained on much larger chunks of data at a time. This parallel processing means they can learn from vast amounts of text more efficiently, using powerful hardware to speed things up. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#foundation-of-transformers-attention-mechanism) Foundation of Transformers: Attention Mechanism At the core of transformers is a technique called the "Attention Mechanism." Initially developed for machine translation, it's now used for various language tasks. * **How It Works:** Imagine the model as focusing its 'attention' on different parts of a sentence to understand its meaning better. For instance, in the sentence "Taylor is setting new benchmarks with her Eras Tour " the model pays more attention to the word "Taylor" when trying to understand who is doing the action. * **Different Flavors:** While there are many types of attention mechanisms, transformers use a specific kind called "Scaled Dot-Product Attention." It's a fancy way of saying the model calculates attention based on a mathematical formula, which involves queries, keys, values, and a scaling factor. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#scaled-dot-product-attention) Scaled Dot-Product Attention The attention mechanism described in the paper [Attention is all you Need](https://arxiv.org/abs/1706.03762) is called _Scaled Dot-Product Attention_, and is defined via the formula below: * Attention(Q,K,V)\=softmax(QKTdk)V\\text{Attention}(Q, K, V) = \\text{softmax}\\left(\\frac{QK^T}{\\sqrt{d\_k}}\\right)V Attention(Q,K,V)\=softmax(dk​​QKT​)V * Here, QQQ, KKK, and VVV represent matrices for queries, keys, and values. How do we get those matrices? We'll understand it soon. But for now, you can know that the formula calculates attention as a weighted sum of values, where the weights are determined by a compatibility function of the query with the corresponding keys. * **Understanding the Components:** * **Queries, Keys, Values:** These are vectors representing different aspects of the input data, that we'll understand now. * **Softmax Function:** This part of the formula helps in normalizing the weights, ensuring they sum up to 1. * **Scaling Factor:** The dk\\sqrt{d\_k}dk​​ term helps in stabilizing the gradients during training, where dk{d\_k}dk​ is the dimension of the key vectors. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#calculating-q-k-and-v) Calculating Q, K, and V Let's break down the process of calculating Q (Query), K (Key), and V (Value) in the context of a self-attention mechanism in a more digestible way. We'll start from the very beginning with a simple example sentence and go through each step. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#step-1-getting-the-vector-embeddings) Step 1: Getting the vector embeddings * For example, let's take this sentence "Manchester United hopes to perform better." * Now the first thing we know is that the sentence is split into words or sub-words, known as tokens. So, we get tokens like \["Manchester", "United", "hopes", "to", "perform", "better"\]. Each token is then mapped to a unique integer for ease of processing. Here, let's say it's \[0, 1, 2, 3, 4, 5\]. * In practice, an embedding matrix is created where each row corresponds to the embedding of a token. The size of this matrix is typically \[vocabulary size x embedding dimension\]. For large models like GPT, the vocabulary size is vast, often in the range of tens of thousands, and the embedding dimension can be large as well, such as 768 or more. * **Note**: Initially, the values in the embedding matrix are randomly initialized. This means that each token is assigned a random vector. During the training phase, these random embeddings are gradually adjusted through [backpropagation](https://youtu.be/IN2XmBhILt4) . Backpropagation is a method used to improve neural networks by going backwards through the network to fine-tune how it makes decisions, based on the errors it made. The model learns to change these embeddings to capture semantic meanings and relationships between words. For example, in our example, we're using random numbers for simplicity. Let's say the embedding dimension is 3. The embeddings for the tokens might look like this after initialization: * Copy Manchester: [ 0.321, -1.024, 0.876] United: [ 1.234, 0.567, -0.890] hopes: [-0.456, 0.789, 1.234] to: [ 1.111, -0.333, 0.222] perform: [-0.777, 0.888, -0.999] better: [ 0.555, -0.666, 0.777] These numbers are arbitrary and for illustrative purposes only. In a real scenario, these embeddings would be learned and adjusted during the training process of the model. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#step-2-calculate-q-k-v) Step 2: Calculate Q, K, V First, we need to define three distinct weight matrices for calculating Q, K, and V. These matrices are randomly initialized. Here we're using NumPy, a powerful & easy to learn library in Python for numerical computing, especially for operations involving arrays and matrices. Example Code: Copy w_q = np.random.random((embedding_dim, 3)) # Weight matrix for Q w_k = np.random.random((embedding_dim, 3)) # Weight matrix for K w_v = np.random.random((embedding_dim, 3)) # Weight matrix for V Each of these weight matrices is then multiplied with the embedding matrix to create Q, K, and V. Example Code: Copy Q = embeddings @ w_q # Query K = embeddings @ w_k # Keys V = embeddings @ w_v # Values The outputs of Q, K, and V will be matrices. This operation is a form of linear transformation where each input (embedding) is transformed into three different spaces represented by Q, K, and V. Let's assume our embeddings look like this (using the embeddings from our previous example): Here's an example of what they might look like: Copy # Sample Output (these values are for illustration and will vary) print("Query (Q):") print(Q) # Output: # [[ 0.456, -0.789, 1.234],\ # [ 1.111, -0.333, 0.222],\ # ... ] print("Keys (K):") print(K) # Output: # [[-0.456, 0.789, -1.234],\ # [-1.111, 0.333, -0.222],\ # ... ] print("Values (V):") print(V) # Output: # [[ 0.123, -0.456, 0.789],\ # [ 0.321, -0.654, 0.987],\ # ... ] #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#step-3-calculate-attention-scores) Step 3: Calculate Attention Scores The attention scores are computed by taking the dot product of the Q and K matrices. This step measures the similarity between each query and key. * The dot product is divided by the square root of the dimensionality of K, i.e. dk\\sqrt{d\_k}dk​​ to keep the gradients stable. * The softmax function is then applied to convert these scores into a probability distribution. * Example Code: Copy def softmax(x: np.ndarray, axis: int) -> np.ndarray: x = np.exp(x - np.amax(x, axis=axis, keepdims=True)) return x / np.sum(x, axis=axis, keepdims=True) scores = Q @ K.T # Dot product between Q and K scores = softmax(scores / np.sqrt(K.shape[1]), axis=1) * The resulting scores are a matrix where each element indicates the attention score, representing the relevance of each word in the context of other words in the sentence. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module#step-4-applying-the-attention-scores-to-values) Step 4: Applying the Attention Scores to Values Finally, the attention scores are multiplied by the V matrix. This operation selectively weighs the importance of each value based on the computed attention scores. Example Code: Copy attention_output = scores @ V The resulting `attention_output` matrix contains the final output of the attention mechanism, where each row represents an aggregated representation of each word, taking into account the context provided by the entire sentence. This process, starting from the creation of Q, K, and V to the computation of attention scores and their application to V, forms the core of the **self-attention mechanism**, allowing models to dynamically focus on different parts of the input sequence based on the context. [PreviousNeural Networks and Transformers (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/neural-networks-and-transformers-bonus-module) [NextMulti-Head Attention and Further Reads (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module) Last updated 1 year ago --- # Multi-Head Attention and Further Reads (Bonus Module) | 10 Days Realtime LLM Bootcamp Building on the foundation of self-attention, Transformers use a more refined approach using the concept of **Multi-Head Attention**. This mechanism enhances the model's ability to focus on different parts of the input for various reasons. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#lets-understand-multi-head-attention) Let's understand Multi-Head Attention. The basic idea of multi-head attention is simple. Think of Multi-Head Attention as having several pairs of eyes (or "heads") looking at the same sentence. Each pair of eyes focuses on different parts of the sentence, capturing various details. * How it makes a difference: Normally, in attention mechanisms, we use one set of Q (query), K (key), and V (value) to understand the sentence. In Multi-Head Attention, we split this process into several smaller parts, each with its own Q, K, and V. This means we can pay attention to different parts of the sentence simultaneously. * This method allows us to get a richer understanding of the sentence. Each head might notice different things - some might focus on grammar, while others might look for emotion or specific keywords. Example in Simple Code: Suppose we decide to have two heads. Here's a simple way to visualize this in code: Copy # Let's say we have 2 heads number_of_heads = 2 # We divide Q, K, and V for each head divided_Q = np.split(Q, number_of_heads) divided_K = np.split(K, number_of_heads) divided_V = np.split(V, number_of_heads) # Now, each head does its own attention attention_results = [] for q, k, v in zip(divided_Q, divided_K, divided_V): attention_results.append(calculate_attention(q, k, v)) # Finally, we combine the results from each head combined_attention = np.concatenate(attention_results) In summary, Multi-Head Attention allows Transformers to process information in parallel, significantly enhancing their ability to understand and interpret complex language data by examining it from multiple perspectives simultaneously. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#sample-implementation) Sample Implementation * **Creating Multiple Heads**: You create several sets of weight matrices (for Q, K, V) corresponding to the number of heads. * **Applying Attention Independently**: Each head performs the attention mechanism independently, as we did with single-head attention. * **Combining Outputs**: Finally, the outputs from all heads are concatenated and possibly linearly transformed to produce the final output. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#role-of-positional-encodings) Role of Positional Encodings Transformers, in addition to Multi-Head Attention, employ Positional Encodings to understand word order, which is vital for processing sentences. This technique assigns a sequence number to each word, such as \[("Building", 1), ("LLM", 2), ("Apps", 3)\], enabling the Transformer to identify the order of words even when processing them all at once. This is crucial because, unlike sequential models like RNNs that naturally read words one by one, Transformers work on entire sentences simultaneously using parallel processing, making them fast but initially unaware of word order – a gap filled by Positional Encodings. Positional Encodings aren’t limited to simple integer incremental sequences like in the case of \[("Building", 1), ("LLM", 2), ("Apps", 3)\]. There are various methods to calculate them. The authors of the Transformer model used a more complex approach. Their method uses sinusoidal waves of different frequencies, allowing the model to attend to relative positions in the input sequence effectively. PE(pos,2i)\=sin⁥(pos100002i/dmodel)PE(pos, 2i) = \\sin\\left(\\frac{pos}{10000^{2i/d\_{\\text{model}}}}\\right) PE(pos,2i)\=sin(100002i/dmodel​pos​) and PE(pos,2i+1)\=cos⁥(pos100002i/dmodel)PE(pos, 2i+1) = \\cos\\left(\\frac{pos}{10000^{2i/d\_{\\text{model}}}}\\right) PE(pos,2i+1)\=cos(100002i/dmodel​pos​) The first formula here is for calculating PE (Positional Encodings) for even indices (2i) and the second one is for odd indices. In these formulas, `pos` refers to the position in the sequence, `i` is the dimension, and dmodeld\_{\\text{model}}dmodel​ is the dimension of the model. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#putting-it-together-transformers-architecture) Putting it together: Transformers Architecture ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FYltiwQ0kolqToxJu84Wa%252Fattention_research_1-768x1082.webp%3Falt%3Dmedia%26token%3D20e14a3f-68b6-4176-8404-575d17161865&width=768&dpr=4&quality=100&sign=493b36ee&sv=2) Source: Attention Is All You Need" by Vaswani et al The Transformer model's architecture, prominently featuring an Encoder and a Decoder, is a marvel of design in language processing. The left side of the diagram is the Encoder: This part of the model processes the input data. The right side is the Decoder, which is responsible for generating the output based on the processed input from the Encoder. By now, you can already recognize a few of the components in this diagram. Before we quickly summarize how it works, let's cover some key components we haven't discussed yet. * **Feed Forward Neural Network**: Present in each layer of both the Encoder and Decoder, these neural networks perform additional transformations on the data. It's crucial for capturing complex patterns in data, making the model better at handling language nuances. * **Add & Norm**: It's a bit like double-checking work. This step, included in every layer, involves adding the original input back into the output (residual connection) and then normalizing it. This is vital for addressing a common issue in training deep networks known as the vanishing gradient problem. In simpler terms, as a network learns, updates to its weights can become very small, almost "vanishing." This makes learning very slow or even stops it. The residual connections help by ensuring that there’s always a flow of gradients, keeping the learning process alive and effective. * **Linear Layer and Softmax in Decoder**: The Decoder ends with a Linear layer and a Softmax function. Hence these components are key to generating the output sequence. The Linear layer predicts the next word, and Softmax turns these predictions into probabilities. This combination ensures the model selects the most appropriate next word, building the output sequence one word at a time. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#bringing-it-all-together) Bringing It All Together 1. **Processing the Input**: In the Encoder, the input sentence is processed with attention mechanisms and positional encodings, capturing its full context in parallel. 2. **Generating the Output**: The Decoder, using the processed data from the Encoder, attentively constructs the output, ensuring each word fits accurately in the sequence. 3. **Parallel Efficiency**: This parallel processing approach marks a significant advancement over older sequential models, making the Transformer faster and more adept at understanding and generating language. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#video-lecture-on-transformers-by-andrej-karpathy) Video Lecture on Transformers by Andrej Karpathy To deepen your grasp of transformers in AI, consider exploring a lecture by Andrej Karpathy, who has made significant contributions to AI, including leading the AI team at Tesla. His session at Stanford University offers a comprehensive look at the role of transformer models in modern AI applications, making it a valuable resource for those keen to understand this advanced technology. **Key Takeaways of the Session:** * Overview of transformers, focusing on their role in generative models and potential in advanced applications like video understanding. * Evolution from conventional neural networks to transformers, highlighting their impact in natural language processing and machine translation. * A detailed look at the transformer's learning process, including multi-headed attention and backpropagation, for efficient data handling. * Discussion on causal self-attention in transformers, crucial for accurate future predictions in applications like those at Tesla. * The transformer's versatility and future potential, with a nod to possible advancements in AI learning and application breadth. [![Logo](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Fwww.youtube.com%2Fs%2Fdesktop%2F9c0f82da%2Fimg%2Ffavicon_144x144.png&width=20&dpr=4&quality=100&sign=26df8f0b&sv=2)Stanford CS25: V2 I Introduction to Transformers w/ Andrej KarpathyYouTube](https://www.youtube.com/watch?v=XfpMkf4rD6E&t=615s) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#bonus-resource-lecture-on-vision-transformers-by-dr.-vijay-s-agneeswaran) Bonus Resource: Lecture on Vision Transformers by Dr. Vijay S Agneeswaran After exploring Transformers architecture, consider viewing a session at IIT Guwahati, which is available on [Pathway's YouTube channel](https://www.youtube.com/@pathwaycom) . This session, featuring Vijay, a Senior Director and ML Research Leader at Microsoft, sheds light on the latest in computer vision, vision transformers, and their role in Large Language Models (LLMs). **Key Takeaways of the Session:** * The transition from traditional convolutional networks to pre-trained transformers in computer vision. * The synergy between these advanced transformers and LLMs leading to enhanced image classification and other tasks. * Insight into Scattering Vision Transformers (SVT), detailing their development, technical aspects, and performance. * Demonstration of SVT's leading performance in tasks like image classification (ImageNet dataset) and instance segmentation (MSCoco dataset). ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/multi-head-attention-and-further-reads-bonus-module#resources-for-further-study) Resources for Further Study Further Study To delve deeper into the intricacies of Transformers and Multi-Head Attention, here are some resources you can explore: 1. [**Original Paper**](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf) : "Attention Is All You Need" by Vaswani et al. - This seminal paper introduced the Transformer model. It's a must-read for understanding the theoretical foundations. 2. [**The Illustrated Transformer**](http://jalammar.github.io/illustrated-transformer/) : An accessible and visually-oriented explanation of how Transformers work, including Multi-Head Attention. 3. [**Jan Chorowski's Course Materials**](https://github.com/janchorowski/dl_uwr/tree/summer2021/Lectures) **:** Dive into a comprehensive collection of materials for Jan's Deep Learning and Neural Networks course at the University of WrocƂaw. 4. [**BERT Explained**](https://blog.research.google/2018/11/open-sourcing-bert-state-of-art-pre.html) : A deep dive into BERT, a groundbreaking Transformer model, which revolutionized the field of NLP. These resources will guide you through the theoretical concepts, practical implementations, and the broader impact of Transformers and Multi-Head Attention in the field of natural language processing. [PreviousAttention and Transformers (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/transforming-vectors-into-llm-responses/attention-and-transformers-bonus-module) [NextLet's Track Our Progress](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/word-vectors-simplified/lets-track-our-progress) Last updated 1 year ago --- # Hands-on Development | 10 Days Realtime LLM Bootcamp Welcome to the final module of this bootcamp! Now, we will guide you through setting up a Retrieval Augmented Generation (RAG) architecture using LLM App, an open-source production framework for building and serving AI applications and LLM-enabled real-time data pipelines. While we're working with this tool, consider starring it on GitHub. It is an effortless way to bookmark it for future and track updates, and it also helps the community discover the resource. * Link to the GitHub repository: [https://github.com/pathwaycom/llm-app](https://github.com/pathwaycom/llm-app) By the end of this module, you'll be able to **build your LLM application** that works with realtime data. This implementation guide is aimed at Mac, Linux, and Windows users. **Note:** If you have already completed your first project by consulting the documentation on the LLM App's open-source repository, that's excellent! In that scenario, you may review the videos in this module for additional perspective and proceed to the '[Final Project + Giveaways](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways) ' module. [PreviousTrack your Progress](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/track-your-progress) [NextPrerequisites](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/prerequisites) Last updated 1 year ago --- # Live Interactions with Jan Chorowski and Adrian Kosowski | Bonus Resource | 10 Days Realtime LLM Bootcamp As a part of this bootcamp, we hosted a captivating **Fireside Chat featuring Jan Chorowski**, CTO of Pathway. Jan is a renowned figure in Artificial Intelligence with a Ph.D. in Neural Networks and a portfolio that includes more than 10k+ citations and collaborations on research papers with AI pioneers like Geoff Hinton and Yoshua Bengio. During this captivating Fireside Chat, our host, Anup Surendran, engaged in a deep exploration of the captivating realm of Large Language Models (LLMs) alongside Jan. Their dynamic discussion spanned a wide spectrum of LLM topics, encompassing their diverse applications, the operational hurdles they face, and the intriguing concept of 'learning to forget.' Moreover, the conversation delved into the real-time capabilities of LLMs and illuminated their paramount importance in the ever-evolving landscape of modern technology. **Key Highlights:** * Gain insights into the evolution of LLMs and their practical applications. * Explore the operational challenges faced by LLMs in real-time scenarios. * Understand the concept of 'learning to forget' and its role in LLM development. * Dive into the discussion on the real-time nature of LLMs and their relevance. * Get valuable answers to audience questions about LLMs, document versioning, and more. Don't miss out on this illuminating Fireside Chat that offers a unique perspective on the evolving world of Large Language Models. Dive into the past conversation to uncover the mysteries and possibilities of LLMs in today's tech-driven world. [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/live-interactions-with-jan-chorowski-and-adrian-kosowski-or-bonus-resource#another-bonus-resource-recorded-interaction-on-real-time-data-processing) Another Bonus Resource: Recorded Interaction on Real-time Data Processing ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- This is a session from Pathway's Archives. It's a live interaction between Jon Krohn (Chief Data Scientist at Nebula | [GitHub](https://github.com/jonkrohn) ) and Adrian Kosowski (CPO at Pathway | [Google Scholar](https://scholar.google.com/citations?user=om8De_0AAAAJ&hl=en) ). Adrian Kosowski, notable for his early PhD at and co-founding Spoj.com, brings over 15 years of diverse research experience to the discussion. It delves into real-time data processing, contrasting stream versus batch processing, and exploring practical ML applications. **Key Highlights:** * Gain a deep understanding of real-time data processing nuances through a discussion on reactive data processing. * Explore the key differences and practical applications of stream versus batch processing. * Understand the role of transformers in data engineering, especially in managing and streaming data. * Discover emerging machine learning tools and approaches that are particularly beneficial for startups. Don't miss these illuminating Fireside Chats that offer unique perspectives on the fast-evolving domains of Large Language Models and Real-time Data Processing. These sessions provide valuable insights into the wonderful world of AI and machine learning. [PreviousHow to Run the Examples](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/how-to-run-the-examples) [NextFinal Project + Giveaways](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways) Last updated 1 year ago --- # Prerequisites | 10 Days Realtime LLM Bootcamp Before we dive in, let's ensure you have all the necessary prerequisites installed on your computer. Not only are these essential for what we're about to embark on, but they'll also be invaluable tools if you decide to contribute to open-source projects in the future. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/prerequisites#git-python-and-pip) Git, Python, and Pip * Python 3.10 or above should be installed on your machine. If not, [Download Python](https://www.python.org/downloads/) . * Pip: Comes pre-installed with Python 3.4+. It is the standard package manager for Python. You can check if it's downloaded by typing the below command in your terminal/command prompt. Copy pip --version * If Pip is not installed, you'll get an error. In that case, you must download and install [Pip](https://pip.pypa.io/en/stable/installation/) to manage project packages. * Git should be installed on your machine. If you've installed XCode (or its Command Line Tools), Git may already be installed. To find out, open a Terminal or Command Prompt, and enter `git --version`. If it's not installed, refer to [this documentation](https://www.atlassian.com/git/tutorials/install-git) and install it. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/prerequisites#openai-api-key-recommended) OpenAI API Key (Recommended) This key is required if you plan to use OpenAI models for embedding and generation. This is a good starting point if you are less confident with using open-source alternatives. If you want to use open-source models, you can find examples like [the one here](https://github.com/pathwaycom/llm-app/blob/main/examples/pipelines/local/app.py) . By default, OpenAI currently offers $5 in free credits for new accounts – i.e., the ones with a new phone number and email ID. Alternatively, you can sign up for free credits on platforms like [Eden AI](https://www.edenai.co/eden-ai-for/developers) . These free credits should suffice for building your project. As we advance, we will use`text-embedding-ada-002` in this coursework for generating the vector embeddings ([OpenAI documentation](https://openai.com/blog/new-and-improved-embedding-model) ) and `gpt-3.5-turbo` for text generation. **To create a new OpenAI API Key:** * [Log in](https://platform.openai.com/login?launch) to the OpenAI website. * Navigate to the [API Key Management](https://platform.openai.com/account/api-keys) page to generate your key. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/prerequisites#note-if-youre-using-windows-os) Note: If you're using Windows OS The example ahead only supports Unix-like systems (such as Linux, macOS, and BSD). But the good news is that you have an easy fix. If you are a Windows user, you can use [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/install) or Dockerize the app to run as a container. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/prerequisites#what-is-docker-and-how-do-you-install-it) What is Docker and how do you install it? Think of Docker as a shipping container for your app. Just as a shipping container can hold all sorts of goods (clothes, electronics, etc.) and be transported anywhere, Docker bundles your app and everything it needs to run into a '**container**.' This makes it easy to share and run your app on any computer. Like Docker, there is a tool called **Conda**, which is showcased in one of the videos above. Conda lets you create separate environments to manage different sets of Python packages, ensuring your code runs the same way on any computer. Conda and Docker aim to solve the problem of "it works on my machine" by isolating your project and its dependencies. * You can download Docker [from here](https://www.docker.com/products/docker-desktop/) . * You can download Conda [from here](https://docs.conda.io/projects/conda/en/stable/user-guide/install/download.html) . Now that we have the prerequisites, let's proceed. 😄 [PreviousHands-on Development](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development) [NextDropbox Retrieval App in 15 Minutes](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes) Last updated 1 year ago --- # Final Project + Giveaways | 10 Days Realtime LLM Bootcamp #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways#welcome-to-the-final-stretch-of-your-bootcamp-journey) Welcome to the Final Stretch of Your Bootcamp Journey! As we approach the conclusion of this mini bootcamp, it's time to transform your acquired knowledge into practical applications. To guide you, we've carefully selected a range of exciting project tracks. These projects are your platform for innovation and making a tangible impact! 🌟 ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways#how-to-successfully-complete-the-bootcamp) How to Successfully Complete the Bootcamp * To qualify for your bootcamp certificate, complete the required MCQs—one in the Vector Embeddings module and another in the RAG module. * Concurrently, you're expected to build a real-time, RAG-based LLM application. * Suppose the idea of creating an LLM application from the ground up (like the one we saw in the Amazon Discounts case) feels overwhelming. In that case, you can build upon the "Dropbox Retrieval App" example discussed earlier by tailoring it to meet specific needs. For example, you can construct an application with substantial business or social value, like the EU AI Act showcase, which successfully repurposed the Dropbox Retrieval App example to simplify understanding complex legal documents in the AI domain. * That said, there are added incentives (beyond learning) to make a novel application using the LLM App. What are those incentives? Let's read. đŸ€© [PreviousLive Interactions with Jan Chorowski and Adrian Kosowski | Bonus Resource](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/live-interactions-with-jan-chorowski-and-adrian-kosowski-or-bonus-resource) [NextPrizes and Giveaways](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/prizes-and-giveaways) Last updated 1 year ago --- # Retrieval Augmented Generation and LLM Architecture | 10 Days Realtime LLM Bootcamp Welcome to the fascinating world of LLM Architecture and Retrieval-Augmented Generation, commonly known as RAG. In the current landscape, the value of Large Language Models (LLMs) in the progression of content understanding and generation is widely acknowledged. However, LLMs come with limitations, such as the production of incorrect information, lack of data source verification, and dependence on outdated data. These shortcomings are particularly consequential for businesses prioritizing real-time, precise, and auditable data—commonly identified as key concerns. **Retrieval Augmented Generation (RAG)** offers a transformative solution to these issues. It elevates the capabilities of LLMs, making them relevant, reliable, and up-to-date. In this module, we're laying the groundwork for an in-depth exploration of specialized techniques to improve pre-trained Large Language Models (LLMs) for particular use cases. Let's start by understanding. * What RAG is and * Why it's a crucial component in the LLM ecosystem. [PreviousTasks in the Excercise](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/prompt-engineering/prompt-engineering-excercise/tasks-in-the-excercise) [NextWhat is Retrieval Augmented Generation (RAG)?](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/what-is-retrieval-augmented-generation-rag) Last updated 1 year ago --- # Dropbox Retrieval App in 15 Minutes | 10 Days Realtime LLM Bootcamp Let's dive into the setup and operation of the Dropbox AI Chat tool, an innovative solution that efficiently searches through extensive, unstructured documents stored in your Dropbox using advanced AI capabilities. What you'll be building can be seen below. In this particular showcase we'll be cloning [this particular repository](https://github.com/pathway-labs/dropbox-ai-chat) to enable you to build a similar application quickly. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252Fa8uFbOlqJnKJZYw7O44i%252Fdropbox-ai-search-tool.gif%3Falt%3Dmedia%26token%3Db902e8b6-35c6-4e51-ac2a-f5d36d3a4fb8&width=768&dpr=4&quality=100&sign=8ec33032&sv=2) Please note, that this project's aim isn't about focusing on Dropbox or any specific tool. Instead, we're here to explore the world of open-source RAG-based applications, which are impactful and blend seamlessly with many popular tools you're already familiar with. Take inspiration from a learner of this bootcamp cohort, where she integrated real-time data from Trello's board and Slack to create a helpful productivity tool ([GitHub link](https://github.com/leabuende/mike-llm-slack-plugin) ). This example illustrates the vast possibilities at your fingertips. In this module, we're offering the starting points to spark your creativity and engineering acumen. 😊 [PreviousPrerequisites](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/prerequisites) [NextBuilding the app without Dockerization](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization) Last updated 1 year ago --- # Final Submission | 10 Days Realtime LLM Bootcamp Congratulations on reaching the final stage of this enriching journey! To officially submit your project, please fill out the following Google Form. This is mandatory to qualify for bootcamp graduation, awards, and other incentives. [PreviousTracks for Submission](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/tracks-for-submission) Last updated 1 year ago --- # In-Context Learning | 10 Days Realtime LLM Bootcamp In this video, you'll be introduced to the concept of in-context learning through prompts. Anup explains how this form of learning is scalable, particularly when dealing with vast amounts of data. This becomes especially relevant when we recall our earlier discussions on Retrieval Augmented Generation (RAG). Understanding in-context learning amplifies the efficacy of technologies like RAG in Large Language Models. [PreviousPrimer to RAG Functioning and LLM Architecture: Pre-trained and Fine-tuned LLMs](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/primer-to-rag-functioning-and-llm-architecture-pre-trained-and-fine-tuned-llms) [NextHigh level LLM Architecture Components for In-context Learning](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/high-level-llm-architecture-components-for-in-context-learning) Last updated 1 year ago --- # High level LLM Architecture Components for In-context Learning | 10 Days Realtime LLM Bootcamp In this brief video, Anup provides a high-level breakdown of how in-context learning operates within an LLM. He'll guide you through the journey of a prompt as it interacts with a vector database or index, undergoes similarity search, feeds context, and eventually results in a coherent LLM output. Understanding this architecture is essential for mastering the interaction between prompts and LLMs, a crucial skill for anyone looking to effectively deploy these models in a variety of settings. As you may have noticed, the concepts explained under 'in-context learning' in the forthcoming sections are essentially what RAG accomplishes. In-context learning allows your LLM to adapt and respond based on not just the pre-trained data, but also from the external, real-time information it retrieves. This is precisely what RAG is designed to do. [PreviousIn-Context Learning](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/in-context-learning) [NextDiving Deeper: LLM Architecture Components](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/diving-deeper-llm-architecture-components) Last updated 2 years ago --- # Understanding Docker | 10 Days Realtime LLM Bootcamp This module will help you build the previous file if you're new to Docker and are struggling to install dependencies on your machine. **First off, a quick recap.** Think of Docker as a shipping container for your app. Just as a shipping container can hold all sorts of goods (clothes, electronics, etc.) and be transported anywhere in the world, Docker bundles your app and everything it needs to run into a '**container**.' This makes it easy to share and run your app on any computer. Given the complexities and manual effort involved in resolving dependency issues in your system, Docker can be a beneficial tool to standardize the development environment among all students. **Why Use Docker?** * **Standardized Environment**: Everyone gets the same set of dependencies, reducing "it works on my machine" issues. * **Isolated**: Doesn't interfere with other projects or system-wide settings. * **Ease of Use**: Running the project becomes much simpler once set up. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/understanding-docker#understanding-key-docker-terminologies) Understanding Key Docker Terminologies * **Docker Image**: Consider this a blueprint or a container snapshot, including the application and its dependencies. You build an image once and use it to create multiple containers. * **Docker Container**: A container is a running instance of an image. It's a lightweight, stand-alone, executable software package with everything needed to run the code. * **CMD**: In Docker, the `CMD` instruction specifies the command to execute when the container starts up. * **Docker Compose:** A tool for defining and running multi-container Docker applications. Using a YAML file (`docker-compose.yml`), it allows you to specify how different containers interact with each other, making it easier to manage multiple containers as a single service. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/understanding-docker#resources-to-understand-docker-better) Resources to Understand Docker Better * Basic Tutorial on Dockerfile: [Here](https://youtu.be/0eMU23VyzR8) * Basic Tutorials on Docker Compose: [Part 1 using (Single Container)](https://youtu.be/600SC33F6Ag) , [Part 2 (using 2 Containers)](https://youtu.be/WBqHr2kPc_A) * Blog on using ChatGPT to build an optimized Docker Image: [3-Minute Read](https://collabnix.com/when-chatgpt-meet-docker-for-the-first-time/) Now let's see the step-by-step implementation. [PreviousBuilding the app without Dockerization](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization) [NextUsing Docker to Build the App](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app) Last updated 1 year ago --- # Primer to RAG Functioning and LLM Architecture: Pre-trained and Fine-tuned LLMs | 10 Days Realtime LLM Bootcamp Welcome back to our module on LLM Architecture and RAG! Up next is a series of learning resources created by Anup Surendran that sets the stage for your journey ahead. This video serves as a primer, acquainting you with key concepts such as pre-training, RLHF (Reinforcement Learning from Human Feedback), fine-tuning, and in-context learning. These aren't just buzzwords; they're your toolkit for unlocking the full potential of Large Language Models. Understanding these terms will be crucial as they lay the groundwork for our upcoming module, which delves into 'In-Context Learning.' So, stay tuned! **Resources for Later:** Once you've completed this module on RAG and LLM Architecture, you can explore these resources for a more comprehensive understanding of RLHF. * For a beginner-friendly introduction, check out the [video by HuggingFace](https://youtu.be/2MBJOuVq380) , which offers an accessible explanation of RLHF concepts. * Additionally, for a slightly deeper dive focusing on the mathematical aspects, consider the [lecture on RLHF](https://youtu.be/SXpJ9EmG3s4?t=2125) available by Stanford Online on their YouTube channel. [PreviousWhat is Retrieval Augmented Generation (RAG)?](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/what-is-retrieval-augmented-generation-rag) [NextIn-Context Learning](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/in-context-learning) Last updated 1 year ago --- # How the Project Works | 10 Days Realtime LLM Bootcamp The project accomplishes its tasks through a series of steps shared below. Make sure you give it a quick read before proceeding to the next step where we explore the repository. **1 - Realtime Indexing:** * Sourcing data: A script called [`discounts-data-generator.py`](https://github.com/Boburmirzo/chatgpt-api-python-sales/blob/main/examples/csv/discounts-data-generator.py) mimics real-time data from external sources. It creates or updates a file named `discounts.csv` with random data. Alongside this, a scheduled task (cron job) using [Python Crontab](https://pypi.org/project/python-crontab/) runs every minute to fetch the latest data from Rainforest API. Crontab is a time-based job scheduler. * Giving the option to choose particular data source(s): With the Streamlit UI provided, either select Rainforest API as a data source or upload a CSV through the UI file-uploader. It then maps each row into a JSONline schema for better managing large data sets. This format helps in managing large datasets by representing each row as a separate JSON object * Chunking: The documents are divided into shorter sections for them to be converted into vector embeddings. * Embedding of data source: These shorter sections are processed through the OpenAI API to generate embeddings. * Real-time Vector Indexing: An index is created based on these embeddings to facilitate quick searching later on. **2 - Query (Prompt) Embedding and Retrieval** * Query Embeddinging: For any question asked by the user, an embedding is generated using the OpenAI API for embeddings, i.e. `text-embedding-ada-002`. * Retrieving: The system compares the vector embedding of the query/prompt and the vector embedding of the data source to find the most relevant information. **3 - Prompt Augmentation and Answer Generation** 1. The query/prompt and the most relevant sections of data are packaged into a message within the token limit. 2. Get Answer from GPT: This message is sent to `gpt-3.5-turbo`, which then provides an answer. [PreviousAmazon Discounts App](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app) [NextStep-by-Step Process](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/step-by-step-process) Last updated 1 year ago --- # Tracks for Submission | 10 Days Realtime LLM Bootcamp #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/tracks-for-submission#hello-future-innovators) Hello Future Innovators! 🌟 As your course instructors, we couldn't be more excited to lead you through the multiple impactful pathways you can venture into for your final projects. So, let's get those creative juices flowing and build something transformative! 🚀 **A Quick Note 📝** Below-mentioned tracks/ideas are not constraints but guiding points, helping you look in multiple directions. For each project, you must try to: * Choose an engaging title that sparks curiosity. * Clearly identify your end-user and the impacted industry. * Feature a user story to give life to your idea. * Specify the business impact your project will have. * Show how your project leverages the power of Realtime LLMs using LLM App. * * * **Crisis Helper 🚹** **Description**: Interested in emergency solutions? Leverage LLM App's real-time alerting and monitoring features to create life-saving systems that operate during emergencies. **Example Output**: Think of a real-time weather and news feed that helps with dynamic evacuation plans or a bot that sends timely alerts based on a user's location. * * * **Rule Keeper ⚖** **Description**: In India's bustling fintech scene, compliance is king but often hard to manage for startups and even enterprises. Use the LLM App to create a tool that helps companies stay on top of evolving regulations without sending data to regulatory bodies. It could also be an explainer for key laws coming up in certain domains. **Example Output**: Think of a system that instantly alerts startup teams about new regulations or potential compliance issues, streamlining their operations and ensuring they thrive in this competitive market. * * * **Team Sync đŸ€** **Description**: Eager to improve workplace collaboration? Utilize LLM App’s real-time data syncing to enhance cross-functional information sharing within an organization. The Dropbox app could be put to good use here. **Example Output**: Picture customer support pulling up the latest bug fixes from the development team’s logs, all in real-time! * * * **Stay Safe 🏭** **Description**: Passionate about safety measures? Make the most of LLM App’s alerting and monitoring features to devise a real-time safety alert system. **Example Output**: Visualize an industrial setting where staff receives immediate alerts about safety incidents, enriched with sensor data and contextual information. * * * **Financial Guru 📈** **Description**: Aspiring to revolutionize the finance sector? Harness LLM App's real-time alerts and scalability to craft a live decision-support system. **Example Output**: Think of a system that alerts the management or sales teams when a high-value deal or investment reaches a critical juncture. * * * **Trend Setter 🎯** **Description**: Fascinated by content trends? Utilize LLM App’s unstructured and real-time data handling to align your content with what's hot in the market. **Example Output**: Imagine an app that delivers real-time ad recommendations based on current market trends. * * * **Bonus Tracks! 🌟** * **Content Cop**: Design a parser that determines the originality of content, be it research papers or problem sets. * **Research Digest**: Overwhelmed by the volume of AI/ML research? Develop a tool that sends weekly summaries of groundbreaking work from your favorite researchers. * **Open Innovation**: Your canvas is blank. Show us your creativity using real-time data and Large Language Models through the LLM App. * * * **Additional Resource 📊** We also provide a Google Sheet ([link](https://docs.google.com/spreadsheets/d/19J7wf01b5AtXjY2NackKOq2f3Vr7W-k97kxfcDZDOcg/edit?usp=sharing) ) as an additional resource to fuel your creative thinking. It also has a bucket list of APIs that provide real-time data. Please be aware that these ideas originated from a weekend brainstorming session by one of the course instructors (Mudit) and are not fully developed solutions. As developers, you may encounter technical challenges that haven't been specifically outlined; your task is to anticipate and navigate them. These ideas are designed to ignite your creativity and potentially guide your thought process, rather than confine your possibilities. So, as you embark on this journey, don't limit yourself—feel free to venture beyond! [PreviousPrizes and Giveaways](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/prizes-and-giveaways) [NextFinal Submission](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/final-submission) Last updated 1 year ago --- # Prizes and Giveaways | 10 Days Realtime LLM Bootcamp Our aim with these awards is to celebrate your accomplishments and inspire you to contribute to the open-source community, complete the bootcamp, and unlock the full potential of Real-Time Large Language Models using the LLM App. To that end, we're thrilled to announce various prizes and incentives. We hope these serve as a valuable addition to your professional profile—like having your standout project featured on Pathway's official blog—or offer personal enrichment through coveted gadgets, ranging from gaming controllers to specialized camera lenses for photography aficionados. 🏆🎼📾 ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/prizes-and-giveaways#incentives-for-bootcamp-graduates) Incentives for Bootcamp Graduates * Completion Reward: Every graduate with a valid submission will receive a cool Pathway T-shirt and a certificate. * Featured Projects: The most impactful projects will be showcased on Pathway's official blog, and read by some of the finest researchers and data professionals. We hope this will be a valuable addition to participants' professional profiles. There's no minimum or maximum number for this, depending on the quality of submissions we receive. **Criteria for Bootcamp Graduation Certificate** * [Fill out the Google Form](https://forms.gle/wr3asGb9RRgnmBxz6) for the final submission. * Learners must submit their real-time RAG-based LLM application using the LLM App via a GitHub repository. * Projects may be inspired by existing examples (e.g. [Dropbox AI Chat](https://github.com/pathway-labs/dropbox-ai-chat) , [Amazon Discounts](https://github.com/Boburmirzo/chatgpt-api-python-sales) , [EU AI Act App](https://www.linkedin.com/posts/richard-pelgrim_concerned-about-how-the-new-eu-ai-act-will-activity-7119992980530208768-kEc4) ) but should primarily focus on leveraging real-time data with the LLM App to unlock the [key benefits of the LLM App](https://github.com/pathwaycom/llm-app#why-llm-app) . ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/prizes-and-giveaways#additional-prizes-for-the-top-10-projects) Additional Prizes for the Top 10 Projects * Top 3 Projects/Individuals: Wireless Microsoft XBOX controller * Next 3: JBL waterproof portable speaker * Next 4: Amazon Basics Phone Camera Lens **Additional Criteria for Blog Feature & Top 10 Prizes** These criteria are only for the blog feature and the top 10 prizes and not for bootcamp graduation: * The project must be novel in nature and not an inspired/cloned version of the Dropbox Retrieval App. * Utilize the LLM App and its real-time data features for eligibility in top rankings. * Ensure the project is open-source with a GitHub repository that includes a project description, a video demo, a clear methodology, and a functional prototype. * Utilize the LLM App to create impactful solutions in areas like customer support automation, data insights, real-time monitoring, and healthcare, or to enable cross-functional collaboration using real-time data; their project can also serve open-ended consumer, developer, or B2B needs across various domains. We're sharing a few tracks that can be a good starting point for you in the next section. * Share the project demo on LinkedIn or Twitter, explaining its key purpose and how you use the LLM App to build a solution that solves business or social challenges. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/prizes-and-giveaways#judging-criteria) Judging Criteria * Code Quality - Code and GitHub Repository are easy to understand and reproduce. * Technical Implementation - RAG/LLM App and its key benefits are leveraged and used appropriately by the developer. * Creativity & originality - The solution is original and differentiates itself from existing solutions and the other submissions. * Social or Business Value - The solution creates additional value beyond competition for businesses, society, developers, or end consumers. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/prizes-and-giveaways#terms-and-conditions-for-all-the-learners) Terms and Conditions for all the Learners * **Ownership & Rights**: While participants maintain ownership of their projects, they grant Pathway, WnCC IIT Bombay, and the AI Community IIT Bombay permission to feature their work on respective blogs if selected for an official blog feature. * **Organizer's Discretion**: The organizers hold the authority to determine winners and reserve the right to disqualify any participant who violates the Terms and Conditions or engages in misconduct. * **Plagiarism**: Plagiarism will result in immediate disqualification. Participants must submit original work. * **Attribution**: If participants utilize open-source resources or someone else's work as a component of their project, proper credit must be given. * **Technical Responsibility**: Participants are responsible for their hardware, software, and internet connectivity. * **T-shirt & Prize Logistics**: For this cohort, WnCC IIT Bombay and the AI Community IIT Bombay are overseeing T-shirt distribution. T-shirts can only be picked up at the IIT Bombay campus within a given timeframe, and on a specific date that will be communicated beforehand. All other prizes, including digital certificates, will be shipped to participants unless located in areas restricted by Indian or French law. [PreviousFinal Project + Giveaways](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways) [NextTracks for Submission](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/final-project-+-giveaways/tracks-for-submission) Last updated 1 year ago --- # Diving Deeper: LLM Architecture Components | 10 Days Realtime LLM Bootcamp In the forthcoming video, we provide a detailed explanation of the essential components that constitute a Large Language Model's architecture. This video aims to extend your comprehension of the LLM architecture, contributing to your foundational understanding of the field. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/diving-deeper-llm-architecture-components#in-this-video-weve-learned-about) In this video, we've learned about * The User Interface Component designed to pose questions * The Storage Layer, which utilises Vector DB or Vector Indexes * The Service, Chain, or Pipeline Layer, which is instrumental in the model's functioning (with a brief mention of the Chain Library used for chaining prompts) * Summary of our learnings around LLM Architecture Components Let's look at a cleaner architecture diagram, and various steps of the pipeline and summarize the advantages of RAG based on what we've understood so far. [PreviousHigh level LLM Architecture Components for In-context Learning](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/high-level-llm-architecture-components-for-in-context-learning) [NextLLM Architecture Diagram and Various Steps](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/llm-architecture-diagram-and-various-steps) Last updated 2 years ago --- # Amazon Discounts App | 10 Days Realtime LLM Bootcamp In this tutorial, we will build an app called "Amazon Discounts App" that can find real-time sales and deals for Amazon products by leveraging the LLM App library. In this case, instead of cloning this Discounts app repository, we'll try to observe and learn how it uses the [LLM App with the Prices API](https://github.com/Boburmirzo/chatgpt-api-python-sales) to give the desired results. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FTJ0Di6p54ewndKUAICMK%252Fdiscounts-tracker-streamlit-1.png%3Falt%3Dmedia%26token%3D6dbdc6d5-3256-46fb-89d0-38bb257aa907&width=768&dpr=4&quality=100&sign=f03c6c0f&sv=2) In this app, we have two data sources: * [Rainforest API for Amazon Prices](https://www.rainforestapi.com/) * Discounts CSV with product and prices data These two sources are used to show how you can upload a CSV along with another data source. While you can use a similar approach with other APIs, even this showcase opens up considerable possibilities in E-Commerce applications. [PreviousUsing Docker to Build the App](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app) [NextHow the Project Works](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/how-the-project-works) Last updated 1 year ago --- # LLM Architecture Diagram and Various Steps | 10 Days Realtime LLM Bootcamp Now that we've explored the various components that make up the architecture of Large Language Models (LLMs), let's dive into how Retrieval-Augmented Generation (RAG) can work synergistically with these components of an LLM architecture. The aim is to show you how RAG can supercharge an LLM's capabilities by seamlessly integrating real-time or static data sources into the information retrieval and generation processes. [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/llm-architecture-diagram-and-various-steps#llm-architecture) LLM Architecture ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ![This diagram shows LLM Architecture and it's various sources. Data Sources: Whether your starting point is cloud storage, Git repositories, or databases like PostgreSQL, the first task is to bring these varied data forms together through pre-configured connectors. Dynamic Vector Indexing: Text from these data sources is broken down into smaller segments and converted into vector representations. Models specialized for text embeddings, such as OpenAI's text-embedding-ada-002, are employed here. These vectors are continuously indexed to facilitate rapid search later on. Query Transformation: A user’s input query is likewise transformed into a compatible vector representation, ensuring that it can be effectively matched with the indexed vectors for data retrieval. Contextual Retrieval: Algorithms like Locality-Sensitive Hashing (LSH) are applied to find the closest matches between the user query and the indexed data vectors, staying within the model's token limitations. Text Generation: With the retrieved context, foundational LLMs like GPT-3.5 Turbo or Llama-2 employ techniques from the Transformer architecture, such as self-attention, to generate an appropriate response. User Experience: Finally, the generated text is presented to the user via interfaces like Streamlit or ChatGPT.](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FAuwpeLdJgPpJbXcZw4YH%252FLLM%2520Architecture%2520Diagram.png%3Falt%3Dmedia%26token%3Da8c3374b-2588-4492-9b4d-c8638e3b3d06&width=768&dpr=4&quality=100&sign=2b5ec7c7&sv=2) LLM Architecture Diagram to show how RAG works with Real-time or Static Data Sources For a nuanced understanding of how Retrieval-Augmented Generation (RAG) optimizes Large Language Models, we'll delve into the essential elements and procedural steps that comprise the LLM architecture. 1. **Data Sources**: Whether your starting point is cloud storage, Git repositories, or databases like PostgreSQL, the first task is to bring these varied data forms together through pre-configured connectors. 2. **Dynamic Vector Indexing**: Text from these data sources is broken down into smaller segments (also called "chunks") and converted into vector representations. Models specialized for text embeddings, such as OpenAI's text-embedding-ada-002, are employed here. These vectors are continuously indexed to facilitate rapid search later on. 3. **Query Transformation**: A user’s input query is likewise transformed into a compatible vector representation, ensuring that it can be effectively matched with the indexed vectors for data retrieval. 4. **Contextual Retrieval**: Algorithms like Locality-Sensitive Hashing (LSH) are applied to find the closest matches between the user query and the indexed data vectors, staying within the model's token limitations. 5. **Text Generation**: With the retrieved context, foundational LLMs like GPT-3.5 Turbo or Llama-2 employ techniques from the Transformer architecture, such as self-attention, to generate an appropriate response. 6. **User Interface**: Finally, the generated text is presented to the user via interfaces like Streamlit or ChatGPT. [PreviousDiving Deeper: LLM Architecture Components](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/diving-deeper-llm-architecture-components) [NextRAG versus Fine-Tuning and Prompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering) Last updated 2 years ago --- # What is Retrieval Augmented Generation (RAG)? | 10 Days Realtime LLM Bootcamp Large Language Models (LLMs) like GPT-4 or Mistral-7b are extraordinary in many ways, yet they come with a set of challenges. For now, let's focus on one specific limitation: the timeliness of their data. Since these models are trained up to a particular cut-off date, they aren't well-suited for real-time or organization-specific information. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252F5togckm6kBQIZFKKZiiN%252Fchatgpt%2520outdated.png%3Falt%3Dmedia%26token%3Db1a4cb47-01f2-46b4-8c7a-0ecdbe6aa43d&width=768&dpr=4&quality=100&sign=3c64d414&sv=2) Imagine you're a developer architecting an LLM-enabled app for Amazon. You aim to support shoppers as they comb through Amazon for the latest deals on jackets. Naturally, you want to furnish them with the most current offers available. After all, nobody wants to rely on outdated information, and the same holds true for data queried from your LLM. This is where Retrieval-Augmented Generation, commonly known as RAG, significantly improves the capabilities of LLMs. In a way that might resemble a resourceful friend in an exam setting or during a speech who—figuratively speaking, of course—swiftly passes you **the most relevant** "cue card" out of a ton of information to help you understand what you should be writing or saying next. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252Fq25FqnT4XBIuNf2g88aa%252Fmaxresdefault-min.jpeg%3Falt%3Dmedia%26token%3D758b517f-89d0-4191-b950-63527cbfee20&width=768&dpr=4&quality=100&sign=974bc47d&sv=2) Perhaps that wasn't the perfect example, but you get the point. 😄 With RAG, efficient retrieval of the most relevant data for your use case ensures the text generated is both **current** and **substantiated**. RAG, as its name indicates, operates through a three-fold process: * **Retrieval:** It sources pertinent information. * **Augmentation:** This information is then added to the model's initial input. * **Generation:** Finally, the LLM utilizes this augmented input to create an informed output. Simply put, RAG empowers LLMs to include real-time, reliable data from external databases in their generated text. For a better explanation, check out this video by Marina Danilevsky, Senior Research Staff Member at IBM Research. She shares two important challenges with LLMs resolved with the help of Retrieval Augmented Generation. _(Credits: IBM Technology)_ Knowing about RAG is essential, particularly if considering implementing an Enterprise LLM Architecture for your organization or a personal project. The efficacy of this architecture depends mainly on the strength and utility of RAG. By now, you should have a basic understanding of RAG and its importance. As we progress through this module, we aim for you to gain a comprehensive grasp of how in-context learning and RAG collectively contribute to making LLMs more effective, current, and enterprise-ready. [PreviousRetrieval Augmented Generation and LLM Architecture](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture) [NextPrimer to RAG Functioning and LLM Architecture: Pre-trained and Fine-tuned LLMs](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/primer-to-rag-functioning-and-llm-architecture-pre-trained-and-fine-tuned-llms) Last updated 1 year ago --- # Building the app without Dockerization | 10 Days Realtime LLM Bootcamp Here, you can explore a video tutorial by Richard Pelgrim, a Developer Advocate in the stream data processing space, demonstrating how they harnessed the Dropbox document sync application to create a RAG app. **Link to the Project** * The repository being referred to can be found here **-** [**https://github.com/pathway-labs/dropbox-ai-chat**](https://github.com/pathway-labs/dropbox-ai-chat) **.** Make sure to star it. ⭐ * If you struggle to build the application with the help of README on the GitHub repo above, the video and the description below should help you. Navigating the maze of new regulations, like the EU AI Act, can be a complex challenge for founders and data practitioners. This app which leverages the Dropbox example, aims to make understanding these regulations more straightforward. Imagine a tool that helps you dissect and comprehend these intricate policies, easing compliance and being informed. As you explore this application, think of the diverse scenarios you can open just with the Dropbox AI Chat example that we're seeing here. Let's analyze a few elements from this video. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#step-1-cloning-the-repository) Step 1: Cloning the Repository Copy git clone https://github.com/pathway-labs/dropbox-ai-chat  cd dropbox-ai-chat ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#step-2-setting-up-environment-variables) Step 2: Setting up Environment Variables Create a `.env` file in the root directory and populate it with your configurations. Make sure to replace `{OPENAI_API_KEY}` it with your actual OpenAI API key. Copy OPENAI_API_TOKEN={OPENAI_API_KEY} HOST=0.0.0.0 PORT=8080 EMBEDDER_LOCATOR=text-embedding-ada-002 EMBEDDING_DIMENSION=1536 MODEL_LOCATOR=gpt-3.5-turbo MAX_TOKENS=200 TEMPERATURE=0.0 DROPBOX_LOCAL_FOLDER_PATH="../../../mnt/c/Users/bumur/Dropbox/documents" Make sure to replace **DROPBOX\_LOCAL\_FOLDER\_PATH** with your local Dropbox folder path; optionally, you can customize other values. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#step-3-optional-creating-a-virtual-environment) Step 3 - Optional: Creating a Virtual Environment In this case, Richard has used Conda so it's not necessary. To create an isolated environment, execute: Copy # Setting Up a Virtual Environment # On macOS and Linux: # Create and activate a virtual environment python -m venv pw-env && source pw-env/bin/activate # On Windows: # Step 1: Create a new virtual environment in a folder named 'pw-env' python -m venv pw-env # Step 2: Activate the virtual environment pw-env\Scripts\activate ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#step-4-installing-the-dependencies) Step 4 - Installing the Dependencies Copy pip install --upgrade -r requirements.txt ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#step-5-running-the-app) Step 5 - Running the App Navigate to the root directory and execute `main.py`. Copy python main.py ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#step-6-launching-ui-with-streamlit) Step 6 - Launching UI with Streamlit Run the Streamlit app using the following command: Copy streamlit run ui.py Access the UI at `http://localhost:8501/` on your browser. With this, your app should be up and running. 😄 ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/building-the-app-without-dockerization#connecting-the-dots) Connecting the Dots If you look closely at the repo and visit [`api.py`](https://github.com/pathway-labs/dropbox-ai-chat/blob/main/api.py) , you'll be able to connect the dots from what we've learned. Here: * The prompt is processed as embeddings and used as embedded\_query. * The data we're getting from our data source, (i.e., Dropbox) is converted into smaller chunks with the help of Pathway (pw) and then converted to embeddings and stored in index. * Using these, we're creating the augmented prompt with the help of retrieved information and feeding that into GPT-3.5 turbo. Copy # Real-time data coming from external unstructured data sources like a PDF file input_data = pw.io.fs.read( dropbox_folder_path, mode="streaming", format="binary", autocommit_duration_ms=50, ) # Chunk input data into smaller documents documents = input_data.select(texts=extract_texts(pw.this.data)) documents = documents.select(chunks=chunk_texts(pw.this.texts)) documents = documents.flatten(pw.this.chunks).rename_columns(chunk=pw.this.chunks) # Compute embeddings for each document using the OpenAI Embeddings API embedded_data = embeddings(context=documents, data_to_embed=pw.this.chunk) # Construct an index on the generated embeddings in real-time index = index_embeddings(embedded_data) # Generate embeddings for the query from the OpenAI Embeddings API embedded_query = embeddings(context=query, data_to_embed=pw.this.query) # Build prompt using indexed data responses = prompt(index, embedded_query, pw.this.query) # Feed the prompt to ChatGPT and obtain the generated answer. response_writer(responses) # Run the pipeline pw.run() Following these steps, you can get the Dropbox AI Chat tool up and running. However, suppose you're **facing issues** downloading the dependencies or running the application **on your machine**. In that case, it might be worthwhile to check the next module, which provides a comprehensive guide for implementing this through Docker. [PreviousDropbox Retrieval App in 15 Minutes](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes) [NextUnderstanding Docker](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/understanding-docker) Last updated 1 year ago --- # RAG versus Fine-Tuning and Prompt Engineering | 10 Days Realtime LLM Bootcamp In the rapidly evolving landscape of Large Language Models (LLMs), achieving cost-efficiency and operational simplicity is critical, and this is where Retrieval-Augmented Generation (RAG) shines. Compared to methods like Fine-tuning and Prompt engineering, RAG stands out due to its advantages in cost-effectiveness, simplicity, and adaptability. Let's individually explore these options to understand where RAG excels. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering#id-1.-fine-tuning-vs-rag) 1\. Fine-Tuning vs RAG For those less familiar with the concept, fine-tuning involves modifying a pre-trained language model (e.g. GPT-3.5 Turbo, Mistral-7b, or Llama-2) with a smaller, targeted dataset to work optimally for specific use cases. Enterprises often prefer to use fine-tuned models along with RAG. While fine-tuning avoids the need to build a model from scratch, it does have its drawbacks, which RAG effectively addresses. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering#data-preparation-challenges) Data Preparation Challenges * Having control over training data permits steps to address biases, yet implementing such measures is far from straightforward. Interventions like altering variable importance or ensuring balanced data distribution demand in-depth data analysis skills. * Furthermore, expertise in the subject matter is essential for accurately annotating data that serves specialized or research-specific functions. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering#cost-efficiency) Cost Efficiency * Retraining and deployment are not only time-consuming but also financially taxing. * For instance, as per the current pricing by OpenAI, using vector embeddings API in RAG models is roughly **80 times** cheaper than commonly utilized fine-tuning APIs. * Consider the need to repeat this process each time your company launches a new product, all to ensure that your teams are not provided with outdated information from your Gen AI model. #### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering#data-freshness) Data Freshness * At the outset, it's only logical to expect that when developing an LLM application, you'd want your large language model to deliver current and pertinent out consistently. * Regarding fine-tuning, the model's accuracy can significantly decline if the data undergoes changes or isn't regularly updated. Consequently, despite the associated challenges, this task must be performed at frequent intervals to maintain the model's efficacy. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering#id-2.-prompt-engineering-vs-rag) 2\. Prompt Engineering vs RAG Prompt engineering might seem like a lighter alternative but comes with its own challenges, such as data privacy, inefficient retrieval of information, and the technical constraint of a token limit. * **Data Privacy**: For organizations handling sensitive information, manually copy-pasting large chunks of data to retrieve a specific piece poses a risk of unintended data exposure. * **Inefficient Retrieval**: Knowing where to find the relevant data becomes crucial when dealing with vast data corpora. Manual prompt engineering lacks the efficiency of automated mechanisms, such as vector indexing in RAG, which enables quick and semantically accurate data retrieval. * **Token Limit Constraints**: Language models have built-in token limitations, restricting the amount of text they can process in a single prompt. This makes it challenging to include all the necessary information in one interaction. In contrast, RAG's approach of storing data in efficient vector indexes circumvents these limitations by facilitating quick and semantically relevant information retrieval, making it a more viable option for dealing with large and complex data sets. [PreviousLLM Architecture Diagram and Various Steps](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/llm-architecture-diagram-and-various-steps) [NextVersatility and Efficiency in Retrieval-Augmented Generation (RAG)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/versatility-and-efficiency-in-retrieval-augmented-generation-rag) Last updated 1 year ago --- # Step-by-Step Process | 10 Days Realtime LLM Bootcamp In this example, we'll dive a bit deeper to show you how this application was made so you can make one of your own. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/step-by-step-process#link-to-the-project) Link to the Project * The repository being referred to can be found [here on GitHub](https://github.com/Boburmirzo/chatgpt-api-python-sales) . * If you have a good experience with open source, visiting the above link should enable you to build a similar project seamlessly. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/step-by-step-process#step-by-step-process-to-build-the-application) Step-by-Step Process to Build the Application A good way to understand the code here would be to read the Streamlit (a Snowflake product) blog below which features the open-source application developed by Bobur. It's a friendly and easy-to-understand blog showing how the application interacts with users via an HTTP REST API. It works in real-time, offering support for various data types like JSON Lines and Rainforest Product API. [https://blog.streamlit.io/build-a-real-time-llm-app-without-vector-databases-using-pathway/blog.streamlit.io](https://blog.streamlit.io/build-a-real-time-llm-app-without-vector-databases-using-pathway/) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/step-by-step-process#video-tutorial) Video Tutorial Once you've read the blog above, check out this video tutorial by Bobur Umurzakov (Developer Advocate at Pathway). He gives a quick walkthrough of the code and the open-source repository. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/step-by-step-process#key-things-to-note) Key things to note * This app is modular; you can add new data sources or interfaces. * You could scale it up to include more advanced features like additional data formats or APIs. * Streamlit and Pathway's LLM App communicate over HTTP REST API, but they can also be integrated in other ways, such as file sharing or inter-process communication. By following this guide, you'll create a versatile application capable of real-time interactions with users, providing them with valuable insights into Amazon discounts. [PreviousHow the Project Works](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/how-the-project-works) [NextHow to Run the Examples](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/how-to-run-the-examples) Last updated 1 year ago --- # Track your Progress | 10 Days Realtime LLM Bootcamp Here is another and final 10-question graded quiz. After this the last task would be to build the hands-on application. This quiz is designed to test your understanding of Retrieval Augmented Generation and does not feature questions from the Bonus section. However, it would be best to quickly revisit the modules if you wish to do so. Ready? 📝 **Here's the link:** [**https://www.flexiquiz.com/SC/N/RAG-quiz**](https://www.flexiquiz.com/SC/N/RAG-quiz) _(Please take the MCQ quiz using the same email ID you used to register for the bootcamp)_ [PreviousUsing kNN and LSH to Enhance Similarity Search in Vector Embeddings (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module) [NextHands-on Development](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development) Last updated 1 year ago --- # How to Run the Examples | 10 Days Realtime LLM Bootcamp Congratulations on coming this far! 🎉 Let's say you want to go beyond the Amazon Discounts App and Dropbox Retrieval App. This module is to make it easy for you to build and run your applications using `examples` on the [LLM App](https://github.com/pathwaycom/llm-app) . ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/how-to-run-the-examples#what-are-the-examples-offered) What are the Examples offered? The repository offers multiple possible use cases under its [`examples`](https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines) folder to illustrate various areas of application. Once you've **cloned the LLM App** repository and set up the **environment variables** (per the steps mentioned on [this link](https://github.com/pathwaycom/llm-app#step-1-clone-the-repository) ), you're all set to run the examples. Below is a table that shares the types of examples you can explore. Example Type What It Does What's Special Good For contextless Answers your questions without looking at any additional data. Simplest example to try. Not RAG based. Beginners to get started. contextful Uses extra documents in a folder to help answer questions. Better answers by using more data. More advanced, detailed answers. contextful\_s3 Like "Contextful," but stores documents in S3 (a cloud storage service). Good for handling a lot of data. Businesses or advanced projects. unstructured Reads different types of files like PDFs, Word docs, etc. Can handle many file formats and unstructured data. Working with various file types. local Runs everything on your own machine without sending data out. Keeps your data private. Those concerned about data privacy. unstructuredtosql Takes data from different files and puts it in a SQL database. Then it uses SQL to answer questions. Great for complex queries. Advanced data manipulation and queries. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/how-to-run-the-examples#simple-way-to-run-the-examples-on-llm-app) Simple Way to Run the Examples on LLM App Considering you've done the steps before, here's a recommended, step-by-step process to run the examples easily: **1 -** Open a terminal and navigate to the LLM App repository folder: Copy cd llm-app **2 -** Choose Your Example. The examples are located in the [`examples`](https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines) folder. Say you want to run the 'alert' example. You have two options here: * **Option 1**: Run the centralized example runner. This allows you to switch between different examples quickly: Copy python run_examples.py alert * * * * **Option 2**: Navigate to the specific pipeline folder and run the example directly. This option is more focused and best if you know exactly which example you're interested in: Copy python examples/pipelines/contextful/app.py That's it! 😄 By following these steps, you're not just running code but actively engaging with the LLM App’s rich feature set, including anything from real-time data syncing to model monitoring. It's a step closer to implementing your LLM application that can have a meaningful impact. [PreviousStep-by-Step Process](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app/step-by-step-process) [NextLive Interactions with Jan Chorowski and Adrian Kosowski | Bonus Resource](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/live-interactions-with-jan-chorowski-and-adrian-kosowski-or-bonus-resource) Last updated 1 year ago --- # Key Benefits of RAG for Enterprise-Grade LLM Applications | 10 Days Realtime LLM Bootcamp Retrieval-augmented generation (RAG) elevates Large Language Models (LLMs) by enhancing their intelligence, efficiency, and relevance. Below, we outline some of the core benefits that will be especially important for you while considering for building an LLM application in a production or enterprise environment. 1. **Real-Time, Human-Like Learning for Trusted and Relevant Information:** By leveraging real-time data feeds, the model can deliver information that is not only current and reliable but also relevant across functions. This capacity for real-time learning mimics how humans naturally acquire and process information, ensuring that the model’s output remains up-to-date and contextually accurate. 2. **Robust Data Governance and Security**: * **Minimized Hallucination**: Real-time data retrieval techniques enhance the model's accuracy, reducing the likelihood of producing misleading or 'hallucinated' content. Plus, this data is sourced from trusted data sources (including unstructured data sources, and not necessarily labeled data sets.) * **PII Management and Hierarchical Access:** Advanced governance protocols ensure the ethical handling of Personally Identifiable Information (PII). Additionally, role-based access controls are in place to limit the availability of sensitive information. For example, if as an employee I inquire about my manager's salary increase, I shouldn't be able to see it. 3. **Clarity on Data Sources**: While generating the responses, the LLMs can site the data source from your data corpus where the information is being retrieved from. The capacity to trace the origins of the data bolsters the LLM's credibility and instills user trust. 4. **Compliance-Ready**: * **Security Measures for AI-Specific Risks**: Standard IT security measures can be adapted to address specific generative AI risks, including features like automated compliance audits or alerts for sensitive data access. * **Regulatory Adaptability**: Given the ever-changing regulations surrounding generative AI, including those like the EU's AI Act, your LLM can be configured to adapt to future compliance requirements. 5. **Streamlined Customization**: Employing RAG means you can say goodbye to the complexities of fine-tuning, extra databases (we'll cover that), or added computational needs, making the customization process both efficient and budget-friendly. This architecture is not just future-proof but also aligns perfectly with real-world needs, striking the right balance between efficiency and reliability. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/key-benefits-of-rag-for-enterprise-grade-llm-applications#lets-understand-this-with-some-real-world-use-case) Let's understand this with some real-world use case * **Customer Support:** For real-time, context-sensitive customer assistance. * **Content Curation:** For summarizing articles, recommending related content, and generating new pieces. * **Healthcare Analytics:** For medical research and drug discovery. * **Supply Chain Management**: For real-time data analysis and decision-making. Let's keep the momentum going as we delve further into the hands-on implementation in the next module! đŸ„ł [PreviousVersatility and Efficiency in Retrieval-Augmented Generation (RAG)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/versatility-and-efficiency-in-retrieval-augmented-generation-rag) [NextSimilarity Search in Vectors (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module) Last updated 1 year ago --- # Versatility and Efficiency in Retrieval-Augmented Generation (RAG) | 10 Days Realtime LLM Bootcamp To further equip you with a comprehensive understanding of RAG and its adaptability, let's address some **frequently asked questions (FAQs)** and challenges developers often encounter. A couple of them were asked to us during the course of this bootcamp. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/versatility-and-efficiency-in-retrieval-augmented-generation-rag#id-1.-which-data-types-can-rag-handle) 1\. Which data types can RAG handle? One of the most compelling features of RAG is its flexibility to work with an array of data types. Whether relational databases, APIs, transcribed audio, or even live feeds from the internet, RAG can seamlessly integrate these into its retrieval mechanism. This adaptability enhances the model's ability to generate contextually accurate and informative responses. * **Data Types Supported:** Relational databases, free-form text, PDFs, APIs, transcribed audio, and streaming platforms like Kafka, Debezium, Redpanda, etc. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/versatility-and-efficiency-in-retrieval-augmented-generation-rag#id-2.-are-vector-databases-necessary-for-rag) 2\. Are Vector Databases necessary for RAG? Short answer – no. Andrej Karpathy humorously addressed this question in a [Tweet](https://twitter.com/karpathy/status/1647374645316968449) , based on his personal project. With [the framework](https://news.ycombinator.com/item?id=36894142) we've outlined ahead, it's possible to efficiently manage vector indexes for production use cases, within program memory. This approach provides persistent storage but also offers essential functionalities for scaling and maintaining such systems in a production environment. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252Fzu2xkVoXuj0F1fRtyNtw%252Fandrej%2520karpathy%2520tweet%2520on%2520vector%2520db.png%3Falt%3Dmedia%26token%3D4e6a005b-d1f4-4b3a-9216-49c4c48d5b04&width=768&dpr=4&quality=100&sign=7bf25398&sv=2) [Tweet Link](https://twitter.com/karpathy/status/1647374645316968449?) If you don't know, a Vector Database is a specialized database that handles vector embeddings. These databases are primarily used to efficiently store, search, and retrieve vectors for their use cases in LLMs and Recommender Systems. Just like the emergence of foundational Large Language Models (LLMs) or what some call the "AI Wave," Retrieval Augmented Generation (RAG) is also a relatively new concept. It gained prominence after the 2020 publication of "[Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401) " by a team at Facebook AI Research. Initially, using vector embeddings led developers to naturally consider the application of dedicated vector databases, frequently seen in Recommender Systems. However, this understanding is quickly evolving. * Introducing any new database into an enterprise environment comes with its own complexities and challenges, making a simpler solution preferable in many instances. * In the hands-on implementation module ahead, we implement a production-grade application without using any Vector DB. While the frameworks we use are compatible with notable vector DBs, our example shows that vector DBs are not mandatory components, especially when using them is a challenge. * Tools like [LLM App](https://github.com/pathwaycom/llm-app) can generate and manage their real-time vector indexes, negating the need for a separate vector database. Additionally, conventional databases like PostgreSQL are expanding their features to include built-in support for vector indexing, thanks to extensions like [PG Vector.](https://github.com/pgvector/pgvector) So, while the allure of vector databases exists, it's critical to understand that they are not the only path to efficient RAG implementation. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/versatility-and-efficiency-in-retrieval-augmented-generation-rag#id-3.-is-a-separate-real-time-processing-framework-needed-for-a-real-time-stream-of-data) 3\. Is a Separate Real-Time Processing Framework Needed for a Real-time Stream of Data? This depends on your choice of RAG framework. Referring to the framework we've used ahead in this curriculum, it natively uses Pathway, an ultra-performant data processing engine ([2023 benchmarks](https://pathway.com/blog/streaming-benchmarks-pathway-fastest-engine-on-the-market) ) suitable for batch and streaming use cases. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/versatility-and-efficiency-in-retrieval-augmented-generation-rag#id-4.-is-the-chatgpt-plugin-for-bing-search-an-example-of-rag) 4\. Is the ChatGPT Plugin for Bing Search an example of RAG? Some may wonder whether certain web search plugins in LLM applications like ChatGPT utilize a Retrieval-Augmented Generation (RAG) approach. For example, a ChatGPT interface with a web search plugin can pull current information online, providing a more accurate and up-to-date response. ![](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2F2242835721-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F6FUyhX0UiywvK1rz0m1N%252Fuploads%252FpG9y3qqEp2c4ucDhW6SI%252FChatGPT%2520Plugin.gif%3Falt%3Dmedia%26token%3D05d1d2b9-bd1c-4349-9b04-51b262c1b0a5&width=768&dpr=4&quality=100&sign=16bc0f8e&sv=2) * **Example:** This plugin lets the model answer questions about things that happened after it was last trained, by retrieving information from a blog on the internet. In a way, these can be considered LLM applications that employ a retrieval-augmented generation strategy. However, they do not retrieve data from a diverse set of data sources, such as a collection of PDFs, links, or Kafka topics. This affects the quality of their responses. In such scenarios, LLMs like ChatGPT can only offer the best answer they find from the specific webpage that the plugin accessed. On the other hand, when you incorporate RAG into your custom LLM application, you benefit from efficient vector embeddings and vector search capabilities. These allow you to extract much more relevant information from a comprehensive data corpus, aiding in identifying the most pertinent answers. By understanding these facets, you're better equipped to leverage the strengths of RAG in your LLM architecture, whether for an enterprise solution or a personal project. [PreviousRAG versus Fine-Tuning and Prompt Engineering](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/rag-versus-fine-tuning-and-prompt-engineering) [NextKey Benefits of RAG for Enterprise-Grade LLM Applications](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/key-benefits-of-rag-for-enterprise-grade-llm-applications) Last updated 1 year ago --- # Using kNN and LSH to Enhance Similarity Search in Vector Embeddings (Bonus Module) | 10 Days Realtime LLM Bootcamp Diving into the intricacies of kNN (k-Nearest-Neighbors) and LSH (Locality Sensitive Hashing), we find ourselves at the intersection of mathematics, data science, and algorithmic strategy. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module#knn-k-nearest-neighbors-in-the-context-of-vector-embeddings) kNN (k-Nearest-Neighbors) in the Context of Vector Embeddings If you've been exploring this domain, you might have come across mentions of kNN (k-Nearest-Neighbors). So, what exactly is kNN? Why do we need it, especially when we already have tools like cosine similarity at our disposal? Let's understand. * **Role of kNN with Vector Embeddings**: Once we have vector representations of data and a measure of similarity like cosine similarity, the next logical step is to leverage these vectors for practical applications. kNN is a powerful algorithm that does just that. Given a query vector, it identifies the 'k' vectors in the dataset that are closest (most similar) to the query. * **Why kNN?**: kNN hinges on the premise that similar data points in a vector space are likely to share attributes or classifications. For instance, in a text classification problem, if a majority of your 'k' nearest vectors correspond to articles about 'F1 Racing', it's likely that the query article is also related to F1 Racing. * **The kNN approach has gained success due to** its straightforward nature. The basic version is not only easy to implement but also offers high accuracy. Unlike many alternative methods, kNN isn't a black box and offers **explainability**, meaning its decision-making process is clear and understandable. This transparency enhances user trust in the system, and thus easier adoption within enterprises. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module#challenges-with-knn) Challenges with kNN * **Computational Intensity**: The brute-force approach of kNN, where every query vector is compared with all vectors in the dataset, is computationally expensive. * For example, using Pathway, often developers with large datasets with high-dimensional data. On such datasets, this naive approach doesn't work. As datasets grow, the time taken for these pairwise comparisons becomes a bottleneck. The standard kNN algorithm compares a query vector to every vector in the dataset, which is demanding in terms of computation, especially as the size of the dataset grows. This method's time complexity is O(dntnq)O(d n\_t n\_q) O(dnt​nq​) where ddd represents the dimensions, and ntn\_tnt​ and nqn\_qnq​ are the numbers of training and query points, respectively. This can become very inefficient with large datasets. * Moreover, managing frequent updates in scenarios with regular data changes is both costly and complex. With new data points coming in, it brings in the requirement for recalculating distances for all queries, which consumes significant resources. In real-time or frequently updated data scenarios, this presents notable challenges. Moreover, modifying or deleting data points necessitates reevaluating all query responses, adding to the complexity. To overcome this, Pathway uses a better approach. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module#leveraging-lsh-to-optimize-knn) Leveraging LSH to Optimize kNN * **What is LSH?**: LSH (Local Sensitive Hashing) is a hashing technique wherein similar data points are probabilistically mapped to the same bucket. Understandably, unlike hash functions used for security, which scatter similar items widely to prevent predictability, LSH clusters related items together to streamline similarity searches. * **How LSH Complements kNN**: LSH enhances kNN's effectiveness by grouping vectors based on similarity into the same or adjacent buckets, utilizing hashing functions defined as hv,b,w(p):hv,b,w(p)\=⌊p⋅v+bA⌋h\_{v,b,w}(p) : h\_{v,b,w}(p) = \\left\\lfloor \\frac{\\mathbf{p} \\cdot \\mathbf{v} + b}{A} \\right\\rfloor hv,b,w​(p):hv,b,w​(p)\=⌊Ap⋅v+b​⌋ where vvv is a random chosen vector and b is a random bias bbb is used to offset the vector, and AAA is the bucket width. This method efficiently narrows down the pool of vectors kNN needs to analyze, boosting the overall speed and performance of the search. If you are keen on diving deeper into the intricacies of how kNN and LSH work together, especially in the context of large-scale datasets, we recommend checking out the below resource by Olivier Ruas on KNN+LSH. It provides a more detailed exploration, complete with visual aids and practical examples. [![Logo](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Fpathway.com%2Ffavicon-96x96.png&width=20&dpr=4&quality=100&sign=cafa16c4&sv=2)Realtime Classification with Nearest Neighbors | PathwayPathway](https://pathway.com/developers/showcases/lsh/lsh_chapter2) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module#incremental-indexing-in-llm-app) Incremental Indexing in [LLM App](https://github.com/pathwaycom/llm-app) Building on similarity search, vector embeddings, and RAG, the next piece offers insights into managing vector indexes in dynamic settings, like an e-commerce platform with constantly changing product data. It delves into LSM (Log-Structured Merge-tree) indexes and how Pathway's indexing approach for its RAG Framework – [LLM App](https://github.com/pathwaycom/llm-app) adapts to streaming (live) data, balancing computational efficiency with user needs. The article includes practical scenarios, such as table joins and real-time alerts in Pathway, enhancing your understanding of indexing in fluid data environments. [![Logo](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/~gitbook/image?url=https%3A%2F%2Fpathway.com%2Ffavicon-96x96.png&width=20&dpr=4&quality=100&sign=cafa16c4&sv=2)Indexes in PathwayPathway](https://pathway.com/developers/tutorials/indexes/) [PreviousSimilarity Search in Vectors (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module) [NextTrack your Progress](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/track-your-progress) Last updated 1 year ago --- # Similarity Search in Vectors (Bonus Module) | 10 Days Realtime LLM Bootcamp We now shift our focus to a more nuanced aspect of natural language processing and machine learning: Similarity Search in Vector Embeddings. The significance of this topic, especially after understanding RAG, cannot be overstated. RAG, with its unique blend of retrieval and generation, underscores the importance of accurately finding and utilizing relevant information from a vast corpus, which is stored as vector indexes. Hence, this principle is closely related to the concept of similarity search in vector embeddings, a cornerstone in understanding and leveraging the full potential of large language models (LLMs). While the upcoming section dives a bit deeper into this topic, it's essential to note that this is a bonus section for a reason. Gaining insights into similarity search and the algorithms in play can enrich your understanding. However, if you're primarily focused on implementing by the end of this course, a detailed grasp isn't mandatory. [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module#similarity-search-in-vector-embeddings) Similarity Search in Vector Embeddings -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By now you know, vector embeddings allow us to represent complex data, such as words or images, in a way that captures their underlying semantics. One of the critical tasks in many applications, from semantic search to recommendation systems to large language models, is determining the similarity between these vector embeddings. Three of the most commonly used metrics to measure the similarity between vectors are Euclidean Distance, Dot Product Similarity, and Cosine Similarity. Here's a closer look at each: ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module#id-1.-euclidean-distance) 1\. Euclidean Distance * **Description**: Represents the straight-line distance between two vectors in a multi-dimensional space. * **Formula**: The Euclidean distance between two vectors, `a` and `b`, is calculated as the square root of the sum of the squared differences between their corresponding components. * **Considerations**: This metric is sensitive to both the magnitudes and the relative location of vectors in space. It's a natural choice when vectors contain information about counts or measurements. For example, it can be used in recommendation systems to measure the absolute difference between embeddings of item purchase frequencies. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module#id-2.-dot-product-similarity) 2\. Dot Product Similarity * **Description**: Calculates the similarity by adding the products of the vectors' corresponding components. * **Formula**: The dot product between vectors `a` and `b` is the sum of the product of their corresponding components. * **Considerations**: This metric considers both the direction and magnitude of vectors. It can be particularly useful in situations where the angle between vectors is of interest, such as in collaborative filtering recommendation systems. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module#id-3.-cosine-similarity) 3\. Cosine Similarity * **Description**: Measures the cosine of the angle between two vectors, focusing purely on the direction and not on the magnitude. * **Formula**: The cosine similarity between vectors `a` and `b` is the dot product of the vectors divided by the product of their magnitudes. * **Considerations**: This metric is not influenced by the magnitude of vectors, making it suitable for tasks like semantic search or document classification, where the direction or angle between vectors is more significant than their length. **Comparing and Summarizing the three options** You now have embeddings for any pair of examples. Similarity Metric Description Formula Correlation with Similarity Euclidean Distance Measures the straight-line distance between two points in space represented by vectors ∑i\=1n(ai−bi)2\\sqrt{\\sum\_{i=1}^{n} (a\_i - b\_i)^2} ∑i\=1n​(ai​−bi​)2​ Inversely related (higher distance means lower similarity) Cosine Similarity Evaluates the cosine of the angle between two vectors, indicating their orientation similarity a⋅b∄a∄∄b∄\\frac{\\mathbf{a} \\cdot \\mathbf{b}}{\\|\\mathbf{a}\\| \\|\\mathbf{b}\\|} ∄a∄∄b∄a⋅b​ Directly related (higher cosine means higher similarity) Dot Product The product of the magnitudes of two vectors and the cosine of the angle between them a1b1+a2b2+
+anbn\=∄a∄∄b∄cos⁥(Ξ)a\_1b\_1 + a\_2b\_2 + \\ldots + a\_nb\_n = \\|\\mathbf{a}\\| \\|\\mathbf{b}\\| \\cos(\\theta) a1​b1​+a2​b2​+
+an​bn​\=∄a∄∄b∄cos(Ξ) Directly related and increases with vector magnitudes ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/similarity-search-in-vectors-bonus-module#see-cosine-similarity-search-in-action) See Cosine Similarity Search in Action _(Credits: Microsoft Reactor)_ You should notice and appreciate that cosine similarity has found extensive applications in areas such as **semantic search** and **document classification**. It provides a robust mechanism to gauge the directional similarity of vectors, which translates to comparing the overall essence or content of documents. Imagine trying to find documents or articles that resonate with a given topic or theme; cosine similarity is your tool of choice. Further, if you've ever used a recommendation system, like those on streaming platforms or e-commerce sites, they might be leveraging cosine similarity. These systems aim to suggest items to users, drawing parallels from their historical behavior and preferences. However, it's crucial to note that cosine similarity may not always be the best fit. In scenarios where the magnitude or 'size' of vectors carries significance, relying solely on cosine similarity might be misleading. Take, for instance, image embeddings that are formulated based on pixel intensities. Here, merely comparing the direction of vectors might not suffice, and the magnitude becomes critical. [PreviousKey Benefits of RAG for Enterprise-Grade LLM Applications](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/key-benefits-of-rag-for-enterprise-grade-llm-applications) [NextUsing kNN and LSH to Enhance Similarity Search in Vector Embeddings (Bonus Module)](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/using-knn-and-lsh-to-enhance-similarity-search-in-vector-embeddings-bonus-module) Last updated 1 year ago --- # Using Docker to Build the App | 10 Days Realtime LLM Bootcamp Welcome to this module on using the **Dockerized Application** to set up and run the Dropbox AI Chat application. Before we begin, it's essential to ensure that you meet the prerequisites and understand each step thoroughly. **Basic Prerequisites:** * Ensure you have **Docker** installed on your machine. * **Dropbox** must also be installed. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app#step-1-cloning-the-repository) Step 1: Cloning the Repository Open your terminal and run: Copy git clone https://github.com/pathway-labs/dropbox-ai-chat  cd dropbox-ai-chat If you have previously cloned an older version of the repository, ensure you're in the correct repository directory and update it using: Copy git pull https://github.com/pathway-labs/dropbox-ai-chat **What this does:** The git pull command will update your local repository with the latest changes from the remote repository. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app#step-2-setting-the-environment-variables) Step 2: Setting the Environment Variables **Overview**: * The `.env` file sets crucial environment variables for your application. * If you're using macOS, the `.env` file might be hidden by default when viewed through Finder but is visible via Terminal. Regardless of the OS, it's important to note that this file plays a pivotal role. * The primary change you'll make in this entire implementation is to the `{PATH_TO_DROPBOX}` variable, which the `.env` file uses. **Understanding** `**DROPBOX_LOCAL_FOLDER_PATH**` **variable used here:** * This variable defines the **relative path** from your project to your Dropbox folder. * If you want to quickly understand how relative path works in the context of Linux, you can check this quick [video by Udacity](https://youtu.be/ephId3mYu9o) or read these comprehensive explanations by [RedHat](https://www.redhat.com/sysadmin/linux-path-absolute-relative) or [Coding Rooms](https://www.codingrooms.com/blog/file-paths) . **Setting the environment variables in MacOS or Linux** 1. Create an `.env` file in the project's root directory using `touch`. Copy touch .env 2. Edit the `.env` file using a text editor like `nano` or `vim`. Copy nano .env 3. Populate the `.env` file with the following content, replacing placeholders with actual values: **Note:** Replace the following while using the environment variables from below: 1. `{OPENAI_API_KEY}` with your OpenAI API key. You can get from [here](https://platform.openai.com/account/api-keys) once you've logged in) 2. `{REPLACE_WITH_DROPBOX_RELATIVE_PATH}` with the relative path where the Dropbox folder is located Copy OPENAI_API_TOKEN={OPENAI_API_KEY} EMBEDDER_LOCATOR=text-embedding-ada-002 EMBEDDING_DIMENSION=1536 MODEL_LOCATOR=gpt-3.5-turbo MAX_TOKENS=200 TEMPERATURE=0.0 DROPBOX_LOCAL_FOLDER_PATH={REPLACE_WITH_DROPBOX_RELATIVE_PATH} 3. **Alternative to using** `**export**`: If `.env` doesn't work for you, you can set these variables directly in your shell using the command below. However, it is important to note that variables set with `export` (Linux/macOS) or `set` (Windows, as seen below) last only for the current session. If you want them to persist, you'll need to add them to shell configuration files like add to `.bashrc` or `.bash_profile` for Linux/macOS, or use System Properties on Windows. Copy export OPENAI_API_TOKEN={OPENAI_API_KEY} export EMBEDDER_LOCATOR=text-embedding-ada-002 export EMBEDDING_DIMENSION=1536 export MODEL_LOCATOR=gpt-3.5-turbo export MAX_TOKENS=200 export TEMPERATURE=0.0 export DROPBOX_LOCAL_FOLDER_PATH={REPLACE_WITH_DROPBOX_RELATIVE_PATH} **Setting the environment variables in Windows:** 1. Create an `.env` file using a text editor of your choice. 2. Populate the `.env` file as shown above. 3. **Alternative**: Use the `set` command in Command Prompt to set environment variables. **Note:** Replace `{OPENAI_API_KEY}` with your OpenAI API key and `{REPLACE_WITH_DROPBOX_RELATIVE_PATH}` with your local Dropbox path. Copy set OPENAI_API_TOKEN={OPENAI_API_KEY} set EMBEDDER_LOCATOR=text-embedding-ada-002 set EMBEDDING_DIMENSION=1536 set MODEL_LOCATOR=gpt-3.5-turbo set MAX_TOKENS=200 set TEMPERATURE=0.0 set DROPBOX_LOCAL_FOLDER_PATH={REPLACE_WITH_DROPBOX_RELATIVE_PATH} ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app#step-3-building-and-starting-containers) Step 3: Building and Starting Containers Now we will build the Docker image and start the containers. Navigate to the cloned directory`dropbox-ai-chat`. Then run these commands. Copy docker-compose build docker-compose up **Behind the Scenes with Docker:** * **Dockerfile**: This file contains instructions that Docker follows to build an image. It's like a blueprint for your application. Docker reads these instructions and creates a Docker image based on them. This image contains everything your app needs to run. * **docker-compose**: It's a tool for defining and running multi-container Docker applications. In our context, `docker-compose` uses the `docker-compose.yml` file to understand how to set up and run the app's services. * When you run `docker-compose up`, it starts the services as defined. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app#step-4-accessing-applications) Step 4: Accessing Applications **What it does**: Opens access to your API and Streamlit UI. * **Access Points for your application**: * API: [`http://localhost:8080/`](http://localhost:8080/) * Streamlit UI: [`http://localhost:8501/`](http://localhost:8501/) ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app#step-5-stopping-containers) Step 5: Stopping Containers To stop the services and remove the containers, execute: Copy docker-compose down By following these steps, you should be able to get both the main application and the Streamlit UI up and running using Docker Compose. ### [](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/using-docker-to-build-the-app#an-interesting-thing-to-notice) 💡 An interesting thing to notice Interestingly if you quickly revisit the [LLM Architecture diagram](https://ai-community-iitb-organization.gitbook.io/10-days-llm-bootcamp/retrieval-augmented-generation-and-llm-architecture/llm-architecture-diagram-and-various-steps) we saw earlier, there was a prompt from a Customer Support Executive trying to understand the product release notes made by the development team. With something like the Dropbox app, that problem can be easily addressed. Now let's look at another example where we use an API as a data source. [PreviousUnderstanding Docker](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/dropbox-retrieval-app-in-15-minutes/understanding-docker) [NextAmazon Discounts App](https://ai-community-iitb-organization.gitbook.io/10-days-realtime-llm-bootcamp/hands-on-development/amazon-discounts-app) Last updated 1 year ago ---