Reference/RAG_tutorial_1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# RAG using Langchain"
      ],
      "metadata": {
        "id": "fJXPuMBgsO_I"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Packages loading & import"
      ],
      "metadata": {
        "id": "J189g9PHscOJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install langchain\n",
        "!pip install langchain_community\n",
        "!pip install langchain_huggingface\n",
        "!pip install langchain_text_splitters\n",
        "!pip install langchain_chroma\n",
        "!pip install rank-bm25\n",
        "!pip install huggingface_hub"
      ],
      "metadata": {
        "id": "LztGdoQClLDK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "import json\n",
        "import bs4\n",
        "import nltk\n",
        "import torch\n",
        "import pickle\n",
        "import numpy as np\n",
        "\n",
        "# from pyserini.index import IndexWriter\n",
        "# from pyserini.search import SimpleSearcher\n",
        "from numpy.linalg import norm\n",
        "from rank_bm25 import BM25Okapi\n",
        "from nltk.tokenize import word_tokenize\n",
        "\n",
        "from langchain_community.llms import Ollama\n",
        "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
        "from langchain.chains import create_retrieval_chain\n",
        "from langchain.vectorstores import Chroma\n",
        "from sentence_transformers import SentenceTransformer\n",
        "from langchain_huggingface import HuggingFaceEmbeddings\n",
        "from langchain_community.embeddings import JinaEmbeddings\n",
        "from langchain.text_splitter import RecursiveCharacterTextSplitter, TokenTextSplitter\n",
        "from langchain.docstore.document import Document\n",
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "from langchain_community.document_loaders import WebBaseLoader\n",
        "from transformers import AutoModel, AutoModelForCausalLM, AutoTokenizer\n",
        "\n",
        "from tqdm import tqdm"
      ],
      "metadata": {
        "id": "R7fVDp_MSM6d"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "nltk.download('punkt')\n",
        "nltk.download('punkt_tab')"
      ],
      "metadata": {
        "id": "KZaXzYlMakAa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Hugging face login\n",
        "- Please apply the model first: https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct\n",
        "- If you haven't been granted access to this model, you can use other LLM model that doesn't have to apply.\n",
        "- You must save the hf token otherwise you need to regenrate the token everytime.\n",
        "- When using Ollama, no login is required to access and utilize the llama model."
      ],
      "metadata": {
        "id": "_rqtezQ8shF0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from huggingface_hub import login\n",
        "\n",
        "hf_token = \"hf_***\" # @param{type:“string”}\n",
        "login(token=hf_token, add_to_git_credential=True)"
      ],
      "metadata": {
        "id": "Wr0_DNixrDT8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!huggingface-cli whoami"
      ],
      "metadata": {
        "id": "zfN6G4fzr8CG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Ollam enviroment setting"
      ],
      "metadata": {
        "id": "DMtZo6OVsUP5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Ollama is a platform designed for running and managing large language models (LLMs) directly **on local devices**, providing a balance between performance, privacy, and control.\n",
        "- There are also other tools support users to manage LLM on local devices and accelerate it like *vllm*, *Llamafile*, *GPT4ALL*...etc."
      ],
      "metadata": {
        "id": "PhZ_q0V4ciJc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install colab-xterm #https://pypi.org/project/colab-xterm/\n",
        "%load_ext colabxterm\n"
      ],
      "metadata": {
        "id": "Y9agmoxFd7um"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Running the below instructions:\n",
        "\n",
        "  ```curl -fsSL https://ollama.com/install.sh | sh```\n",
        "\n",
        "- If you idle for a long time, the connection would be closed forcedly. If so, run \"ollama serve\" again.\n",
        "\n",
        "  ```ollama serve```\n",
        "\n",
        "- Then <font color=#FF0000>**execute this block again**</font> to download the LLM.(Ollama library: https://ollama.com/library)\n",
        "  - In this tutorial, we'll use the model llama3.2:1b.\n",
        "\n",
        "  ```ollama pull llama3.2:1b```"
      ],
      "metadata": {
        "id": "WDrr_YcB-ddC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%xterm"
      ],
      "metadata": {
        "id": "10NK09FKd_th"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Ollama testing"
      ],
      "metadata": {
        "id": "22kvHBerCCqm"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "v_hBBGBOnH4P"
      },
      "outputs": [],
      "source": [
        "# Setting up the model that this tutorial will use\n",
        "MODEL = \"llama3.2:1b\" # https://ollama.com/library/llama3.2:3b\n",
        "EMBED_MODEL = \"jinaai/jina-embeddings-v2-base-en\""
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize an instance of the Ollama model\n",
        "llm = Ollama(model=MODEL)\n",
        "# Invoke the model to generate responses\n",
        "response = llm.invoke(\"What is the capital of Taiwan?\")\n",
        "print(response)"
      ],
      "metadata": {
        "id": "T2sq6FoDWjVl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Build a simple RAG system by using LangChain"
      ],
      "metadata": {
        "id": "X5ndgmF6wYHY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize the Llama 3 model\n",
        "llm_model = Ollama(model=MODEL)\n",
        "\n",
        "# Create an embedding model\n",
        "model_kwargs = {'trust_remote_code': True}\n",
        "encode_kwargs = {'normalize_embeddings': False}\n",
        "embeddings_model = HuggingFaceEmbeddings(\n",
        "    model_name=EMBED_MODEL,\n",
        "    model_kwargs=model_kwargs,\n",
        "    encode_kwargs=encode_kwargs\n",
        ")"
      ],
      "metadata": {
        "id": "NmPvFlGxe-mr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Prompt setting\n",
        "system_prompt = (\n",
        "    \"Use the given context to answer the question. \"\n",
        "    \"If you don't know the answer, say you don't know. \"\n",
        "    \"Use three sentence maximum and keep the answer concise. \"\n",
        "    \"Context: {context}\"\n",
        ")\n",
        "prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", system_prompt),\n",
        "        (\"human\", \"{input}\"),\n",
        "    ]\n",
        ")\n",
        "# print(prompt)"
      ],
      "metadata": {
        "id": "-oar4tEQONkl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- For the vectorspace, the common algorithm would be used like Faiss, Chroma...(https://python.langchain.com/docs/integrations/vectorstores/) to deal with the extreme huge database."
      ],
      "metadata": {
        "id": "rG9tEFr8wvLX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Prepare documents\n",
        "documents = [\n",
        "    Document(page_content=\"The capital of Florida is Tallahassee.\", metadata={\"id\": 0}),\n",
        "    Document(page_content=\"Florida is known for its beautiful beaches and warm climate.\", metadata={\"id\": 1}),\n",
        "    Document(page_content=\"The largest city in Florida by population is Jacksonville.\", metadata={\"id\": 2}),\n",
        "    Document(page_content=\"The President of Miami Dade College is President Madeline Pumariega.\", metadata={\"id\": 3}),\n",
        "    Document(page_content=\"The Provost of Miami Dade College is Dr. Malou C. Harrison.\", metadata={\"id\": 4}),\n",
        "    Document(page_content=\"Dr. Ernesto Lee is an AI and Data Analytics Professor on the Kendall Campus at Miami Dade College.\", metadata={\"id\": 5})\n",
        "]\n"
      ],
      "metadata": {
        "id": "Ugm57Sh8h9_r"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create Chroma vector store\n",
        "# search_type could be “similarity” (default), “mmr”, or “similarity_score_threshold”\n",
        "vector_store = Chroma.from_documents(documents, embedding=embeddings_model)\n",
        "retriever = vector_store.as_retriever(search_type=\"mmr\", search_kwargs={\"k\": 3, \"fetch_k\": 5})"
      ],
      "metadata": {
        "id": "A3Rn4z4siC6e"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load the QA chain\n",
        "question_answer_chain = create_stuff_documents_chain(llm=llm_model, prompt=prompt) # Create a chain for passing a list of Documents to a model.\n",
        "# print(question_answer_chain)\n",
        "\n",
        "chain = create_retrieval_chain(retriever=retriever, combine_docs_chain=question_answer_chain) # Create retrieval chain that retrieves documents and then passes them on.\n",
        "# print(chain)"
      ],
      "metadata": {
        "id": "yVbi3irCUQn9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Use the QA chain to retrieve relevant documents and generate a response\n",
        "queries = [\n",
        "    \"What is the capital of Florida?\",\n",
        "    \"Who is the President of Miami Dade College?\",\n",
        "    \"Who is the Provost of Miami Dade College?\",\n",
        "    \"Who is Dr. Ernesto Lee?\"\n",
        "]\n",
        "\n",
        "for query in queries:\n",
        "    response = chain.invoke({\"input\": query})\n",
        "    print(f\"Query: {query}\\nResponse: {response}\\n\")\n"
      ],
      "metadata": {
        "id": "6fLKCHWkizlr"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}