rag_lab.ipynb

{
 "metadata": {
  "kernelspec": {
   "display_name": "Streamlit Notebook",
   "name": "streamlit"
  }
 },
 "nbformat_minor": 5,
 "nbformat": 4,
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5b47bedf-bba1-4f6d-8159-199a0231288e",
   "metadata": {
    "name": "Intro",
    "collapsed": false
   },
   "source": "## Introduction ##\n\n### To reduce hallucinations (i.e. incorrect responses), LLMs can be combined with private datasets. \n#### Today, the most common approach for reducing hallucinations without having to change the model (e.g. fine-tuning) is the **Retrieval Augmented Generation (RAG) framework**.\n#### RAG allows you to \"ground\" the model's responses by making a set of relevant documents available to the LLM as context in the response."
  },
  {
   "cell_type": "markdown",
   "id": "ec6dcdb4-b59e-42a2-ade3-39ac8e75a63b",
   "metadata": {
    "name": "cell23",
    "collapsed": false
   },
   "source": "## Organize Documents and Create Pre-Processing Functions\n\n### Step 1. Download example documents\n\n* Mondracer Infant Bike\n* Premium Bycycle User Guide\n* Ski Boots TDBootz Special\n* The Ultimate Downhill Bike"
  },
  {
   "cell_type": "markdown",
   "id": "48787803-b165-48bf-9f54-919fc866433b",
   "metadata": {
    "name": "cell17",
    "collapsed": false
   },
   "source": "### Step 2. Create a table function that will read the PDF documents and split them in chunks"
  },
  {
   "cell_type": "code",
   "id": "c6bc3f83-6e8d-4db8-8305-25868941d355",
   "metadata": {
    "language": "sql",
    "name": "cell10",
    "codeCollapsed": false
   },
   "outputs": [],
   "source": "-- Create the function pdf_text_chunker\n\ncreate or replace function pdf_text_chunker(file_url string)\nreturns table (chunk varchar)\nlanguage python\nruntime_version = '3.9'\nhandler = 'pdf_text_chunker'\npackages = ('snowflake-snowpark-python','PyPDF2', 'langchain')\nas\n$$\nfrom snowflake.snowpark.types import StringType, StructField, StructType\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom snowflake.snowpark.files import SnowflakeFile\nimport PyPDF2, io\nimport logging\nimport pandas as pd\n\nclass pdf_text_chunker:\n\n    def read_pdf(self, file_url: str) -> str:\n    \n        logger = logging.getLogger(\"udf_logger\")\n        logger.info(f\"Opening file {file_url}\")\n    \n        with SnowflakeFile.open(file_url, 'rb') as f:\n            buffer = io.BytesIO(f.readall())\n            \n        reader = PyPDF2.PdfReader(buffer)   \n        text = \"\"\n        for page in reader.pages:\n            try:\n                text += page.extract_text().replace('\\n', ' ').replace('\\0', ' ')\n            except:\n                text = \"Unable to Extract\"\n                logger.warn(f\"Unable to extract from file {file_url}, page {page}\")\n        \n        return text\n\n    def process(self,file_url: str):\n\n        text = self.read_pdf(file_url)\n        \n        text_splitter = RecursiveCharacterTextSplitter(\n            chunk_size = 4000, #Adjust this as you see fit\n            chunk_overlap  = 400, #This let's text have some form of overlap. Useful for keeping chunks contextual\n            length_function = len\n        )\n    \n        chunks = text_splitter.split_text(text)\n        df = pd.DataFrame(chunks, columns=['chunks'])\n        \n        yield from df.itertuples(index=False, name=None)\n$$;",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "280d9d9a-370d-4338-bb32-6a1b0a5a56b9",
   "metadata": {
    "name": "cell24",
    "collapsed": false
   },
   "source": "### Step 3. Create a Stage with Directory Table where you will be uploading your documents"
  },
  {
   "cell_type": "code",
   "id": "a78d6a68-0b58-4414-9c5a-14424e9e6aa5",
   "metadata": {
    "language": "sql",
    "name": "cell11"
   },
   "outputs": [],
   "source": "-- Create a Stage with Directory Table where you will be uploading your documents\n\ncreate or replace stage docs ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE') DIRECTORY = ( ENABLE = true );",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "16f9d512-9393-4f20-989a-443e6cd3340e",
   "metadata": {
    "name": "cell14",
    "collapsed": false
   },
   "source": "Upload the documents to the stage\n\n* Click on the plus (+) icon and the select Add Data\n![](https://github.com/Infostrux-Solutions/rag_lab/blob/main/upload1.png?raw=true, \"upload\")\n\n* Click on \"Load data into stage\"\n![](https://github.com/Infostrux-Solutions/rag_lab/blob/main/upload2.png?raw=true, \"upload\")\n\n* Select your database, schema and stage\n![](https://github.com/Infostrux-Solutions/rag_lab/blob/main/upload_pdf_to_stage.png?raw=true, \"upload\")"
  },
  {
   "cell_type": "markdown",
   "id": "d09b717b-1ecf-4a63-b4db-cf152f57541c",
   "metadata": {
    "name": "cell26",
    "collapsed": false
   },
   "source": "### Step 4. Check files has been successfully uploaded"
  },
  {
   "cell_type": "code",
   "id": "0a16bfb5-ef5f-48a7-b615-c36b2bd9aaab",
   "metadata": {
    "language": "sql",
    "name": "cell9",
    "codeCollapsed": false
   },
   "outputs": [],
   "source": "ls @docs;",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "29d9ef29-ac5e-440d-9211-96b420453e63",
   "metadata": {
    "name": "cell16",
    "collapsed": false
   },
   "source": " ## Build the Vector Store\n #### In this step we are going to leverage our document processing functions to prepare documents before turning the text into embeddings using Snowflake Cortex. These embeddings will be stored in a Snowflake Table using the new native VECTOR data type."
  },
  {
   "cell_type": "markdown",
   "id": "868b8689-acaa-46fd-af8c-e096cd683bd0",
   "metadata": {
    "name": "cell18",
    "collapsed": false
   },
   "source": "Step 1. Create the table where we are going to store the chunks and vectors for each PDF. Note here the usage of the new **VECTOR** data type:"
  },
  {
   "cell_type": "code",
   "id": "36eebbd6-07ad-4063-a9ce-90023e673d50",
   "metadata": {
    "language": "sql",
    "name": "cell12"
   },
   "outputs": [],
   "source": "-- Create the table where we are going to store the chunks and vectors for each PDF.\n-- Note here the usage of the new VECTOR data type:\n\ncreate or replace TABLE DOCS_CHUNKS_TABLE ( \n    RELATIVE_PATH VARCHAR(16777216), -- Relative path to the PDF file\n    SIZE NUMBER(38,0), -- Size of the PDF\n    FILE_URL VARCHAR(16777216), -- URL for the PDF\n    SCOPED_FILE_URL VARCHAR(16777216), -- Scoped url (you can choose which one to keep depending on your use case)\n    CHUNK VARCHAR(16777216), -- Piece of text\n    CHUNK_VEC VECTOR(FLOAT, 768) );  -- Embedding using the VECTOR data type\n",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "45017f17-baa2-4a83-96c6-21a161d32bf7",
   "metadata": {
    "name": "cell19",
    "collapsed": false
   },
   "source": "Step 2. Use the function previously created to process the PDF files, extract the chunks and created the embeddings. Insert that info in the table we have just created:"
  },
  {
   "cell_type": "code",
   "id": "682b7ebe-a92e-4624-8e53-0659b1453c8a",
   "metadata": {
    "language": "sql",
    "name": "cell13"
   },
   "outputs": [],
   "source": "insert into docs_chunks_table (relative_path, size, file_url,\n                            scoped_file_url, chunk, chunk_vec)\n    select relative_path, \n            size,\n            file_url, \n            build_scoped_file_url(@docs, relative_path) as scoped_file_url,\n            func.chunk as chunk,\n            SNOWFLAKE.CORTEX.EMBED_TEXT_768('e5-base-v2',chunk) as chunk_vec\n    from \n        directory(@docs),\n        TABLE(pdf_text_chunker(build_scoped_file_url(@docs, relative_path))) as func;",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "e0fa37c7-49ae-43b8-b101-99e7bdc2e386",
   "metadata": {
    "name": "cell25",
    "collapsed": false
   },
   "source": "### Explanation of the previous code:\n\nThe insert statement is reading the records from the docs_stream stream and it is generating a table calling the table function pdf_text_chunker, where we get the chunk that is the piece of text from the PDF.\n\nThe chunk text is passed to Snowflake Cortex to generate the embeddings with this code:\n\n```sql\nSNOWFLAKE.CORTEX.EMBED_TEXT_768('e5-base-v2',chunk) as chunk_vec\n```\n\nThat code is calling the embed_text function using the e5-base-v2 transformer and returning an embedding vector.\n\nIf you check the **docs_chunks_table table**, you should see the PDFs has been processed\n\n```sql\nselect relative_path, size, chunk, chunk_vec from docs_chunks_table limit 5;\n```"
  },
  {
   "cell_type": "code",
   "id": "1b95680f-6b88-4bad-81c3-69fde53b2f05",
   "metadata": {
    "language": "sql",
    "name": "cell1",
    "codeCollapsed": false
   },
   "outputs": [],
   "source": "-- If you check the docs_chunks_table table, you should see the PDFs has been processed\n-- (And you can see the CHUNK_VEC columns that contains the embedings using the VECTOR data type)\n\nselect relative_path, size, chunk, chunk_vec from docs_chunks_table limit 5;",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "594861c1-0ec1-43bd-9895-fbac288c473d",
   "metadata": {
    "name": "cell21",
    "collapsed": false
   },
   "source": "And you can see the CHUNK_VEC columns that contains the embedings using the VECTOR data type.\n\nYour PDF files has been chunked and each chunk has an embedding. We can check how many chunks we got for each file using this query"
  },
  {
   "cell_type": "code",
   "id": "bccfc1de-d6b7-4422-b145-ab3dcf3974fe",
   "metadata": {
    "language": "sql",
    "name": "cell4",
    "codeCollapsed": false
   },
   "outputs": [],
   "source": "-- Your PDF files has been chunked and each chunk has an embedding.\n-- We can check how many chunks we got for each file using this query\n\nselect relative_path, count(*) as num_chunks \n    from docs_chunks_table\n    group by relative_path;",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "473cf501-4958-44e0-ab42-43c94e731012",
   "metadata": {
    "name": "cell22",
    "collapsed": false
   },
   "source": "## Build Chat UI and Chat (Retrieval and Generation) Logic\n\nTo make it easy for anyone to ask questions against the vector store, let's create a fairly simple front-end using Streamlit. As part of the app, we will provide the end-user with a toggle that allows testing of LLM responses with and without access to the context to observe the differences.\n\n1. Click on the Streamlit tab on the left\n2. Clickn on + Streamlit App button on the right\n3. Give the App a name (CC_CORTEX_APP in my example)\n4. Select the warehouse to run the App (a Small WH will be enough)\n5. Choose the CC_QUICKSTART_CORTEX_DOCS database and DATA schema\n\n![](https://github.com/Infostrux-Solutions/rag_lab/blob/main/create_streamlit.png?raw=true, \"Streamlit\")\n\nThe Streamlit app comes with a default template you can delete and replace with this code which includes the front-end elements:\n\n```python\nimport streamlit as st # Import python packages\nfrom snowflake.snowpark.context import get_active_session\nsession = get_active_session() # Get the current credentials\n\nimport pandas as pd\n\npd.set_option(\"max_colwidth\",None)\nnum_chunks = 3 # Num-chunks provided as context. Play with this to check how it affects your accuracy\n\ndef create_prompt (myquestion, rag):\n\n    if rag == 1:    \n\n        cmd = \"\"\"\n         with results as\n         (SELECT RELATIVE_PATH,\n           VECTOR_COSINE_SIMILARITY(docs_chunks_table.chunk_vec,\n                    SNOWFLAKE.CORTEX.EMBED_TEXT_768('e5-base-v2', ?)) as similarity,\n           chunk\n         from docs_chunks_table\n         order by similarity desc\n         limit ?)\n         select chunk, relative_path from results \n         \"\"\"\n    \n        df_context = session.sql(cmd, params=[myquestion, num_chunks]).to_pandas()      \n        \n        context_lenght = len(df_context) -1\n\n        prompt_context = \"\"\n        for i in range (0, context_lenght):\n            prompt_context += df_context._get_value(i, 'CHUNK')\n\n        prompt_context = prompt_context.replace(\"'\", \"\")\n        relative_path =  df_context._get_value(0,'RELATIVE_PATH')\n    \n        prompt = f\"\"\"\n          'You are an expert assistance extracting information from context provided. \n           Answer the question based on the context. Be concise and do not hallucinate. \n           If you don´t have the information just say so.\n          Context: {prompt_context}\n          Question:  \n           {myquestion} \n           Answer: '\n           \"\"\"\n        cmd2 = f\"select GET_PRESIGNED_URL(@docs, '{relative_path}', 360) as URL_LINK from directory(@docs)\"\n        df_url_link = session.sql(cmd2).to_pandas()\n        url_link = df_url_link._get_value(0,'URL_LINK')\n\n    else:\n        prompt = f\"\"\"\n         'Question:  \n           {myquestion} \n           Answer: '\n           \"\"\"\n        url_link = \"None\"\n        relative_path = \"None\"\n        \n    return prompt, url_link, relative_path\n\ndef complete(myquestion, model_name, rag = 1):\n\n    prompt, url_link, relative_path =create_prompt (myquestion, rag)\n    cmd = f\"\"\"\n             select SNOWFLAKE.CORTEX.COMPLETE(?,?) as response\n           \"\"\"\n    \n    df_response = session.sql(cmd, params=[model_name, prompt]).collect()\n    return df_response, url_link, relative_path\n\ndef display_response (question, model, rag=0):\n    response, url_link, relative_path = complete(question, model, rag)\n    res_text = response[0].RESPONSE\n    st.markdown(res_text)\n    if rag == 1:\n        display_url = f\"Link to [{relative_path}]({url_link}) that may be useful\"\n        st.markdown(display_url)\n\n#Main code\n\nst.title(\"Asking Questions to Your Own Documents with Snowflake Cortex:\")\nst.write(\"\"\"You can ask questions and decide if you want to use your documents for context or allow the model to create their own response.\"\"\")\nst.write(\"This is the list of documents you already have:\")\ndocs_available = session.sql(\"ls @docs\").collect()\nlist_docs = []\nfor doc in docs_available:\n    list_docs.append(doc[\"name\"])\nst.dataframe(list_docs)\n\n#Here you can choose what LLM to use. Please note that they will have different cost & performance\nmodel = st.sidebar.selectbox('Select your model:',(\n                                    'mixtral-8x7b',\n                                    'snowflake-arctic',\n                                    'mistral-large',\n                                    'llama3-8b',\n                                    'llama3-70b',\n                                    'reka-flash',\n                                     'mistral-7b',\n                                     'llama2-70b-chat',\n                                     'gemma-7b'))\n\nquestion = st.text_input(\"Enter question\", placeholder=\"Is there any special lubricant to be used with the premium bike?\", label_visibility=\"collapsed\")\n\nrag = st.sidebar.checkbox('Use your own documents as context?')\n\nprint (rag)\n\nif rag:\n    use_rag = 1\nelse:\n    use_rag = 0\n\nif question:\n    display_response (question, model, use_rag)\n```"
  },
  {
   "cell_type": "markdown",
   "id": "2ddcf81c-99fd-4e7c-8a36-ea4d422a220c",
   "metadata": {
    "name": "cell20",
    "collapsed": false
   },
   "source": "## Build a ChatBot UI that Remember Previous Conversations\n\nIn the previous section we have created a simple interface where we can ask questions about our documents and select the LLM running within Snowflake Cortex to answer the question. We have seen that when no context from our documents is provided, we just get a general answer, versus a specific answer related to our documents when we use context from the PDFs. But what if we want to have a conversation sytle?\n\nFirst let´s create the new Streamlit App and then we will discuss each of the steps. Give it a name and create it within the database and schema that we are using in this lab.\n\n![](https://github.com/Infostrux-Solutions/rag_lab/blob/main/chatbot_streamlit.png?raw=true, \"Streamlit\")\n\n```python\nimport streamlit as st # Import python packages\nfrom snowflake.snowpark.context import get_active_session\nsession = get_active_session() # Get the current credentials\n\nimport pandas as pd\n\npd.set_option(\"max_colwidth\",None)\n\n### Default Values\n#model_name = 'mistral-7b' #Default but we allow user to select one\nnum_chunks = 3 # Num-chunks provided as context. Play with this to check how it affects your accuracy\nslide_window = 7 # how many last conversations to remember. This is the slide window.\n#debug = 1 #Set this to 1 if you want to see what is the text created as summary and sent to get chunks\n#use_chat_history = 0 #Use the chat history by default\n\n### Functions\n\ndef main():\n    \n    st.title(f\":speech_balloon: Chat Document Assistant with Snowflake Cortex\")\n    st.write(\"This is the list of documents you already have and that will be used to answer your questions:\")\n    docs_available = session.sql(\"ls @docs\").collect()\n    list_docs = []\n    for doc in docs_available:\n        list_docs.append(doc[\"name\"])\n    st.dataframe(list_docs)\n\n    config_options()\n    init_messages()\n     \n    # Display chat messages from history on app rerun\n    for message in st.session_state.messages:\n        with st.chat_message(message[\"role\"]):\n            st.markdown(message[\"content\"])\n    \n    # Accept user input\n    if question := st.chat_input(\"What do you want to know about your products?\"):\n        # Add user message to chat history\n        st.session_state.messages.append({\"role\": \"user\", \"content\": question})\n        # Display user message in chat message container\n        with st.chat_message(\"user\"):\n            st.markdown(question)\n        # Display assistant response in chat message container\n        with st.chat_message(\"assistant\"):\n            message_placeholder = st.empty()\n    \n            question = question.replace(\"'\",\"\")\n    \n            with st.spinner(f\"{st.session_state.model_name} thinking...\"):\n                response = complete(question)\n                res_text = response[0].RESPONSE     \n            \n                res_text = res_text.replace(\"'\", \"\")\n                message_placeholder.markdown(res_text)\n        \n        st.session_state.messages.append({\"role\": \"assistant\", \"content\": res_text})\n\n\ndef config_options():\n\n\n    \n    st.sidebar.selectbox('Select your model:',(\n                                    'mixtral-8x7b',\n                                    'snowflake-arctic',\n                                    'mistral-large',\n                                    'llama3-8b',\n                                    'llama3-70b',\n                                    'reka-flash',\n                                     'mistral-7b',\n                                     'llama2-70b-chat',\n                                     'gemma-7b'), key=\"model_name\")\n                                           \n    # For educational purposes. Users can chech the difference when using memory or not\n    st.sidebar.checkbox('Do you want that I remember the chat history?', key=\"use_chat_history\", value = True)\n\n    st.sidebar.checkbox('Debug: Click to see summary generated of previous conversation', key=\"debug\", value = True)\n    st.sidebar.button(\"Start Over\", key=\"clear_conversation\")\n    st.sidebar.expander(\"Session State\").write(st.session_state)\n\n\ndef init_messages():\n\n    # Initialize chat history\n    if st.session_state.clear_conversation or \"messages\" not in st.session_state:\n        st.session_state.messages = []\n\n    \ndef get_similar_chunks (question):\n\n    cmd = \"\"\"\n        with results as\n        (SELECT RELATIVE_PATH,\n           VECTOR_COSINE_SIMILARITY(docs_chunks_table.chunk_vec,\n                    SNOWFLAKE.CORTEX.EMBED_TEXT_768('e5-base-v2', ?)) as similarity,\n           chunk\n        from docs_chunks_table\n        order by similarity desc\n        limit ?)\n        select chunk, relative_path from results \n    \"\"\"\n    \n    df_chunks = session.sql(cmd, params=[question, num_chunks]).to_pandas()       \n\n    df_chunks_lenght = len(df_chunks) -1\n\n    similar_chunks = \"\"\n    for i in range (0, df_chunks_lenght):\n        similar_chunks += df_chunks._get_value(i, 'CHUNK')\n\n    similar_chunks = similar_chunks.replace(\"'\", \"\")\n             \n    return similar_chunks\n\n\ndef get_chat_history():\n#Get the history from the st.session_stage.messages according to the slide window parameter\n    \n    chat_history = []\n    \n    start_index = max(0, len(st.session_state.messages) - slide_window)\n    for i in range (start_index , len(st.session_state.messages) -1):\n         chat_history.append(st.session_state.messages[i])\n\n    return chat_history\n\n    \ndef summarize_question_with_history(chat_history, question):\n# To get the right context, use the LLM to first summarize the previous conversation\n# This will be used to get embeddings and find similar chunks in the docs for context\n\n    prompt = f\"\"\"\n        Based on the chat history below and the question, generate a query that extend the question\n        with the chat history provided. The query should be in natual language. \n        Answer with only the query. Do not add any explanation.\n        \n        <chat_history>\n        {chat_history}\n        </chat_history>\n        <question>\n        {question}\n        </question>\n        \"\"\"\n    \n    cmd = \"\"\"\n            select snowflake.cortex.complete(?, ?) as response\n          \"\"\"\n    df_response = session.sql(cmd, params=[st.session_state.model_name, prompt]).collect()\n    sumary = df_response[0].RESPONSE     \n\n    if st.session_state.debug:\n        st.sidebar.text(\"Summary to be used to find similar chunks in the docs:\")\n        st.sidebar.caption(sumary)\n\n    sumary = sumary.replace(\"'\", \"\")\n\n    return sumary\n\ndef create_prompt (myquestion):\n\n    if st.session_state.use_chat_history:\n        chat_history = get_chat_history()\n\n        if chat_history != []: #There is chat_history, so not first question\n            question_summary = summarize_question_with_history(chat_history, myquestion)\n            prompt_context =  get_similar_chunks(question_summary)\n        else:\n            prompt_context = get_similar_chunks(myquestion) #First question when using history\n    else:\n        prompt_context = get_similar_chunks(myquestion)\n        chat_history = \"\"\n  \n    prompt = f\"\"\"\n           You are an expert chat assistance that extracs information from the CONTEXT provided\n           between <context> and </context> tags.\n           You offer a chat experience considering the information included in the CHAT HISTORY\n           provided between <chat_history> and </chat_history> tags..\n           When ansering the question contained between <question> and </question> tags\n           be concise and do not hallucinate. \n           If you don´t have the information just say so.\n           \n           Do not mention the CONTEXT used in your answer.\n           Do not mention the CHAT HISTORY used in your asnwer.\n           \n           <chat_history>\n           {chat_history}\n           </chat_history>\n           <context>          \n           {prompt_context}\n           </context>\n           <question>  \n           {myquestion}\n           </question>\n           Answer: \n           \"\"\"\n\n    return prompt\n\n\ndef complete(myquestion):\n\n    prompt =create_prompt (myquestion)\n    cmd = \"\"\"\n            select snowflake.cortex.complete(?, ?) as response\n          \"\"\"\n    \n    df_response = session.sql(cmd, params=[st.session_state.model_name, prompt]).collect()\n    return df_response\n\nif __name__ == \"__main__\":\n    main()\n```    \n"
  },
  {
   "cell_type": "code",
   "id": "8f895d26-142a-41f9-929f-a4644f77933e",
   "metadata": {
    "language": "sql",
    "name": "cell5",
    "codeCollapsed": false
   },
   "outputs": [],
   "source": "-- optional: automatica processing of new documents\ncreate or replace stream docs_stream on stage docs;\n\ncreate or replace task task_extract_chunk_vec_from_pdf \n    warehouse = INSTRUCTOR_WH\n    schedule = '1 minute'\n    when system$stream_has_data('docs_stream')\n    as\n\n    insert into docs_chunks_table (relative_path, size, file_url,\n                            scoped_file_url, chunk, chunk_vec)\n    select relative_path, \n            size,\n            file_url, \n            build_scoped_file_url(@docs, relative_path) as scoped_file_url,\n            func.chunk as chunk,\n            SNOWFLAKE.CORTEX.EMBED_TEXT_768('e5-base-v2',chunk) as chunk_vec\n    from \n        docs_stream,\n        TABLE(pdf_text_chunker(build_scoped_file_url(@docs, relative_path))) as func;\n\nalter task task_extract_chunk_vec_from_pdf resume;",
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "4be8f895-d852-4b9c-aa6a-ff0b9dd7ea37",
   "metadata": {
    "language": "sql",
    "name": "cell6",
    "codeCollapsed": false
   },
   "outputs": [],
   "source": "alter task task_extract_chunk_vec_from_pdf suspend;",
   "execution_count": null
  }
 ]
}