feat: update documentation (#158)

* ⚡ override setup method use lanarky favicon * 🐛 fix stream_response * ✨ add `system` param * ✨ add StreamingClient * 📝 update getting started doc * 🔧 update nav * 🔥 remove print * 📝 minor update * 📝 add boilerplate doc structure * ✨ add WebSocketClient 🔧 update dependencies * 📝 add streaming and websocket docs * ✨ add stream_response to WebsocketClient * 📝 update openai adapter docs ⚡ send END event for openai websocket factory endpoint 🔧 update navigation features 📝 update openai api router doc * 📝 complete openai adapter docs - 🔧 update nav - 📝 update langchain adapter docs * 🔧 update css for mobile devices * 🐛 fix `build_factory_websocket_endpoint` in langchain adapter * 📝 complete langchain adapter docs * 🔥 remove deployment section will be added in future release
ajndkr · Nov 28, 2023 · 221c06b · 221c06b
1 parent 4d6cea3
commit 221c06b
Show file tree

Hide file tree

Showing 27 changed files with 1,383 additions and 89 deletions.
diff --git a/.coveragerc b/.coveragerc
@@ -1,3 +1,4 @@
 [run]
 omit =
+    lanarky/clients.py
     lanarky/adapters/*/__init__.py
diff --git a/docs/getting-started.md b/docs/getting-started.md
@@ -3,22 +3,18 @@ hide:
   - navigation
 ---
 
-This is a quick tutorial on getting started with Lanarky.
+Let's build our first LLM microservice with Lanarky!
 
-We will use LangChain as the LLM tooling framework and OpenAI as the LLM provider to
-build our first LLM microservice.
-
-## Install Dependencies
+We need to first install some extra dependencies as we will use OpenAI as the LLM
+provider.
 
 <!-- termynal -->
 
 ```
-$ pip install lanarky[langchain,openai]
+$ pip install lanarky[openai]
 ```
 
-## Example
-
-We will use the `ConversationChain` from LangChain library to build our first LLM microservice.
+## Application
 
 !!! info
 
@@ -28,27 +24,26 @@ We will use the `ConversationChain` from LangChain library to build our first LL
 ```python
 import os
 
-from langchain.chains import ConversationChain
-from langchain.chat_models import ChatOpenAI
-
 from lanarky import Lanarky
-from lanarky.adapters.langchain.routing import LangchainAPIRouter
+from lanarky.adapters.openai.resources import ChatCompletionResource
+from lanarky.adapters.openai.routing import OpenAIAPIRouter
 
 os.environ["OPENAI_API_KEY"] = "add-your-openai-api-key-here"
 
 app = Lanarky()
-langchain_router = LangchainAPIRouter()
+router = OpenAIAPIRouter()
 
 
-@langchain_router.post("/chat")
-def chat(streaming: bool = True) -> ConversationChain:
-    return ConversationChain(llm=ChatOpenAI(streaming=streaming))
+@router.post("/chat")
+def chat(stream: bool = True) -> ChatCompletionResource:
+    system = "You are a sassy assistant"
+    return ChatCompletionResource(stream=stream, system=system)
 
 
-app.include_router(langchain_router)
+app.include_router(router)
 ```
 
-Run the application:
+Run application:
 
 <!-- termynal -->
 
@@ -57,56 +52,81 @@ $ pip install uvicorn
 $ uvicorn app:app --reload
 ```
 
-View the Swagger docs at [http://localhost:8000/docs](http://localhost:8000/docs).
+!!! tip
 
-## Testing
+    Swagger docs will be available at [http://localhost:8000/docs](http://localhost:8000/docs).
 
-<!-- termynal -->
+## Client
 
-```
-$ pip install httpx-sse
-```
+Now that the application script is running, we will setup a client script for testing.
 
-Create `client.py` script:
+Create `client.py`:
 
 ```python
 import click
-import httpx
-from httpx_sse import connect_sse
+
+from lanarky.clients import StreamingClient
 
 
 @click.command()
 @click.option("--input", required=True)
-@click.option("--streaming", is_flag=True)
-def main(input: str, streaming: bool):
-    url = f"http://localhost:8000/chat?streaming={str(streaming).lower()}"
-    with httpx.Client() as client:
-        with connect_sse(
-            client,
-            "POST",
-            url,
-            json={"input": input},
-        ) as event_source:
-            for sse in event_source.iter_sse():
-                print(sse.event, sse.data)
+@click.option("--stream", is_flag=True)
+def main(input: str, stream: bool):
+    client = StreamingClient()
+    for event in client.stream_response(
+        "POST",
+        "/chat",
+        params={"stream": str(stream).lower()},
+        json={"messages": [dict(role="user", content=input)]},
+    ):
+        print(f"{event.event}: {event.data}")
 
 
 if __name__ == "__main__":
     main()
 ```
 
-Stream output:
+Since we have exposed only `stream` as the query parameter, we can test 2 scenarios:
+
+1. Recieve output as it is generated:
 
 <!-- termynal -->
 
 ```
-$ python client.py --input hi --streaming
+$ python client.py --input "hi" --stream
+completion:
+completion: Well
+completion: ,
+completion:  hello
+completion:  there
+completion: !
+completion:  How
+completion:  can
+completion:  I
+completion:  sass
+completion: ...
+completion:  I
+completion:  mean
+completion:  assist
+completion:  you
+completion:  today
+completion: ?
 ```
 
-Recieve all output at once:
+2.  Recieve all output at once:
 
 <!-- termynal -->
 
 ```
-$ python client.py --input hi
+$ python client.py --input "hi"
+completion: Oh, hello there! What can I sass...I mean assist you with today?
 ```
+
+## Next Steps
+
+Congrats on building your first LLM microservice with Lanarky!
+
+Now that you have a basic understanding of how Lanarky works, let's learn more about
+the core concepts of Lanarky.
+
+[Let's Learn!](./learn/index.md){ .md-button .md-button--primary }
diff --git a/docs/learn/adapters/index.md b/docs/learn/adapters/index.md
@@ -0,0 +1,14 @@
+# Adapters
+
+The **Adapters API** allows Lanarky users to build microservices using popular LLM frameworks.
+
+We will cover the following adapters in depth:
+
+- [OpenAI](./openai/index.md): build microservices using the
+  [OpenAI Python SDK](https://platform.openai.com/docs/api-reference?lang=python)
+- [LangChain](./langchain/index.md): build microservices using the
+  [LangChain](https://www.langchain.com/)
+
+!!! note "Note from Author"
+
+    The **Adapters API** is still in active development. I will add more adapters in the future.
diff --git a/docs/learn/adapters/langchain/callbacks.md b/docs/learn/adapters/langchain/callbacks.md
@@ -0,0 +1,48 @@
+Lanarky offers a collection of callback handlers for LangChain. These callback
+handlers are useful in executing intermediate callback events related to your LangChain
+microservice.
+
+Lanarky offers callback handlers for both streaming and WebSockets. We will take a look at
+both of them in this guide.
+
+!!! note
+
+    All callback handlers can be imported from the `lanarky.adapters.langchain.callbacks`
+    module.
+
+## Tokens
+
+- `TokenStreamingCallbackHandler`: handles streaming of the intermediate tokens over HTTP
+- `TokenWebSocketCallbackHandler`: handles streaming of the intermediate tokens over WebSockets
+
+Both callback handlers offer token streaming in two modes: `text` and `json`. In `text` mode,
+the callback handlers will use raw token string as event data. In `json` mode, the callback
+handlers will use a JSON object containing the token string as event data.
+
+These callback handlers are useful for all chains where the `llm` component supports streaming.
+
+## Source Documents
+
+- `SourceDocumentStreamingCallbackHandler`: handles streaming of the source documents
+  over HTTP
+- `SourceDocumentWebSocketCallbackHandler`: handles streaming of the source documents
+  over WebSockets
+
+The source documents are sent at the end of a chain execution as a `source_documents` event.
+
+These callback handlers are useful for retrieval-based chains like `RetrievalQA`.
+
+## Agents
+
+- `FinalTokenStreamingCallbackHandler`: handles streaming of the final answer tokens over HTTP
+- `FinalTokenWebSocketCallbackHandler`: handles streaming of the final answer tokens over WebSockets
+
+Both callback handlers are extension of the token streaming callback handlers where the tokens are
+streamed only when the LLM agent has reached the final step of its execution.
+
+These callback handlers are useful for all agent types like `ZeroShotAgent`.
+
+!!! note
+
+    The callback handlers also inherit some functionality of the `FinalStreamingStdOutCallbackHandler`
+    callback handler. Check out [LangChain Docs](https://api.python.langchain.com/en/latest/callbacks/langchain.callbacks.streaming_stdout_final_only.FinalStreamingStdOutCallbackHandler.html) to know more.
diff --git a/docs/learn/adapters/langchain/dependency.md b/docs/learn/adapters/langchain/dependency.md
@@ -0,0 +1,103 @@
+---
+hide:
+  - toc
+---
+
+FastAPI offers a powerful [Dependency Injection](https://fastapi.tiangolo.com/tutorial/dependencies/)
+system that allows you to inject dependencies into your API endpoints. Lanarky extends this functionality
+by offering LangChain as a dependency.
+
+!!! example "Experimental"
+
+    LLM-based dependency injection is an experimental feature. We will add more functionality
+    based on community feedback and viable use cases. If you have ideas or suggestions, we
+    would love to hear from you. Feel free to open an issue on
+    [GitHub](https://github.com/ajndkr/lanarky/issues/new/choose).
+
+Let's take a look at how we can use LangChain as a dependency.
+
+```python
+import os
+
+from langchain.chains import LLMChain
+from langchain.chat_models import ChatOpenAI
+from langchain.prompts import (
+    ChatPromptTemplate,
+    HumanMessagePromptTemplate,
+    PromptTemplate,
+)
+
+from lanarky import Lanarky
+from lanarky.adapters.langchain.dependencies import Depends
+
+os.environ["OPENAI_API_KEY"] = "add-your-openai-api-key-here"
+
+
+app = Lanarky()
+
+
+def chain_factory(temperature: float = 0.0, verbose: bool = False) -> LLMChain:
+    return LLMChain(
+        llm=ChatOpenAI(temperature=temperature),
+        prompt=ChatPromptTemplate.from_messages(
+            [
+                HumanMessagePromptTemplate(
+                    prompt=PromptTemplate.from_template("Respond in JSON: {input}")
+                ),
+            ]
+        ),
+        verbose=verbose,
+    )
+
+
+@app.post("/")
+async def endpoint(outputs: dict = Depends(chain_factory)):
+    return outputs["text"]
+```
+
+In the above example, we pass `chain_factory` as a dependency to the endpoint. The endpoint
+exposes the dependency function arguments as query parameters. This allows us to configure
+the dependency at runtime.
+
+To test the above endpoint, let's create a client script:
+
+```python
+import click
+import httpx
+
+
+@click.command()
+@click.option("--input", required=True)
+def main(input: str):
+    url = "http://localhost:8000/"
+
+    with httpx.Client() as client:
+        response = client.post(url, json={"input": input})
+        if response.status_code == 200:
+            data = response.json()
+            print(f"Received: {data}")
+        else:
+            print(response.text)
+
+
+if __name__ == "__main__":
+    main()
+```
+
+First, start the server:
+
+```bash
+uvicorn app:app
+```
+
+Then, run the client script:
+
+<!-- termynal -->
+
+```
+$ python client.py --input "Who won the world series in 2020?"
+Received: {
+  "team": "Los Angeles Dodgers",
+  "year": 2020
+}
+```