pass chain as a parameter in websocket for each user chat session #98
Replies: 1 comment 1 reply
-
|
Hi @raphael2692, do you maybe have an example of other ways to implement websocket applications? I'm not an expert with websockets, so I'll need more to work with. |
Beta Was this translation helpful? Give feedback.
-
|
hello @ajndkr , thanks for reaching out. First of all, I am no expert either, but I'd glad to work on this with you and improve. I set up an example of what I intended (I just edited some lines from https://fastapi.tiangolo.com/advanced/websockets/) from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.responses import HTMLResponse
from langchain.memory import ConversationBufferMemory
from langchain import OpenAI, LLMChain, PromptTemplate
def create_custom_chain():
template = """You are a chatbot having a conversation with a human.
{chat_history}
Human: {human_input}
Chatbot:"""
prompt = PromptTemplate(
input_variables=["chat_history", "human_input"], template=template
)
memory = ConversationBufferMemory(memory_key="chat_history")
return LLMChain(
llm=OpenAI(streaming=True),
prompt=prompt,
verbose=True,
memory=memory,
)
app = FastAPI()
html = """
<!DOCTYPE html>
<html>
<head>
<title>Chat</title>
</head>
<body>
<h1>WebSocket Chat</h1>
<h2>Your ID: <span id="ws-id"></span></h2>
<form action="" onsubmit="sendMessage(event)">
<input type="text" id="messageText" autocomplete="off"/>
<button>Send</button>
</form>
<ul id='messages'>
</ul>
<script>
var client_id = Date.now()
document.querySelector("#ws-id").textContent = client_id;
var ws = new WebSocket(`ws://localhost:8000/ws/${client_id}`);
ws.onmessage = function(event) {
var messages = document.getElementById('messages')
var message = document.createElement('li')
var content = document.createTextNode(event.data)
message.appendChild(content)
messages.appendChild(message)
};
function sendMessage(event) {
var input = document.getElementById("messageText")
ws.send(input.value)
input.value = ''
event.preventDefault()
}
</script>
</body>
</html>
"""
class ConnectionManager:
def __init__(self):
self.active_connections: list[WebSocket] = []
async def connect(self, websocket: WebSocket):
await websocket.accept()
self.active_connections.append(websocket)
def disconnect(self, websocket: WebSocket):
self.active_connections.remove(websocket)
async def send_personal_message(self, message: str, websocket: WebSocket):
await websocket.send_text(message)
async def broadcast(self, message: str):
for connection in self.active_connections:
await connection.send_text(message)
manager = ConnectionManager()
@app.get("/")
async def get():
return HTMLResponse(html)
@app.websocket("/ws/{client_id}")
async def websocket_endpoint(websocket: WebSocket, client_id: int):
chain = create_custom_chain()
await manager.connect(websocket)
try:
while True:
data = await websocket.receive_text()
await manager.send_personal_message(chain.predict(human_input=data), websocket)
await manager.broadcast(f"Client #{client_id} says: {data}")
except WebSocketDisconnect:
manager.disconnect(websocket)
await manager.broadcast(f"Client #{client_id} left the chat")in the example each ws has an id parameter, so i can pass a custom chaid for each new ws istance (imagine a website where users can setup a custom chatbot and chat with it). The only problem is this should be adapted to stream the answer instead of waiting for the chain to finish. Let me know if this is interesting :) |
Beta Was this translation helpful? Give feedback.
-
hello, i have a newbie question...I want to create a chat app that supports websocket in order to get fast reponse time. I also wat the enduser be able to have custom chat session with a custom chain.
I have seen the examples, it seems that the chain object is passed as a parameter when the ws is created. Spawning a new WS for each user conversation session seems a bad way to handle this situation, what should be the correct way?
Thank you very much :)
Beta Was this translation helpful? Give feedback.
All reactions