Goals of this project

1. Deploy LLMs with opeanai compatible API which can be used as drop in replacement for actual openai api for which access is paid and you have little control over it

2. Learn to develop Langchain based application which will use these APIs

3. Compare different LLMs for the same langchain agents, the quality of response and other KPIs (TBD)

4. Understand the cost of LLM+application

Appoches for LLM deployment

1. FastChat: provides easy to deploy interface for most populat LLMs (Vicuna, maybe LLaMA), supports PERF
2. Find other easy methods ... TBD

Which LLMs to try as backend

1. Vicuna (because it is safest with FastChat, but not very good quality with 7b model, especially the Oneshot_REACT agents fail to follow REACT framework)
2. LLaMA-2
3. Dolly-2
4. Falcon