Open-ChatGPT is a open-source library that allows you to train a hyper-personalized ChatGPT-like ai model using your own data and the least amount of compute possible.
Open-ChatGPT is a general system framework for enabling an end-to-end training experience for ChatGPT-like models. It can automatically take your favorite pre-trained large language models though an OpenAI InstructGPT style three stages to produce your very own high-quality ChatGPT-style model.
We have Impleamented RLHF (Reinforcement Learning with Human Feedback) powered by transformer library and DeepsSpeed. It supports distributed training and offloading, which can fit extremly large models.
If you like the project, please show your support by leaving a star ⭐.
- [2023/04] We released RLHF(Reinforcement Learning with Human Feedback) Pipeline .
- [2023/03] We released the code OpenChatGPT: An Open-Source libraray to train ChatBot like ChatGPT.
git clone https://github.com/jianzhnie/open-chatgpt.git
pip install -r requirements.txtPEFT
- If you would like to use LORA along with other parameter-efficient methods, please install peft as an additional dependency.
DeepSpeed
- If you want to accelerate LLM training using techniques such as pipeline parallelism, gradient checkpointing, and tensor fusion. Please install DeepSpeed.
Our purpose is to make this repo even better. If you are interested in contributing, please refer to HERE for instructions in contribution.
Open-ChatGPT is released under the Apache 2.0 license.
We appreciate the work by many open-source contributors, especially:
Please cite the repo if you use the data or code in this repo.
@misc{open-chatgpt,
author = {jianzhnie},
title = {Open-ChatGPT, An open-source implementation of ChatGPT},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/jianzhnie/open-chatgpt}},
}