LinChance-Fine-tuning-System

Read this in other languages: English, 中文.

View Demo · Report Bug · Request Feature

Content

• LinChance-Fine-tuning-System
  • Content
    • Project Introduction
    • Environment Setup
    • Installation Steps
      • 1. Clone the Repository:
      • 2. Execute the following commands:
      • 3. Run the startup command:
      • 4. ngrok Intranet Penetration
      • 4. WandB Fine-tuning Output Information Network Charting
      • 5. Dataset Format
    • Web UI Page Operations
      • Click to Download Models
      • Upload or Choose Existing Datasets
      • Start Fine-tuning with Default Script Parameters
      • Switch Between Chinese and English in Fine-tuning Parameter Information
      • Automatically Optimize and Display Hyperparameters
      • Customize and Save Fine-tuning Script Parameters for Viewing and Starting Fine-tuning
      • Resume Fine-tuning Progress
      • Display Loss Graph After Fine-tuning Completion
      • Test the Trained Model with Dialogue
      • Forceful Exit and Restart Button
    • Supported Models
  • Likelihood Lab
  • Contributors
  • License

Project Introduction

• LinChance Fine-tuning System is a model fine-tuning web UI created using Streamlit combined with LLaMA-Factory in the Autodl 3090 24G experimental environment.
• It utilizes ngrok for intranet penetration to enable internet access to Autodl services.
• The system features a simple and elegant fine-tuning interface implemented using Streamlit components and methods. It incorporates the modelscope method for quick model downloads, allowing users to customize fine-tuning parameters. Users can choose existing datasets or upload private datasets for convenient and private fine-tuning using the Lora method. This design aims to provide a user-friendly experience, especially for beginners.
• The system employs Linux subprocess methods to run multiple Python processes for fine-tuning scripts and Streamlit Web UI.

Environment Setup

Rent a GPU machine with 24GB VRAM, such as a 3090 card, on the autodl platform. Choose the following image: PyTorch --> 2.0.0 --> 3.8(ubuntu20.04) --> 11.8, as shown in the image below.

Next, open the JupyterLab on the rented server and open the terminal to start environment configuration, model downloads, and run the demo.

pip source replacement and dependency installation:

# Upgrade pip
python -m pip install --upgrade pip
# Change the pypi source to speed up library installation
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

pip install modelscope

Installation Steps

1. Clone the Repository:

git clone https://github.com/Joe-2002/LinChance-Fine-tuning-System.git  

2. Execute the following commands:

cd /root/LinChance Fine-tuning System
pip install -r requirements.txt

3. Run the startup command:

python -m streamlit run main.py

4. ngrok Intranet Penetration

1. First, check the installation command on the Ngrok official website. We'll use the example of a Linux system for installation, which can be done through various methods including downloading the compressed package, APT installation, and Snap installation. Here, we use APT installation, execute the following command:

   curl -s https://ngrok-agent.s3.amazonaws.com/ngrok.asc | sudo tee /etc/apt/trusted.gpg.d/ngrok.asc >/dev/null && echo "deb https://ngrok-agent.s3.amazonaws.com buster main" | sudo tee /etc/apt/sources.list.d/ngrok.list && sudo apt update && sudo apt install ngrok

2. After Ngrok is installed, register an account on its official website. Then, obtain the Authtoken in the Your Authtoken menu. This Authtoken is used for user identity verification. Set the Authtoken locally with the following command:

   ngrok config add-authtoken your-ngrok-authtoken # Replace this with your Authtoken

3. Next, execute the following command to proxy the local Streamlit service through Ngrok.

   ngrok http 8501 # streamlit default port is 8501

4. Access the link shown in the image below to open the Web UI page:

   

4. WandB Fine-tuning Output Information Network Charting

1. Install WandB using the following command:

   pip install wandb

2. Once the installation is complete, you can access the WandB Quickstart guide at the following link: WandB Quickstart

3. Register an account on the WandB official website and obtain the API key for that account. Then, execute the following command in the terminal:

   wandb login

5. Dataset Format

By default, the lima dataset is selected. After uploading a dataset, change its name to lima (since this repository is developed based on LlaMa Factory, the modified name can be directly recognized and used).

[
  {
    "instruction": "",
    "input": "",
    "output": "",
    "history": ""
  },
  {
    "instruction": "",
    "input": "",
    "output": "",
    "history": ""
  },
]

Web UI Page Operations

Click to Download Models

Upload or Choose Existing Datasets

Start Fine-tuning with Default Script Parameters

Switch Between Chinese and English in Fine-tuning Parameter Information

Automatically Optimize and Display Hyperparameters

Customize and Save Fine-tuning Script Parameters for Viewing and Starting Fine-tuning

Resume Fine-tuning Progress

Display Loss Graph After Fine-tuning Completion

Test the Trained Model with Dialogue

Forceful Exit and Restart Button

Before usage, please read the warning message. Use with caution!!!

Supported Models

• ChatGLM3
  • ChatGLM3-6B-chat
• Baichuan2
  • Baichuan2-7B-chat
  • Baichuan2-13B-chat
• Mistral
  • Mistral-7B-v 0.1
• Llama2
  • LlaMa2-7B-chat

Likelihood Lab

Likelihood Lab is a public AI lab initiated by Maxwell (Mingwen) Liu, the general manager of Guangzhou Shining Midas Investment Management Co. Ltd. and AI leader of Revenco Conglomerate. The main research fields are FinTech, Energy, and Robotics. Other co-founders of the lab include Andrew Chen from MIT, Elvis Zhang from Stanford, Xingyu Fu from the School of Mathematics SYSU, Tanli Zuo from the School of Computer Science SYSU.

Contributors

Kechen Li

Contact Email: likechen@linchance.com

Affiliation: Likelihood Lab

Kechen Li

License