Skip to content

Latest commit

 

History

History
20 lines (14 loc) · 467 Bytes

README.md

File metadata and controls

20 lines (14 loc) · 467 Bytes

Mixtral 8X7B — Deploying an Open AI Agent

The goal of this repo is to show how to deploy an AI agent using the new Open source model Mistral 8X7B on an instance (AWS, RunPod).

1. Create an instance (RUNPOD)

The instance must have the following characteristics:

  • 160 GB VRAM
  • 8 max
  • 250 GB RAM 24 vCPU

2. Install the dependencies

!pip install -qU \
    transformers==4.36.1 \
    accelerate==0.25.0 \
    duckduckgo_search==4.1.0