Skip to content

Home

Jump to bottom
ThiloteE edited this page Aug 28, 2024 · 16 revisions

GPT4All Wiki

GPT4All - Your private local LLM environment, brought to you by NOMIC.

Welcome to the GPT4All Wiki! We're excited to bring you an open-source project that allows you to run large language models (LLMs) privately on your own computer. With GPT4All, you can chat with LLMs and integrate them into your workflow without relying on cloud services.

System Requirements & Installation

What Operating Systems are supported?

GPT4All is designed for Windows, macOS, and Linux users.

What are the minimum requirements?

  • CPU: GPT4All installers require your CPU has AVX/AVX2 instruction sets.
  • Resolution: You need a display resolution of at least 1280x720.
  • Memory: At least 8 GB of system RAM.
  • OS: A recent Operating System...
    • Windows 10 or later
    • macOS High Sierra (10.13) or later
    • Ubuntu 22.04 LTS or later

What hardware is recommended?

RAM

Have enough of it, because ...

  • the large language model (LLM) should fit into RAM completely. Reason being: Trying to load a model that does not fit into your RAM triggers your machine to utilize the swap space (assuming there is one) on your harddrive (SSD/HDD) and that will slow down speed of inference substantially. In short: RAM is faster than your harddrive (HDD/SSD).
  • chatting with the model adds to the context, which is mapped into RAM. The longer the conversation, the more RAM is required.
  • more RAM will allow you to run larger models with larger context.

GPU

Have one with lots of VRAM, because ...

  • GPU are very fast at inferenceing LLMs and in most cases faster than a regular CPU / RAM combo.
  • We recommend at least 8GB of VRAM.

Have one that is supported by the GPU backends:

  • Nvidia
    • CUDA backend
      • will run any .gguf quantized models.
      • available for the LocalDocs feature
    • Vulkan Backend
      • will run .gguf quantized models of fp16, Q4_0, Q4_1.
  • AMD
    • Vulkan Backend
      • will run .gguf quantized models of fp16, Q4_0, Q4_1.

Feature matrix:

CPU (AVX/AVX2) CPU (ARM NEON) Metal Vulkan/Kompute Cuda
GGUF
Q4_0, Q4_1 & F16		 ✅ 🚫 ✅ ✅ ✅
GGUF
K-quants		 ✅ 🚫 ✅ 🚫 ✅
GGUF
I-quants		 ✅ 🐢 🚫 ✅ 🚫 ✅🐢
GGUF
K cache quants		 ❓ 🚫 ❓ ❓ ❓
Multi-GPU N/A N/A ❓ ❓ ❓
  • ✅: feature works
  • 🚫: feature does not work
  • ❓: unknown, please contribute if you can test it youself
  • 🐢: feature is slow

Where can I find the Installers?

Need more help?

We're here to help!

Clone this wiki locally