Private AI Backend Service

---

Topos

A personal intelligence service, using your own computer to power private conversations with friends, family, and coworkers, collect/store your own private data, and use AI privately. Runs great with the chat arena app available on desktop and mobile.

Tech: ollama, postgres, FastAPI, nixOS, huggingface-transformers

(MacOS) Easy Install With .dmg

(Experimental): This is new, and should work on most MacOS machines! Simply double click the topos.dmg file, and drag the app into your Applications directory. You should be able to launch the Topos service anywhere from your machine.

Install with nix (Recommended)

If nix is not installed:

1. Install nix: macos/linux: sh <(curl -L https://nixos.org/nix/install) windows: sh <(curl -L https://nixos.org/nix/install) --daemon
2. Run Topos and all its dependencies:

   nix run github:jonnyjohnson1/topos-cli
   

   This will start all services including Topos, Postgres, Kafka, and Ollama.

Development

Clone the repository:

git clone https://github.com/jonnyjohnson1/topos-cli
cd topos-cli

For development, you have several options:

Build Binary

First build topos binary (only usable on machines with nix installed)

nix build .#topos

run built binary

./result/bin/topos

Dev Shell

nix develop
topos run

Poetry Shell

nix develop .#poetry

Install Instructions

requires brew install just requires brew install poetry

Graph Database - Install Neo4j

Install Neo4j Database on Osx

brew install neo4j brew services start neo4j

Topos

Step 1: Install Topos

install the topos package with the command just build

Step 2: Set the Spacy Model Size

Set the size of the spacy model you wish to use on your system. There are 'small', 'med', 'large', and 'trf'.

Use the tag like this. topos set --spacy small topos set --spacy trf

Step 3: Start Topos on your local machine

topos run topos run --local

Step 4a (zrok): Set up web proxy

We are going to expose our backend service to a public network so our phone/tablet can use it. In this case, we use zrok. Below is are the guides to set up ngrok.

zrok is opensourced and free. ngrok has a gated requests/month under its free tier, then requires you pay for it.

1. Be sure you have the topos server running already in another terminal.
2. Install zrok command
3. zrok enable <given_key>
4. zrok status should show you information
5. Route local path through zrok: zrok share public http://0.0.0.0:13341 This will take you to a new screen with an https:// at the top. Insert this url into the field under settings-> "Api Endpoints" -> "Custom API"
6. After you've insert it into the field, press the test button, and "hello world" should appear next to the button.

[ ] Enable permanent sharing of zrok url docs (requires Docker)

Step 4b (ngrok): Set up web proxy