API needed for ollama

[abdessalaam]

My ollama is running remotely so it is secured by an API key. ( in the format Authorization: Bearer YOUR_KEY).
Would it be possible to add it in a similar way to how OPEN_AI key is specified in .env?

Thanks!

[icereed]

Interesting use case. I just checked the library that I use for Ollama. It should be possible. Will try to implement it within the next week.

[abdessalaam]

Thank you!

[icereed]

@abdessalaam I created a custom Docker image icereed/paperless-gpt:ollama-bearer with the changes of #4.
You can specify the bearer token via OLLAMA_BEARER_TOKEN. Can you test this as I don't have a setup for this?

[abdessalaam]

@icereed thank you!
I have the following error:

[DEBUG] POST https://my.ollama-api.url/api/chat
paperless-gpt        | 2024-10-04T10:48:53.599893490Z 2024/10/04 10:48:53 Request with bearer Beare... token to POST https://my.ollama-api.url/api/chat
paperless-gpt        | 2024-10-04T10:48:53.699604567Z 2024/10/04 10:48:53 Error processing document 4: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.699805982Z 2024/10/04 10:48:53 Error processing document 6: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.724979380Z 2024/10/04 10:48:53 Error processing document 7: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.726294368Z 2024/10/04 10:48:53 Error processing document 11: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.727619895Z 2024/10/04 10:48:53 Error processing document 10: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.728729478Z 2024/10/04 10:48:53 Error processing document 9: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.729307234Z 2024/10/04 10:48:53 Error processing document 1: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.729416081Z 2024/10/04 10:48:53 Error processing document 8: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.729442490Z 2024/10/04 10:48:53 Error processing documents: Document 4: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T10:48:53.729456560Z [GIN] 2024/10/04 - 10:48:53 | 500 |  202.632935ms |  xx.xx.xx.xx | POST     "/api/generate-suggestions"

[icereed]

@abdessalaam it seems the hook that I used wasn't working as expected. I implemented now a custom http client to inject the bearer token. I added also a unit test for that use case. Please pull again the same image and kindly run your test again. 🚀

[abdessalaam]

I think the bearer token call works, but perhaps the response is nor formatted correctly? When I run it with open-ai it works fine (with the same docker image).

paperless-gpt        | 2024-10-04T12:54:12.035667778Z [GIN] 2024/10/04 - 12:54:12 | 200 |   109.98123ms |  87.236.142.199 | GET      "/api/documents"
paperless-gpt        | 2024-10-04T12:54:15.677965074Z [GIN] 2024/10/04 - 12:54:15 | 200 |      64.909µs |  87.236.142.199 | GET      "/"
paperless-gpt        | 2024-10-04T12:54:15.840781277Z [GIN] 2024/10/04 - 12:54:15 | 200 |     603.845µs |  87.236.142.199 | GET      "/assets/index-CcrhXiwk.js"
paperless-gpt        | 2024-10-04T12:54:15.977299806Z [GIN] 2024/10/04 - 12:54:15 | 200 |     255.784µs |  87.236.142.199 | GET      "/assets/index-01ocH0eB.css"
paperless-gpt        | 2024-10-04T12:54:16.294315534Z [GIN] 2024/10/04 - 12:54:16 | 200 |      36.709µs |  87.236.142.199 | GET      "/api/filter-tag"
paperless-gpt        | 2024-10-04T12:54:16.401458944Z [GIN] 2024/10/04 - 12:54:16 | 200 |  105.826652ms |  87.236.142.199 | GET      "/api/documents"
paperless-gpt        | 2024-10-04T12:54:16.465867553Z [GIN] 2024/10/04 - 12:54:16 | 200 |   32.042114ms |  87.236.142.199 | GET      "/api/tags"
paperless-gpt        | 2024-10-04T12:54:16.627675041Z [GIN] 2024/10/04 - 12:54:16 | 200 |      97.807µs |  87.236.142.199 | GET      "/vite.svg"
paperless-gpt        | 2024-10-04T12:54:31.939603700Z 2024/10/04 12:54:31 Using bearer token for OLLAMA authentication
paperless-gpt        | 2024-10-04T12:54:32.010514120Z 2024/10/04 12:54:32 Processing Document 1...
paperless-gpt        | 2024-10-04T12:54:32.010565068Z 2024/10/04 12:54:32 Processing Document 3...
paperless-gpt        | 2024-10-04T12:54:32.116136167Z 2024/10/04 12:54:32 Error processing document 3: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T12:54:32.116330432Z 2024/10/04 12:54:32 Error processing document 1: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T12:54:32.116353242Z 2024/10/04 12:54:32 Error processing documents: Document 3: Error getting response from LLM: invalid character '<' looking for beginning of value
paperless-gpt        | 2024-10-04T12:54:32.116354162Z [GIN] 2024/10/04 - 12:54:32 | 500 |  177.019855ms |  xx.xx.xx.xx | POST     "/api/generate-suggestions" 

[icereed]

Can you explain your setup a bit more? Is there some reverse proxy like NGINX in front of Ollama? Maybe I can replicate the setup locally to test.

[abdessalaam]

I have implemented this llamatunnel so I run ollama with OpenWebUI on my local mac, but then make them available through a cloudflare tunnel over the internet, so I can connect to my local models from anywhere, as long as my mac is on.
Since that setup already created an api endpoint for ollama (with an added API token, because it is exposed over the internet) I thought I could use this API point.
My paperless (and paperless-gpt) are on my server, and there is no ollama running on that server – my mac is a little bit more powerful machine, so I only run ollama locally.

[icereed]

Interesting. As a workaround: Can you run paperless-gpt on your machine and let it interface with the paperless instance on your server?

[abdessalaam]

Ok, I will try that!

[icereed]

@abdessalaam did it work out for you? I don't want to leave you hanging.

[abdessalaam]

@abdessalaam did it work out for you? I don't want to leave you hanging.

Thank you for checking 😀 It works this way!

[icereed]

Awesome 👏