[Bug]: Update from 2.5.0, restart setup, most data present

[dennisvandehoef]

🔍 Bug Summary

Partial data los on update

📖 Description

I updated from 2.5.0 to 2.6.4.
After restarting my docker container I was guided to the new setup page (wich looks good by the way).

Klicking trough the tabs, makes me realise that most settings (openAI key, prompt, processed documents) seem to be present.

Except for:

• username/password
• Enable/Disable AI Functions

docker-compose file, as you can see I store the data outside of the container.

  paperless-ai:
    image: clusterzx/paperless-ai
    container_name: paperless-ai
    restart: unless-stopped
    depends_on:
      - paperless
    cap_drop:
      - ALL
    security_opt:
      - no-new-privileges=true
    environment:
      - PUID=1000
      - PGID=1000
    ports:
      - 8001:3000
    volumes:
      - ./paperless-ai/data:/app/data

🔄 Steps to Reproduce

Update from 2.5.0 to 2.6.4.

✅ Expected Behavior

No data loss

❌ Actual Behavior

Small data los

📜 Docker Logs

paperless-ai  | 2025-02-15T14:10:34: PM2 log: Launching in no daemon mode
paperless-ai  | 2025-02-15T14:10:34: PM2 log: App [paperless-ai:0] starting in -cluster mode-
paperless-ai  | 2025-02-15T14:10:34: PM2 log: App [paperless-ai:0] online
paperless-ai  | Loading .env from: /app/data/.env
paperless-ai  | Loaded environment variables: {
paperless-ai  |   PAPERLESS_API_URL: 'http://paperless:8000/api',
paperless-ai  |   PAPERLESS_API_TOKEN: '---REDACTED----',
paperless-ai  |   LIMIT_FUNCTIONS: {
paperless-ai  |     activateTagging: 'yes',
paperless-ai  |     activateCorrespondents: 'yes',
paperless-ai  |     activateDocumentType: 'yes',
paperless-ai  |     activateTitle: 'yes',
paperless-ai  |     activateCustomFields: 'no'
paperless-ai  |   }
paperless-ai  | }
paperless-ai  | Server running on port 3000
paperless-ai  | Validating Paperless config for: http://paperless:8000/api/documents/
paperless-ai  | Paperless validation error: connect ECONNREFUSED 172.18.0.7:8000
paperless-ai  | Configuration validation failed: Invalid Paperless configuration
paperless-ai  | Setup not completed. Visit http://your-ip-or-host.com:3000/setup to complete setup.
paperless-ai  | Current config TAGS: []
paperless-ai  | Current config PROMPT_TAGS: []
paperless-ai  | Current config TAGS: []
paperless-ai  | Current config PROMPT_TAGS: []
paperless-ai  | Setup request received: {
paperless-ai  |   username: 'dennis',
paperless-ai  |   password: '---REDACTED----',
paperless-ai  |   confirmPassword: '---REDACTED----',
paperless-ai  |   paperlessUrl: 'http://paperless:8000',
paperless-ai  |   paperlessToken: '---REDACTED----',
paperless-ai  |   paperlessUsername: 'dennis',
paperless-ai  |   aiProvider: 'openai',
paperless-ai  |   openaiKey: '---REDACTED----',
paperless-ai  |   openaiModel: 'gpt-4o-mini',
paperless-ai  |   ollamaUrl: 'http://localhost:11434',
paperless-ai  |   ollamaModel: 'llama3.2',
paperless-ai  |   customBaseUrl: '',
paperless-ai  |   customApiKey: '',
paperless-ai  |   customModel: '',
paperless-ai  |   useExistingData: 'yes',
paperless-ai  |   scanInterval: '*/30 * * * *',
paperless-ai  |   showTags: 'no',
paperless-ai  |   tags: '',
paperless-ai  |   aiProcessedTag: 'no',
paperless-ai  |   aiTagName: 'ai-processed',
paperless-ai  |   usePromptTags: 'no',
paperless-ai  |   promptTags: '',
paperless-ai  |   activateTagging: 'on',
paperless-ai  |   activateCorrespondents: 'on',
paperless-ai  |   activateDocumentType: 'on',
paperless-ai  |   activateTitle: 'on',
paperless-ai  |   customFields: '{"custom_fields":[]}',
paperless-ai  |   systemPrompt: 'You are a personalized document analyzer. Your task is to analyze documents and extract relevant information.\n' +
paperless-ai  |     '\n' +
paperless-ai  |     'Analyze the document content and extract the following information into a structured JSON object:\n' +
paperless-ai  |     '\n' +
paperless-ai  |     '1. title: Create a concise, meaningful title for the document\n' +
paperless-ai  |     '2. correspondent: Identify the sender/institution but do not include addresses\n' +
paperless-ai  |     '3. tags: Select up to 3 relevant thematic tags\n' +
paperless-ai  |     '4. document_date: Extract the document date (format: YYYY-MM-DD)\n' +
paperless-ai  |     '5. document_type: Determine a precise type that classifies the document (e.g. Invoice, Contract, Employer, Information and so on)\n' +
paperless-ai  |     '      \n' +
paperless-ai  |     'Important rules for the analysis:\n' +
paperless-ai  |     '\n' +
paperless-ai  |     'For tags:\n' +
paperless-ai  |     '- FIRST check the existing tags before suggesting new ones\n' +
paperless-ai  |     '- preffer known tags over creating a new one\n' +
paperless-ai  |     '- Use only relevant categories\n' +
paperless-ai  |     '- Maximum 3 tags per document, less if sufficient (at least 1)\n' +
paperless-ai  |     '- Avoid too general or too specific tags\n' +
paperless-ai  |     '- Use only the most important information for tag creation\n' +
paperless-ai  |     "- don't use date, year or cities as a tag\n" +
paperless-ai  |     '- never return a known correspondent as tag\n' +
paperless-ai  |     '- tags should fit the document type, for example the topic Datenschutz might be in a document often, but still is not relevant\n' +
paperless-ai  |     '\n' +
paperless-ai  |     'For the title:\n' +
paperless-ai  |     '- Short and concise, NO ADDRESSES\n' +
paperless-ai  |     '- Contains the most important identification features\n' +
paperless-ai  |     '- For invoices/orders, mention invoice/order number if available\n' +
paperless-ai  |     '\n' +
paperless-ai  |     'For the correspondent:\n' +
paperless-ai  |     '- FIRST check and use the known correspondents before suggesting new ones\n' +
paperless-ai  |     '- Identify the sender or institution\n' +
paperless-ai  |     '- When generating the correspondent, always create the shortest possible form of the company name (e.g. "Amazon" instead of "Amazon EU SARL, German branch")\n' +
paperless-ai  |     '- do not use an e-maill adress or domain as corospondent\n' +
paperless-ai  |     "- don't add VVaG, GmbH and other operating forms\n" +
paperless-ai  |     '\n' +
paperless-ai  |     'For the document date:\n' +
paperless-ai  |     '- Extract the date of the document\n' +
paperless-ai  |     '- Use the format YYYY-MM-DD\n' +
paperless-ai  |     '- If multiple dates are present, use the most relevant one\n' +
paperless-ai  |     '\n' +
paperless-ai  |     'for all texts: The output language is german\n'
paperless-ai  | }
paperless-ai  | Validating Paperless config for: http://paperless:8000/api/documents/
paperless-ai  | Validating API permissions for http://paperless:8000/api/correspondents/
paperless-ai  | API permissions validated for correspondents, 200
paperless-ai  | Validating API permissions for http://paperless:8000/api/tags/
paperless-ai  | API permissions validated for tags, 200
paperless-ai  | Validating API permissions for http://paperless:8000/api/documents/
paperless-ai  | API permissions validated for documents, 200
paperless-ai  | Validating API permissions for http://paperless:8000/api/document_types/
paperless-ai  | API permissions validated for document_types, 200
paperless-ai  | Validating API permissions for http://paperless:8000/api/custom_fields/
paperless-ai  | API permissions validated for custom_fields, 200
paperless-ai  | Validating API permissions for http://paperless:8000/api/users/
paperless-ai  | API permissions validated for users, 200
paperless-ai  | [DEBUG] [15.02.25, 14:24] OpenAI request sent
paperless-ai  | Validating Paperless config for: http://paperless:8000/api/documents/
paperless-ai  | [DEBUG] [15.02.25, 14:24] OpenAI request sent
paperless-ai  | [DEBUG] User dennis added
paperless-ai  | 2025-02-15T14:24:24: PM2 log: App name:paperless-ai id:0 disconnected
paperless-ai  | 2025-02-15T14:24:24: PM2 log: App [paperless-ai:0] exited with code [0] via signal [SIGINT]
paperless-ai  | 2025-02-15T14:24:24: PM2 log: App [paperless-ai:0] will restart in 100ms
paperless-ai  | 2025-02-15T14:24:24: PM2 log: App [paperless-ai:0] starting in -cluster mode-
paperless-ai  | 2025-02-15T14:24:24: PM2 log: App [paperless-ai:0] online
paperless-ai  | Loading .env from: /app/data/.env
paperless-ai  | Loaded environment variables: {
paperless-ai  |   PAPERLESS_API_URL: 'http://paperless:8000/api',
paperless-ai  |   PAPERLESS_API_TOKEN: '---REDACTED----',
paperless-ai  |   LIMIT_FUNCTIONS: {
paperless-ai  |     activateTagging: 'yes',
paperless-ai  |     activateCorrespondents: 'yes',
paperless-ai  |     activateDocumentType: 'yes',
paperless-ai  |     activateTitle: 'yes',
paperless-ai  |     activateCustomFields: 'no'
paperless-ai  |   }
paperless-ai  | }
paperless-ai  | Server running on port 3000
paperless-ai  | Validating Paperless config for: http://paperless:8000/api/documents/
paperless-ai  | [DEBUG] [15.02.25, 14:24] OpenAI request sent
paperless-ai  | [DEBUG] Found own user ID: 3
paperless-ai  | Configured scan interval: */30 * * * *
paperless-ai  | Starting initial scan at 2025-02-15T14:24:26.935Z
paperless-ai  | [DEBUG] Found own user ID: 3
paperless-ai  | [DEBUG] Fetched page 1, got 100 tags. [DEBUG] Total so far: 100
paperless-ai  | [DEBUG] Fetched page 1, got 100 documents. [DEBUG] Total so far: 100
paperless-ai  | [DEBUG] Fetched page 2, got 74 tags. [DEBUG] Total so far: 174
paperless-ai  | [DEBUG] Fetched page 2, got 100 documents. [DEBUG] Total so far: 200
paperless-ai  | [DEBUG] Fetched page 3, got 100 documents. [DEBUG] Total so far: 300
paperless-ai  | [DEBUG] Fetched page 4, got 40 documents. [DEBUG] Total so far: 340
paperless-ai  | [DEBUG] Finished fetching. Found 340 documents.
paperless-ai  | Login attempt for user: dennis
paperless-ai  | Password validation result: true
paperless-ai  | 2025-02-15T14:25:04: PM2 log: [PM2][WORKER] Reset the restart delay, as app paperless-ai has been up for more than 30000ms

📜 Paperless-ngx Logs

🖼️ Screenshots of your settings page

No response

🖥️ Desktop Environment

Other

💻 OS Version

No response

🌐 Browser

None

🔢 Browser Version

No response

🌐 Mobile Browser

No response

📝 Additional Information

• I have checked existing issues and this is not a duplicate
• I am sure that this problem is affecting everyone, not only me
• I have tried debugging this issue on my own
• I can provide a fix and submit a PR

📌 Extra Notes

No response

[clusterzx]

You should not need to setup the app again after update. I can see in your logs that the setup process popped off because it could not connect to paperless-ngx. Make sure to run paperless-ai only after paperless-ngx is up and running and reachable.

[dennisvandehoef]

Whould it not be better to show an error in that case? Instead of guiding into setup again?
Offering the user to setup again if needed.