roadmap: Jan refactors /messages, /threads to Cortex

[dan-menlo]

Goal

• Jan should be a thin UI layer
• Cortex should be an API Platform that has full OpenAI compatibility
• We need to itemize and shift functionality to Cortex (e.g. Threads, Messages)

From my POV, Jan should only be a UI and "Packaging" layer (i.e. Mobile, Desktop, Server)
All other API functionality and state should shift to Cortex

Tasklist

Frontend

• Itemize functionality that needs to shift to Cortex

Backend

• epic: Jan integrates Cortex.cpp #3825
• planning: Migrate Threads, Messages to Cortex, deprecates Conversation Extension #3904
• roadmap: Cortex API supports /threads, /messages cortex.cpp#1567

Administrative

• Dan/Nicole to link Internal/Admin issues

Product Marketing

• @eckartal to link Content Calendar issues

Resources

[gabrielle-ong]

WIP - Preliminary notes from last week with Louis

Model Hub

• Updated models, delist
• Jan dogfoods this to display on UI
• future: eg Cortex remote models list

View RAM, CPU usage

• requires reimplementing cortex ps for each engine
• Can be deployed first but currently combines all engines together resulting in inaccurate

Hardware, eg GPU support

• Cortex.cpp planning: Cortex Hardware API #1165
• Major Epic, needs thorough planning
• cortex.cpp needs hardware detection, currently doesnt support multi-GPU
• Major usage for Jan

Support Tensorrt-llm engines [Prioritised?]

• Need to define Model artifact, model.yml
• need to bundle artifacts (needs investigating)

multi-modal - cortex.cpp needs to be able to pull 2 models at the same time

• Need to define model.yml for the models

Reference discussions:

• planning: Jan's path to cortex.cpp? #3690

[imtuyethan]

• Bugs will be scoped in QA: Jan 0.5.12 Release Sign-off #4291 if there's any