planning: Cortex.cpp features needed to fully support Jan

[gabrielle-ong]

WIP list

Model Hub

• Updated models, delist
• Jan dogfoods this to display on UI
• future: eg Cortex remote models list

View RAM, CPU usage

• requires reimplementing cortex ps for each engine
• Can be deployed first but currently combines all engines together resulting in inaccurate

Hardware, eg GPU support

• planning: Cortex Hardware API #1165
• Major Epic, needs thorough planning
• cortex.cpp needs hardware detection, currently doesnt support multi-GPU
• Major usage for Jan

Support Tensorrt-llm engines [Prioritised?]

• Need to define Model artifact, model.yml
• need to bundle artifacts (needs investigating)

multi-modal - cortex.cpp needs to be able to pull 2 models at the same time

• Need to define model.yml for the models

To add (priority, needed, blockers, effort)

[gabrielle-ong]

Closing in favour of Jan issues

• 3895
• planning: Jan's path to cortex.cpp? jan#3690