Jan: Jan and cortex.cpp Architecture

[dan-homebrew]

Overview

• Clear architecture needed to align team and community
• Jan, "Cortex Platform", and cortex.cpp
• Refactoring existing components powering Jan into smaller products

Ideal Outcome

• Jan is a UI
• Cortex Platform is a full DB-driven Application, that implements the full OpenAI API
• cortex.cpp is a C++ engine to run models, does not have a DB

Both cortex.cpp and Cortex Platform are embeddable in other apps, i.e. cortex.cpp -> Cortex Platform -> Jan

Related Issues

• Discussion: Jan Data Structures #3541
• epic: Change Jan API server to use cortex.cpp #3487
• feat: Jan supports most llama.cpp params #3508

[freelerobot]

Consider a user who uses both Cortex and Jan.
e.g. Uses cortex for extensions like continue.dev
e.g. Uses Jan UI as a personal assistant, for writing emails

• Can we they run both simultaneously?
• Is Cortex a shared process and port? OR does Jan run its own process/port of Cortex.
• How are model binaries shared?
• If a user downloads a model on Jan vs Cortex, where does it go? Does the other have access to it?

cc @dan-homebrew