Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit 82812a6

Browse files
hiento09hiento09
committed
docs: installation linux instruction
1 parent 7e83ab6 commit 82812a6

File tree

4 files changed

+175
-106
lines changed

4 files changed

+175
-106
lines changed

‎docs/docs/installation.mdx

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
title: Installation
3+
description: Cortex Installation
4+
slug: "installation"
5+
---
6+
7+
import Tabs from '@theme/Tabs';
8+
import TabItem from '@theme/TabItem';
9+
import Admonition from '@theme/Admonition';
10+
11+
:::warning
12+
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
13+
:::
14+
15+
## Cortex.cpp Installation
16+
### Cortex.cpp offers four installer types
17+
- Network Installers download a minimal system and require an internet connection to fetch packages during installation.
18+
- Local Installers include all necessary packages, enabling offline installation without internet access.
19+
- Dockerfile Installers are used to build a Docker image for Cortex.
20+
- Binary files without package management.
21+
22+
### Cortex.cpp supports three channels
23+
- Stable: The latest stable release on github.
24+
- Beta: The release candidate for the next stable release, available on github release with the tag `vx.y.z-rc1`
25+
- Nightly: The nightly build of the latest code on dev branch, available on [discord](https://discord.com/channels/1107178041848909847/1283654073488379904).
26+
27+
### Download URLs
28+
29+
| Type | Version | Operating System | Download Link |
30+
|----------|---------------|------------------|---------------------------------------------------------------------------------------------------|
31+
| **Local** | **Stable** | **Windows** | [Download](https://app.cortexcpp.com/download/latest/windows-amd64-local) |
32+
| **Local** | **Stable** | **Linux** | [Download](https://app.cortexcpp.com/download/latest/linux-amd64-local) |
33+
| **Local** | **Stable** | **MacOS** | [Download](https://app.cortexcpp.com/download/latest/mac-universal-local) |
34+
35+
:::info
36+
For other versions, please look at [cortex.cpp repo](https://github.com/janhq/cortex.cpp) or each installer page.
37+
:::
38+
39+
40+
## Minimum Requirements
41+
42+
### OS
43+
- MacOS 12 or later
44+
- Windows 10 or later
45+
- Linux: Ubuntu 20.04 or later, Debian 11 or later (For other distributions, please use the Dockerfile installer or binary files, we have not tested on other distributions yet.)
46+
47+
### Hardware
48+
#### CPU
49+
:::info
50+
- Cortex.cpp supports a processor that can handle AVX2. For the full list, please see [here](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2).
51+
- We support older processors with AVX, AVX-512 and non-AVX, though this is not recommended.
52+
:::
53+
54+
##### Intel CPU
55+
- Haswell processors (Q2 2013) and newer.
56+
- Tiger Lake (Q3 2020) and newer for Celeron and Pentium processors.
57+
##### AMD CPU
58+
- Excavator processors (Q2 2015) and newer.
59+
60+
#### RAM
61+
:::info
62+
We support DDR2 RAM as the minimum requirement but recommend using newer generations of RAM for improved performance.
63+
:::
64+
- 8GB for running up to 3B models (int4).
65+
- 16GB for running up to 7B models (int4).
66+
- 32GB for running up to 13B models (int4).
67+
68+
#### GPU
69+
:::info
70+
Having at least 6GB VRAM when using NVIDIA, AMD, or Intel Arc GPUs is recommended.
71+
:::
72+
- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
73+
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
74+
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
75+
:::info
76+
- [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) version 470.63.01 or higher.
77+
*Note: Users must install the driver by themselves.*
78+
- [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) version 12.0 or higher. *Note: Cortex.cpp will automatically detect and install the required version of cudart to the user's machine.*
79+
:::
80+
#### Disk
81+
- At least 10GB for app storage and model download.

‎docs/docs/installation/docker.mdx

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,26 @@
1-
21
---
32
title: Docker
43
description: Install Cortex using Docker.
54
---
65

6+
import Tabs from '@theme/Tabs';
7+
import TabItem from '@theme/TabItem';
8+
import Admonition from '@theme/Admonition';
9+
710
:::warning
811
🚧 **Cortex.cpp is currently in development.** The documentation describes the intended functionality, which may not yet be fully implemented.
912
:::
1013

11-
# Setting Up Cortex with Docker
14+
## Setting Up Cortex with Docker
1215

1316
This guide walks you through the setup and running of Cortex using Docker.
1417

15-
## Prerequisites
18+
### Prerequisites
1619

1720
- Docker or Docker Desktop
1821
- `nvidia-container-toolkit` (for GPU support)
1922

20-
## Setup Instructions
23+
### Setup Instructions
2124

2225
1. **Clone the Cortex Repository**
2326
```bash
@@ -64,11 +67,11 @@ This guide walks you through the setup and running of Cortex using Docker.
6467
cortex --help
6568
```
6669

67-
## Usage
70+
### Usage
6871

6972
With Docker running, you can use the following commands to interact with Cortex. Ensure the container is running and `curl` is installed on your machine.
7073

71-
### 1. List Available Engines
74+
#### 1. List Available Engines
7275

7376
```bash
7477
curl --request GET --url http://localhost:39281/v1/engines --header "Content-Type: application/json"
@@ -98,7 +101,7 @@ curl --request GET --url http://localhost:39281/v1/engines --header "Content-Typ
98101
}
99102
```
100103

101-
### 2. Pull Models from Hugging Face
104+
#### 2. Pull Models from Hugging Face
102105

103106
- Open a terminal and run `websocat ws://localhost:39281/events` to capture download events, follow [this instruction](https://github.com/vi/websocat?tab=readme-ov-file#installation) to install `websocat`.
104107
- In another terminal, pull models using the commands below.
@@ -118,7 +121,7 @@ curl --request GET --url http://localhost:39281/v1/engines --header "Content-Typ
118121
curl --request GET --url http://localhost:39281/v1/models
119122
```
120123

121-
### 3. Start a Model and Send an Inference Request
124+
#### 3. Start a Model and Send an Inference Request
122125

123126
- **Start the model:**
124127
```bash
@@ -140,7 +143,7 @@ curl --request GET --url http://localhost:39281/v1/engines --header "Content-Typ
140143
}'
141144
```
142145

143-
### 4. Stop a Model
146+
#### 4. Stop a Model
144147

145148
- To stop a running model, use:
146149
```bash

‎docs/docs/installation/linux.mdx

Lines changed: 78 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -13,119 +13,102 @@ import Admonition from '@theme/Admonition';
1313
:::
1414

1515
## Cortex.cpp Installation
16-
Cortex.cpp offers two installer types:
17-
- Network Installers download a minimal system and require an internet connection to fetch packages during installation.
18-
- Local Installers include all necessary packages, enabling offline installation without internet access.
1916
:::info
20-
Before installation, make sure that you have met the required [dependencies](#dependencies) and [hardware](#hardware) to run Cortex.
17+
Before installation, make sure that you have met the [minimum requirements](/docs/installation#minimum-requirements) to run Cortex.
18+
This instruction is for stable releases. For beta and nightly releases, please replace `cortex` with `cortex-beta` and `cortex-nightly`, respectively.
2119
:::
22-
1. Download the Linux installer:
23-
- `.deb`:
24-
- Stable: https://github.com/janhq/cortex.cpp/releases
25-
- Beta: https://github.com/janhq/cortex.cpp/releases
26-
- Nightly: https://github.com/janhq/cortex.cpp/releases
27-
- `.appImage`:
28-
- Stable: https://github.com/janhq/cortex.cpp/releases
29-
- Beta: https://github.com/janhq/cortex.cpp/releases
30-
- Nightly: https://github.com/janhq/cortex.cpp/releases
31-
32-
2. Ensure that Cortex.cpp is sucessfulyy installed:
33-
```bash
34-
# Stable
35-
cortex
3620

37-
# Beta
38-
cortex-beta
21+
### Prerequisites
22+
- OpenMPI
3923

40-
# Nightly
41-
cortex-nightly
42-
```
24+
### Install Cortex.cpp
25+
26+
1. Download the Linux installer:
27+
- From release: https://github.com/janhq/cortex.cpp/releases
28+
- From quick download links:
29+
- Local installer `.deb`:
30+
- Stable: https://app.cortexcpp.com/download/latest/linux-amd64-local
31+
- Beta: https://app.cortexcpp.com/download/beta/linux-amd64-local
32+
- Nightly: https://app.cortexcpp.com/download/nightly/linux-amd64-local
33+
- Network installer `.deb`:
34+
- Stable: https://app.cortexcpp.com/download/latest/linux-amd64-network
35+
- Beta: https://app.cortexcpp.com/download/beta/linux-amd64-network
36+
- Nightly: https://app.cortexcpp.com/download/nightly/linux-amd64-network
37+
- Binary:
38+
- Stable: https://app.cortexcpp.com/download/latest/linux-amd64-binary
39+
- Beta: https://app.cortexcpp.com/download/beta/linux-amd64-binary
40+
- Nightly: https://app.cortexcpp.com/download/nightly/linux-amd64-binary
41+
42+
2. Install Cortex.cpp using the following command:
43+
```bash
44+
# Installer
45+
sudo apt install ./cortex-<version>-linux-amd64-network-installer.deb
46+
47+
48+
# Binary
49+
tar -xvf cortex-<version>-linux-amd64.tar.gz
50+
cd cortex
51+
sudo mv cortex /usr/bin/cortex
52+
sudo chmod +x /usr/bin/Cortexs
53+
sudo mv cortex-server /usr/bin/cortex-server
54+
55+
## For binary, you need to install engine manually after extracting the binary
56+
cortex engines install llama-cpp
57+
```
58+
59+
3. Ensure that Cortex.cpp is sucessfulyy installed:
60+
```bash
61+
# Stable
62+
cortex -v
63+
```
4364

4465
### Data Folder
4566
By default, Cortex.cpp is installed in the following directory:
4667
```
4768
# Binary Location
48-
/usr/bin/cortexcpp<env>
69+
/usr/bin/cortex
70+
/usr/bin/cortex-server
4971
5072
# Application Data (Engines, Models and Logs folders)
51-
/home/<username>/.cortexcpp<env>
73+
/home/<username>/cortexcpp
74+
75+
# Configuration File
76+
/home/<username>/.cortexrc
5277
```
78+
5379
## Uninstall Cortex.cpp
5480
```bash
5581
# Stable version
5682
sudo apt remove cortexcpp
57-
58-
# Beta version
59-
sudo apt remove cortexcpp-beta
60-
61-
# Nightly version
62-
sudo apt remove cortexcpp-nightly
6383
```
6484

6585
## Build from Source
66-
1. Clone the Cortex.cpp repository [here](https://github.com/janhq/cortex.cpp).
67-
2. Navigate to the `engine > vcpkg` folder.
68-
3. Configure the vpkg:
6986

70-
```bash
71-
cd vcpkg
72-
./bootstrap-vcpkg.sh
73-
vcpkg install
74-
```
75-
4. Build the Cortex.cpp inside the `build` folder:
76-
77-
```bash
78-
mkdir build
79-
cd build
80-
cmake .. -DCMAKE_TOOLCHAIN_FILE=path_to_vcpkg_folder/vcpkg/scripts/buildsystems/vcpkg.cmake
81-
make -j4
82-
```
83-
5. Use Visual Studio with the C++ development kit to build the project using the files generated in the `build` folder.
84-
6. Verify that Cortex.cpp is installed correctly by getting help information.
85-
86-
```sh
87-
# Get the help information
88-
cortex -h
89-
```
90-
91-
## Prerequisites
92-
### Dependencies
93-
- Node.js version 18 and higher
94-
- NPM
95-
- Homebrew
87+
### Prerequisites
9688
- OpenMPI
97-
98-
### Hardware
99-
#### Operating System
100-
- Debian-based (Supports `.deb` and `AppImage` )
101-
- Ubuntu version 22.04 and higher
102-
#### CPU
103-
:::info
104-
- Cortex.cpp supports a processor that can handle AVX2. For the full list, please see [here](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2).
105-
- We support older processors with AVX and AVX-512, though this is not recommended.
106-
:::
107-
##### Intel CPU
108-
- Haswell processors (Q2 2013) and newer.
109-
- Tiger Lake (Q3 2020) and newer for Celeron and Pentium processors.
110-
##### AMD CPU
111-
- Excavator processors (Q2 2015) and newer.
112-
#### RAM
113-
:::info
114-
We support DDR2 RAM as the minimum requirement but recommend using newer generations of RAM for improved performance.
115-
:::
116-
- 8GB for running up to 3B models (int4).
117-
- 16GB for running up to 7B models (int4).
118-
- 32GB for running up to 13B models (int4).
119-
#### GPU
120-
:::info
121-
Having at least 6GB VRAM when using NVIDIA, AMD, or Intel Arc GPUs is recommended.
122-
:::
123-
- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
124-
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
125-
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
126-
:::info
127-
- [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) version 470.63.01 or higher.
128-
- [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) version 12.3 or higher.
129-
:::
130-
#### Disk
131-
- At least 10GB for app storage and model download.
89+
- CMake >= 3.10
90+
- gcc/g++ >= 9
91+
- ninja-build
92+
93+
### Build Cortex.cpp
94+
95+
1. **Clone the Cortex Repository**
96+
```bash
97+
git clone https://github.com/janhq/cortex.cpp.git
98+
cd cortex.cpp
99+
git submodule update --init
100+
```
101+
2. Build the Cortex.cpp :
102+
103+
```bash
104+
cd engine
105+
make configure-vcpkg
106+
make build CMAKE_EXTRA_FLAGS="-DCORTEX_CPP_VERSION=$(git rev-parse HEAD) -DCMAKE_BUILD_TEST=OFF -DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake
107+
```
108+
109+
3. Verify that Cortex.cpp is builded correctly by getting help information.
110+
111+
```sh
112+
# Get the help information
113+
./build/cortex -h
114+
```

‎docs/sidebars.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,8 @@ const sidebars: SidebarsConfig = {
3535
type: "category",
3636
label: "Installation",
3737
link: {
38-
type: "generated-index",
38+
type: "doc",
39+
id: "installation"
3940
},
4041
collapsed: true,
4142
items: [
@@ -68,7 +69,8 @@ const sidebars: SidebarsConfig = {
6869
type: "category",
6970
label: "Architecture",
7071
link: {
71-
type: "generated-index",
72+
type: "doc",
73+
id: "architecture"
7274
},
7375
collapsed: true,
7476
items: [

0 commit comments

Comments
 (0)