Skip to content

Commit

Permalink
Merge pull request #1621 from janhq/docs/installation
Browse files Browse the repository at this point in the history
Docs: installation
  • Loading branch information
gabrielle-ong authored Nov 5, 2024
2 parents c44fb59 + a04a93a commit 0942fc1
Show file tree
Hide file tree
Showing 8 changed files with 515 additions and 362 deletions.
7 changes: 5 additions & 2 deletions docs/docs/architecture/cortexrc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,17 @@ import TabItem from "@theme/TabItem";
Cortex.cpp supports reading its configuration from a file called `.cortexrc`. Using this file, you can also change the data folder, Cortex.cpp API server port, and host.

## File Location

The configuration file is stored in the following locations:

- **Windows**: `C:\Users\<username>\.cortexrc`
- **Linux**: `/home/<username>/.cortexrc`
- **macOS**: `/Users/<username>/.cortexrc`

## Configuration Parameters

You can configure the following parameters in the `.cortexrc` file:
| Parameter | Description | Default Value |
| Parameter | Description | Default Value |
|------------------|--------------------------------------------------|--------------------------------|
| `dataFolderPath` | Path to the folder where `.cortexrc` located. | User's home folder. |
| `apiServerHost` | Host address for the Cortex.cpp API server. | `127.0.0.1` |
Expand All @@ -37,6 +39,7 @@ You can configure the following parameters in the `.cortexrc` file:
| `huggingFaceToken` | HuggingFace token. | Empty string |

Example of the `.cortexrc` file:

```
logFolderPath: /Users/<username>/cortexcpp
logLlamaCppPath: ./logs/cortex.log
Expand All @@ -49,4 +52,4 @@ apiServerPort: 39281
checkedForUpdateAt: 1730501224
latestRelease: v1.0.1
huggingFaceToken: ""
```
```
92 changes: 16 additions & 76 deletions docs/docs/architecture/data-folder.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Data Folder
description: Cortex.cpp's data folder.
title: Data Folder and App Folder
description: Cortex.cpp's data folder and app folder.
slug: "data-folder"
---

Expand All @@ -17,37 +17,25 @@ When you install Cortex.cpp, three types of files will be generated on your devi
- **Configuration Files**
- **Data Folder**

## Binary Files
## Binary Files - under the App Folder
These are the executable files of the Cortex.cpp application. The file format varies depending on the operating system:

- **Windows**: `.exe`
- Stable: `C:\Users\<username>\AppData\Local\cortexcpp\cortex.exe`
- Beta: `C:\Users\<username>\AppData\Local\cortexcpp-beta\cortex-beta.exe`
- Nighty: `C:\Users\<username>\AppData\Local\cortexcpp-nightly\cortex-nightly.exe`
- **Linux**: `.deb` or `.fedora`
- Stable: `/usr/bin/cortexcpp`
- Beta: `/usr/bin/cortexcpp-beta`
- Nighty: `/usr/bin/cortexcpp-nightly`
- **macOS**: `.pkg`
- Stable: `/usr/local/bin/cortexcpp`
- Beta: `/home/<username>/.cortexrc-beta`
- Nighty: `/home/<username>/.cortexrc-nightly`
- **Windows**:
- cli: `C:\Users\<username>\AppData\Local\cortexcpp\cortex.exe`
- server: `C:\Users\<username>\AppData\Local\cortexcpp\cortex-server.exe`
- **Linux**:
- cli: `/usr/bin/cortex`
- server: `/usr/bin/cortex-server`
- **macOS**:
- cli: `/usr/local/bin/cortex`
- server: `/usr/local/bin/cortex-server`

## Cortex.cpp Data Folder
The data folder stores the engines, models, and logs required by Cortex.cpp. This folder is located at:

- **Windows**:
- Stable: `C:\Users\<username>\.cortexcpp`
- Beta: `C:\Users\<username>\.cortexcpp-beta`
- Nighty: `C:\Users\<username>\.cortexcpp-nightly`
- **Linux**:
- Stable: `/home/<username>/.cortexcpp<env>`
- Beta: `/home/<username>/.cortexcpp-beta`
- Nighty: `/home/<username>/.cortexcpp-nightly`
- **macOS**:
- Stable: `/Users/<username>\.cortexcpp<env>`
- Beta: `/Users/<username>/.cortexcpp-beta`
- Nighty: `/Users/<username>/.cortexcpp-nightly`
- **Windows**: `C:\Users\<username>\cortexcpp`
- **Linux**: `/home/<username>/cortexcpp`
- **macOS**: `/Users/<username>\cortexcpp`

### Folder Structure
The Cortex.cpp data folder typically follows this structure:
Expand Down Expand Up @@ -77,57 +65,9 @@ The Cortex.cpp data folder typically follows this structure:
└── llamacpp
```
</TabItem>
<TabItem value="Beta" label="Beta">
```yaml
~/.cortex-beta
├── models/
│ └── model.list
│ └── huggingface.co/
│ └── <repo_name>/
└── <branch_name>/
└── model.yaml
└── model.gguf
│ └── cortex.so/
│ └── <repo_name>/
│ └── <branch_name>/
└── ...engine_files
└── model.yaml
│ └── imported/
└── imported_model.yaml
├── logs/
│ └── cortex.txt
└── cortex-cli.txt
└── engines/
└── llamacpp
```
</TabItem>
<TabItem value="Nightly" label="Nightly">
```yaml
~/.cortex-nightly
├── models/
│ └── model.list
│ └── huggingface.co/
│ └── <repo_name>/
└── <branch_name>/
└── model.yaml
└── model.gguf
│ └── cortex.so/
│ └── <repo_name>/
│ └── <branch_name>/
└── ...engine_files
└── model.yaml
│ └── imported/
└── imported_model.yaml
├── logs/
│ └── cortex.txt
└── cortex-cli.txt
└── engines/
└── llamacpp
```
</TabItem>
</Tabs>

#### `.cortexcpp<env>`
#### `cortexcpp`
The main directory that stores all Cortex-related files, located in the user's home directory.
#### `models/`
Contains the AI models used by Cortex for processing and generating responses.
Expand Down
109 changes: 109 additions & 0 deletions docs/docs/installation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
title: Installation
description: Cortex Installation
slug: "installation"
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';

:::warning
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

## Cortex.cpp Installation
### Cortex.cpp offers four installer types
- Network Installers download a minimal system and require an internet connection to fetch packages during installation.
- Local Installers include all necessary packages, enabling offline installation without internet access.
- Dockerfile Installers are used to build a Docker image for Cortex.
- Binary files without package management.

### Cortex.cpp supports three channels

- Stable: The latest stable release on github.
- Beta: The release candidate for the next stable release, available on github release with the tag `vx.y.z-rc1`
- Nightly: The nightly build of the latest code on dev branch, available on [discord](https://discord.com/channels/1107178041848909847/1283654073488379904).

For more information, please check out [different channels](#different-channels).

### Download URLs

| Type | Version | Operating System | Download Link |
|----------|---------------|------------------|---------------------------------------------------------------------------------------------------|
| **Local** | **Stable** | **Windows** | [Download](https://app.cortexcpp.com/download/latest/windows-amd64-local) |
| **Local** | **Stable** | **Linux** | [Download](https://app.cortexcpp.com/download/latest/linux-amd64-local) |
| **Local** | **Stable** | **MacOS** | [Download](https://app.cortexcpp.com/download/latest/mac-universal-local) |

:::info
For other versions, please look at [cortex.cpp repo](https://github.com/janhq/cortex.cpp) or each installer page.
:::


## Minimum Requirements

### OS
- MacOS 12 or later
- Windows 10 or later
- Linux: Ubuntu 20.04 or later, Debian 11 or later (For other distributions, please use the Dockerfile installer or binary files, we have not tested on other distributions yet.)

### Hardware
#### CPU
:::info
- Cortex.cpp supports a processor that can handle AVX2. For the full list, please see [here](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2).
- We support older processors with AVX, AVX-512 and non-AVX, though this is not recommended.
:::

##### Intel CPU
- Haswell processors (Q2 2013) and newer.
- Tiger Lake (Q3 2020) and newer for Celeron and Pentium processors.
##### AMD CPU
- Excavator processors (Q2 2015) and newer.

#### RAM
:::info
We support DDR2 RAM as the minimum requirement but recommend using newer generations of RAM for improved performance.
:::
- 8GB for running up to 3B models (int4).
- 16GB for running up to 7B models (int4).
- 32GB for running up to 13B models (int4).

#### GPU
:::info
Having at least 6GB VRAM when using NVIDIA, AMD, or Intel Arc GPUs is recommended.
:::
- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
:::info
- [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) version 470.63.01 or higher.
*Note: Users must install the driver by themselves.*
- [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) version 12.0 or higher. *Note: Cortex.cpp will automatically detect and install the required version of cudart to the user's machine.*
:::
#### Disk
- At least 10GB for app storage and model download.

## Different channels

Different channels have different features, stability levels, binary file name, app folder and data folder.

### Stable
- App name: `cortexcpp`
- Binary file name: `cortex`, `cortex-server` for linux and mac; `cortex.exe`, `cortex-server.exe` for windows.;
- App folder (Windows Only): `C:\Users\<username>\AppData\Local\cortexcpp`
- Data folder: `~/cortexcpp` for linux and mac; `C:\Users\<username>\cortexcpp` for windows.
- Uninstaller script (Mac Only): `cortex-uninstall.sh`

### Beta
- App name: `cortexcpp-beta`
- Binary file name: `cortex-beta`, `cortex-server-beta` for linux and mac; `cortex-beta.exe`, `cortex-server-beta.exe` for windows.;
- App folder (Windows Only): `C:\Users\<username>\AppData\Local\cortexcpp-beta`
- Data folder: `~/cortexcpp-beta` for linux and mac; `C:\Users\<username>\cortexcpp-beta` for windows.
- Uninstaller script (Mac Only): `cortex-beta-uninstall.sh`

### Nightly
- App name: `cortexcpp-nightly`
- Binary file name: `cortex-nightly`, `cortex-server-nightly` for linux and mac; `cortex-nightly.exe`, `cortex-server-nightly.exe` for windows.;
- App folder (Windows Only): `C:\Users\<username>\AppData\Local\cortexcpp-nightly`
- Data folder: `~/cortexcpp-nightly` for linux and mac; `C:\Users\<username>\cortexcpp-nightly` for windows.
- Uninstaller script (Mac Only): `cortex-nightly-uninstall.sh`
Loading

0 comments on commit 0942fc1

Please sign in to comment.