-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'dev' into more-groq-models
- Loading branch information
Showing
27 changed files
with
295 additions
and
344 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| # Jan - Turn your computer into an AI computer | ||
| # Jan - Local AI Assistant | ||
|
|
||
|  | ||
|  | ||
|
|
||
| <p align="center"> | ||
| <!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --> | ||
|
|
@@ -12,25 +12,35 @@ | |
| </p> | ||
|
|
||
| <p align="center"> | ||
| <a href="https://jan.ai/guides">Getting Started</a> | ||
| <a href="https://jan.ai/docs/quickstart">Getting Started</a> | ||
| - <a href="https://jan.ai/docs">Docs</a> | ||
| - <a href="https://github.com/janhq/jan/releases">Changelog</a> | ||
| - <a href="https://github.com/janhq/jan/issues">Bug reports</a> | ||
| - <a href="https://discord.gg/AsJ8krTT3N">Discord</a> | ||
| </p> | ||
|
|
||
| > [!Warning] >**Jan is currently in Development**: Expect breaking changes and bugs! | ||
| <p align="center"> | ||
| ⚠️ <b> Jan is currently in Development</b>: Expect breaking changes and bugs! | ||
| </p> | ||
|
|
||
|
|
||
| Jan is an open-source ChatGPT alternative that runs 100% offline on your computer. | ||
| Jan is a ChatGPT-alternative that runs 100% offline on your device. Our goal is to make it easy for a layperson to download and run LLMs and use AI with **full control** and **privacy**. | ||
|
|
||
| **Jan runs on any hardware.** From PCs to multi-GPU clusters, Jan supports universal architectures: | ||
| Jan is powered by [Cortex](https://github.com/janhq/cortex.cpp), our embeddable local AI engine that runs on any hardware. | ||
| From PCs to multi-GPU clusters, Jan & Cortex supports universal architectures: | ||
|
|
||
| - [x] NVIDIA GPUs (fast) | ||
| - [x] Apple M-series (fast) | ||
| - [x] Apple Intel | ||
| - [x] Linux Debian | ||
| - [x] Windows x64 | ||
|
|
||
| #### Features: | ||
| - [Model Library](https://jan.ai/docs/models/manage-models#add-models) with popular LLMs like Llama, Gemma, Mistral, or Qwen | ||
| - Connect to [Remote AI APIs](https://jan.ai/docs/remote-models/openai) like Groq and OpenRouter | ||
| - Local API Server with OpenAI-equivalent API | ||
| - [Extensions](https://jan.ai/docs/extensions) for customizing Jan | ||
|
|
||
| ## Download | ||
|
|
||
| <table> | ||
|
|
@@ -74,7 +84,40 @@ Jan is an open-source ChatGPT alternative that runs 100% offline on your compute | |
| </td> | ||
| </tr> | ||
| <tr style="text-align:center"> | ||
| <td style="text-align:center"><b>Experimental (Nightly Build)</b></td> | ||
| <td style="text-align:center"><b>Beta (Preview)</b></td> | ||
| <td style="text-align:center"> | ||
| <a href='https://app.jan.ai/download/beta/win-x64'> | ||
| <img src='https://github.com/janhq/jan/blob/dev/docs/static/img/windows.png' style="height:14px; width: 14px" /> | ||
| <b>jan.exe</b> | ||
| </a> | ||
| </td> | ||
| <td style="text-align:center"> | ||
| <a href='https://app.jan.ai/download/beta/mac-x64'> | ||
| <img src='https://github.com/janhq/jan/blob/dev/docs/static/img/mac.png' style="height:15px; width: 15px" /> | ||
| <b>Intel</b> | ||
| </a> | ||
| </td> | ||
| <td style="text-align:center"> | ||
| <a href='https://app.jan.ai/download/beta/mac-arm64'> | ||
| <img src='https://github.com/janhq/jan/blob/dev/docs/static/img/mac.png' style="height:15px; width: 15px" /> | ||
| <b>M1/M2/M3/M4</b> | ||
| </a> | ||
| </td> | ||
| <td style="text-align:center"> | ||
| <a href='https://app.jan.ai/download/beta/linux-amd64-deb'> | ||
| <img src='https://github.com/janhq/jan/blob/dev/docs/static/img/linux.png' style="height:14px; width: 14px" /> | ||
| <b>jan.deb</b> | ||
| </a> | ||
| </td> | ||
| <td style="text-align:center"> | ||
| <a href='https://app.jan.ai/download/beta/linux-amd64-appimage'> | ||
| <img src='https://github.com/janhq/jan/blob/dev/docs/static/img/linux.png' style="height:14px; width: 14px" /> | ||
| <b>jan.AppImage</b> | ||
| </a> | ||
| </td> | ||
| </tr> | ||
| <tr style="text-align:center"> | ||
| <td style="text-align:center"><b>Nightly Build (Experimental)</b></td> | ||
| <td style="text-align:center"> | ||
| <a href='https://app.jan.ai/download/nightly/win-x64'> | ||
| <img src='https://github.com/janhq/jan/blob/dev/docs/static/img/windows.png' style="height:14px; width: 14px" /> | ||
|
|
@@ -108,79 +151,64 @@ Jan is an open-source ChatGPT alternative that runs 100% offline on your compute | |
| </tr> | ||
| </table> | ||
|
|
||
| Download the latest version of Jan at https://jan.ai/ or visit the **[GitHub Releases](https://github.com/janhq/jan/releases)** to download any previous release. | ||
| Download the latest version of Jan at https://jan.ai/ or visit the [GitHub Releases](https://github.com/janhq/jan/releases) to download any previous release. | ||
|
|
||
| ## Demo | ||
|
|
||
|  | ||
| https://github.com/user-attachments/assets/c3592fa2-c504-4d9d-a885-7e00122a50f3 | ||
|
|
||
| _Realtime Video: Jan v0.4.3-nightly on a Mac M1, 16GB Sonoma 14_ | ||
| *Real-time Video: Jan v0.5.7 on a Mac M2, 16GB Sonoma 14.2* | ||
|
|
||
| ## Quicklinks | ||
|
|
||
| #### Jan | ||
| ### Jan | ||
|
|
||
| - [Jan website](https://jan.ai/) | ||
| - [Jan Website](https://jan.ai/) | ||
| - [Jan GitHub](https://github.com/janhq/jan) | ||
| - [User Guides](https://jan.ai/guides/) | ||
| - [Developer docs](https://jan.ai/developer/) | ||
| - [API reference](https://jan.ai/api-reference/) | ||
| - [Specs](https://jan.ai/docs/) | ||
|
|
||
| #### Nitro | ||
|
|
||
| Nitro is a high-efficiency C++ inference engine for edge computing. It is lightweight and embeddable, and can be used on its own within your own projects. | ||
|
|
||
| - [Nitro Website](https://nitro.jan.ai) | ||
| - [Nitro GitHub](https://github.com/janhq/nitro) | ||
| - [Documentation](https://nitro.jan.ai/docs) | ||
| - [API Reference](https://nitro.jan.ai/api-reference) | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| As Jan is in development mode, you might get stuck on a broken build. | ||
|
|
||
| To reset your installation: | ||
| - [Documentation](https://jan.ai/docs) | ||
| - [Jan Changelog](https://jan.ai/changelog) | ||
| - [Jan Blog](https://jan.ai/blog) | ||
|
|
||
| 1. Use the following commands to remove any dangling backend processes: | ||
|
|
||
| ```sh | ||
| ps aux | grep nitro | ||
| ``` | ||
| ### Cortex.cpp | ||
| Jan is powered by **Cortex.cpp**. It is a C++ command-line interface (CLI) designed as an alternative to [Ollama](https://ollama.com/). By default, it runs on the llama.cpp engine but also supports other engines, including ONNX and TensorRT-LLM, making it a multi-engine platform. | ||
|
|
||
| Look for processes like "nitro" and "nitro_arm_64," and kill them one by one with: | ||
|
|
||
| ```sh | ||
| kill -9 <PID> | ||
| ``` | ||
|
|
||
| 2. **Remove Jan from your Applications folder and Cache folder** | ||
|
|
||
| ```bash | ||
| make clean | ||
| ``` | ||
|
|
||
| This will remove all build artifacts and cached files: | ||
|
|
||
| - Delete Jan extension from your `~/jan/extensions` folder | ||
| - Delete all `node_modules` in current folder | ||
| - Clear Application cache in `~/Library/Caches/jan` | ||
|
|
||
| - [Cortex Website](https://cortex.so/) | ||
| - [Cortex GitHub](https://github.com/janhq/cortex.cpp) | ||
| - [Documentation](https://cortex.so/docs/) | ||
| - [Models Library](https://cortex.so/models) | ||
| - API Reference: *Under development* | ||
|
|
||
| ## Requirements for running Jan | ||
|
|
||
| - MacOS: 13 or higher | ||
| - Windows: | ||
| - **MacOS**: 13 or higher | ||
| - **Windows**: | ||
| - Windows 10 or higher | ||
| - To enable GPU support: | ||
| - Nvidia GPU with CUDA Toolkit 11.7 or higher | ||
| - Nvidia driver 470.63.01 or higher | ||
| - Linux: | ||
| - **Linux**: | ||
| - glibc 2.27 or higher (check with `ldd --version`) | ||
| - gcc 11, g++ 11, cpp 11 or higher, refer to this [link](https://jan.ai/guides/troubleshooting/gpu-not-used/#specific-requirements-for-linux) for more information | ||
| - To enable GPU support: | ||
| - Nvidia GPU with CUDA Toolkit 11.7 or higher | ||
| - Nvidia driver 470.63.01 or higher | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| As Jan is in development mode, you might get stuck on a some common issues: | ||
| - [Troubleshooting a broken build](https://jan.ai/docs/troubleshooting#broken-build) | ||
| - [Troubleshooting NVIDIA GPU](https://jan.ai/docs/troubleshooting#troubleshooting-nvidia-gpu) | ||
| - [Troubleshooting Something's Amiss](https://jan.ai/docs/troubleshooting#somethings-amiss) | ||
|
|
||
|
|
||
| If you can't find what you need in our troubleshooting guide, feel free reach out to us for extra help: | ||
| 1. Copy your [error logs & device specifications](https://jan.ai/docs/troubleshooting#how-to-get-error-logs). | ||
| 2. Go to our [Discord](https://discord.com/invite/FTk2MvZwJH) & send it to **#🆘|get-help** channel for further support. | ||
|
|
||
| *Check the logs to ensure the information is what you intend to send. Note that we retain your logs for only 24 hours, so report any issues promptly.* | ||
|
|
||
|
|
||
| ## Contributing | ||
|
|
||
| Contributions are welcome! Please read the [CONTRIBUTING.md](CONTRIBUTING.md) file | ||
|
|
@@ -209,11 +237,7 @@ Contributions are welcome! Please read the [CONTRIBUTING.md](CONTRIBUTING.md) fi | |
|
|
||
| This will start the development server and open the desktop app. | ||
|
|
||
| 3. (Optional) **Run the API server without frontend** | ||
|
|
||
| ```bash | ||
| yarn dev:server | ||
| ``` | ||
|
|
||
| ### For production build | ||
|
|
||
|
|
@@ -225,102 +249,6 @@ make build | |
|
|
||
| This will build the app MacOS m1/m2 for production (with code signing already done) and put the result in `dist` folder. | ||
|
|
||
| ### Docker mode | ||
|
|
||
| - Supported OS: Linux, WSL2 Docker | ||
| - Pre-requisites: | ||
|
|
||
| - Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu. | ||
|
|
||
| ```bash | ||
| curl -fsSL https://get.docker.com -o get-docker.sh | ||
| sudo sh ./get-docker.sh --dry-run | ||
| ``` | ||
|
|
||
| - If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation. | ||
|
|
||
| - Run Jan in Docker mode | ||
| > User can choose between `docker-compose.yml` with latest prebuilt docker image or `docker-compose-dev.yml` with local docker build | ||
|
|
||
| | Docker compose Profile | Description | | ||
| | ---------------------- | -------------------------------------------- | | ||
| | `cpu-fs` | Run Jan in CPU mode with default file system | | ||
| | `cpu-s3fs` | Run Jan in CPU mode with S3 file system | | ||
| | `gpu-fs` | Run Jan in GPU mode with default file system | | ||
| | `gpu-s3fs` | Run Jan in GPU mode with S3 file system | | ||
|
|
||
| | Environment Variable | Description | | ||
| | ----------------------- | ------------------------------------------------------------------------------------------------------- | | ||
| | `S3_BUCKET_NAME` | S3 bucket name - leave blank for default file system | | ||
| | `AWS_ACCESS_KEY_ID` | AWS access key ID - leave blank for default file system | | ||
| | `AWS_SECRET_ACCESS_KEY` | AWS secret access key - leave blank for default file system | | ||
| | `AWS_ENDPOINT` | AWS endpoint URL - leave blank for default file system | | ||
| | `AWS_REGION` | AWS region - leave blank for default file system | | ||
| | `API_BASE_URL` | Jan Server URL, please modify it as your public ip address or domain name default http://localhost:1377 | | ||
|
|
||
| - **Option 1**: Run Jan in CPU mode | ||
|
|
||
| ```bash | ||
| # cpu mode with default file system | ||
| docker compose --profile cpu-fs up -d | ||
| # cpu mode with S3 file system | ||
| docker compose --profile cpu-s3fs up -d | ||
| ``` | ||
|
|
||
| - **Option 2**: Run Jan in GPU mode | ||
|
|
||
| - **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output | ||
|
|
||
| ```bash | ||
| nvidia-smi | ||
| # Output | ||
| +---------------------------------------------------------------------------------------+ | ||
| | NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 | | ||
| |-----------------------------------------+----------------------+----------------------+ | ||
| | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | | ||
| | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | ||
| | | | MIG M. | | ||
| |=========================================+======================+======================| | ||
| | 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A | | ||
| | 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default | | ||
| | | | N/A | | ||
| +-----------------------------------------+----------------------+----------------------+ | ||
| | 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A | | ||
| | 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default | | ||
| | | | N/A | | ||
| +-----------------------------------------+----------------------+----------------------+ | ||
| | 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A | | ||
| | 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default | | ||
| | | | N/A | | ||
| +-----------------------------------------+----------------------+----------------------+ | ||
| +---------------------------------------------------------------------------------------+ | ||
| | Processes: | | ||
| | GPU GI CI PID Type Process name GPU Memory | | ||
| | ID ID Usage | | ||
| |=======================================================================================| | ||
| ``` | ||
|
|
||
| - **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0) | ||
|
|
||
| - **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`) | ||
|
|
||
| - **Step 4**: Run command to start Jan in GPU mode | ||
|
|
||
| ```bash | ||
| # GPU mode with default file system | ||
| docker compose --profile gpu-fs up -d | ||
| # GPU mode with S3 file system | ||
| docker compose --profile gpu-s3fs up -d | ||
| ``` | ||
|
|
||
| This will start the web server and you can access Jan at `http://localhost:3000`. | ||
|
|
||
| > Note: RAG feature is not supported in Docker mode with s3fs yet. | ||
|
|
||
| ## Acknowledgements | ||
|
|
||
| Jan builds on top of other open-source projects: | ||
|
|
@@ -334,18 +262,18 @@ Jan builds on top of other open-source projects: | |
|
|
||
| - Bugs & requests: file a GitHub ticket | ||
| - For discussion: join our Discord [here](https://discord.gg/FTk2MvZwJH) | ||
| - For business inquiries: email [email protected] | ||
| - For business inquiries: email [email protected] | ||
| - For jobs: please email [email protected] | ||
|
|
||
| ## Trust & Safety | ||
|
|
||
| Beware of scams. | ||
| Beware of scams! | ||
|
|
||
| - We will never ask you for personal info | ||
| - We are a free product; there's no paid version | ||
| - We don't have a token or ICO | ||
| - We are not actively fundraising or seeking donations | ||
| - We will never request your personal information. | ||
| - Our product is completely free; no paid version exists. | ||
| - We do not have a token or ICO. | ||
| - We are a [bootstrapped company](https://en.wikipedia.org/wiki/Bootstrapping), and don't have any external investors (*yet*). We're open to exploring opportunities with strategic partners want to tackle [our mission](https://jan.ai/about#mission) together. | ||
|
|
||
| ## License | ||
|
|
||
| Jan is free and open source, under the AGPLv3 license. | ||
| Jan is free and open source, under the **AGPLv3** license. | ||
Oops, something went wrong.