Skip to content

Commit

Permalink
Merge pull request #158 from janhq/improvement/landing-page
Browse files Browse the repository at this point in the history
web: nitro landing page
  • Loading branch information
tikikun authored Nov 21, 2023
2 parents 5a71fb6 + 6a84f9b commit 9dbe4e6
Show file tree
Hide file tree
Showing 61 changed files with 1,357 additions and 788 deletions.
3 changes: 2 additions & 1 deletion docs/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,5 @@
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.env
yarn.lock
.env
2 changes: 1 addition & 1 deletion docs/babel.config.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module.exports = {
presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
presets: [require.resolve("@docusaurus/core/lib/babel/preset")],
};
9 changes: 7 additions & 2 deletions docs/docs/features/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ curl http://localhost:3928/v1/chat/completions \
}'

```

</div>

<div style={{ width: '50%', float: 'right', clear: 'right' }}>
Expand All @@ -42,6 +43,7 @@ curl https://api.openai.com/v1/chat/completions \
]
}'
```

</div>

This command sends a request to your local LLM, querying about the winner of the 2020 World Series.
Expand Down Expand Up @@ -77,6 +79,7 @@ curl http://localhost:3928/v1/chat/completions \
}'

```

</div>

<div style={{ width: '50%', float: 'right', clear: 'right' }}>
Expand Down Expand Up @@ -106,6 +109,7 @@ curl https://api.openai.com/v1/chat/completions \
]
}'
```

</div>

### Chat Completion Response
Expand Down Expand Up @@ -138,6 +142,7 @@ Below are examples of responses from both the Nitro server and OpenAI:
}
}
```

</div>

<div style={{ width: '50%', float: 'right', clear: 'right' }}>
Expand Down Expand Up @@ -166,7 +171,7 @@ Below are examples of responses from both the Nitro server and OpenAI:
}
}
```
</div>

</div>

The chat completion feature in Nitro showcases compatibility with OpenAI, making the transition between using OpenAI and local AI models more straightforward. For further details and advanced usage, please refer to the [API reference](https://nitro.jan.ai/api).
The chat completion feature in Nitro showcases compatibility with OpenAI, making the transition between using OpenAI and local AI models more straightforward. For further details and advanced usage, please refer to the [API reference](https://nitro.jan.ai/api-reference).
9 changes: 6 additions & 3 deletions docs/docs/features/embed.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ curl http://localhost:3928/v1/embeddings \
}'

```

</div>
<div style={{ width: '50%', float: 'right', clear: 'right' }}>

Expand All @@ -39,6 +40,7 @@ curl https://api.openai.com/v1/embeddings \
"encoding_format": "float"
}'
```

</div>

## Embedding Reponse
Expand All @@ -61,8 +63,9 @@ The example response used the output from model [llama2 Chat 7B Q5 (GGUF)](https
"object": "embedding"
}
]
}
}
```

</div>

<div style={{ width: '50%', float: 'right', clear: 'right' }}>
Expand All @@ -83,7 +86,7 @@ The example response used the output from model [llama2 Chat 7B Q5 (GGUF)](https


```
</div>

</div>

The embedding feature in Nitro demonstrates a high level of compatibility with OpenAI, simplifying the transition between using OpenAI and local AI models. For more detailed information and advanced use cases, refer to the comprehensive [API Reference]((https://nitro.jan.ai/api)).
The embedding feature in Nitro demonstrates a high level of compatibility with OpenAI, simplifying the transition between using OpenAI and local AI models. For more detailed information and advanced use cases, refer to the comprehensive [API Reference](https://nitro.jan.ai/api-reference)).
15 changes: 11 additions & 4 deletions docs/docs/new/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,9 @@ curl http://localhost:3928/v1/chat/completions \
},
]
}'

```

</div>

<div style={{ width: '50%', float: 'right', clear: 'right' }}>
Expand All @@ -63,6 +64,7 @@ curl https://api.openai.com/v1/chat/completions \
]
}'
```

</div>

- **Extends OpenAI's API with helpful model methods:**
Expand All @@ -80,7 +82,7 @@ curl https://api.openai.com/v1/chat/completions \
### Multi-modal Capabilities

- **Coming Soon**: Expansion to multi-modal functionalities - enabling Nitro to process and generate images, and audio.
- **Features to Expect**:
- **Features to Expect**:
- Large Language-and-Vision Assistant.
- Speech recognition and transcription.

Expand All @@ -90,25 +92,30 @@ curl https://api.openai.com/v1/chat/completions \
- **Detailed Specifications**: For an in-depth understanding of Nitro's internal workings, components, and design philosophy, refer to our [Architecture Specifications](architecture.md).

## Support

### GitHub Issue Tracking

- **Report Problems**: Encounter an issue with Nitro? File a [GitHub issue](https://github.com/janhq/nitro). Please include detailed error logs and steps to reproduce the problem.

### Discord Community

- **Join the Conversation**: Discuss Nitro development and seek peer support in our [#nitro-dev](https://discord.gg/FTk2MvZwJH) channel on Discord.

## Contributing

### How to Contribute

Nitro welcomes contributions in various forms, not just coding. Here are some ways you can get involved:

- **Understand Nitro**: Start with the [Getting Started](nitro/overview) guide. Found an issue or have a suggestion? [Open an issue](https://github.com/janhq/nitro/issues) to let us know.
- **Understand Nitro**: Start with the [Getting Started](/new/quickstart) guide. Found an issue or have a suggestion? [Open an issue](https://github.com/janhq/nitro/issues) to let us know.

- **Feature Development**: Engage with community feature requests. Bring ideas to life by opening a [pull request](https://github.com/janhq/nitro/pulls) for features that interest you.

### Links

- [Nitro GitHub Repository](https://github.com/janhq/nitro)

## Acknowledgements

- [drogon](https://github.com/drogonframework/drogon): The fast C++ web framework
- [llama.cpp](https://github.com/ggerganov/llama.cpp): Inference of LLaMA model in pure C/C++
- [llama.cpp](https://github.com/ggerganov/llama.cpp): Inference of LLaMA model in pure C/C++
10 changes: 1 addition & 9 deletions docs/docs/new/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ title: Architecture
### Details element example

## Key Concepts

## Inference Server

An inference server is a type of server designed to process requests for running large language models and to return predictions. This server acts as the backbone for AI-powered applications, providing real-time execution of models to analyze data and make decisions.
Expand All @@ -24,15 +25,6 @@ Parallel processing involves executing multiple computations simultaneously. For
Drogon is an HTTP application framework based on C++14/17, designed for its speed and simplicity. Utilizing a non-blocking I/O and event-driven architecture, Drogon manages HTTP requests efficiently for high-performance and scalable applications.

- **Event Loop**: Drogon uses an event loop to wait for and dispatch events or messages within a program. This allows for handling many tasks asynchronously, without relying on multi-threading.

- **Threads**: While the event loop allows for efficient task management, Drogon also employs threads to handle parallel operations. These "drogon threads" process multiple tasks concurrently.

- **Asynchronous Operations**: The framework supports non-blocking operations, permitting the server to continue processing other tasks while awaiting responses from databases or external services.

- **Scalability**: Drogon's architecture is built to scale, capable of managing numerous connections at once, suitable for applications with high traffic loads.



We should only have 1 document
- [ ] Refactor system/architecture
- [ ] Refactor system/key-concepts
79 changes: 41 additions & 38 deletions docs/docs/new/build-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,21 @@ git clone --recurse https://github.com/janhq/nitro
If you don't have git, you can download the source code as a file archive from [Nitro GitHub](https://github.com/janhq/nitro). Each [release](https://github.com/caddyserver/caddy/releases) also has source snapshots.

## Install Dependencies

Next, let's install the necessary dependencies.

- **On MacOS with Apple Silicon:**
```bash
./install_deps.sh
```

```bash
./install_deps.sh
```

- **On Windows:**

```bash
cmake -S ./nitro_deps -B ./build_deps/nitro_deps
cmake --build ./build_deps/nitro_deps --config Release
```
```bash
cmake -S ./nitro_deps -B ./build_deps/nitro_deps
cmake --build ./build_deps/nitro_deps --config Release
```

This creates a `build_deps` folder.

Expand All @@ -37,66 +39,67 @@ Now, let's generate the build files.
- **On MacOS, Linux, and Windows:**
```bash
mkdir build && cd build
cmake ..
```
```bash
mkdir build && cd build
cmake ..
```
- **On MacOS with Intel processors:**
```bash
mkdir build && cd build
cmake -DLLAMA_METAL=OFF ..
```
```bash
mkdir build && cd build
cmake -DLLAMA_METAL=OFF ..
```
- **On Linux with CUDA:**
```bash
mkdir build && cd build
cmake -DLLAMA_CUBLAS=ON ..
```
```bash
mkdir build && cd build
cmake -DLLAMA_CUBLAS=ON ..
```
## Build the Application
Time to build Nitro!
- **On MacOS:**
```bash
make -j $(sysctl -n hw.physicalcpu)
```
```bash
make -j $(sysctl -n hw.physicalcpu)
```
- **On Linux:**
```bash
make -j $(%NUMBER_OF_PROCESSORS%)
```
```bash
make -j $(%NUMBER_OF_PROCESSORS%)
```
- **On Windows:**
```bash
cmake --build . --config Release
```
```bash
cmake --build . --config Release
```
## Start process
Finally, let's start Nitro.

- **On MacOS and Linux:**

```bash
./nitro
```
```bash
./nitro
```

- **On Windows:**

```bash
cd Release
copy ..\..\build_deps\_install\bin\zlib.dll .
nitro.exe
```
```bash
cd Release
copy ..\..\build_deps\_install\bin\zlib.dll .
nitro.exe
```

To verify if the build was successful:

```bash
curl http://localhost:3928/healthz
```
```
20 changes: 13 additions & 7 deletions docs/docs/new/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
---
title: Quickstart
---

## Step 1: Install Nitro

### For Linux and MacOS

Open your terminal and enter the following command. This will download and install Nitro on your system.
```bash
curl -sfL https://raw.githubusercontent.com/janhq/nitro/main/install.sh -o /tmp/install.sh && chmod +x /tmp/install.sh && sudo bash /tmp/install.sh --gpu && rm /tmp/install.sh
```

```bash
curl -sfL https://raw.githubusercontent.com/janhq/nitro/main/install.sh -o /tmp/install.sh && chmod +x /tmp/install.sh && sudo bash /tmp/install.sh --gpu && rm /tmp/install.sh
```

### For Windows

Open PowerShell and execute the following command. This will perform the same actions as for Linux and MacOS but is tailored for Windows.
```bash
powershell -Command "& { Invoke-WebRequest -Uri 'https://raw.githubusercontent.com/janhq/nitro/main/install.bat' -OutFile 'install.bat'; .\install.bat --gpu; Remove-Item -Path 'install.bat' }"
```

```bash
powershell -Command "& { Invoke-WebRequest -Uri 'https://raw.githubusercontent.com/janhq/nitro/main/install.bat' -OutFile 'install.bat'; .\install.bat --gpu; Remove-Item -Path 'install.bat' }"
```

> **NOTE:**Installing Nitro will add new files and configurations to your system to enable it to run.
Expand All @@ -24,6 +29,7 @@ For a manual installation process, see: [Install from Source](install.md)
Next, we need to download a model. For this example, we'll use the [Llama2 7B chat model](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/tree/main).

- Create a `/model` and navigate into it:

```bash
mkdir model && cd model
wget -O llama-2-7b-model.gguf https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_M.gguf?download=true
Expand Down Expand Up @@ -78,4 +84,4 @@ curl http://localhost:3928/v1/chat/completions \

This command sends a request to Nitro, asking it about the 2020 World Series winner.

- As you can see, A key benefit of Nitro is its alignment with [OpenAI's API structure](https://platform.openai.com/docs/guides/text-generation?lang=curl). Its inference call syntax closely mirrors that of OpenAI's API, facilitating an easier shift for those accustomed to OpenAI's framework.
- As you can see, A key benefit of Nitro is its alignment with [OpenAI's API structure](https://platform.openai.com/docs/guides/text-generation?lang=curl). Its inference call syntax closely mirrors that of OpenAI's API, facilitating an easier shift for those accustomed to OpenAI's framework.
Loading

0 comments on commit 9dbe4e6

Please sign in to comment.