Doc update for Ollama Support. (#40)

* Doc update for Ollama Support. Signed-off-by: Tomoya Fujita <[email protected]> * address review comment. Signed-off-by: Tomoya Fujita <[email protected]> --------- Signed-off-by: Tomoya Fujita <[email protected]>
fujitatomoya · Sep 14, 2024 · 0fb1a0e · 0fb1a0e
1 parent b5d412a
commit 0fb1a0e
Show file tree

Hide file tree

Showing 5 changed files with 107 additions and 60 deletions.
diff --git a/README.md b/README.md
@@ -2,16 +2,19 @@
 
 # ros2ai 🤖
 
-[ros2ai](https://github.com/fujitatomoya/ros2ai) is a <span style="color:red">next-generation</span> [ROS 2](https://github.com/ros2) command line interface extension with [OpenAI](https://openai.com/)
+[ros2ai](https://github.com/fujitatomoya/ros2ai) is a <span style="color:red">next-generation</span> [ROS 2](https://github.com/ros2) command line interface extension with [OpenAI](https://openai.com/) and [Ollama](https://github.com/ollama/ollama).
 
 see [overview slide deck](https://raw.githack.com/fujitatomoya/ros2ai/rolling/doc/overview.html) for more information.
 
+TODO(@fujitatomoya): insert overview diagram here.
+
 ## Motivation
 
 - (Just for fun 😝)
 - Getting answers against the questions directly without browsing, clicking and typing many times.
 - Easy to use for everyone, especially for [ROS 2](https://github.com/ros2) beginners and students who do not really know [ros2cli](https://github.com/ros2/ros2cli).
 - Multiple language support.
+- Multiple LLMs support (with [OpenAI Python API](https://github.com/openai/openai-python))
 
 ## Demo 🖥️
 
@@ -39,26 +42,26 @@ docker run -it --rm -e OPENAI_API_KEY=$OPENAI_API_KEY tomoyafujita/ros2ai:humble
 ```
 
 > [!NOTE]
-> `OPENAI_API_KEY` environmental variable must be set
+> `OPENAI_API_KEY` environmental variable is not required if using Ollama.
 
 https://github.com/fujitatomoya/ros2ai/assets/43395114/2af4fd44-2ccf-472c-9153-c3c19987dc96
 
 ### Required Package
 
 - `rolling` / `jazzy`
 
-```bash
-pip install openai --break-system-packages
-```
+  ```bash
+  pip install openai ollama --break-system-packages
+  ```
 
-> [!NOTE]
-> see [PEP 668 – Marking Python base environments as “externally managed”](PEP 668 – Marking Python base environments as “externally managed”) why `--break-system-packages` is required.
+  > [!NOTE]
+  > see [PEP 668 – Marking Python base environments as “externally managed”](PEP 668 – Marking Python base environments as “externally managed”) why `--break-system-packages` is required.
 
 - `iron` / `humble`
 
-```bash
-pip install openai
-```
+  ```bash
+  pip install openai ollama
+  ```
 
 ### Build
 
@@ -77,21 +80,38 @@ colcon build --symlink-install --packages-select ros2ai
 
 ### Prerequisites
 
-- `ros2ai` requires [OpenAI API key](https://platform.openai.com/docs/overview)
+As described in overview diagram, `ros2ai` uses only OpenAI Python API but backend implementation can be replaced with Ollama that provides the compatible API with OpenAI.
+User need to choose either of them is used by `ros2ai` as following configuration, otherwise it falls back to use OpenAI.
 
-```bash
-export OPENAI_API_KEY='your-api-key-here'
-```
+- [OpenAI](https://openai.com/)
+
+  `ros2ai` requires [OpenAI API key](https://platform.openai.com/docs/overview)
+
+  ```bash
+  export OPENAI_API_KEY='your-api-key-here'
+  ```
+
+  > [!CAUTION]
+  > Do not share or expose your OpenAI API key.
+
+- [Ollama](https://github.com/ollama/ollama)
 
-> [!CAUTION]
-> Do not share or expose your OpenAI API key.
+  `ros2ai` requires [Ollama](https://github.com/ollama/ollama) service running, see more details for [How to install Ollama to Linux](https://github.com/ollama/ollama/blob/main/docs/linux.md)
+
+  ```bash
+  ### Install Ollama and System Service
+  sudo curl -fsSL https://ollama.com/install.sh | sh
+  ### Check ollama.service is up and running
+  systemctl status ollama
+  ```
 
 #### Optional
 
 | environmental variable | default                     | Note                   |
 | :----------------------| :-------------------------- | :--------------------- |
-| OPENAI_MODEL_NAME      | 'gpt-4o'                    | AI model to be used.   |
-| OPENAI_ENDPOINT        | 'https://api.openai.com/v1' | API endpoint URL.      |
+| OPENAI_API_KEY         | None.                       | Required for OpenAI    |
+| OPENAI_MODEL_NAME      | 'gpt-4o'                    | AI model to be used. e.g) llama3.1 (Ollama) |
+| OPENAI_ENDPOINT        | 'https://api.openai.com/v1' | API endpoint URL. e.g) http://localhost:11434/v1 (Ollama) |
 | OPENAI_TEMPERATURE     | 0.5                         | [OpenAI temperature](https://platform.openai.com/docs/guides/text-generation/how-should-i-set-the-temperature-parameter) |
 
 > [!NOTE]
@@ -101,65 +121,79 @@ export OPENAI_API_KEY='your-api-key-here'
 
 #### Basics
 
+- configuration
+
+  ```bash
+  ### OpenAI configuration
+  export OPENAI_API_KEY=<YOUR_API_KEY>
+  export OPENAI_MODEL_NAME=gpt-4
+  export OPENAI_ENDPOINT=https://api.openai.com/v1
+
+  ### Ollama
+  unset OPENAI_API_KEY
+  export OPENAI_MODEL_NAME=llama3.1
+  export OPENAI_ENDPOINT=http://localhost:11434/v1
+  ```
+
 - `status` to check OpenAI API key is valid.
 
-```bash
-root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai status -v
------ api_model: gpt-4
------ api_endpoint: https://api.openai.com/v1
------ api_token: None
-As an artificial intelligence, I do not have a physical presence, so I can't be "in service" in the traditional sense. But I am available to assist you 24/7.
-[SUCCESS] Valid OpenAI API key.
-```
+  ```bash
+  root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai status -v
+  ----- api_model: gpt-4
+  ----- api_endpoint: https://api.openai.com/v1
+  ----- api_token: None
+  As an artificial intelligence, I do not have a physical presence, so I can't be "in service" in the traditional sense. But I am available to assist you 24/7.
+  [SUCCESS] Valid OpenAI API key.
+  ```
 
 - `query` to ask any questions to ROS 2 assistant.
 
-```bash
-root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai query "Tell me how to check the available topics?"
-To check the available topics in ROS 2, you can use the following command in the terminal:
+  ```bash
+  root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai query "Tell me how to check the available topics?"
+  To check the available topics in ROS 2, you can use the following command in the terminal:
 
----
-ros2 topic list
----
+  ---
+  ros2 topic list
+  ---
 
-After you enter this command, a list of all currently active topics in your ROS2 system will be displayed. This list includes all topics that nodes in your system are currently publishing to or subscribing from.
-```
+  After you enter this command, a list of all currently active topics in your ROS2 system will be displayed. This list includes all topics that nodes in your system are currently publishing to or subscribing from.
+  ```
 
 - `exec` that ROS 2 assistant can execute the appropriate command based on your request.
 
-```bash
-root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai exec "give me all nodes"
-/talker
-root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai exec "what topics available"
-/chatter
-/parameter_events
-/rosout
-root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai exec "give me detailed info for topic /chatter"
-Type: std_msgs/msg/String
-Publisher count: 1
-Subscription count: 0
-```
+  ```bash
+  root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai exec "give me all nodes"
+  /talker
+  root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai exec "what topics available"
+  /chatter
+  /parameter_events
+  /rosout
+  root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai exec "give me detailed info for topic /chatter"
+  Type: std_msgs/msg/String
+  Publisher count: 1
+  Subscription count: 0
+  ```
 
 #### Multiple Language
 
 - Japanese (could be any language ❓)
 
-```bash
-root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai query "パラメータのリスト取得方法を教えて"
-ROS 2のパラメータリストを取得するには、コマンドラインインターフェース(CLI)を使います。具体的には、次のコマンドを使用します：
+  ```bash
+  root@tomoyafujita:~/docker_ws/ros2_colcon# ros2 ai query "パラメータのリスト取得方法を教えて"
+  ROS 2のパラメータリストを取得するには、コマンドラインインターフェース(CLI)を使います。具体的には、次のコマンドを使用します：
 
----code
-ros2 param list
----
+  ---code
+  ros2 param list
+  ---
 
-このコマンドは、現在動作しているすべてのノードのパラメーターをリストアップします。特定のノードのパラメータだけを見たい場合には、以下のようにノード名を指定することもできます。
+  このコマンドは、現在動作しているすべてのノードのパラメーターをリストアップします。特定のノードのパラメータだけを見たい場合には、以下のようにノード名を指定することもできます。
 
----code
-ros2 param list /node_name
----
+  ---code
+  ros2 param list /node_name
+  ---
 
-このようにして、ROS2のパラメータリストの取得を行うことが可能です。なお、上述したコマンドはシェルから直接実行してください。
-```
+  このようにして、ROS2のパラメータリストの取得を行うことが可能です。なお、上述したコマンドはシェルから直接実行してください。
+  ```
 
 ## Reference
 

diff --git a/ros2ai/api/utils.py b/ros2ai/api/utils.py
@@ -35,7 +35,7 @@ def run_command(*, command, argv = None, prefix = None):
     except Exception as ex:
         print(f"Subprocess generates exception: {ex}")
 
-def run_executable(*, command, argv = None, prefix=None):
+def run_executable(*, command, argv = None, prefix=None) -> int:
     """
     Execute command line interface with arguments. (Interactive)
 

diff --git a/ros2ai/verb/exec.py b/ros2ai/verb/exec.py
@@ -75,6 +75,8 @@ def main(self, *, args):
             original = client.get_result(), substring = 'ros2')
         command_str = remove_backticks(command_str)
         if not args.dry_run:
-            run_executable(command = command_str)
+            return_code = run_executable(command = command_str)
+            return return_code
         else:
             print(f"Command Candidate: '{command_str}'")
+            return 0
diff --git a/ros2ai/verb/query.py b/ros2ai/verb/query.py
@@ -75,3 +75,4 @@ def main(self, *, args):
             print(f"System role:\n{system_role}")
         else:
             client.print_result()
+        return 0
diff --git a/scripts/verification.sh b/scripts/verification.sh
@@ -40,6 +40,7 @@ command_list=(
 # Function Definitions #
 ########################
 
+# Note: trap cannot handle return code bu signals only. this only works with general error signal.
 function exit_trap() {
     # shellcheck disable=SC2317  # Don't warn about unreachable commands in this function
     if [ $? != 0 ]; then
@@ -48,6 +49,14 @@ function exit_trap() {
     fi
 }
 
+function check_and_exit() {
+    # shellcheck disable=SC2317  # Don't warn about unreachable commands in this function
+    if [ $? -ne 0 ]; then
+        echo "Error: [$BASH_COMMAND] failed with exit code $?"
+        exit $?
+    fi
+}
+
 function check_user_setting () {
     trap exit_trap ERR
     echo "[${FUNCNAME[0]}]: checking user setting and configuration."
@@ -65,6 +74,7 @@ function verify_ros2ai() {
     for command in "${command_list[@]}"; do
         echo "----- $command"
         eval $command
+        check_and_exit
     done
     echo "----- all ros2ai commands return successfully!!! -----"
 }