Merge branch 'refs/heads/main' into eugene/TLK-1864_models_refactoring

README.md

@@ -13,6 +13,7 @@ Toolkit is a deployable all-in-one RAG application that enables users to quickly
   - [How to add tools](/docs/custom_tool_guides/tool_guide.md)
   - [How to add auth to your tools](/docs/custom_tool_guides/tool_auth_guide.md)
   - [How to setup Google Drive](/docs/custom_tool_guides/google_drive.md)
+  - [How to setup Slack Tool](/docs/custom_tool_guides/slack.md)
   - [How to setup Google Text-to-Speech](/docs/text_to_speech.md)
   - [How to add authentication](/docs/auth_guide.md)
   - [How to deploy toolkit services](/docs/service_deployments.md)
@@ -24,22 +25,33 @@ Toolkit is a deployable all-in-one RAG application that enables users to quickly
 ![](/docs/assets/toolkit.gif)
 
 ## Try Now:
+There are two main ways for quickly running Toolkit: local and cloud. See the specific instructions given below. 
+### Local
+*You will need to have [Docker](https://www.docker.com/products/docker-desktop/) and [Docker-compose >= 2.22](https://docs.docker.com/compose/install/) installed. [Go here for a more detailed setup.](/docs/setup.md)*  
+Note: to include community tools when building locally, set the `INSTALL_COMMUNITY_DEPS` build arg in the `docker-compose.yml` to `true`.
 
-*You will need to have [Docker](https://www.docker.com/products/docker-desktop/) and [Docker-compose >= 2.22](https://docs.docker.com/compose/install/) installed. [Go here for a more detailed setup.](/docs/setup.md)*
+Both options will make the frontend available at http://localhost:4000.
 
-There are two main ways of running Toolkit:
+#### Using `make`
+Use the provided Makefile to simplify and automate your development workflow with Cohere Toolkit, including Docker Compose management, testing, linting, and environment setup.
+```bash
+git clone https://github.com/cohere-ai/cohere-toolkit.git
+cd cohere-toolkit
+make first-run
+```
 
-**Docker Compose setup**
 
-Note: to include community tools when building locally, set the `INSTALL_COMMUNITY_DEPS` build arg in the `docker-compose.yml` to `true`.
 
+#### Docker Compose only
+Use Docker Compose directly if you want to quickly spin up and manage your container environment without the additional automation provided by the Makefile.
 ```bash
 git clone https://github.com/cohere-ai/cohere-toolkit.git
 cd cohere-toolkit
-make first-run
+docker compose up
+docker compose run --build backend alembic -c src/backend/alembic.ini upgrade head
 ```
-
-## GitHub Codespaces
+### Cloud
+#### GitHub Codespaces
 
 To run this project using GitHub Codespaces, please refer to our [Codespaces Setup Guide](/docs/github_codespaces.md).
 

docs/custom_tool_guides/slack.md

@@ -0,0 +1,131 @@
+# Slack Tool Setup
+
+To set up the Slack tool you will need a Slack application. Follow the steps below to set it up:
+
+## 1. Create a Slack App
+
+Head to the [Slack API](https://api.slack.com/apps) and create a new app.
+After creating the app, you will see the `App Credentials` section. Copy the `Client ID` and `Client Secret` values.
+That will be used for the environment variables specified above.
+
+## 2. Set up OAuth & Permissions
+OAuth flow is required to authenticate users with Slack. 
+To enable it please set the following redirect URL to your app's settings:
+```bash
+    https://<your_backend_url>/v1/tool/auth
+```
+Please note that for the local development you will need to enable HTTPS. 
+See the [Setup HTTPS for Local Development](#5-setup-https-for-local-development) section for more details.
+If you are using a local https setup, redirect url should be 
+```
+ https://localhost:8000/v1/tool/auth
+```
+Also, you can set up a proxy, such as [ngrok](https://ngrok.com/docs/getting-started/), to expose your local server to the internet.
+
+The Slack tool uses User Token Scopes to access the user's Slack workspace.
+The required and the default permission scope is `search:read`.
+Set it in the `OAuth & Permissions` section of your Slack app settings.
+
+To work with the Slack Tool Advanced token security via token rotation is required.
+To enable it, go to the `OAuth & Permissions` section of your Slack app settings and click 'Opt in' button in the 'Advanced token security via token rotation' section.
+
+More information about the OAuth flow can be found [here](https://api.slack.com/authentication/oauth-v2).
+
+## 3. Set Up Environment Variables
+
+Then set the following environment variables. You can either set the below values in your `secrets.yaml` file:
+```bash
+slack:
+    client_id: <your_client_id from step 1>
+    client_secret: <your_client_secret from step 1>
+```
+or update your `.env` configuration to contain:
+```bash
+SLACK_CLIENT_ID=<your_client_id from step 1>
+SLACK_CLIENT_SECRET=<your_client_secret from step 1>
+```
+
+## 4. Enable the Slack Tool in the Frontend
+
+To enable the Slack tool in the frontend, you will need to modify the `src/community/config/tools.py` file. Add the `TOOL_SLACK_ID` to the `AGENT_SETTINGS_TOOLS` list.
+
+```typescript
+export const AGENT_SETTINGS_TOOLS = [
+  TOOL_HYBRID_WEB_SEARCH_ID,
+  TOOL_PYTHON_INTERPRETER_ID,
+  TOOL_WEB_SCRAPE_ID,
+  TOOL_SLACK_ID,
+];
+```
+
+To enable the Slack tool in the frontend for Base Agent, you will need to modify the `src/community/config/tools.py` file. Remove the `TOOL_SLACK_ID` from the `BASE_AGENT_EXCLUDED_TOOLS` list.
+By default, the Slack Tool is disabled for the Base Agent. Also if you need to exclude some tool from the Base Agent just add it to the `BASE_AGENT_EXCLUDED_TOOLS` list.
+```typescript
+export const BASE_AGENT_EXCLUDED_TOOLS = [];
+```
+
+## 5. Setup HTTPS for Local Development
+
+To enable HTTPS for local development, the self-signed certificate needs to be generated.
+Run the following command in the project root directory to generate the certificate and key:
+
+```bash
+ openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
+```
+
+Then, update the backend Docker configuration(src/backend/Dockerfile) to use the generated certificate.
+Just change next lines in the Dockerfile:
+```Dockerfile
+COPY pyproject.toml poetry.lock cert.pem key.pem ./ 
+```
+and 
+```Dockerfile
+CMD uvicorn backend.main:app --reload --host 0.0.0.0 --port ${PORT} --timeout-keep-alive 300 --ssl-keyfile /workspace/key.pem --ssl-certfile /workspace/cert.pem
+```
+Change NEXT_PUBLIC_API_HOSTNAME environment variable in the .env `https` protocol:
+```bash
+NEXT_PUBLIC_API_HOSTNAME=https://localhost:8000
+```
+
+or in the configurations.yaml file:
+
+```yaml
+auth:
+  backend_hostname: https://localhost:8000
+```
+
+To run the Frontend with HTTPS, update the `start` script in the `package.json` file:
+
+```json
+"scripts": {
+    "dev": "next dev --port 4000 --experimental-https",
+..........
+}
+```
+
+Add the following line to the 'docker-compose.yml' file to the frontend environment variables:
+
+```yaml
+    NEXT_PUBLIC_API_HOSTNAME=https://localhost:8000
+```
+
+and change the API_HOSTNAME to
+
+```yaml
+    API_HOSTNAME: https://localhost:8000
+```
+also change the src/interfaces/assistants_web/.env.development file env variables to use https.
+
+## 6. Run the Backend and Frontend
+
+run next command to start the backend and frontend:
+
+```bash
+make dev
+```
+
+## 7. Troubleshooting
+
+If you encounter any issues with OAuth, please check the following [link](https://api.slack.com/authentication/oauth-v2#errors)
+For example, if you see the invalid_team_for_non_distributed_app error,
+please ensure the app is distributed or try logging in with the workspace owner's account.

pyproject.toml

@@ -57,6 +57,7 @@ llama-index = "^0.11.10"
 llama-index-llms-cohere = "^0.3.0"
 llama-index-embeddings-cohere = "^0.2.1"
 google-cloud-texttospeech = "^2.18.0"
+slack-sdk = "^3.33.1"
 
 [tool.poetry.group.dev]
 optional = true

src/backend/config/configuration.template.yaml

@@ -34,6 +34,9 @@ tools:
       - tavily_web_search
   python_interpreter:
     url: http://terrarium:8080
+  slack:
+    user_scopes:
+      - search:read
 feature_flags:
   # Experimental features
   use_agents_view: false

src/backend/config/secrets.template.yaml

@@ -31,6 +31,9 @@ tools:
   google_web_search:
     api_key:
     cse_id:
+  slack:
+    client_id:
+    client_secret:
 auth:
   secret_key:
   google_oauth:

src/backend/config/settings.py

@@ -146,6 +146,24 @@ class GDriveSettings(BaseSettings, BaseModel):
     )
 
 
+class SlackSettings(BaseSettings, BaseModel):
+    model_config = SETTINGS_CONFIG
+    client_id: Optional[str] = Field(
+        default=None,
+        validation_alias=AliasChoices("SLACK_CLIENT_ID", "client_id"),
+    )
+    client_secret: Optional[str] = Field(
+        default=None,
+        validation_alias=AliasChoices("SLACK_CLIENT_SECRET", "client_secret"),
+    )
+    user_scopes: Optional[str] = Field(
+        default=None,
+        validation_alias=AliasChoices(
+            "SLACK_USER_SCOPES", "scopes"
+        ),
+    )
+
+
 class TavilyWebSearchSettings(BaseSettings, BaseModel):
     model_config = SETTINGS_CONFIG
     api_key: Optional[str] = Field(
@@ -195,10 +213,13 @@ class ToolSettings(BaseSettings, BaseModel):
     brave_web_search: Optional[BraveWebSearchSettings] = Field(
         default=BraveWebSearchSettings()
     )
-
     hybrid_web_search: Optional[HybridWebSearchSettings] = Field(
         default=HybridWebSearchSettings()
     )
+    slack: Optional[SlackSettings] = Field(
+        default=SlackSettings()
+    )
+
 
 
 class DatabaseSettings(BaseSettings, BaseModel):