Initial commit for elizaos clone with orbisdb and lit protocol

.env.example

@@ -0,0 +1,100 @@
+# Required environment variables
+DISCORD_APPLICATION_ID=
+DISCORD_API_TOKEN= # Bot token
+OPENAI_API_KEY=sk-* # OpenAI API key, starting with sk-
+REDPILL_API_KEY= # REDPILL API Key
+GROK_API_KEY= # GROK API Key
+GROQ_API_KEY=gsk_*
+OPENROUTER_API_KEY=
+GOOGLE_GENERATIVE_AI_API_KEY= # Gemini API key
+
+ELEVENLABS_XI_API_KEY= # API key from elevenlabs
+
+# ELEVENLABS SETTINGS
+ELEVENLABS_MODEL_ID=eleven_multilingual_v2
+ELEVENLABS_VOICE_ID=21m00Tcm4TlvDq8ikWAM
+ELEVENLABS_VOICE_STABILITY=0.5
+ELEVENLABS_VOICE_SIMILARITY_BOOST=0.9
+ELEVENLABS_VOICE_STYLE=0.66
+ELEVENLABS_VOICE_USE_SPEAKER_BOOST=false
+ELEVENLABS_OPTIMIZE_STREAMING_LATENCY=4
+ELEVENLABS_OUTPUT_FORMAT=pcm_16000
+
+TWITTER_DRY_RUN=false
+TWITTER_USERNAME= # Account username
+TWITTER_PASSWORD= # Account password
+TWITTER_EMAIL= # Account email
+TWITTER_COOKIES= # Account cookies
+
+X_SERVER_URL=
+XAI_API_KEY=
+XAI_MODEL=
+
+#POST INTERVAL RANDOM MIN-MAX MINUTES
+POST_INTERVAL_MIN=  #90 #Default
+POST_INTERVAL_MAX= #180 #Default
+
+
+#USE IMAGE GEN
+IMAGE_GEN= #TRUE
+
+#Leave blank to use local embeddings
+USE_OPENAI_EMBEDDING=  #TRUE
+
+#OpenRouter (Use one model for everything or set individual for small, medium, large tasks)
+#leave blank to use defaults hermes 70b for small tasks & 405b for medium/large tasks
+OPENROUTER_MODEL=
+SMALL_OPENROUTER_MODEL=
+MEDIUM_OLLAMA_MODEL=
+LARGE_OLLAMA_MODEL=
+
+
+#Set to Use for New OLLAMA provider
+OLLAMA_SERVER_URL=   #Leave blank for default localhost:11434
+OLLAMA_MODEL=
+OLLAMA_EMBEDDING_MODEL=     #default mxbai-embed-large
+#To use custom model types for different tasks set these
+SMALL_OLLAMA_MODEL=         #default llama3.2
+MEDIUM_OLLAMA_MODEL=        #default herems3
+LARGE_OLLAMA_MODEL=         #default hermes3:70b
+
+# For asking Claude stuff
+ANTHROPIC_API_KEY=
+
+# Heurist API (Get API Key at https://heurist.ai/dev-access)
+HEURIST_API_KEY=
+SMALL_HEURIST_LANGUAGE_MODEL=
+MEDIUM_HEURIST_LANGUAGE_MODEL=
+LARGE_HEURIST_LANGUAGE_MODEL=
+HEURIST_IMAGE_MODEL=
+
+WALLET_PRIVATE_KEY=EXAMPLE_WALLET_PRIVATE_KEY
+WALLET_PUBLIC_KEY=EXAMPLE_WALLET_PUBLIC_KEY
+
+BIRDEYE_API_KEY=
+
+SOL_ADDRESS=So11111111111111111111111111111111111111112
+SLIPPAGE=1
+BASE_MINT=So11111111111111111111111111111111111111112
+RPC_URL=https://api.mainnet-beta.solana.com
+HELIUS_API_KEY=
+
+
+## Telegram
+TELEGRAM_BOT_TOKEN=
+
+TOGETHER_API_KEY=
+SERVER_PORT=3000
+
+# Starknet
+STARKNET_ADDRESS=
+STARKNET_PRIVATE_KEY=
+
+# for RAG
+CONTEXT_ID=
+TABLE_ID=
+ETHEREUM_PRIVATE_KEY=
+LIT_TOKEN_ID=
+ORBIS_SEED=
+ORBIS_GATEWAY_URL="http://localhost:7008"
+CERAMIC_NODE_URL="https://ceramic-orbisdb-mainnet-direct.hirenodes.io/"

.gitignore

@@ -0,0 +1,7 @@
+*.ts
+!index.ts
+!character.ts
+.env
+node_modules
+
+db.sqlite

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 ai16z
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,221 @@
+# Eliza Starter with Protected Vector Database
+
+This concept is an iteration of the [Eliza Starter](https://github.com/elizaOS/eliza-starter) that integrates the following:
+
+- Uses a [fork of OrbisDB](https://github.com/mzkrasner/orbisdb) to create vectorized conversation logs
+- Uses [Lit Protocol](https://www.litprotocol.com/) to provide programmatic encryption and decryption of the vector database entries
+
+## Running the app
+
+In this section I will walk you through how to deploy and run this app.
+
+### Prerequisites
+
+To run this app, you need the following:
+
+1. An [OpenAI](https://platform.openai.com/) API key
+2. A modified [OrbisDB] instance (outlined below)
+3. Docker
+4. A [Lit](https://www.litprotocol.com/) token ID (also shown below)
+
+## Initial Setup
+
+To run the app locally, follow these steps:
+
+1. Clone this repo and install the dependencies
+
+```sh
+git clone https://github.com/ceramicstudio/eliza-with-lit-orbisdb
+cd eliza-with-lit-orbisdb
+pnpm install
+```
+
+2. In a separate terminal, clone this modified version of OrbisDB and install the dependencies
+
+```sh
+git clone https://github.com/mzkrasner/orbisdb
+cd orbisdb
+npm install
+```
+
+3. In your orbisdb terminal, start the database process
+
+```sh
+# Ensure that you have your Docker Daemon running in the background first
+npm run dev
+```
+
+Your OrbisDB instance will need to initially be configured using the GUI running on `localhost:7008`. Navigate to this address in your browser and follow these steps:
+
+a. For "Ceramic node URL" enter the following value: `https://ceramic-orbisdb-mainnet-direct.hirenodes.io/`
+
+This is the default public Ceramic node that the hosted Orbis Studio uses. We will leverage this to avoid the setup of a local Ceramic node
+
+b. For "Ceramic Seed" simply click "generate a new one" and go to the next page
+
+c. For "Database configuration" enter the following:
+
+```sh
+User=postgres
+Database=postgres
+Password=postgres
+Host=localhost
+Port=5432
+```
+
+These are the default configurations your Docker image is using. Entering them here enables you to configure your OrbisDB instance to index Ceramic stream data using your dockerized Postgres instance.
+
+Go to the next page
+
+d. Click next on the presets page (do not select anything)
+
+e. Connect with your Metamask account and click "Get started". Keep the Orbis Studio UI in your browser as we will navigate back to it later
+
+4. Go to your `orbis-lit-langchain` terminal and copy the example env file
+
+```sh
+cp .env.example.local .env
+```
+
+5. Navigate to your browser running the OrbisDB UI and create a new context. You can call this anything you want. Once saved, click into your new context and copy the value prefixed with "k" into your `.env` file
+
+```sh
+CONTEXT_ID="<your-context-id>"
+```
+
+Contexts allow developers to organize their data across different applications or projects. When this application uses the Orbis SDK to write embeddings, it will leverage this context when making the write request
+
+6. Next, we will create an OrbisDB seed to self-authenticate onto the Ceramic Network using the Orbis SDK
+
+```sh
+pnpm gen-seed
+```
+
+This is the seed the Orbis SDK will use to self-authenticate your OrbisDB client onto the Ceramic Network in order to perform writes.
+
+Copy only the array of numbers into your `.env` file
+
+```sh
+# enter as a string like "[2, 19, 140, 10...]"
+ORBIS_SEED="your-array-here"
+```
+
+Make sure the final number in your array does not contain a trailing comma
+
+7. Copy an active and funded OpenAI API key into your `.env` file next to `OPENAI_API_KEY`
+
+8. Choose or create a dummy Metamask address and claim Lit Protocol Testnet tokens using that address by visiting `https://chronicle-yellowstone-faucet.getlit.dev/`
+
+9. Navigate to `https://explorer.litprotocol.com/` in your browser and sign in with the same dummy address as the previous step. Once signed in, click "Mint a new PKP". After minting, copy the value under "Token ID" into your `.env` file
+
+```sh
+LIT_TOKEN_ID="<your-token-id>"
+```
+
+10. Grab the private key from your dummy Metamask wallet (used in the two steps above) and enter it into your `.env` file
+
+```sh
+ETHEREUM_PRIVATE_KEY="<your-private-key>"
+```
+
+11. Finally, deploy your OrbisDB data model we will use to create and query via vector search
+
+```sh
+pnpm deploy-model
+```
+
+This creates a new "table" in your OrbisDB instance by creating a Ceramic model stream using the model definition found in [scripts/deploy-model.mjs](./scripts/deploy-model.mjs).
+
+Copy the value prefixed with "k" into your `.env` file
+
+```sh
+TABLE_ID="<your-table-id>"
+```
+
+## Current Flow
+
+As you'll notice in the [src/index.ts](/src/index.ts) file, a new `StorageService` instance is created within the `handleUserInput` method to manage the conversation between the agent and the user, which entails:
+
+1. Vectorizing the initial user input
+2. Encrypting the input's plaintext values with Lit
+3. Storing the embedding + encrypted plaintext on OrbisDB
+
+After these steps, `getEmbeddingContext` is used to search the OrbisDB vectors to get relevant logs, and decrypting them if the default access control conditions are met. This is loaded as context when prompting the agent.
+
+Finally, after obtaining the response from the agent, the same steps are taken to vectorize, encrypt, and store the agent's response in OrbisDB
+
+### Considerations
+
+At the moment, the protected database flow is only integrated into conversations. However, this can be modified to query OrbisDB when responding to posts on X, making new posts, or performing other agent actions.
+
+## Access control
+
+At the moment, very simple access control conditions are being leveraged based on whether the wallet trying to read the data contains >=0.000001 ETH (found in the [access service](./src/services/access.service.ts)).
+
+```typescript
+const accessControlConditions = [
+  {
+    contractAddress: "",
+    standardContractType: "",
+    chain: "ethereum",
+    method: "eth_getBalance",
+    parameters: [":userAddress", "latest"],
+    returnValueTest: {
+      comparator: ">=",
+      value: "1000000000000", // 0.000001 ETH
+    },
+  },
+];
+```
+
+There is a wide array of access control conditions you can use or create. For more information, visit [Lit's Access Control documentation](https://developer.litprotocol.com/sdk/access-control/intro).
+
+## Edit the character files
+
+Open `agent/src/character.ts` to modify the default character. Uncomment and edit.
+
+### Custom characters
+
+To load custom characters instead:
+- Use `pnpm start --characters="path/to/your/character.json"`
+- Multiple character files can be loaded simultaneously
+
+### Add clients
+
+```diff
+- clients: [],
++ clients: ["twitter", "discord"],
+```
+
+## Duplicate the .env.example template
+
+```bash
+cp .env.example .env
+```
+
+\* Fill out the .env file with your own values.
+
+### Add login credentials and keys to .env
+
+```diff
+-DISCORD_APPLICATION_ID=
+-DISCORD_API_TOKEN= # Bot token
++DISCORD_APPLICATION_ID="000000772361146438"
++DISCORD_API_TOKEN="OTk1MTU1NzcyMzYxMT000000.000000.00000000000000000000000000000000"
+...
+-OPENROUTER_API_KEY=
++OPENROUTER_API_KEY="sk-xx-xx-xxx"
+...
+-TWITTER_USERNAME= # Account username
+-TWITTER_PASSWORD= # Account password
+-TWITTER_EMAIL= # Account email
++TWITTER_USERNAME="username"
++TWITTER_PASSWORD="password"
++TWITTER_EMAIL="[email protected]"
+```
+
+## Install dependencies and start your agent
+
+```bash
+pnpm i && pnpm start
+```