Skip to content

Commit 05ee292

Browse files
NotoriousArnavNotoriousArnav
committedMar 6, 2024
Added 2 tools
- Image Generation - Code Generation
1 parent c030c11 commit 05ee292

File tree

6 files changed

+207
-0
lines changed

6 files changed

+207
-0
lines changed
 

‎cli.py

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
#TODO: Build a CLI Front end.

‎prompt.py

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
B_INST, E_INST = "<|im_start|>user\n", "<|im_end|>\n" #Instruction begin and end
2+
B_SYS, E_SYS = "<|im_start|>system\n", "<|im_end|>\n" #System message begin and end
3+
4+
# Define a new system prompt
5+
sys_msg = B_SYS + """You are Assistant named Aditya. Assistant is a expert JSON builder designed to assist with a wide range of tasks.
6+
7+
Assistant is able to respond to the User and use tools using JSON strings that contain "action" and "action_input" parameters.
8+
9+
All of Assistant's communication is performed using this JSON format. The assistant NEVER outputs anything other than a json object with an action and action_input fields!
10+
11+
Assistant can also use tools by responding to the user with tool use instructions in the same "action" and "action_input" JSON format. Tools available to Assistant are:
12+
13+
> duckduckgo_search: A wrapper around DuckDuckGo Search. Useful for when you need to answer questions about current events. Input should be a search query.
14+
> wikipedia: A wrapper around Wikipedia Search. Useful for when you need to answer questions about some topic. Input should be a search query.
15+
> code_block: A Markdown wrapper around Input Python Code BLock. Useful for when you need to provide a Python Code block in Markdown. Input should be a Code Block. Example: print("Hello World!") without any "```"
16+
> Word Length Tool: Use this tool when you need to find the length of a given word
17+
> Calculator: Use this tool when you need to evaluate a mathematical expression. The calculator uses python's numexpr library so make sure your inputs to this tool are in the correct format.
18+
> remember: Use this tool when you need to Value. The tool stores the value in the Conversational Window Buffer Memory.
19+
20+
Here is an example of a previous conversation between User and Assistant:
21+
---
22+
User: Hey how are you today?
23+
Assistant: ```json
24+
{{"action": "Final Answer",
25+
"action_input": "I'm good thanks, how are you?"}}
26+
```
27+
User: I'm great, what is the length of the word educate?
28+
Assistant: ```json
29+
{{"action": "Word Length Tool",
30+
"action_input": "educate"}}
31+
```
32+
User: 7
33+
Assistant: ```json
34+
{{"action": "Final Answer",
35+
"action_input": "It looks like the answer is 7!"}}
36+
```
37+
User: Who is Olivia Wilde's husband?
38+
Assistant: ```json
39+
{{"action": "duckduckgo_search",
40+
"action_input": "Who is Olivia Wilde's husband"}}
41+
```
42+
User: September 21, 2022: Olivia Wilde says kids\' happiness remains top priority for her and Jason Sudeikis. During an appearance on The Kelly Clarkson Show, Wilde talked about the ups and downs of ... Olivia Wilde is "quietly dating again" following her November 2022 split from. Harry Styles, a source exclusively tells Life & Style. "The man she\'s with is \'normal\' by Hollywood ... Wilde honored Sudeikis with a sweet tribute on his 42nd birthday. "I have approximately one billion pictures of this guy, my partner in life-crime, who was born on this day in 1975, but this one ... November 18, 2022: Olivia Wilde and Harry Styles are "taking a break". After nearly two years together, Wilde and Styles are pressing pause on their romance. Multiple sources confirmed exclusively ... Wilde told Allure in October 2013 that when she first met Sudeikis she "thought he was so charming." "He\'s a great dancer, and I\'m a sucker for great dancers," she said at the time. "But he didn\'t ...
43+
Assistant: ```json
44+
{{"action": "Final Answer",
45+
"action_input": "Jason Sudeikis"}}
46+
```
47+
User: What is his age raised to the 3rd power?
48+
Assistant: ```json
49+
{{"action": "duckduckgo_search",
50+
"action_input": "What is Jason Sudeikis age?"}}
51+
```
52+
User: Jason Sudeikis (born September 18, 1975, Fairfax, Virginia, U.S.) American comedian, actor, and writer who first garnered attention for his work (2003-13) on the TV show Saturday Night Live ( SNL) and later starred in the hugely popular series Ted Lasso (2020-23). Early life and improv comedy Parents Jason Sudeikis and Olivia Wilde\'s 2 Kids: All About Daisy and Otis Jason Sudeikis and Olivia Wilde welcomed two children together before their split in 2020 By Jacqueline Weiss... The Ted of 2013 is a classic Jason Sudeikis character of the time. Sudeikis came up on Saturday Night Live , where he made the most of his 6-foot height and square jaw by largely playing jerks. Sudeikis looks a little bleary-eyed after flying in from Los Angeles with his children, Otis and Daisy, aged nine and six, but he is assiduously polite and attentive. His language can be quaintly... Jason Sudeikis rose to fame on "Saturday Night Live" with his comedic talent, and his children seemed to have inherited it. Sudeikis, who shares kids Otis, 9, and Daisy, 7, with ex Olivia ...
53+
Assistant: ```json
54+
{{"action": "Calculator",
55+
"action_input": "47**3"}}
56+
```
57+
User: 103823
58+
Assistant: ```json
59+
{{"action": "Final Answer",
60+
"action_input": "103823"}}
61+
```
62+
---
63+
Assistant is very intelligent and knows it's limitations, so it will always try to use a tool when applicable, even if Assistant thinks it knows the answer!
64+
Notice that after Assistant uses a tool, User will give the output of that tool. Then this output can be returned as a final answer.
65+
Assistant will only use the available tools and NEVER a tool not listed. If the User's question does not require the use of a tool, Assistant will use the "Final Answer" action to give a normal response.
66+
""" + E_SYS

‎starcoder_langchain_poc.py

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
from langchain_community.llms import HuggingFaceEndpoint
2+
3+
sc2_3b = HuggingFaceEndpoint(
4+
repo_id = "bigcode/starcoder2-3b",
5+
temperature= 0.1
6+
)
7+
8+
sc2_15b = HuggingFaceEndpoint(
9+
repo_id = "bigcode/starcoder2-15b",
10+
temperature= 0.1
11+
)

‎tools/__init__.py

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
"""
2+
Tools Folder.
3+
This will contain all the Tools that Are Required to make this ChatBot Kiccck the Ass of ChatGPT!
4+
"""
5+
from langchain_community.tools import DuckDuckGoSearchRun
6+
from .starcoder import StarCoder_2_15B, StarCoder_2_3B
7+
from .imagegen import *
8+
9+
tools = [
10+
DuckDuckGoSearchRun(),
11+
StarCoder_2_15B(),
12+
StarCoder_2_3B(),
13+
StableDiffusionLofiGirl(),
14+
StudioGhibliArt(),
15+
StableDiffusion()
16+
]

‎tools/imagegen.py

+80
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
from langchain_community.tools import BaseTool
2+
# from PIL import Image
3+
import requests
4+
import os
5+
import io
6+
7+
class StableDiffusionLofiGirl(BaseTool):
8+
name = "Stable Diffusion with Lofi Girl Adapter Image Generation"
9+
description = "Generate images using the Stable Diffusion model with the Lofi Girl adapter and upload them to 0x0.st. Returns the URL of the uploaded image."
10+
11+
def _run(self, prompt: str, token='') -> str:
12+
if not token:
13+
token = os.getenv("HUGGINGFACEHUB_API_TOKEN")
14+
api_url = "https://api-inference.huggingface.co/models/Norod78/SDXL-LofiGirl-Lora"
15+
headers = {"Authorization": f"Bearer {token}"}
16+
response = requests.post(api_url, headers=headers, json={"inputs": prompt})
17+
image_bytes = response.content
18+
19+
upload_url = "https://0x0.st/"
20+
files = {'file': image_bytes}
21+
upload_response = requests.post(upload_url, files=files, timeout=120)
22+
uploaded_image_url = upload_response.text.strip()
23+
24+
return uploaded_image_url
25+
26+
async def _arun(self, *args, **kwargs):
27+
raise NotImplementedError()
28+
29+
class StableDiffusion(BaseTool):
30+
name = "Stable Diffusion Image Generation"
31+
description = "Generate images using the Stable Diffusion model and upload them to 0x0.st. Returns the URL of the uploaded image."
32+
33+
def _run(self, prompt: str, token='') -> str:
34+
if not token:
35+
token = os.getenv("HUGGINGFACEHUB_API_TOKEN")
36+
api_url = "https://api-inference.huggingface.co/models/stabilityai/stable-diffusion-xl-base-1.0"
37+
headers = {"Authorization": f"Bearer {token}"}
38+
response = requests.post(api_url, headers=headers, json={"inputs": prompt})
39+
image_bytes = response.content
40+
41+
upload_url = "https://0x0.st/"
42+
files = {'file': image_bytes}
43+
upload_response = requests.post(upload_url, files=files, timeout=120)
44+
uploaded_image_url = upload_response.text.strip()
45+
46+
return uploaded_image_url
47+
48+
async def _arun(self, *args, **kwargs):
49+
raise NotImplementedError()
50+
51+
class StudioGhibliArt(BaseTool):
52+
name = "Studio Ghilbli Art"
53+
description = "Image Generation using Stable Diffusion. Uses a LoRa that produces the Output in Studio Ghibli Art Style. Returns an Image URL"
54+
55+
def _run(self, prompt:str, token='') -> str:
56+
if not token:
57+
token = os.getenv("HUGGINGFACEHUB_API_TOKEN")
58+
API_URL = "https://api-inference.huggingface.co/models/KappaNeuro/studio-ghibli-style"
59+
headers = {"Authorization": f"Bearer {token}"}
60+
response = requests.post(
61+
API_URL,
62+
headers=headers,
63+
json={
64+
"inputs": f"Studio Ghibli Style -{prompt}.Studio Ghibli Cel Style"
65+
},
66+
timeout=120
67+
)
68+
data = io.BytesIO(response.content)
69+
response = requests.post(
70+
'https://0x0.st/',
71+
files={'file':data},
72+
timeout = 120
73+
)
74+
return str(response.content.strip())
75+
76+
async def _arun(self, *args, **kwargs):
77+
raise NotImplementedError()
78+
79+
#TODO: Add Other Loras. Refer: https://huggingface.co/spaces/multimodalart/LoraTheExplorer
80+

‎tools/starcoder.py

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
from langchain_community.llms import HuggingFaceEndpoint
2+
from langchain_community.tools import BaseTool
3+
4+
5+
class StarCoder_2_15B(BaseTool):
6+
name = "starcoder2-15b"
7+
description = """StarCoder2-15b is an AI Assistant that knows Several Programming languages.
8+
Use this Assistant's Help to Generate Code.
9+
This is assistant is Trained on 15 Billion parameters
10+
Beware: This Tool can sometimes generate code that might not be correct or repetative, so you need to verify that before you return this to user
11+
"""
12+
llm = HuggingFaceEndpoint(repo_id="bigcode/starcoder2-3b")
13+
14+
def _run(self, prompt: str) -> str:
15+
return self.llm.invoke(prompt)
16+
17+
def _arun(self, prompt: str) -> str:
18+
raise NotImplementedError()
19+
20+
class StarCoder_2_3B(BaseTool):
21+
name = "starcoder2-3b"
22+
description = """StarCoder2-3b is an AI Assistant that knows Several Programming languages.
23+
Use this Assistant's Help to Generate Code.
24+
This is assistant is Trained on 3 Billion parameters
25+
Beware: This Tool can sometimes generate code that might not be correct or repetative, so you need to verify that before you return this to user
26+
"""
27+
llm = HuggingFaceEndpoint(repo_id="bigcode/starcoder2-3b")
28+
29+
def _run(self, prompt: str, **kwargs) -> str:
30+
return self.llm.invoke(prompt)
31+
32+
def _arun(self, prompt: str, **kwargs) -> str:
33+
raise NotImplementedError()

0 commit comments

Comments
 (0)
Please sign in to comment.