Skip to content

Commit

Permalink
Semantic Cache with MemoryDB
Browse files Browse the repository at this point in the history
  • Loading branch information
@LakshmiPeri23
LakshmiPeri23 committed Sep 30, 2024
1 parent ed5ee08 commit d1a39f6
Show file tree
Hide file tree
Showing 2 changed files with 64 additions and 177 deletions.
7 changes: 6 additions & 1 deletion libs/aws/langchain_aws/cache.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -266,5 +266,10 @@ def update(self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE) -> N
)
llm_cache = self._get_llm_cache(llm_string)

metadata = {"llm_string": llm_string, "prompt": prompt}
metadata = {
"llm_string": llm_string,
"prompt": prompt,
"return_val": dumps([g for g in return_val]),
}
llm_cache.add_texts(texts=[prompt], metadatas=[metadata])

234 changes: 58 additions & 176 deletions samples/inmemory/Semanticcache.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,14 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"id": "41f56fd0-efc1-4edb-9a99-d41de0954949",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"env: MEMORYDB_HOST=semanticcache.ghlaqp.clustercfg.memorydb.us-east-1.amazonaws.com\n",
"env: MEMORYDB_PORT=6379\n"
]
}
],
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%env MEMORYDB_HOST=semanticcache.ghlaqp.clustercfg.memorydb.us-east-1.amazonaws.com\n",
"%env MEMORYDB_HOST=your_cluster_endpoint\n",
"%env MEMORYDB_PORT=6379"
]
},
Expand Down Expand Up @@ -52,9 +45,11 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"id": "d10cf385-93ac-4b03-8532-30efebfb6061",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import os\n",
Expand All @@ -76,9 +71,11 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"id": "ccfa25cf-26aa-45f9-9a2a-df1a1606fa68",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# create the Anthropic Model\n",
Expand All @@ -92,9 +89,11 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"id": "2481182f-3432-4e59-a14e-b00f31d0b84f",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# use the Anthropic Claude model\n",
Expand All @@ -106,9 +105,11 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"id": "6ba15759-b194-4fd8-a9a6-1be417b47887",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# create a Titan Embeddings client\n",
Expand All @@ -125,29 +126,12 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"id": "27319029-fba1-401e-84b1-f63cfcfb1f17",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 60.8 ms, sys: 0 ns, total: 60.8 ms\n",
"Wall time: 67.2 ms\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%time\n",
"memorydb_host = os.environ.get(\"MEMORYDB_HOST\", \"localhost\")\n",
Expand All @@ -168,34 +152,12 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"id": "68fe670e-9c9c-4d2f-9c39-29c2279a9512",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Here are some key details about missions to the Moon:\n",
"\n",
"- The first human missions to the Moon were the crewed missions of the United States Apollo program between 1968-1972. A total of 12 astronauts walked on the lunar surface during the six successful NASA landing missions.\n",
"\n",
"- On July 20, 1969, Apollo 11 marked the first crewed landing on the Moon. Neil Armstrong became the first person to walk on the lunar surface, famously saying \"That's one small step for man, one giant leap for mankind.\"\n",
"\n",
"- The last crewed landing was Apollo 17 in December 1972. Since then, numerous uncrewed spacecraft have explored the Moon from orbit.\n",
"\n",
"- Some key uncrewed lunar missions include the Soviet Luna program, NASA's Lunar Reconnaissance Orbiter mapping the Moon since 2009, and recent missions like China's Chang'e 4 which landed on the far side of the Moon in 2019.\n",
"\n",
"- NASA is currently working towards the Artemis program which aims to land the first woman and next man on the lunar surface by 2024 and establish a sustainable human presence on the Moon by the end of the decade.\n",
"\n",
"- Future crewed lunar missions are planned not just for exploration but to test technologies and resources needed for eventual human exploration of Mars.\n",
"\n",
"So in summary, after the pioneering Apollo missions, there was a lull before robotic missions resumed lunar exploration paving the way for a new era of human Moon landings.\n",
"CPU times: user 44.2 ms, sys: 0 ns, total: 44.2 ms\n",
"Wall time: 24.3 s\n"
]
}
],
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%time\n",
"response=llm.invoke(\"Tell me about mission to moon\")\n",
Expand All @@ -212,9 +174,11 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"id": "4f00da6f-a370-418f-a305-fc2a59f0fc78",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"set_llm_cache(\n",
Expand All @@ -223,36 +187,22 @@
")"
]
},
{
"cell_type": "markdown",
"id": "0ac943fd-6ffe-4c99-ac5c-e9fbf93b7e44",
"metadata": {},
"source": [
"#### As a question to the LLM and Re-run the same block to see the improvemnt in response time. "
]
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"id": "0ad4cc96-69eb-4a47-ab6f-b21d5592d9fc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Here are some key details about missions to the Moon:\n",
"\n",
"- The first human missions to the Moon were the crewed missions of the United States Apollo program between 1968-1972. A total of 12 astronauts walked on the lunar surface during the six successful NASA landing missions.\n",
"\n",
"- On July 20, 1969, Apollo 11 marked the first crewed landing on the Moon. Neil Armstrong became the first person to walk on the lunar surface, famously saying \"That's one small step for man, one giant leap for mankind.\"\n",
"\n",
"- The last crewed landing was Apollo 17 in December 1972. Since then, numerous uncrewed spacecraft have explored the Moon from orbit.\n",
"\n",
"- Some key robotic lunar missions include the Soviet Luna program, NASA's Lunar Reconnaissance Orbiter and Lunar Crater Observation and Sensing Satellite, India's Chandrayaan-1, and China's Chang'e program which landed rovers on the Moon.\n",
"\n",
"- NASA's Artemis program aims to land the first woman and next man on the lunar surface by 2024 and establish sustainable lunar exploration with the eventual goal of sending astronauts to Mars.\n",
"\n",
"- Private companies like SpaceX, Blue Origin and others are also working on lunar lander vehicles for NASA's commercial lunar payload services.\n",
"\n",
"- Future international goals include establishing a crewed lunar base and testing technologies and resources utilization on the Moon as a stepping stone for eventual Mars exploration.\n",
"CPU times: user 159 ms, sys: 37.5 ms, total: 196 ms\n",
"Wall time: 320 ms\n"
]
}
],
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%time\n",
"response=llm.invoke(\"Tell me about mission to moon\")\n",
Expand All @@ -261,30 +211,10 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"id": "2c668188-5658-492d-8fce-bea9063d2cfc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The telescope was first invented in the Netherlands in the early 17th century, around 1608. However, there is some debate over who should get credit as the original inventor.\n",
"\n",
"The main contenders are:\n",
"\n",
"1) Hans Lippershey - A Dutch eyeglass maker who is often credited as the first to apply for a patent for his refracting telescope in 1608, though his patent request was denied.\n",
"\n",
"2) Zacharias Janssen - Another Dutch spectacle maker who may have invented a compound microscope and telescope around 1590, though the exact dates are uncertain.\n",
"\n",
"3) Jacob Metius - A Dutch professor and maker of optical instruments who claimed to have invented the telescope in 1608, around the same time as Lippershey.\n",
"\n",
"While there is no consensus on who was truly first, most historians give credit to Hans Lippershey as the first person known to have successfully applied for a patent for the refracting telescope in 1608. Galileo Galilei later made improvements and was one of the first to use the telescope for astronomical observations in 1609.\n",
"CPU times: user 6.98 ms, sys: 0 ns, total: 6.98 ms\n",
"Wall time: 82.6 ms\n"
]
}
],
"outputs": [],
"source": [
"%%time\n",
"response=llm.invoke(\"Who first invented a telescope\")\n",
Expand All @@ -293,30 +223,10 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"id": "6665d884-08fe-46a8-a6b4-fcf42df44d43",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The invention of the modern car is credited to several pioneers who contributed different components and concepts over time. However, some key figures stand out:\n",
"\n",
"1. Karl Benz (1844-1929) - A German engineer who patented the first gasoline-powered automobile in 1886. His three-wheeled Benz Patent Motorwagen is widely regarded as the first modern automobile.\n",
"\n",
"2. Gottlieb Daimler (1834-1900) - Another German engineer who independently designed and built a high-speed petrol engine in 1885, which he fitted to a bicycle, creating the first motorcycle. He later collaborated with Wilhelm Maybach to build the first four-wheeled automobile in 1889.\n",
"\n",
"3. Henry Ford (1863-1947) - An American engineer and industrialist who didn't invent the automobile itself, but revolutionized its production with the introduction of the assembly line and mass production techniques. His Model T, introduced in 1908, made cars affordable for the masses.\n",
"\n",
"4. Nicolas Joseph Cugnot (1725-1804) - A French engineer who built a steam-powered tricycle in 1769, which is considered the first self-propelled vehicle, although it was not a modern automobile.\n",
"\n",
"While many individuals contributed to the development of the automobile, Karl Benz is widely regarded as the inventor of the modern gasoline-powered automobile as we know it today.\n",
"CPU times: user 6.93 ms, sys: 57 µs, total: 6.99 ms\n",
"Wall time: 92.3 ms\n"
]
}
],
"outputs": [],
"source": [
"%%time\n",
"response=llm.invoke(\"Who first invented a car\")\n",
Expand All @@ -325,50 +235,22 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"id": "7d645f3c-d721-490c-a734-8c22158b227f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The telescope was first invented in the Netherlands in the early 17th century, around 1608. However, there is some debate over who should get credit as the original inventor.\n",
"\n",
"The main contenders are:\n",
"\n",
"1) Hans Lippershey - A Dutch eyeglass maker who is often credited as the first to apply for a patent for his refracting telescope in 1608, though his patent request was denied.\n",
"\n",
"2) Zacharias Janssen - Another Dutch spectacle maker who may have invented a compound microscope and telescope around 1590, though the exact dates are uncertain.\n",
"\n",
"3) Jacob Metius - A Dutch professor and maker of optical instruments who claimed to have invented the telescope in 1608, around the same time as Lippershey.\n",
"\n",
"While there is no consensus on who was truly first, most historians give credit to Hans Lippershey as the first person known to have successfully applied for a patent for the refracting telescope in 1608. Galileo Galilei later made improvements and was one of the first to use the telescope for astronomical observations in 1609.\n",
"CPU times: user 6.25 ms, sys: 417 µs, total: 6.67 ms\n",
"Wall time: 106 ms\n"
]
}
],
"outputs": [],
"source": [
"%%time\n",
"respone3=llm.invoke(\"Who first a Telescope\")\n",
"print(respone3.content)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6732c3e5-eee0-49cf-94b0-924e66606143",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": "conda_python3",
"language": "python",
"name": "python3"
"name": "conda_python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -380,7 +262,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
"version": "3.10.14"
}
},
"nbformat": 4,
Expand Down

0 comments on commit d1a39f6

Please sign in to comment.