Skip to content

Commit

Permalink
Merge pull request #682 from Teradata/consumer-compl-gemini
Browse files Browse the repository at this point in the history 
Consumer complaints with google gemini
  • Loading branch information
@DallasBowden
DallasBowden authored Aug 21, 2024
2 parents d9a57c1 + 500a8b6 commit b26fd36
Show file tree
Hide file tree
Showing 45 changed files with 10,758 additions and 495 deletions.
Binary file removed UseCases/Complaints_Analysis_GenAI/audio_files/Conversation_5.mp3
View file
Binary file not shown.
4 changes: 0 additions & 4 deletions UseCases/Complaints_Analysis_GenAI/requirements.txt
View file

This file was deleted.

86 changes: 42 additions & 44 deletions ...enAI/Complaint_Analysis_Customer360.ipynb → ...rock/Complaint_Analysis_Customer360.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,19 +53,8 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install -r requirements.txt --quiet"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61ae8bca-87dd-4771-a7c4-4f7c2c0cc641",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!pip install pyopenssl cryptography --force-reinstall --quiet"
"%%capture\n",
"!pip install --upgrade -r requirements.txt --quiet"
]
},
{
Expand Down Expand Up @@ -318,20 +307,23 @@
"source": [
"# Sentiment\n",
"for i in tqdm(range(len(pd_df))):\n",
" prompt = f'''\n",
" User prompt: \n",
" The following is text from a review:\n",
" try:\n",
" prompt = f'''\n",
" User prompt: \n",
" The following is text from a review:\n",
"\n",
" “{pd_df['consumer_complaint_narrative'][i]}”\n",
" “{pd_df['consumer_complaint_narrative'][i]}”\n",
"\n",
" Categorize the review as one of the following:\n",
" Categorize the review as one of the following:\n",
"\n",
" Positive\n",
" Negative\n",
" Neutral\n",
" '''\n",
" Positive\n",
" Negative\n",
" Neutral\n",
" '''\n",
"\n",
" pd_df['Sentiment'][i] = ai21(prompt = prompt)"
" pd_df['Sentiment'][i] = ai21(prompt = prompt)\n",
" except:\n",
" pass"
]
},
{
Expand All @@ -343,23 +335,26 @@
"source": [
"# Topic\n",
"for i in tqdm(range(len(pd_df))):\n",
" prompt = f'''\n",
" User prompt:\n",
" The following is text from a complaint:\n",
" try:\n",
" prompt = f'''\n",
" User prompt:\n",
" The following is text from a complaint:\n",
"\n",
" “{pd_df['consumer_complaint_narrative'][i]}”\n",
"\n",
" “{pd_df['consumer_complaint_narrative'][i]}”\n",
" Identify the topic of the complaint and categorize into one of the following topics. Only output one of the following options:\n",
"\n",
" Identify the topic of the complaint and categorize into one of the following topics. Only output one of the following options:\n",
" - Mortgage Application\n",
" - Payment Trouble\n",
" - Mortgage Closing\n",
" - Report Inaccuracy\n",
" - Payment Struggle\n",
"\n",
" - Mortgage Application\n",
" - Payment Trouble\n",
" - Mortgage Closing\n",
" - Report Inaccuracy\n",
" - Payment Struggle\n",
" \n",
" Return just one of the above options\n",
"'''\n",
" pd_df['Topic'][i] = ai21(prompt = prompt)"
" Return just one of the above options\n",
" '''\n",
" pd_df['Topic'][i] = ai21(prompt = prompt)\n",
" except:\n",
" pass"
]
},
{
Expand All @@ -371,13 +366,16 @@
"source": [
"# Summary\n",
"for i in tqdm(range(len(pd_df))):\n",
" prompt = f'''\n",
" The following is text from a Bank Review:\n",
" “{pd_df['consumer_complaint_narrative'][i]}”\n",
" Summarize the Bank Review in one sentence\n",
" '''\n",
" try:\n",
" prompt = f'''\n",
" The following is text from a Bank Review:\n",
" “{pd_df['consumer_complaint_narrative'][i]}”\n",
" Summarize the Bank Review in one sentence\n",
" '''\n",
"\n",
" pd_df['Summary'][i] = ai21(prompt = prompt)"
" pd_df['Summary'][i] = ai21(prompt = prompt)\n",
" except:\n",
" pass"
]
},
{
Expand Down Expand Up @@ -435,7 +433,7 @@
"id": "06cbb0f4-d026-4a0b-8ab8-6982b7f7777a",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Save the results back to Vantage.</p>"
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Now the results can be saved back to Vantage.</p>"
]
},
{
Expand Down
23 changes: 2 additions & 21 deletions ..._GenAI/Complaint_Analysis_Dashboard.ipynb → ...edrock/Complaint_Analysis_Dashboard.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"source": [
"<header>\n",
" <p style=\"font-size: 36px; font-family: Arial; color: #F0F0F0; background-color: #00233c; padding: 20pt;\">\n",
" Customer Complaint Analyzer with Gen-AI\n",
" Customer Complaint Analyzer with Gen-AI (Amazon Bedrock)\n",
" <br>\n",
" <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
" </p>\n",
Expand Down Expand Up @@ -45,25 +45,6 @@
" <img src=\"./images/architecture_bedrock_audio.png\" alt=\"Speech Recognition\" style=\"width: 100%; height: auto; margin-bottom: 10px;\" />\n",
" <p><em>Speech Analysis with AWS Bedrock</em></p>\n",
" <div style=\"text-align: right;\">\n",
" <a href=\"./Speech_Recognition_AWS_Bedrock.ipynb\" style=\"padding: 10px; background-color: #00233c; color: #FFFFFF; text-decoration: none; border-radius: 5px;\">\n",
" Open Demo\n",
" </a>\n",
" </div>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"id": "b836f94d-9d43-4ce8-a681-88b8298d3a02",
"metadata": {},
"source": [
"<!-- Tile 3 -->\n",
"<div style=\"border: 1px solid #00233c; padding: 20px; background-color: #F5F5F5; margin-bottom: 20px;\">\n",
" <h2>Speech Analysis with Gemini</h2>\n",
" <p style='font-size:16px;font-family:Arial;color:#00233C'>This demo shows Speech Recognition using <b>Teradata Vantage</b> on Google and <b>Google Gemini LLM model</b> is a cutting-edge technology that leverages the power of cloud computing and advanced algorithms to accurately recognize and transcribe spoken words. This innovative solution integrates the capabilities of Teradata Vantage, Google Cloud, and Gemini to provide a robust and efficient speech recognition system.</p>\n",
" <img src=\"./images/voice-note.jpg\" alt=\"Speech Recognition\" style=\"width: 100%; height: auto; margin-bottom: 10px;\" />\n",
" <p><em>Speech Analysis with Google Gemini</em></p>\n",
" <div style=\"text-align: right;\">\n",
" <a href=\"./Speech_Recognition.ipynb\" style=\"padding: 10px; background-color: #00233c; color: #FFFFFF; text-decoration: none; border-radius: 5px;\">\n",
" Open Demo\n",
" </a>\n",
Expand All @@ -83,7 +64,7 @@
" <img src=\"./images/sentiment_analysis.png\" alt=\"Sentiment Analysis Bedrock\" style=\"width: 100%; height: auto; margin-bottom: 10px;\" />\n",
" <p><em>Sentiment Analysis Using Vantage and Amazon Bedrock</em></p>\n",
" <div style=\"text-align: right;\">\n",
" <a href=\"./Sentiment_Analysis_Bedrock.ipynb\" style=\"padding: 10px; background-color: #00233c; color: #FFFFFF; text-decoration: none; border-radius: 5px;\">\n",
" <a href=\"./Sentiment_Analysis.ipynb\" style=\"padding: 10px; background-color: #00233c; color: #FFFFFF; text-decoration: none; border-radius: 5px;\">\n",
" Open Demo\n",
" </a>\n",
" </div>\n",
Expand Down
84 changes: 35 additions & 49 deletions ...lysis_GenAI/Complaint_Summarization.ipynb → ...nAI_Bedrock/Complaint_Summarization.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,19 +64,9 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install -r requirements.txt --quiet"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61ae8bca-87dd-4771-a7c4-4f7c2c0cc641",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!pip install pyopenssl cryptography --force-reinstall --quiet"
"%%capture\n",
"\n",
"!pip install --upgrade -r requirements.txt --quiet"
]
},
{
Expand Down Expand Up @@ -135,6 +125,7 @@
"outputs": [],
"source": [
"%run -i ../startup.ipynb\n",
"\n",
"eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n",
"print(eng)\n",
"execute_sql('''SET query_band='DEMO=Complaint_Summarization.ipynb;' UPDATE FOR SESSION;''')"
Expand Down Expand Up @@ -284,7 +275,7 @@
"metadata": {},
"outputs": [],
"source": [
"df = DataFrame(in_schema('DEMO_ComplaintAnalysis', 'Consumer_Complaints')).to_pandas(num_rows = 200)\n",
"df = DataFrame(in_schema('DEMO_ComplaintAnalysis', 'Consumer_Complaints')).to_pandas(num_rows = 20)\n",
"df['Summary'] = \"\"\n",
"df['Reasoning with Chain of Thought'] = \"\""
]
Expand All @@ -297,33 +288,36 @@
"outputs": [],
"source": [
"for i in tqdm(range(len(df))):\n",
" prompt = f'''\n",
" The following is text from a Bank Review:\n",
" \n",
" “{df['consumer_complaint_narrative'][i]}”\n",
" \n",
" Give me reasoning as well as summary for this review.\n",
" \n",
" Instructions for Reasoning:\n",
" - Give me Reasoning in short\n",
" - Only one sentence reasoning\n",
" Instructions for Summary:\n",
" - A short one sentence Summary of everything the review states.\n",
" \n",
" My output comes in the format:\n",
" Summary: ,\n",
" Reasoning: ,\n",
" '''\n",
" try:\n",
" prompt = f'''\n",
" The following is text from a Bank Review:\n",
"\n",
" output = mistral(prompt = prompt)\n",
" summary = re.search('Summary:(.*)', output).group(1)\n",
" if summary == \"\":\n",
" summary = re.search('Summary:\\n(.*)', output).group(1)\n",
" reasoning = re.search('Reasoning:(.*)', output).group(1)\n",
" if reasoning == \"\":\n",
" reasoning = re.search('Reasoning:\\n(.*)', output).group(1)\n",
" df['Summary'][i] = summary\n",
" df['Reasoning with Chain of Thought'][i] = reasoning"
" “{df['consumer_complaint_narrative'][i]}”\n",
"\n",
" Give me reasoning as well as summary for this review.\n",
"\n",
" Instructions for Reasoning:\n",
" - Give me Reasoning in short\n",
" - Only one sentence reasoning\n",
" Instructions for Summary:\n",
" - A short one sentence Summary of everything the review states.\n",
"\n",
" My output comes in the format:\n",
" Summary: ,\n",
" Reasoning: ,\n",
" '''\n",
"\n",
" output = mistral(prompt = prompt)\n",
" summary = re.search('Summary:(.*)', output).group(1)\n",
" if summary == \"\":\n",
" summary = re.search('Summary:\\n(.*)', output).group(1)\n",
" reasoning = re.search('Reasoning:(.*)', output).group(1)\n",
" if reasoning == \"\":\n",
" reasoning = re.search('Reasoning:\\n(.*)', output).group(1)\n",
" df['Summary'][i] = summary\n",
" df['Reasoning with Chain of Thought'][i] = reasoning\n",
" except:\n",
" pass"
]
},
{
Expand Down Expand Up @@ -390,7 +384,7 @@
"id": "06cbb0f4-d026-4a0b-8ab8-6982b7f7777a",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Save the results back to Vantage.</p>"
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Now the results can be saved back to Vantage.</p>"
]
},
{
Expand Down Expand Up @@ -467,14 +461,6 @@
" </div>\n",
"</footer>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ff3c6ae7-6de8-4d81-9aba-e1cd28fc7dd7",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
Loading

0 comments on commit b26fd36

Please sign in to comment.