4_ChatGPT Prompt to get Datasets for Machine Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7de14f8b",
   "metadata": {},
   "source": [
    "In the age of AI, many of our tasks have been automated especially after the launch of ChatGPT. One such tool that uses the power of ChatGPT to ease data manipulation task in Python is PandasAI. It leverages the power of ChatGPT to generate Python code and executes it. The output of the generated code is returned. Pandas AI helps performing tasks involving pandas library without explicitly writing lines of code. In this jupyter notebook, we will discuss about how one can use Pandas AI to simplify data manipulation.\n",
    "\n",
    "What is Pandas AI?\n",
    "\n",
    "Using generative AI models from OpenAI, Pandas AI is a pandas library addition. With simply a text prompt, you can produce insights from your dataframe. It utilises the <b>OpenAI-developed text-to-query generative AI<b>. \n",
    "    \n",
    "The preparation of the data for analysis is a labor-intensive process for data scientists and analysts. Now they can carry on with their data analysis. Data experts may now leverage many of the methods and techniques they have studied to cut down on the time needed for data preparation thanks to Pandas AI. PandasAI should be used in conjunction with Pandas, not as a substitute for Pandas. Instead of having to manually traverse the dataset and react to inquiries about it, you can ask PandasAI these questions, and it will provide you answers in the form of Pandas DataFrames. Pandas AI wants to make it possible for you to visually communicate with a machine that will then deliver the desired results rather than having to program the work yourself. To do this, it uses the OpenAI GPT API to generate the code using Pandas library in Python and run this code in the background. The results are then returned which can be saved inside a variable.\n",
    "\n",
    "How Can I use Pandas AI in my projects\n",
    "1. Install and Import of Pandas AI library in python environment\n",
    "Execute the following command in your jupyter notebook to install pandasai library in python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "93c20dd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f9c5774c",
   "metadata": {},
   "outputs": [],
   "source": [
    "openai.api_key = \"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\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a4125ed6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message):\n",
    "\tresponse = openai.ChatCompletion.create(\n",
    "\t\tmodel=\"gpt-3.5-turbo\",\n",
    "\t\tmessages=[\n",
    "\t\t\t{\"role\": \"user\", \"content\": f\"{message}\"},\n",
    "\t\t]\n",
    "\t)\n",
    "\treturn response['choices'][0]['message']['content']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "784b5ed6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Subject: Request for Information on Upcoming Conference\n",
      "\n",
      "Dear [Conference Organizer's Name],\n",
      "\n",
      "I hope this letter finds you well. I am writing to inquire about an upcoming conference that has been generating a lot of buzz in our industry. As an enthusiastic professional eager to expand my knowledge and network, I am keen to learn more about this event and explore the possibilities it offers.\n",
      "\n",
      "Firstly, I would appreciate if you could provide me with information regarding the conference's theme, objectives, and target audience. Understanding the core focus of the event will help me evaluate its relevance to my current professional goals. Additionally, I would like to know if there are any industry experts or renowned speakers who have been confirmed as presenters or panelists. This knowledge will assist me in gauging the expertise and quality of the discussions and sessions.\n",
      "\n",
      "Furthermore, could you kindly supply the dates, location, and duration of the conference? Considering my work commitments and travel arrangements, having this information will enable me to plan my attendance effectively. I would also like to inquire about the registration fees and any early-bird discounts available for participants. Understanding the financial aspects will assist me in securing the necessary funds for my attendance.\n",
      "\n",
      "Lastly, I would be grateful if you could provide me with details about any networking opportunities or interactive sessions planned within the conference. Building connections and engaging in meaningful discussions with industry professionals are essential components for maximizing the benefits of such events, and I would like to know if these are included in the conference itinerary.\n",
      "\n",
      "Please feel free to include any additional information or promotional material that will help to familiarize me with the conference. Should there be any updates or changes in the event's schedule, kindly keep me informed.\n",
      "\n",
      "Thank you for your attention to this matter. I eagerly await your response with the requested details. I believe that attending this conference will be a valuable experience that will contribute significantly to my professional growth.\n",
      "\n",
      "Yours sincerely,\n",
      "\n",
      "[Your Name]\n",
      "[Your Contact Information]\n"
     ]
    }
   ],
   "source": [
    "res = chat('letter')\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "520ef543",
   "metadata": {},
   "source": [
    "Prompts to Gather/Generate Datasets for Machine Learning\n",
    "\n",
    "# Prompt 1:\n",
    "Create a list of datasets that can be used to train {topic} models. Ensure that the datasets are available in CSV format. The objective is to use this dataset to learn about {topic}. Also, provide links to the dataset if possible. Create the list in tabular form with the following columns: Dataset name, dataset, URL, dataset description"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ea4cebbd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset Name | Dataset URL | Dataset Description\n",
      "--- | --- | --- \n",
      "Titanic: Machine Learning from Disaster | https://www.kaggle.com/c/titanic/data | Contains information about passengers aboard the RMS Titanic such as age, sex, passenger class, and survival status. The objective is to predict whether a passenger survived or not.\n",
      "Bank Marketing Dataset | https://archive.ics.uci.edu/ml/datasets/Bank+Marketing | This dataset is related to a direct marketing campaign of a Portuguese banking institution. It includes attributes like age, job, marital status, and previous marketing outcomes. The goal is to predict whether a client will subscribe to a term deposit or not.\n",
      "Breast Cancer Wisconsin (Diagnostic) | https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic) | This dataset contains features computed from digitized images of fine needle aspirates (FNA) of breast masses. The task is to predict whether a mass is benign or malignant.\n",
      "Australian Credit Approval | https://archive.ics.uci.edu/ml/datasets/Statlog+(Australian+Credit+Approval) | This dataset contains information about credit card applications. Features include attributes like income, employment status, and credit account balances. The objective is to predict whether an application will be approved or not.\n",
      "Default of Credit Card Clients | https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients | This dataset contains information on credit card clients in Taiwan. Features include repayment status, credit limit, and payment history. The goal is to predict whether a client will default on the next payment or not.\n",
      "Pima Indians Diabetes Database | https://www.kaggle.com/uciml/pima-indians-diabetes-database | This dataset contains data from female Pima Indian adults. Features include measurements like glucose concentration, blood pressure, and BMI. The objective is to predict whether a person has diabetes or not.\n",
      "German Credit | https://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data) | This dataset includes information about male individuals who have applied for credit in Germany. Features comprise attributes like age, job, and credit history. The task is to predict whether a credit applicant is a good or bad credit risk.\n",
      "Online Shoppers Purchasing Intention Dataset | https://archive.ics.uci.edu/ml/datasets/Online+Shoppers+Purchasing+Intention+Dataset | This dataset contains features extracted from the browsing behavior of online shoppers. Attributes include the number of pages viewed, duration, and page type. The objective is to predict whether an online shopper will make a purchase or not.\n"
     ]
    }
   ],
   "source": [
    "prompt ='''\n",
    "Create a list of datasets that can be used to train logistic regression models.\n",
    "Ensure that the datasets are available in CSV format.\n",
    "The objective is to use this dataset to learn about logistic regression models\n",
    "and related nuances such as training the models. Also provide links to the dataset if possible.\n",
    "Create the list in tabular form with following columns:\n",
    "Dataset name, dataset, URL, dataset description\n",
    "'''\n",
    "res = chat(prompt)\n",
    "print(res)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de062b57",
   "metadata": {},
   "source": [
    "# Prompt 2:\n",
    "Generate a dummy dataset to train and test a {machine learning model name} for educational purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "30be45ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sure! Here's an example of a dummy dataset that you can use to train and test a logistic regression model. It consists of two numerical features and a binary target variable.\n",
      "\n",
      "```plaintext\n",
      "feature1,feature2,target\n",
      "1.2,3.4,1\n",
      "4.5,6.7,0\n",
      "2.1,5.6,1\n",
      "5.5,7.8,0\n",
      "3.6,4.5,0\n",
      "6.7,8.9,1\n",
      "4.3,2.1,1\n",
      "2.8,6.3,0\n",
      "```\n",
      "\n",
      "You can copy this data into a text editor and save it as a CSV file named \"dummy_dataset.csv\".\n"
     ]
    }
   ],
   "source": [
    "res = chat('generate a dummy dataset to train and test a logistic regression model\\\n",
    "for educational purposes. Ensure that the dataset is available in csv format')\n",
    "print(res)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79fad6d2",
   "metadata": {},
   "source": [
    "# Prompt 3:\n",
    "List down datasets to practice {topic}, and the if possible also attach dataset links and descriptions. Create the list in tabular format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "9f817a45",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "| Dataset Name                    | Dataset Link                                                                                                          | Description                                                                                                                         |\n",
      "|-----------------------------|-----------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|\n",
      "| COCO (Common Objects in Context) | [Link](http://cocodataset.org/#home)                                                                              | COCO is a large-scale object detection, segmentation, and captioning dataset with over 200,000 labeled images and 80 object categories.                                                                                                         |\n",
      "| Pascal VOC Dataset                     | [Link](http://host.robots.ox.ac.uk/pascal/VOC/)                                                               | Pascal VOC dataset contains labeled images for object detection and classification. It consists of 20 object classes and offers benchmark task datasets for object detection, segmentation, and classification. |\n",
      "| ImageNet                            | [Link](http://www.image-net.org/)                                                                               | ImageNet is a large dataset of over 14 million annotated images. While it is widely used for image classification, it can also be used for object detection tasks by bounding box annotations.                      |\n",
      "| Open Images Dataset                   | [Link](https://storage.googleapis.com/openimages/web/index.html)                                           | Open Images is a dataset containing millions of labeled images across thousands of classes. It is a great resource for object detection as it covers a wide range of object categories.              |\n",
      "| KITTI Vision Benchmark Suite   | [Link](http://www.cvlibs.net/datasets/kitti/)                                                                      | KITTI dataset focuses on autonomous driving scenarios and provides labeled data for object detection, tracking, road segmentation, and more.                                                               |\n",
      "| Microsoft COCO Text                      | [Link](https://visualstudio.microsoft.com/services/intellicode/)                                        | COCO Text dataset provides images with per-character bounding boxes and segmentation masks for text regions. It is specialized for document analysis and scene text recognition tasks.                        |\n",
      "| MIO-TCD                             | [Link](http://podoce.dinf.usherbrooke.ca/challenge/dataset/)                                            | The MIO-TCD dataset focuses on traffic and vehicle object detection. It contains over 9,200 images labeled with 20 vehicle classes and 7 non-vehicle classes in the context of traffic scenes.      |\n",
      "| WIDER FACE                          | [Link](http://shuoyang1213.me/WIDERFACE/)                                                                | WIDER FACE dataset is dedicated to face detection. It consists of 32,203 images and provides bounding box annotations for faces in various poses, occlusion levels, and different environmental settings.  |\n"
     ]
    }
   ],
   "source": [
    "prompt ='''\n",
    "List down datasets to practice object detection,\n",
    "if possible also attach dataset links and description.\n",
    "Create the list in tabular format\n",
    "'''\n",
    "res = chat(prompt)\n",
    "print(res)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "330e9e71",
   "metadata": {},
   "source": [
    "# Prompt 4:\n",
    "Create a list of datasets for practicing on {topic}. Make sure they are available in CSV format. Also, provide links to the dataset.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b409a666",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. Hindi-English Parallel Corpora: This dataset contains parallel sentences in English and Hindi languages. It consists of multiple domains such as News, Medical, Tourism, and more. The dataset is available in text format and can be found at: https://huggingface.co/datasets/cc_mt\n",
      "\n",
      "2. IIT Bombay English-Hindi Parallel Corpus: This dataset is provided by the Center for Indian Language Technology at IIT Bombay. It contains approximately 1.5 million sentence pairs in English and Hindi. The dataset is available in text format and can be found at: http://www.cfilt.iitb.ac.in/iitb_parallel/ \n",
      "\n",
      "3. Indic NLP Library Parallel Corpora: This repository contains parallel corpora for multiple Indian languages, including English to Hindi translation datasets. It includes different domains such as movies, law, science, and more. The dataset is available in text format and can be accessed at: https://indicnlp.ai4bharat.org/\n",
      "\n",
      "4. Common Crawl Corpus: Common Crawl dataset provides a large-scale web crawl corpus in multiple languages. It includes English-Hindi parallel data and can be extracted from the corpus using appropriate filtering techniques. The dataset is available in text format and can be accessed at: http://commoncrawl.org/\n",
      "\n",
      "5. Opus Corpus: The Opus corpus is a collection of translated texts from a wide range of domains. It includes parallel corpora for various language pairs, including English to Hindi translation. The dataset is available in text format and can be found at: https://github.com/uhh-lt/OPUS-MT-train/tree/master/data\n",
      "\n",
      "Note: Please ensure to review the licensing and usage terms specified for each dataset before using them for machine translation tasks.\n"
     ]
    }
   ],
   "source": [
    "prompt =\"\"\"\n",
    "Create a list of datasets for practicing on machine translation from english to hindi.\n",
    "Make sure they are available in text format.\n",
    "Also, provide links to the dataset.\n",
    "\"\"\"\n",
    "res = chat(prompt)\n",
    "print(res)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "198764fa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. Food-101 Dataset: A large-scale dataset with 101 food categories for image recognition, containing 101,000 images. Each class consists of 1,000 images. [Link](https://www.vision.ee.ethz.ch/datasets_extra/food-101/)\n",
      "\n",
      "2. UEC Food 256 Dataset: A dataset with 256 food categories for recognition, comprising of 28,673 images. [Link](http://foodcam.mobi/dataset256.html)\n",
      "\n",
      "3. Food-50 Dataset: A dataset with 50 food categories, containing 25,000 images. Each class consists of 500 images. [Link](https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-50/)\n",
      "\n",
      "4. Stanford SNAP Food Dataset: A dataset with 11 food categories and a total of 310,000 images. [Link](https://snap.stanford.edu/data/vw-food-image-recognition.html)\n",
      "\n",
      "5. ETHZ Food-101N Dataset: A variant of the Food-101 dataset, but with noisy annotations provided by the crowd. Contains 108,175 images in 101 categories. [Link](https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101-n/)\n",
      "\n",
      "6. Chinese Food Dataset: A comprehensive dataset with 100 food categories from Chinese cuisine, comprising 101,000 images. [Link](https://www.kaggle.com/vernonli/chinese-food-dataset)\n",
      "\n",
      "7. FIDS30 Dataset: A dataset with 30 food categories, containing 28,959 images collected from various online platforms. [Link](https://github.com/Murgio/FIDS30)\n",
      "\n",
      "8. Food-5K Dataset: A dataset with 2,500 food images divided into 5 categories (bread, dairy product, dessert, fried food, and meat). [Link](https://mmspg.epfl.ch/food-image-datasets)\n",
      "\n",
      "9. Food11 Dataset: A dataset with 16643 images from 11 food categories. Each class consists of 1500 images. [Link](https://www.kaggle.com/ayushggarg/food11-image-dataset)\n",
      "\n",
      "10. FoodCam Dataset: A dataset of food images containing approximately 231,000 images of 61 different food types. [Link](http://foodcam.mobi/dataset.html)\n"
     ]
    }
   ],
   "source": [
    "prompt =\"\"\"\n",
    "Create a list of food image datasets for binary image classification.\n",
    "Also, provide links to the dataset.\n",
    "\"\"\"\n",
    "res = chat(prompt)\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc746dfe",
   "metadata": {},
   "source": [
    "https://pypi.org/project/pandasai/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf4cafca",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}