python/samples/getting_started/03-prompt-function-inline.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3c93ac5b",
   "metadata": {},
   "source": [
    "# Running Prompt Functions Inline\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad85226c",
   "metadata": {},
   "source": [
    "Import Semantic Kernel SDK from pypi.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a94d2b07",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: if using a virtual environment, do not run this cell\n",
    "%pip install -U semantic-kernel\n",
    "from semantic_kernel import __version__\n",
    "\n",
    "__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c704e54b",
   "metadata": {},
   "source": [
    "Initial configuration for the notebook to run properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ebb95f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make sure paths are correct for the imports\n",
    "\n",
    "import os\n",
    "import sys\n",
    "\n",
    "notebook_dir = os.path.abspath(\"\")\n",
    "parent_dir = os.path.dirname(notebook_dir)\n",
    "grandparent_dir = os.path.dirname(parent_dir)\n",
    "\n",
    "\n",
    "sys.path.append(grandparent_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14eb77b5",
   "metadata": {},
   "source": [
    "### Configuring the Kernel\n",
    "\n",
    "Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
    "\n",
    "**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
    "\n",
    "#### Option 1: using OpenAI\n",
    "\n",
    "Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
    "\n",
    "```\n",
    "GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
    "OPENAI_API_KEY=\"sk-...\"\n",
    "OPENAI_ORG_ID=\"\"\n",
    "OPENAI_CHAT_MODEL_ID=\"\"\n",
    "OPENAI_TEXT_MODEL_ID=\"\"\n",
    "OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
    "```\n",
    "The names should match the names used in the `.env` file, as shown above.\n",
    "\n",
    "#### Option 2: using Azure OpenAI\n",
    "\n",
    "Add your [Azure OpenAI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
    "\n",
    "```\n",
    "GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
    "AZURE_OPENAI_API_KEY=\"...\"\n",
    "AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
    "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
    "AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
    "AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
    "AZURE_OPENAI_API_VERSION=\"...\"\n",
    "```\n",
    "The names should match the names used in the `.env` file, as shown above.\n",
    "\n",
    "For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "40201641",
   "metadata": {},
   "source": [
    "The [previous notebook](./02-running-prompts-from-file.ipynb)\n",
    "showed how to define a semantic function using a prompt template stored on a file.\n",
    "\n",
    "In this notebook, we'll show how to use the Semantic Kernel to define functions inline with your python code. This can be useful in a few scenarios:\n",
    "\n",
    "- Dynamically generating the prompt using complex rules at runtime\n",
    "- Writing prompts by editing Python code instead of TXT files.\n",
    "- Easily creating demos, like this document\n",
    "\n",
    "Prompt templates are defined using the SK template language, which allows to reference variables and functions. Read [this doc](https://aka.ms/sk/howto/configurefunction) to learn more about the design decisions for prompt templating.\n",
    "\n",
    "For now we'll use only the `{{$input}}` variable, and see more complex templates later.\n",
    "\n",
    "Almost all semantic function prompts have a reference to `{{$input}}`, which is the default way\n",
    "a user can import content from the context variables.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d90b0c13",
   "metadata": {},
   "source": [
    "Prepare a semantic kernel instance first, loading also the AI service settings defined in the [Setup notebook](00-getting-started.ipynb):\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0377f42e",
   "metadata": {},
   "source": [
    "Let's define our kernel for this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "462b281a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantic_kernel.kernel import Kernel\n",
    "\n",
    "kernel = Kernel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "734d121f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from services import Service\n",
    "\n",
    "from samples.service_settings import ServiceSettings\n",
    "\n",
    "service_settings = ServiceSettings.create()\n",
    "\n",
    "# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
    "selectedService = (\n",
    "    Service.AzureOpenAI\n",
    "    if service_settings.global_llm_service is None\n",
    "    else Service(service_settings.global_llm_service.lower())\n",
    ")\n",
    "print(f\"Using service type: {selectedService}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06740170",
   "metadata": {},
   "source": [
    "We now configure our Chat Completion service on the kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3712b7c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove all services so that this cell can be re-run without restarting the kernel\n",
    "kernel.remove_all_services()\n",
    "\n",
    "service_id = None\n",
    "if selectedService == Service.OpenAI:\n",
    "    from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
    "\n",
    "    service_id = \"default\"\n",
    "    kernel.add_service(\n",
    "        OpenAIChatCompletion(\n",
    "            service_id=service_id,\n",
    "        ),\n",
    "    )\n",
    "elif selectedService == Service.AzureOpenAI:\n",
    "    from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
    "\n",
    "    service_id = \"default\"\n",
    "    kernel.add_service(\n",
    "        AzureChatCompletion(\n",
    "            service_id=service_id,\n",
    "        ),\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "589733c5",
   "metadata": {},
   "source": [
    "Let's use a prompt to create a semantic function used to summarize content, allowing for some creativity and a sufficient number of tokens.\n",
    "\n",
    "The function will take in input the text to summarize.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae29c207",
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantic_kernel.connectors.ai.open_ai import AzureChatPromptExecutionSettings, OpenAIChatPromptExecutionSettings\n",
    "from semantic_kernel.prompt_template import InputVariable, PromptTemplateConfig\n",
    "\n",
    "prompt = \"\"\"{{$input}}\n",
    "Summarize the content above.\n",
    "\"\"\"\n",
    "\n",
    "if selectedService == Service.OpenAI:\n",
    "    execution_settings = OpenAIChatPromptExecutionSettings(\n",
    "        service_id=service_id,\n",
    "        ai_model_id=\"gpt-3.5-turbo\",\n",
    "        max_tokens=2000,\n",
    "        temperature=0.7,\n",
    "    )\n",
    "elif selectedService == Service.AzureOpenAI:\n",
    "    execution_settings = AzureChatPromptExecutionSettings(\n",
    "        service_id=service_id,\n",
    "        ai_model_id=\"gpt-35-turbo\",\n",
    "        max_tokens=2000,\n",
    "        temperature=0.7,\n",
    "    )\n",
    "\n",
    "prompt_template_config = PromptTemplateConfig(\n",
    "    template=prompt,\n",
    "    name=\"summarize\",\n",
    "    template_format=\"semantic-kernel\",\n",
    "    input_variables=[\n",
    "        InputVariable(name=\"input\", description=\"The user input\", is_required=True),\n",
    "    ],\n",
    "    execution_settings=execution_settings,\n",
    ")\n",
    "\n",
    "summarize = kernel.add_function(\n",
    "    function_name=\"summarizeFunc\",\n",
    "    plugin_name=\"summarizePlugin\",\n",
    "    prompt_template_config=prompt_template_config,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f26b90c4",
   "metadata": {},
   "source": [
    "Set up some content to summarize, here's an extract about Demo, an ancient Greek poet, taken from Wikipedia (https://en.wikipedia.org/wiki/Demo_(ancient_Greek_poet)).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "314557fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_text = \"\"\"\n",
    "Demo (ancient Greek poet)\n",
    "From Wikipedia, the free encyclopedia\n",
    "Demo or Damo (Greek: Δεμώ, Δαμώ; fl. c. AD 200) was a Greek woman of the Roman period, known for a single epigram, engraved upon the Colossus of Memnon, which bears her name. She speaks of herself therein as a lyric poetess dedicated to the Muses, but nothing is known of her life.[1]\n",
    "Identity\n",
    "Demo was evidently Greek, as her name, a traditional epithet of Demeter, signifies. The name was relatively common in the Hellenistic world, in Egypt and elsewhere, and she cannot be further identified. The date of her visit to the Colossus of Memnon cannot be established with certainty, but internal evidence on the left leg suggests her poem was inscribed there at some point in or after AD 196.[2]\n",
    "Epigram\n",
    "There are a number of graffiti inscriptions on the Colossus of Memnon. Following three epigrams by Julia Balbilla, a fourth epigram, in elegiac couplets, entitled and presumably authored by \"Demo\" or \"Damo\" (the Greek inscription is difficult to read), is a dedication to the Muses.[2] The poem is traditionally published with the works of Balbilla, though the internal evidence suggests a different author.[1]\n",
    "In the poem, Demo explains that Memnon has shown her special respect. In return, Demo offers the gift for poetry, as a gift to the hero. At the end of this epigram, she addresses Memnon, highlighting his divine status by recalling his strength and holiness.[2]\n",
    "Demo, like Julia Balbilla, writes in the artificial and poetic Aeolic dialect. The language indicates she was knowledgeable in Homeric poetry—'bearing a pleasant gift', for example, alludes to the use of that phrase throughout the Iliad and Odyssey.[a][2]\n",
    "\"\"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "bf0f2330",
   "metadata": {},
   "source": [
    "...and run the summary function:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b0e3b0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "summary = await kernel.invoke(summarize, input=input_text)\n",
    "\n",
    "print(summary)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1c2c1262",
   "metadata": {},
   "source": [
    "# Using ChatCompletion for Semantic Plugins\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "29b59b28",
   "metadata": {},
   "source": [
    "You can also use chat completion models (like `gpt-35-turbo` and `gpt4`) for creating plugins. Normally you would have to tweak the API to accommodate for a system and user role, but SK abstracts that away for you by using `kernel.add_service` and `AzureChatCompletion` or `OpenAIChatCompletion`\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4777f447",
   "metadata": {},
   "source": [
    "Here's one more example of how to write an inline Semantic Function that gives a TLDR for a piece of text using a ChatCompletion model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c5886aeb",
   "metadata": {},
   "outputs": [],
   "source": [
    "kernel.remove_all_services()\n",
    "\n",
    "service_id = None\n",
    "if selectedService == Service.OpenAI:\n",
    "    from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
    "\n",
    "    service_id = \"default\"\n",
    "    kernel.add_service(\n",
    "        OpenAIChatCompletion(\n",
    "            service_id=service_id,\n",
    "        ),\n",
    "    )\n",
    "elif selectedService == Service.AzureOpenAI:\n",
    "    from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
    "\n",
    "    service_id = \"default\"\n",
    "    kernel.add_service(\n",
    "        AzureChatCompletion(\n",
    "            service_id=service_id,\n",
    "        ),\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea8128c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantic_kernel.connectors.ai.open_ai import AzureChatPromptExecutionSettings, OpenAIChatPromptExecutionSettings\n",
    "\n",
    "prompt = \"\"\"\n",
    "{{$input}}\n",
    "\n",
    "Give me the TLDR in 5 words or less.\n",
    "\"\"\"\n",
    "\n",
    "text = \"\"\"\n",
    "    1) A robot may not injure a human being or, through inaction,\n",
    "    allow a human being to come to harm.\n",
    "\n",
    "    2) A robot must obey orders given it by human beings except where\n",
    "    such orders would conflict with the First Law.\n",
    "\n",
    "    3) A robot must protect its own existence as long as such protection\n",
    "    does not conflict with the First or Second Law.\n",
    "\"\"\"\n",
    "\n",
    "if selectedService == Service.OpenAI:\n",
    "    execution_settings = OpenAIChatPromptExecutionSettings(\n",
    "        service_id=service_id,\n",
    "        ai_model_id=\"gpt-3.5-turbo\",\n",
    "        max_tokens=2000,\n",
    "        temperature=0.7,\n",
    "    )\n",
    "elif selectedService == Service.AzureOpenAI:\n",
    "    execution_settings = AzureChatPromptExecutionSettings(\n",
    "        service_id=service_id,\n",
    "        ai_model_id=\"gpt-35-turbo\",\n",
    "        max_tokens=2000,\n",
    "        temperature=0.7,\n",
    "    )\n",
    "\n",
    "prompt_template_config = PromptTemplateConfig(\n",
    "    template=prompt,\n",
    "    name=\"tldr\",\n",
    "    template_format=\"semantic-kernel\",\n",
    "    input_variables=[\n",
    "        InputVariable(name=\"input\", description=\"The user input\", is_required=True),\n",
    "    ],\n",
    "    execution_settings=execution_settings,\n",
    ")\n",
    "\n",
    "tldr_function = kernel.add_function(\n",
    "    function_name=\"tldrFunction\",\n",
    "    plugin_name=\"tldrPlugin\",\n",
    "    prompt_template_config=prompt_template_config,\n",
    ")\n",
    "\n",
    "summary = await kernel.invoke(tldr_function, input=text)\n",
    "\n",
    "print(f\"Output: {summary}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}