week1/day5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a98030af-fcd1-4d63-a36e-38ba053498fa",
   "metadata": {},
   "source": [
    "# A full business solution\n",
    "\n",
    "## Now we will take our project from Day 1 to the next level\n",
    "\n",
    "### BUSINESS CHALLENGE:\n",
    "\n",
    "Create a product that builds a Brochure for a company to be used for prospective clients, investors and potential recruits.\n",
    "\n",
    "We will be provided a company name and their primary website.\n",
    "\n",
    "See the end of this notebook for examples of real-world business applications.\n",
    "\n",
    "And remember: I'm always available if you have problems or ideas! Please do reach out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d5b08506-dc8b-4443-9201-5f1848161363",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "# If these fail, please check you're running from an 'activated' environment with (llms) in the command prompt\n",
    "\n",
    "import os\n",
    "import requests\n",
    "import json\n",
    "from typing import List\n",
    "from dotenv import load_dotenv\n",
    "from bs4 import BeautifulSoup\n",
    "from IPython.display import Markdown, display, update_display\n",
    "from openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "fc5d8880-f2ee-4c06-af16-ecbc0262af61",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize and constants\n",
    "MODEL = \"llama3.2\"\n",
    "openai = OpenAI(base_url=\"http://localhost:11434/v1\", api_key=\"ollama\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "106dd65e-90af-4ca8-86b6-23a41840645b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# A class to represent a Webpage\n",
    "\n",
    "# Some websites need you to use proper headers when fetching them:\n",
    "headers = {\n",
    " \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36\"\n",
    "}\n",
    "\n",
    "class Website:\n",
    "    \"\"\"\n",
    "    A utility class to represent a Website that we have scraped, now with links\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, url):\n",
    "        self.url = url\n",
    "        response = requests.get(url, headers=headers)\n",
    "        self.body = response.content\n",
    "        soup = BeautifulSoup(self.body, 'html.parser')\n",
    "        self.title = soup.title.string if soup.title else \"No title found\"\n",
    "        if soup.body:\n",
    "            for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
    "                irrelevant.decompose()\n",
    "            self.text = soup.body.get_text(separator=\"\\n\", strip=True)\n",
    "        else:\n",
    "            self.text = \"\"\n",
    "        links = [link.get('href') for link in soup.find_all('a')]\n",
    "        self.links = [link for link in links if link]\n",
    "\n",
    "    def get_contents(self):\n",
    "        return f\"Webpage Title:\\n{self.title}\\nWebpage Contents:\\n{self.text}\\n\\n\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "e30d8128-933b-44cc-81c8-ab4c9d86589a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['https://edwarddonner.com/',\n",
       " 'https://edwarddonner.com/connect-four/',\n",
       " 'https://edwarddonner.com/outsmart/',\n",
       " 'https://edwarddonner.com/about-me-and-about-nebula/',\n",
       " 'https://edwarddonner.com/posts/',\n",
       " 'https://edwarddonner.com/',\n",
       " 'https://news.ycombinator.com',\n",
       " 'https://nebula.io/?utm_source=ed&utm_medium=referral',\n",
       " 'https://www.prnewswire.com/news-releases/wynden-stark-group-acquires-nyc-venture-backed-tech-startup-untapt-301269512.html',\n",
       " 'https://patents.google.com/patent/US20210049536A1/',\n",
       " 'https://www.linkedin.com/in/eddonner/',\n",
       " 'https://edwarddonner.com/2025/01/23/llm-workshop-hands-on-with-agents-resources/',\n",
       " 'https://edwarddonner.com/2025/01/23/llm-workshop-hands-on-with-agents-resources/',\n",
       " 'https://edwarddonner.com/2024/12/21/llm-resources-superdatascience/',\n",
       " 'https://edwarddonner.com/2024/12/21/llm-resources-superdatascience/',\n",
       " 'https://edwarddonner.com/2024/11/13/llm-engineering-resources/',\n",
       " 'https://edwarddonner.com/2024/11/13/llm-engineering-resources/',\n",
       " 'https://edwarddonner.com/2024/10/16/from-software-engineer-to-ai-data-scientist-resources/',\n",
       " 'https://edwarddonner.com/2024/10/16/from-software-engineer-to-ai-data-scientist-resources/',\n",
       " 'https://edwarddonner.com/',\n",
       " 'https://edwarddonner.com/connect-four/',\n",
       " 'https://edwarddonner.com/outsmart/',\n",
       " 'https://edwarddonner.com/about-me-and-about-nebula/',\n",
       " 'https://edwarddonner.com/posts/',\n",
       " 'mailto:hello@mygroovydomain.com',\n",
       " 'https://www.linkedin.com/in/eddonner/',\n",
       " 'https://twitter.com/edwarddonner',\n",
       " 'https://www.facebook.com/edward.donner.52']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ed = Website(\"https://edwarddonner.com\")\n",
    "ed.links"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "707310fb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Webpage Title:\n",
      "Home - Edward Donner\n",
      "Webpage Contents:\n",
      "Home\n",
      "Connect Four\n",
      "Outsmart\n",
      "An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n",
      "About\n",
      "Posts\n",
      "Well, hi there.\n",
      "I’m Ed. I like writing code and experimenting with LLMs, and hopefully you’re here because you do too. I also enjoy DJing (but I’m badly out of practice), amateur electronic music production (\n",
      "very\n",
      "amateur) and losing myself in\n",
      "Hacker News\n",
      ", nodding my head sagely to things I only half understand.\n",
      "I’m the co-founder and CTO of\n",
      "Nebula.io\n",
      ". We’re applying AI to a field where it can make a massive, positive impact: helping people discover their potential and pursue their reason for being. Recruiters use our product today to source, understand, engage and manage talent. I’m previously the founder and CEO of AI startup untapt,\n",
      "acquired in 2021\n",
      ".\n",
      "We work with groundbreaking, proprietary LLMs verticalized for talent, we’ve\n",
      "patented\n",
      "our matching model, and our award-winning platform has happy customers and tons of press coverage.\n",
      "Connect\n",
      "with me for more!\n",
      "January 23, 2025\n",
      "LLM Workshop – Hands-on with Agents – resources\n",
      "December 21, 2024\n",
      "Welcome, SuperDataScientists!\n",
      "November 13, 2024\n",
      "Mastering AI and LLM Engineering – Resources\n",
      "October 16, 2024\n",
      "From Software Engineer to AI Data Scientist – resources\n",
      "Navigation\n",
      "Home\n",
      "Connect Four\n",
      "Outsmart\n",
      "An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n",
      "About\n",
      "Posts\n",
      "Get in touch\n",
      "ed [at] edwarddonner [dot] com\n",
      "www.edwarddonner.com\n",
      "Follow me\n",
      "LinkedIn\n",
      "Twitter\n",
      "Facebook\n",
      "Subscribe to newsletter\n",
      "Type your email…\n",
      "Subscribe\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(ed.get_contents())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1771af9c-717a-4fca-bbbe-8a95893312c3",
   "metadata": {},
   "source": [
    "## First step: Have GPT-4o-mini figure out which links are relevant\n",
    "\n",
    "### Use a call to gpt-4o-mini to read the links on a webpage, and respond in structured JSON.  \n",
    "It should decide which links are relevant, and replace relative links such as \"/about\" with \"https://company.com/about\".  \n",
    "We will use \"one shot prompting\" in which we provide an example of how it should respond in the prompt.\n",
    "\n",
    "This is an excellent use case for an LLM, because it requires nuanced understanding. Imagine trying to code this without LLMs by parsing and analyzing the webpage - it would be very hard!\n",
    "\n",
    "Sidenote: there is a more advanced technique called \"Structured Outputs\" in which we require the model to respond according to a spec. We cover this technique in Week 8 during our autonomous Agentic AI project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "6957b079-0d96-45f7-a26a-3487510e9b35",
   "metadata": {},
   "outputs": [],
   "source": [
    "link_system_prompt = \"You are provided with a list of links found on a webpage. \\\n",
    "You are able to decide which of the links would be most relevant to include in a brochure about the company, \\\n",
    "such as links to an About page, or a Company page, or Careers/Jobs pages.\\n\"\n",
    "link_system_prompt += \"You should respond in JSON as in this example:\"\n",
    "link_system_prompt += \"\"\"\n",
    "{\n",
    "    \"links\": [\n",
    "        {\"type\": \"about page\", \"url\": \"https://full.url/goes/here/about\"},\n",
    "        {\"type\": \"careers page\": \"url\": \"https://another.full.url/careers\"}\n",
    "    ]\n",
    "}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "b97e4068-97ed-4120-beae-c42105e4d59a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are provided with a list of links found on a webpage. You are able to decide which of the links would be most relevant to include in a brochure about the company, such as links to an About page, or a Company page, or Careers/Jobs pages.\n",
      "You should respond in JSON as in this example:\n",
      "{\n",
      "    \"links\": [\n",
      "        {\"type\": \"about page\", \"url\": \"https://full.url/goes/here/about\"},\n",
      "        {\"type\": \"careers page\": \"url\": \"https://another.full.url/careers\"}\n",
      "    ]\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(link_system_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8e1f601b-2eaf-499d-b6b8-c99050c9d6b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_links_user_prompt(website):\n",
    "    user_prompt = f\"Here is the list of links on the website of {website.url} - \"\n",
    "    user_prompt += \"please decide which of these are relevant web links for a brochure about the company, respond with the full https URL in JSON format. \\\n",
    "Do not include Terms of Service, Privacy, email links.\\n\"\n",
    "    user_prompt += \"Links (some might be relative links):\\n\"\n",
    "    user_prompt += \"\\n\".join(website.links)\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "6bcbfa78-6395-4685-b92c-22d592050fd7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here is the list of links on the website of https://edwarddonner.com - please decide which of these are relevant web links for a brochure about the company, respond with the full https URL in JSON format. Do not include Terms of Service, Privacy, email links.\n",
      "Links (some might be relative links):\n",
      "https://edwarddonner.com/\n",
      "https://edwarddonner.com/connect-four/\n",
      "https://edwarddonner.com/outsmart/\n",
      "https://edwarddonner.com/about-me-and-about-nebula/\n",
      "https://edwarddonner.com/posts/\n",
      "https://edwarddonner.com/\n",
      "https://news.ycombinator.com\n",
      "https://nebula.io/?utm_source=ed&utm_medium=referral\n",
      "https://www.prnewswire.com/news-releases/wynden-stark-group-acquires-nyc-venture-backed-tech-startup-untapt-301269512.html\n",
      "https://patents.google.com/patent/US20210049536A1/\n",
      "https://www.linkedin.com/in/eddonner/\n",
      "https://edwarddonner.com/2025/01/23/llm-workshop-hands-on-with-agents-resources/\n",
      "https://edwarddonner.com/2025/01/23/llm-workshop-hands-on-with-agents-resources/\n",
      "https://edwarddonner.com/2024/12/21/llm-resources-superdatascience/\n",
      "https://edwarddonner.com/2024/12/21/llm-resources-superdatascience/\n",
      "https://edwarddonner.com/2024/11/13/llm-engineering-resources/\n",
      "https://edwarddonner.com/2024/11/13/llm-engineering-resources/\n",
      "https://edwarddonner.com/2024/10/16/from-software-engineer-to-ai-data-scientist-resources/\n",
      "https://edwarddonner.com/2024/10/16/from-software-engineer-to-ai-data-scientist-resources/\n",
      "https://edwarddonner.com/\n",
      "https://edwarddonner.com/connect-four/\n",
      "https://edwarddonner.com/outsmart/\n",
      "https://edwarddonner.com/about-me-and-about-nebula/\n",
      "https://edwarddonner.com/posts/\n",
      "mailto:hello@mygroovydomain.com\n",
      "https://www.linkedin.com/in/eddonner/\n",
      "https://twitter.com/edwarddonner\n",
      "https://www.facebook.com/edward.donner.52\n"
     ]
    }
   ],
   "source": [
    "print(get_links_user_prompt(ed))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "a29aca19-ca13-471c-a4b4-5abbfa813f69",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_links(url):\n",
    "    website = Website(url)\n",
    "    response = openai.chat.completions.create(\n",
    "        model=MODEL,\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": link_system_prompt},\n",
    "            {\"role\": \"user\", \"content\": get_links_user_prompt(website)}\n",
    "      ],\n",
    "        response_format={\"type\": \"json_object\"}\n",
    "    )\n",
    "    result = response.choices[0].message.content\n",
    "    return json.loads(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "74a827a0-2782-4ae5-b210-4a242a8b4cc2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/',\n",
       " '/models',\n",
       " '/datasets',\n",
       " '/spaces',\n",
       " '/posts',\n",
       " '/docs',\n",
       " '/enterprise',\n",
       " '/pricing',\n",
       " '/login',\n",
       " '/join',\n",
       " '/spaces',\n",
       " '/models',\n",
       " '/Qwen/QwQ-32B',\n",
       " '/sesame/csm-1b',\n",
       " '/google/gemma-3-27b-it',\n",
       " '/deepseek-ai/DeepSeek-R1',\n",
       " '/SparkAudio/Spark-TTS-0.5B',\n",
       " '/models',\n",
       " '/spaces/sesame/csm-1b',\n",
       " '/spaces/Wan-AI/Wan2.1',\n",
       " '/spaces/Qwen/QwQ-32B-Demo',\n",
       " '/spaces/ASLP-lab/DiffRhythm',\n",
       " '/spaces/nanotron/ultrascale-playbook',\n",
       " '/spaces',\n",
       " '/datasets/FreedomIntelligence/medical-o1-reasoning-SFT',\n",
       " '/datasets/facebook/natural_reasoning',\n",
       " '/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k',\n",
       " '/datasets/open-r1/codeforces-cots',\n",
       " '/datasets/SmallDoge/SmallThoughts',\n",
       " '/datasets',\n",
       " '/join',\n",
       " '/pricing#endpoints',\n",
       " '/pricing#spaces',\n",
       " '/pricing',\n",
       " '/enterprise',\n",
       " '/enterprise',\n",
       " '/enterprise',\n",
       " '/enterprise',\n",
       " '/enterprise',\n",
       " '/enterprise',\n",
       " '/enterprise',\n",
       " '/allenai',\n",
       " '/facebook',\n",
       " '/amazon',\n",
       " '/google',\n",
       " '/Intel',\n",
       " '/microsoft',\n",
       " '/grammarly',\n",
       " '/Writer',\n",
       " '/docs/transformers',\n",
       " '/docs/diffusers',\n",
       " '/docs/safetensors',\n",
       " '/docs/huggingface_hub',\n",
       " '/docs/tokenizers',\n",
       " '/docs/trl',\n",
       " '/docs/transformers.js',\n",
       " '/docs/smolagents',\n",
       " '/docs/peft',\n",
       " '/docs/datasets',\n",
       " '/docs/text-generation-inference',\n",
       " '/docs/accelerate',\n",
       " '/models',\n",
       " '/datasets',\n",
       " '/spaces',\n",
       " '/tasks',\n",
       " 'https://ui.endpoints.huggingface.co',\n",
       " '/chat',\n",
       " '/huggingface',\n",
       " '/brand',\n",
       " '/terms-of-service',\n",
       " '/privacy',\n",
       " 'https://apply.workable.com/huggingface/',\n",
       " 'mailto:press@huggingface.co',\n",
       " '/learn',\n",
       " '/docs',\n",
       " '/blog',\n",
       " 'https://discuss.huggingface.co',\n",
       " 'https://status.huggingface.co/',\n",
       " 'https://github.com/huggingface',\n",
       " 'https://twitter.com/huggingface',\n",
       " 'https://www.linkedin.com/company/huggingface/',\n",
       " '/join/discord']"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Anthropic has made their site harder to scrape, so I'm using HuggingFace..\n",
    "\n",
    "huggingface = Website(\"https://huggingface.co\")\n",
    "huggingface.links"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "d3d583e2-dcc4-40cc-9b28-1e8dbf402924",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'links': [{'type': 'About page', 'url': 'https://huggingface.co/'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/allenai'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/facebook'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/amazon'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/google'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/Intel'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/microsoft'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/grammarly'},\n",
       "  {'type': 'Company page', 'url': 'https://huggingface.co/Writer'}]}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now get the links\n",
    "get_links(\"https://huggingface.co\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d74128e-dfb6-47ec-9549-288b621c838c",
   "metadata": {},
   "source": [
    "## Second step: make the brochure!\n",
    "\n",
    "Assemble all the details into another prompt to GPT4-o"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "85a5b6e2-e7ef-44a9-bc7f-59ede71037b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_all_details(url):\n",
    "    result = \"Landing page:\\n\"\n",
    "    result += Website(url).get_contents()\n",
    "    links = get_links(url)\n",
    "    print(\"Found links:\", links)\n",
    "    for link in links[\"links\"]:\n",
    "        result += f\"\\n\\n{link['type']}\\n\"\n",
    "        result += Website(link[\"url\"]).get_contents()\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "5099bd14-076d-4745-baf3-dac08d8e5ab2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found links: {'links': [{'type': 'about page', 'url': 'https://huggingface.co/'}, {'type': 'company page', 'url': 'https://huggingface.co/brand'}, {'type': ' careers/jobs page', 'url': 'https://apply.workable.com/huggingface/'}, {'type': 'GitHub repository', 'url': 'https://github.com/huggingface'}]}\n",
      "Landing page:\n",
      "Webpage Title:\n",
      "Hugging Face – The AI community building the future.\n",
      "Webpage Contents:\n",
      "Hugging Face\n",
      "Models\n",
      "Datasets\n",
      "Spaces\n",
      "Posts\n",
      "Docs\n",
      "Enterprise\n",
      "Pricing\n",
      "Log In\n",
      "Sign Up\n",
      "The AI community building the future.\n",
      "The platform where the machine learning community collaborates on models, datasets, and applications.\n",
      "Explore AI Apps\n",
      "or\n",
      "Browse 1M+ models\n",
      "Trending on\n",
      "this week\n",
      "Models\n",
      "Qwen/QwQ-32B\n",
      "Updated\n",
      "4 days ago\n",
      "•\n",
      "296k\n",
      "•\n",
      "2.19k\n",
      "sesame/csm-1b\n",
      "Updated\n",
      "1 day ago\n",
      "•\n",
      "677\n",
      "google/gemma-3-27b-it\n",
      "Updated\n",
      "3 days ago\n",
      "•\n",
      "85.2k\n",
      "•\n",
      "575\n",
      "deepseek-ai/DeepSeek-R1\n",
      "Updated\n",
      "19 days ago\n",
      "•\n",
      "2.52M\n",
      "•\n",
      "11.3k\n",
      "SparkAudio/Spark-TTS-0.5B\n",
      "Updated\n",
      "8 days ago\n",
      "•\n",
      "8.91k\n",
      "•\n",
      "403\n",
      "Browse 1M+ models\n",
      "Spaces\n",
      "Running\n",
      "on\n",
      "Zero\n",
      "276\n",
      "276\n",
      "Sesame CSM\n",
      "🌱\n",
      "Conversational speech generation\n",
      "Running\n",
      "1.18k\n",
      "1.18k\n",
      "Wan2.1\n",
      "💻\n",
      "Wan: Open and Advanced Large-Scale Video Generative Models\n",
      "Running\n",
      "408\n",
      "408\n",
      "QwQ 32B Demo\n",
      "🌖\n",
      "Generate text responses to user prompts\n",
      "Running\n",
      "on\n",
      "Zero\n",
      "439\n",
      "439\n",
      "Di♪♪Rhythm\n",
      "🎶\n",
      "Blazingly Fast and Embarrassingly Simple Song Generation\n",
      "Running\n",
      "2.25k\n",
      "2.25k\n",
      "The Ultra-Scale Playbook\n",
      "🌌\n",
      "The ultimate guide to training LLM on large GPU Clusters\n",
      "Browse 400k+ applications\n",
      "Datasets\n",
      "FreedomIntelligence/medical-o1-reasoning-SFT\n",
      "Updated\n",
      "21 days ago\n",
      "•\n",
      "28.1k\n",
      "•\n",
      "459\n",
      "facebook/natural_reasoning\n",
      "Updated\n",
      "22 days ago\n",
      "•\n",
      "11.1k\n",
      "•\n",
      "404\n",
      "Congliu/Chinese-DeepSeek-R1-Distill-data-110k\n",
      "Updated\n",
      "22 days ago\n",
      "•\n",
      "8.03k\n",
      "•\n",
      "531\n",
      "open-r1/codeforces-cots\n",
      "Updated\n",
      "1 day ago\n",
      "•\n",
      "1.5k\n",
      "•\n",
      "42\n",
      "SmallDoge/SmallThoughts\n",
      "Updated\n",
      "about 13 hours ago\n",
      "•\n",
      "1.51k\n",
      "•\n",
      "30\n",
      "Browse 250k+ datasets\n",
      "The Home of Machine Learning\n",
      "Create, discover and collaborate on ML better.\n",
      "The collaboration platform\n",
      "Host and collaborate on unlimited public models, datasets and applications.\n",
      "Move faster\n",
      "With the HF Open source stack.\n",
      "Explore all modalities\n",
      "Text, image, video, audio or even 3D.\n",
      "Build your portfolio\n",
      "Share your work with the world and build your ML profile.\n",
      "Sign Up\n",
      "Accelerate your ML\n",
      "We provide paid Compute and Enterprise solutions.\n",
      "Compute\n",
      "Deploy on optimized\n",
      "Inference Endpoints\n",
      "or update your\n",
      "Spaces applications\n",
      "to a GPU in a few clicks.\n",
      "View pricing\n",
      "Starting at $0.60/hour for GPU\n",
      "Enterprise\n",
      "Give your team the most advanced platform to build AI with enterprise-grade security, access controls and\n",
      "\t\t\tdedicated support.\n",
      "Getting started\n",
      "Starting at $20/user/month\n",
      "Single Sign-On\n",
      "Regions\n",
      "Priority Support\n",
      "Audit Logs\n",
      "Resource Groups\n",
      "Private Datasets Viewer\n",
      "More than 50,000 organizations are using Hugging Face\n",
      "Ai2\n",
      "Enterprise\n",
      "non-profit\n",
      "•\n",
      "396 models\n",
      "•\n",
      "2.82k followers\n",
      "AI at Meta\n",
      "Enterprise\n",
      "company\n",
      "•\n",
      "2.07k models\n",
      "•\n",
      "5.07k followers\n",
      "Amazon\n",
      "company\n",
      "•\n",
      "11 models\n",
      "•\n",
      "2.86k followers\n",
      "Google\n",
      "company\n",
      "•\n",
      "968 models\n",
      "•\n",
      "9.42k followers\n",
      "Intel\n",
      "company\n",
      "•\n",
      "219 models\n",
      "•\n",
      "2.34k followers\n",
      "Microsoft\n",
      "company\n",
      "•\n",
      "363 models\n",
      "•\n",
      "10.3k followers\n",
      "Grammarly\n",
      "Enterprise\n",
      "company\n",
      "•\n",
      "10 models\n",
      "•\n",
      "135 followers\n",
      "Writer\n",
      "Enterprise\n",
      "company\n",
      "•\n",
      "20 models\n",
      "•\n",
      "249 followers\n",
      "Our Open Source\n",
      "We are building the foundation of ML tooling with the community.\n",
      "Transformers\n",
      "141,228\n",
      "State-of-the-art ML for PyTorch, TensorFlow, JAX\n",
      "Diffusers\n",
      "28,024\n",
      "State-of-the-art Diffusion models in PyTorch\n",
      "Safetensors\n",
      "3,163\n",
      "Safe way to store/distribute neural network weights\n",
      "Hub Python Library\n",
      "2,429\n",
      "Python client to interact with the Hugging Face Hub\n",
      "Tokenizers\n",
      "9,475\n",
      "Fast tokenizers optimized for research & production\n",
      "TRL\n",
      "12,501\n",
      "Train transformers LMs with reinforcement learning\n",
      "Transformers.js\n",
      "13,202\n",
      "State-of-the-art ML running directly in your browser\n",
      "smolagents\n",
      "14,774\n",
      "Smol library to build great agents in Python\n",
      "PEFT\n",
      "17,760\n",
      "Parameter-efficient finetuning for large language models\n",
      "Datasets\n",
      "19,793\n",
      "Access & share datasets for any ML tasks\n",
      "Text Generation Inference\n",
      "9,880\n",
      "Serve language models with TGI optimized toolkit\n",
      "Accelerate\n",
      "8,479\n",
      "Train PyTorch models with multi-GPU, TPU, mixed precision\n",
      "System theme\n",
      "Website\n",
      "Models\n",
      "Datasets\n",
      "Spaces\n",
      "Tasks\n",
      "Inference Endpoints\n",
      "HuggingChat\n",
      "Company\n",
      "About\n",
      "Brand assets\n",
      "Terms of service\n",
      "Privacy\n",
      "Jobs\n",
      "Press\n",
      "Resources\n",
      "Learn\n",
      "Documentation\n",
      "Blog\n",
      "Forum\n",
      "Service Status\n",
      "Social\n",
      "GitHub\n",
      "Twitter\n",
      "LinkedIn\n",
      "Discord\n",
      "\n",
      "\n",
      "\n",
      "about page\n",
      "Webpage Title:\n",
      "Hugging Face – The AI community building the future.\n",
      "Webpage Contents:\n",
      "Hugging Face\n",
      "Models\n",
      "Datasets\n",
      "Spaces\n",
      "Posts\n",
      "Docs\n",
      "Enterprise\n",
      "Pricing\n",
      "Log In\n",
      "Sign Up\n",
      "The AI community building the future.\n",
      "The platform where the machine learning community collaborates on models, datasets, and applications.\n",
      "Explore AI Apps\n",
      "or\n",
      "Browse 1M+ models\n",
      "Trending on\n",
      "this week\n",
      "Models\n",
      "Qwen/QwQ-32B\n",
      "Updated\n",
      "4 days ago\n",
      "•\n",
      "296k\n",
      "•\n",
      "2.19k\n",
      "sesame/csm-1b\n",
      "Updated\n",
      "1 day ago\n",
      "•\n",
      "677\n",
      "google/gemma-3-27b-it\n",
      "Updated\n",
      "3 days ago\n",
      "•\n",
      "85.2k\n",
      "•\n",
      "575\n",
      "deepseek-ai/DeepSeek-R1\n",
      "Updated\n",
      "19 days ago\n",
      "•\n",
      "2.52M\n",
      "•\n",
      "11.3k\n",
      "SparkAudio/Spark-TTS-0.5B\n",
      "Updated\n",
      "8 days ago\n",
      "•\n",
      "8.91k\n",
      "•\n",
      "403\n",
      "Browse 1M+ models\n",
      "Spaces\n",
      "Running\n",
      "on\n",
      "Zero\n",
      "276\n",
      "276\n",
      "Sesame CSM\n",
      "🌱\n",
      "Conversational speech generation\n",
      "Running\n",
      "1.18k\n",
      "1.18k\n",
      "Wan2.1\n",
      "💻\n",
      "Wan: Open and Advanced Large-Scale Video Generative Models\n",
      "Running\n",
      "408\n",
      "408\n",
      "QwQ 32B Demo\n",
      "🌖\n",
      "Generate text responses to user prompts\n",
      "Running\n",
      "on\n",
      "Zero\n",
      "439\n",
      "439\n",
      "Di♪♪Rhythm\n",
      "🎶\n",
      "Blazingly Fast and Embarrassingly Simple Song Generation\n",
      "Running\n",
      "2.25k\n",
      "2.25k\n",
      "The Ultra-Scale Playbook\n",
      "🌌\n",
      "The ultimate guide to training LLM on large GPU Clusters\n",
      "Browse 400k+ applications\n",
      "Datasets\n",
      "FreedomIntelligence/medical-o1-reasoning-SFT\n",
      "Updated\n",
      "21 days ago\n",
      "•\n",
      "28.1k\n",
      "•\n",
      "459\n",
      "facebook/natural_reasoning\n",
      "Updated\n",
      "22 days ago\n",
      "•\n",
      "11.1k\n",
      "•\n",
      "404\n",
      "Congliu/Chinese-DeepSeek-R1-Distill-data-110k\n",
      "Updated\n",
      "22 days ago\n",
      "•\n",
      "8.03k\n",
      "•\n",
      "531\n",
      "open-r1/codeforces-cots\n",
      "Updated\n",
      "1 day ago\n",
      "•\n",
      "1.5k\n",
      "•\n",
      "42\n",
      "SmallDoge/SmallThoughts\n",
      "Updated\n",
      "about 13 hours ago\n",
      "•\n",
      "1.51k\n",
      "•\n",
      "30\n",
      "Browse 250k+ datasets\n",
      "The Home of Machine Learning\n",
      "Create, discover and collaborate on ML better.\n",
      "The collaboration platform\n",
      "Host and collaborate on unlimited public models, datasets and applications.\n",
      "Move faster\n",
      "With the HF Open source stack.\n",
      "Explore all modalities\n",
      "Text, image, video, audio or even 3D.\n",
      "Build your portfolio\n",
      "Share your work with the world and build your ML profile.\n",
      "Sign Up\n",
      "Accelerate your ML\n",
      "We provide paid Compute and Enterprise solutions.\n",
      "Compute\n",
      "Deploy on optimized\n",
      "Inference Endpoints\n",
      "or update your\n",
      "Spaces applications\n",
      "to a GPU in a few clicks.\n",
      "View pricing\n",
      "Starting at $0.60/hour for GPU\n",
      "Enterprise\n",
      "Give your team the most advanced platform to build AI with enterprise-grade security, access controls and\n",
      "\t\t\tdedicated support.\n",
      "Getting started\n",
      "Starting at $20/user/month\n",
      "Single Sign-On\n",
      "Regions\n",
      "Priority Support\n",
      "Audit Logs\n",
      "Resource Groups\n",
      "Private Datasets Viewer\n",
      "More than 50,000 organizations are using Hugging Face\n",
      "Ai2\n",
      "Enterprise\n",
      "non-profit\n",
      "•\n",
      "396 models\n",
      "•\n",
      "2.82k followers\n",
      "AI at Meta\n",
      "Enterprise\n",
      "company\n",
      "•\n",
      "2.07k models\n",
      "•\n",
      "5.07k followers\n",
      "Amazon\n",
      "company\n",
      "•\n",
      "11 models\n",
      "•\n",
      "2.86k followers\n",
      "Google\n",
      "company\n",
      "•\n",
      "968 models\n",
      "•\n",
      "9.42k followers\n",
      "Intel\n",
      "company\n",
      "•\n",
      "219 models\n",
      "•\n",
      "2.34k followers\n",
      "Microsoft\n",
      "company\n",
      "•\n",
      "363 models\n",
      "•\n",
      "10.3k followers\n",
      "Grammarly\n",
      "Enterprise\n",
      "company\n",
      "•\n",
      "10 models\n",
      "•\n",
      "135 followers\n",
      "Writer\n",
      "Enterprise\n",
      "company\n",
      "•\n",
      "20 models\n",
      "•\n",
      "249 followers\n",
      "Our Open Source\n",
      "We are building the foundation of ML tooling with the community.\n",
      "Transformers\n",
      "141,228\n",
      "State-of-the-art ML for PyTorch, TensorFlow, JAX\n",
      "Diffusers\n",
      "28,024\n",
      "State-of-the-art Diffusion models in PyTorch\n",
      "Safetensors\n",
      "3,163\n",
      "Safe way to store/distribute neural network weights\n",
      "Hub Python Library\n",
      "2,429\n",
      "Python client to interact with the Hugging Face Hub\n",
      "Tokenizers\n",
      "9,475\n",
      "Fast tokenizers optimized for research & production\n",
      "TRL\n",
      "12,501\n",
      "Train transformers LMs with reinforcement learning\n",
      "Transformers.js\n",
      "13,202\n",
      "State-of-the-art ML running directly in your browser\n",
      "smolagents\n",
      "14,774\n",
      "Smol library to build great agents in Python\n",
      "PEFT\n",
      "17,760\n",
      "Parameter-efficient finetuning for large language models\n",
      "Datasets\n",
      "19,793\n",
      "Access & share datasets for any ML tasks\n",
      "Text Generation Inference\n",
      "9,880\n",
      "Serve language models with TGI optimized toolkit\n",
      "Accelerate\n",
      "8,479\n",
      "Train PyTorch models with multi-GPU, TPU, mixed precision\n",
      "System theme\n",
      "Website\n",
      "Models\n",
      "Datasets\n",
      "Spaces\n",
      "Tasks\n",
      "Inference Endpoints\n",
      "HuggingChat\n",
      "Company\n",
      "About\n",
      "Brand assets\n",
      "Terms of service\n",
      "Privacy\n",
      "Jobs\n",
      "Press\n",
      "Resources\n",
      "Learn\n",
      "Documentation\n",
      "Blog\n",
      "Forum\n",
      "Service Status\n",
      "Social\n",
      "GitHub\n",
      "Twitter\n",
      "LinkedIn\n",
      "Discord\n",
      "\n",
      "\n",
      "\n",
      "company page\n",
      "Webpage Title:\n",
      "Brand assets - Hugging Face\n",
      "Webpage Contents:\n",
      "Hugging Face\n",
      "Models\n",
      "Datasets\n",
      "Spaces\n",
      "Posts\n",
      "Docs\n",
      "Enterprise\n",
      "Pricing\n",
      "Log In\n",
      "Sign Up\n",
      "Hugging Face · Brand assets\n",
      "HF Logos\n",
      ".svg\n",
      ".png\n",
      ".ai\n",
      ".svg\n",
      ".png\n",
      ".ai\n",
      ".svg\n",
      ".png\n",
      ".ai\n",
      "HF Colors\n",
      "#FFD21E\n",
      "#FF9D00\n",
      "#6B7280\n",
      "HF Bio\n",
      "Hugging Face is the collaboration platform for the machine learning community.\n",
      "\n",
      "The Hugging Face Hub works as a central place where anyone can share, explore, discover, and experiment with open-source ML. HF empowers the next generation of machine learning engineers, scientists, and end users to learn, collaborate and share their work to build an open and ethical AI future together.\n",
      "\n",
      "With the fast-growing community, some of the most used open-source ML libraries and tools, and a talented science team exploring the edge of tech, Hugging Face is at the heart of the AI revolution.\n",
      "Copy to clipboard\n",
      "HF Universe\n",
      "Find other assets available for use from the Hugging Face brand universe\n",
      "here\n",
      ".\n",
      "System theme\n",
      "Website\n",
      "Models\n",
      "Datasets\n",
      "Spaces\n",
      "Tasks\n",
      "Inference Endpoints\n",
      "HuggingChat\n",
      "Company\n",
      "About\n",
      "Brand assets\n",
      "Terms of service\n",
      "Privacy\n",
      "Jobs\n",
      "Press\n",
      "Resources\n",
      "Learn\n",
      "Documentation\n",
      "Blog\n",
      "Forum\n",
      "Service Status\n",
      "Social\n",
      "GitHub\n",
      "Twitter\n",
      "LinkedIn\n",
      "Discord\n",
      "\n",
      "\n",
      "\n",
      " careers/jobs page\n",
      "Webpage Title:\n",
      "Hugging Face - Current Openings\n",
      "Webpage Contents:\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "GitHub repository\n",
      "Webpage Title:\n",
      "Hugging Face · GitHub\n",
      "Webpage Contents:\n",
      "Skip to content\n",
      "Navigation Menu\n",
      "Toggle navigation\n",
      "Sign in\n",
      "huggingface\n",
      "Product\n",
      "GitHub Copilot\n",
      "Write better code with AI\n",
      "Security\n",
      "Find and fix vulnerabilities\n",
      "Actions\n",
      "Automate any workflow\n",
      "Codespaces\n",
      "Instant dev environments\n",
      "Issues\n",
      "Plan and track work\n",
      "Code Review\n",
      "Manage code changes\n",
      "Discussions\n",
      "Collaborate outside of code\n",
      "Code Search\n",
      "Find more, search less\n",
      "Explore\n",
      "All features\n",
      "Documentation\n",
      "GitHub Skills\n",
      "Blog\n",
      "Solutions\n",
      "By company size\n",
      "Enterprises\n",
      "Small and medium teams\n",
      "Startups\n",
      "Nonprofits\n",
      "By use case\n",
      "DevSecOps\n",
      "DevOps\n",
      "CI/CD\n",
      "View all use cases\n",
      "By industry\n",
      "Healthcare\n",
      "Financial services\n",
      "Manufacturing\n",
      "Government\n",
      "View all industries\n",
      "View all solutions\n",
      "Resources\n",
      "Topics\n",
      "AI\n",
      "DevOps\n",
      "Security\n",
      "Software Development\n",
      "View all\n",
      "Explore\n",
      "Learning Pathways\n",
      "Events & Webinars\n",
      "Ebooks & Whitepapers\n",
      "Customer Stories\n",
      "Partners\n",
      "Executive Insights\n",
      "Open Source\n",
      "GitHub Sponsors\n",
      "Fund open source developers\n",
      "The ReadME Project\n",
      "GitHub community articles\n",
      "Repositories\n",
      "Topics\n",
      "Trending\n",
      "Collections\n",
      "Enterprise\n",
      "Enterprise platform\n",
      "AI-powered developer platform\n",
      "Available add-ons\n",
      "Advanced Security\n",
      "Enterprise-grade security features\n",
      "Copilot for business\n",
      "Enterprise-grade AI features\n",
      "Premium Support\n",
      "Enterprise-grade 24/7 support\n",
      "Pricing\n",
      "Search or jump to...\n",
      "Search code, repositories, users, issues, pull requests...\n",
      "Search\n",
      "Clear\n",
      "Search syntax tips\n",
      "Provide feedback\n",
      "We read every piece of feedback, and take your input very seriously.\n",
      "Include my email address so I can be contacted\n",
      "Cancel\n",
      "Submit feedback\n",
      "Saved searches\n",
      "Use saved searches to filter your results more quickly\n",
      "Cancel\n",
      "Create saved search\n",
      "Sign in\n",
      "Sign up\n",
      "Reseting focus\n",
      "You signed in with another tab or window.\n",
      "Reload\n",
      "to refresh your session.\n",
      "You signed out in another tab or window.\n",
      "Reload\n",
      "to refresh your session.\n",
      "You switched accounts on another tab or window.\n",
      "Reload\n",
      "to refresh your session.\n",
      "Dismiss alert\n",
      "Hugging Face\n",
      "The AI community building the future.\n",
      "Verified\n",
      "We've verified that the organization\n",
      "huggingface\n",
      "controls the domain:\n",
      "huggingface.co\n",
      "Learn more about verified organizations\n",
      "46.2k\n",
      "followers\n",
      "NYC + Paris\n",
      "https://huggingface.co/\n",
      "X\n",
      "@huggingface\n",
      "Overview\n",
      "Repositories\n",
      "Projects\n",
      "Packages\n",
      "People\n",
      "Sponsoring\n",
      "0\n",
      "More\n",
      "Overview\n",
      "Repositories\n",
      "Projects\n",
      "Packages\n",
      "People\n",
      "Sponsoring\n",
      "Pinned\n",
      "Loading\n",
      "transformers\n",
      "transformers\n",
      "Public\n",
      "🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.\n",
      "Python\n",
      "141k\n",
      "28.3k\n",
      "diffusers\n",
      "diffusers\n",
      "Public\n",
      "🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX.\n",
      "Python\n",
      "28k\n",
      "5.8k\n",
      "datasets\n",
      "datasets\n",
      "Public\n",
      "🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools\n",
      "Python\n",
      "19.8k\n",
      "2.8k\n",
      "peft\n",
      "peft\n",
      "Public\n",
      "🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.\n",
      "Python\n",
      "17.8k\n",
      "1.8k\n",
      "accelerate\n",
      "accelerate\n",
      "Public\n",
      "🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support\n",
      "Python\n",
      "8.5k\n",
      "1k\n",
      "optimum\n",
      "optimum\n",
      "Public\n",
      "🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools\n",
      "Python\n",
      "2.8k\n",
      "509\n",
      "Repositories\n",
      "Loading\n",
      "Type\n",
      "Select type\n",
      "Forks\n",
      "Archived\n",
      "Mirrors\n",
      "Templates\n",
      "Language\n",
      "Select language\n",
      "All\n",
      "C\n",
      "C#\n",
      "C++\n",
      "Cuda\n",
      "Dockerfile\n",
      "Go\n",
      "Handlebars\n",
      "HTML\n",
      "Java\n",
      "JavaScript\n",
      "Jupyter Notebook\n",
      "Kotlin\n",
      "Lua\n",
      "MDX\n",
      "Mustache\n",
      "Nix\n",
      "Python\n",
      "Rust\n",
      "Shell\n",
      "Smarty\n",
      "Swift\n",
      "TypeScript\n",
      "Sort\n",
      "Select order\n",
      "Last updated\n",
      "Name\n",
      "Stars\n",
      "Showing 10 of 300 repositories\n",
      "diffusers\n",
      "Public\n",
      "🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX.\n",
      "huggingface/diffusers’s past year of commit activity\n",
      "Python\n",
      "28,024\n",
      "Apache-2.0\n",
      "5,758\n",
      "442\n",
      "(10 issues need help)\n",
      "173\n",
      "Updated\n",
      "Mar 15, 2025\n",
      "trl\n",
      "Public\n",
      "Train transformer language models with reinforcement learning.\n",
      "huggingface/trl’s past year of commit activity\n",
      "Python\n",
      "12,501\n",
      "Apache-2.0\n",
      "1,688\n",
      "301\n",
      "70\n",
      "Updated\n",
      "Mar 15, 2025\n",
      "nanotron\n",
      "Public\n",
      "Minimalistic large language model 3D-parallelism training\n",
      "huggingface/nanotron’s past year of commit activity\n",
      "Python\n",
      "1,684\n",
      "Apache-2.0\n",
      "164\n",
      "57\n",
      "(20 issues need help)\n",
      "41\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "transformers\n",
      "Public\n",
      "🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.\n",
      "huggingface/transformers’s past year of commit activity\n",
      "Python\n",
      "141,228\n",
      "Apache-2.0\n",
      "28,285\n",
      "1,028\n",
      "(2 issues need help)\n",
      "601\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "lerobot\n",
      "Public\n",
      "🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning\n",
      "huggingface/lerobot’s past year of commit activity\n",
      "Python\n",
      "10,149\n",
      "Apache-2.0\n",
      "1,125\n",
      "143\n",
      "110\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "blog\n",
      "Public\n",
      "Public repo for HF blog posts\n",
      "huggingface/blog’s past year of commit activity\n",
      "Jupyter Notebook\n",
      "2,775\n",
      "840\n",
      "152\n",
      "(1 issue needs help)\n",
      "71\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "smolagents\n",
      "Public\n",
      "🤗 smolagents: a barebones library for agents. Agents write python code to call tools and orchestrate other agents.\n",
      "huggingface/smolagents’s past year of commit activity\n",
      "Python\n",
      "14,774\n",
      "Apache-2.0\n",
      "1,297\n",
      "105\n",
      "91\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "meshgen\n",
      "Public\n",
      "A blender addon for generating meshes with AI\n",
      "huggingface/meshgen’s past year of commit activity\n",
      "Python\n",
      "498\n",
      "MIT\n",
      "28\n",
      "8\n",
      "0\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "optimum-habana\n",
      "Public\n",
      "Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)\n",
      "huggingface/optimum-habana’s past year of commit activity\n",
      "Python\n",
      "175\n",
      "Apache-2.0\n",
      "238\n",
      "17\n",
      "(1 issue needs help)\n",
      "27\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "hub-docs\n",
      "Public\n",
      "Docs of the Hugging Face Hub\n",
      "huggingface/hub-docs’s past year of commit activity\n",
      "Handlebars\n",
      "355\n",
      "Apache-2.0\n",
      "283\n",
      "99\n",
      "28\n",
      "Updated\n",
      "Mar 14, 2025\n",
      "View all repositories\n",
      "People\n",
      "View all\n",
      "Top languages\n",
      "Loading…\n",
      "Most used topics\n",
      "Loading…\n",
      "Footer\n",
      "© 2025 GitHub, Inc.\n",
      "Footer navigation\n",
      "Terms\n",
      "Privacy\n",
      "Security\n",
      "Status\n",
      "Docs\n",
      "Contact\n",
      "Manage cookies\n",
      "Do not share my personal information\n",
      "You can’t perform that action at this time.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(get_all_details(\"https://huggingface.co\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "9b863a55-f86c-4e3f-8a79-94e24c1a8cf2",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_prompt = \"You are an assistant that analyzes the contents of several relevant pages from a company website \\\n",
    "and creates a short brochure about the company for prospective customers, investors and recruits. Respond in markdown.\\\n",
    "Include details of company culture, customers and careers/jobs if you have the information.\"\n",
    "\n",
    "# Or uncomment the lines below for a more humorous brochure - this demonstrates how easy it is to incorporate 'tone':\n",
    "\n",
    "# system_prompt = \"You are an assistant that analyzes the contents of several relevant pages from a company website \\\n",
    "# and creates a short humorous, entertaining, jokey brochure about the company for prospective customers, investors and recruits. Respond in markdown.\\\n",
    "# Include details of company culture, customers and careers/jobs if you have the information.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "6ab83d92-d36b-4ce0-8bcc-5bb4c2f8ff23",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_brochure_user_prompt(company_name, url):\n",
    "    user_prompt = f\"You are looking at a company called: {company_name}\\n\"\n",
    "    user_prompt += f\"Here are the contents of its landing page and other relevant pages; use this information to build a short brochure of the company in markdown.\\n\"\n",
    "    user_prompt += get_all_details(url)\n",
    "    user_prompt = user_prompt[:5_000] # Truncate if more than 5,000 characters\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd909e0b-1312-4ce2-a553-821e795d7572",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found links: {'links': [{'type': 'blog', 'url': 'https://discuss.huggingface.co'}, {'type': 'status', 'url': 'https://status.huggingface.co/'}, {'type': 'github', 'url': 'https://github.com/huggingface'}, {'type': 'twitter', 'url': 'https://twitter.com/huggingface'}, {'type': 'linkedin company page', 'url': 'https://www.linkedin.com/company/huggingface/'}, {'type': 'press email', 'url': 'mailto:press@huggingface.co'}]}\n",
      "An error occurred: No connection adapters were found for 'mailto:press@huggingface.co'\n"
     ]
    }
   ],
   "source": [
    "get_brochure_user_prompt(\"HuggingFace\", \"https://huggingface.co\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "e44de579-4a1a-4e6a-a510-20ea3e4b8d46",
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_brochure(company_name, url):\n",
    "    response = openai.chat.completions.create(\n",
    "        model=MODEL,\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": get_brochure_user_prompt(company_name, url)}\n",
    "          ],\n",
    "    )\n",
    "    result = response.choices[0].message.content\n",
    "    display(Markdown(result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e093444a-9407-42ae-924a-145730591a39",
   "metadata": {},
   "outputs": [],
   "source": [
    "create_brochure(\"HuggingFace\", \"https://huggingface.co\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61eaaab7-0b47-4b29-82d4-75d474ad8d18",
   "metadata": {},
   "source": [
    "## Finally - a minor improvement\n",
    "\n",
    "With a small adjustment, we can change this so that the results stream back from OpenAI,\n",
    "with the familiar typewriter animation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51db0e49-f261-4137-aabe-92dd601f7725",
   "metadata": {},
   "outputs": [],
   "source": [
    "def stream_brochure(company_name, url):\n",
    "    stream = openai.chat.completions.create(\n",
    "        model=MODEL,\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": get_brochure_user_prompt(company_name, url)}\n",
    "          ],\n",
    "        stream=True\n",
    "    )\n",
    "    \n",
    "    response = \"\"\n",
    "    display_handle = display(Markdown(\"\"), display_id=True)\n",
    "    for chunk in stream:\n",
    "        response += chunk.choices[0].delta.content or ''\n",
    "        response = response.replace(\"```\",\"\").replace(\"markdown\", \"\")\n",
    "        update_display(Markdown(response), display_id=display_handle.display_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56bf0ae3-ee9d-4a72-9cd6-edcac67ceb6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "stream_brochure(\"HuggingFace\", \"https://huggingface.co\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdb3f8d8-a3eb-41c8-b1aa-9f60686a653b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Try changing the system prompt to the humorous version when you make the Brochure for Hugging Face:\n",
    "\n",
    "stream_brochure(\"HuggingFace\", \"https://huggingface.co\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a27bf9e0-665f-4645-b66b-9725e2a959b5",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left;\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#181;\">Business applications</h2>\n",
    "            <span style=\"color:#181;\">In this exercise we extended the Day 1 code to make multiple LLM calls, and generate a document.\n",
    "\n",
    "This is perhaps the first example of Agentic AI design patterns, as we combined multiple calls to LLMs. This will feature more in Week 2, and then we will return to Agentic AI in a big way in Week 8 when we build a fully autonomous Agent solution.\n",
    "\n",
    "Generating content in this way is one of the very most common Use Cases. As with summarization, this can be applied to any business vertical. Write marketing content, generate a product tutorial from a spec, create personalized email content, and so much more. Explore how you can apply content generation to your business, and try making yourself a proof-of-concept prototype. See what other students have done in the community-contributions folder -- so many valuable projects -- it's wild!</span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14b2454b-8ef8-4b5c-b928-053a15e0d553",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left;\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#900;\">Before you move to Week 2 (which is tons of fun)</h2>\n",
    "            <span style=\"color:#900;\">Please see the week1 EXERCISE notebook for your challenge for the end of week 1. This will give you some essential practice working with Frontier APIs, and prepare you well for Week 2.</span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17b64f0f-7d33-4493-985a-033d06e8db08",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left;\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#f71;\">A reminder on 3 useful resources</h2>\n",
    "            <span style=\"color:#f71;\">1. The resources for the course are available <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">here.</a><br/>\n",
    "            2. I'm on LinkedIn <a href=\"https://www.linkedin.com/in/eddonner/\">here</a> and I love connecting with people taking the course!<br/>\n",
    "            3. I'm trying out X/Twitter and I'm at <a href=\"https://x.com/edwarddonner\">@edwarddonner<a> and hoping people will teach me how it's done..  \n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f48e42e-fa7a-495f-a5d4-26bfc24d60b6",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left;\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#090;\">Finally! I have a special request for you</h2>\n",
    "            <span style=\"color:#090;\">\n",
    "                My editor tells me that it makes a MASSIVE difference when students rate this course on Udemy - it's one of the main ways that Udemy decides whether to show it to others. If you're able to take a minute to rate this, I'd be so very grateful! And regardless - always please reach out to me at ed@edwarddonner.com if I can help at any point.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8d3e1a1-ba54-4907-97c5-30f89a24775b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}