LLM Engineering: Master AI, Large Language Models & Agents

This course provides a comprehensive guide to mastering AI, focusing on Large Language Models (LLMs) and agents. It covers essential concepts, practical coding exercises, and comparisons of leading LLMs to equip participants with the skills needed to build and deploy LLM-based solutions.

Enroll in the course on Udemy

Table of Contents

• LLM Engineering: Master AI, Large Language Models & Agents
  • Table of Contents
  • Week 1 - Build Your First LLM Product: Exploring Top Models & Transformers
    • Day 1
    • Day 2
    • Day 3
    • Day 4
    • Day 5
  • Week 2 - Build a Multi-Modal Chatbot: LLMs, Gradio UI, And Agents
    • Day 6

Week 1 - Build Your First LLM Product: Exploring Top Models & Transformers

Day 1

What I did today:

• Introduced to the course and its objectives.
• Learned about the basics of Large Language Models (LLMs).
• Used Ollama to run LLMs locally.
• Wrote code to call OpenAI's frontier models.
• Distinguished between System and User prompts.
• Learned summarization techniques applicable to many commercial problems.

Resources:

• day1.ipynb

Day 2

What I did today:

• Reviewed the installation and setup of Ollama.
• Upgraded the Day 1 project to use an open-source model running locally via Ollama.
• Implemented a website summarizer using Llama 3.2.
• Explored alternative approaches using the OpenAI client library to call Ollama.
• Experimented with the DeepSeek reasoning model.

Resources:

• day2 EXERCISE.ipynb
• day2 notes.ipynb

Day 3

What I did today:

• Reflected on the capabilities of six leading LLMs, emphasizing their power and convergence in performance.
• Discussed evolving factors that differentiate these models, such as price and specific features.
• Conducted a fun, unscientific leadership challenge between GPT-4, Claude 3 Opus, and Gemini 1.5 Pro.
• Analyzed the pitches made by Alex (GPT-4), Blake (Claude 3 Opus), and Charlie (Gemini 1.5 Pro) for leadership.
• Prepared for the next session, which will delve into the technical aspects of LLMs, including Transformers, tokens, context windows, parameters, and API costs.

Resources:

• day3 notes.ipynb

Day 4

What I did today:

• Covered essential concepts like tokens, tokenization, context windows, and API costs.
• Clarified the difference between the chat interface cost and the API cost.
• Discussed the challenge of counting letters in a tokenized text.
• Explained why some models were able to answer the "how many A's" question.
• Practiced writing code to call the OpenAI API and local models like Llama.
• Compared and contrasted different frontier LLMs.

Resources:

• day4 notes.ipynb

Day 5

What I did today:

• Successfully completed the first week, gaining a comprehensive understanding of Transformer models, tokenization techniques, and context window limitations.

• Explored various frontier AI models, assessing their capabilities and constraints in real-world applications.

• Developed practical experience with the OpenAI API, implementing streaming responses and markdown formatting.

• Designed and built a personal AI tutor tool, applying multi-shot prompting to enhance interactions.

• Experimented with system prompts to refine model responses based on tone, character, and instruction adherence.

• Integrated the Llama API to facilitate efficient local model interactions.

• Outlined key objectives for the upcoming week, including multi-model API usage, agent development, and UI implementation with Gradio.

• day5.ipynb

• day5 notes.ipynb

• week1 EXERCISE.ipynb

Week 2 - Build a Multi-Modal Chatbot: LLMs, Gradio UI, And Agents

Day 6

What I did today:

• Successfully configured API keys for Anthropic (Claude) and Google (Gemini), expanding the toolkit for LLM development.

• Demonstrated the ability to integrate and utilize OpenAI, Anthropic, and Google APIs within a JupyterLab environment, including setting parameters and streaming responses.

• Implemented real-time LLM output by streaming responses from Claude and OpenAI, effectively handling markdown formatting.

• Constructed multi-turn adversarial conversations between GPT-4-mini and Claude-3-haiku, showcasing the manipulation of message lists and system prompts.

• Explored and compared the API structures and functionalities of OpenAI, Claude, and Gemini, highlighting their differences and similarities.

• Applied temperature control to influence the creativity and randomness of LLM outputs, showcasing practical parameter adjustments.

• Designed and executed a joke generation experiment to compare the humor capabilities of different LLMs, providing insights into their creative outputs.

• Reviewed and understood the key components of Transformers, including context windows, tokens, and API costs, reinforcing foundational knowledge.

• day1.ipynb

• day1 notes.ipynb