diff --git a/content/page/fallschool/Chapter5/img/arthur.jpg b/content/page/fallschool/Chapter5/img/arthur.jpg new file mode 100644 index 0000000..d5e9904 Binary files /dev/null and b/content/page/fallschool/Chapter5/img/arthur.jpg differ diff --git a/content/page/fallschool/Chapter5/index.md b/content/page/fallschool/Chapter5/index.md index e03d16c..cf29bb2 100644 --- a/content/page/fallschool/Chapter5/index.md +++ b/content/page/fallschool/Chapter5/index.md @@ -1,5 +1,5 @@ --- -title: "Chapter 05 - Logging and Analyzing Execution" +title: "Chapter 05 - Create your own LLM assistant" date: 2023-15-05T10:35:35-06:00 subtitle: "" tags: ["Research"] @@ -12,8 +12,7 @@ resources: src: "buttons.json" ---
-On Chapter 5, you will learn how to log and analyze the execution of the milk delivery task. -This session emphasizes the importance of data logging and how it can be used to improve robotic performance in future tasks. +In Chapter 5, you will head into generative Large Language Models (LLMs) and how to fine-tune them. With Retreival Augmented Generation (RAG) you create a specialized assistant that serves as a companion for robot programming. For Entering Chapter five click here: Chapter 5! @@ -28,22 +27,32 @@ For Entering Chapter five click here:

Welcome to the fifth day of our hands-on course!

-Today, you will focus on the critical aspect of logging and analyzing the robot's actions during the milk delivery task. This knowledge will help you identify areas for improvement and optimize future robotic tasks. +Today, you will head into generative Large Language Models (LLMs) and how to fine-tune them. With Retreival Augmented Generation (RAG) you create a specialized assistant that serves as a companion for robot programming. The software used is completely open-source and can be installed on your personal machines. For this course we offer the webservice RAGflow, utilizing models from Ollama to digest and extend its knowledge. -**Goal**: By the end of the session, you will be able to set up logging to capture execution data and analyze this data to enhance task performance. +**Goal**: By the end of this session, you will know how to fine-tune an existing LLM with a knowledgebase in the fashion of RAG, and define the assistants behavior through (initial) prompt-engineering. ## Prerequisites -- ? +- Laptop with internet connection + +(Optional) If run on your own machine: +- 16GB RAM +- GPU recommended ## Theoretical Background -- We will discuss logging frameworks and how they capture important execution data. -- You’ll learn how analysis of logs can reveal inefficiencies and inform better planning algorithms. +- Build your LLM knowledgebase with the content of the past weeks lectures. +- Prompt-engineer to constraint and form an assistants behavior. +- Refine your model to improve the assistent. ## Step-by-Step Hands-On Exercises -1. **Set Up Logging**: Configure logging to capture the robot's actions during the milk delivery task. -2. **Analyze Logs**: Examine the collected logs to identify any inefficiencies or issues during task execution. -3. **Discussion**: Discuss how the insights gained from log analysis can lead to improvements in robotic planning and execution. -4. **Code Examples**: Work with scripts for logging and analyzing critical data points. +1. **Scope and Recap**: We will get an overview of useful lecture material, presented over the past week, to refresh your memory and collect that as training data. +2. **Introduction to RAGflow**: What is Retreival Augmented Generation and how can you set it up for any kind of application you need? +3. **Discuss first impressions**: Gather our first ideas on strength and weakness of generative LLMs and RAG. +4. **Refine your assistant**: Exceed boundaries and try to break the system, explore creative ways of forming your assistant. +5. **Share experiences**: Condense the experience we made by sharing them with your peers. + +Access to RAGFlow +--- +TBD Interactive Actions and/or Examples --- @@ -51,14 +60,13 @@ Interactive Actions and/or Examples {{}} ## Summary -By the end of this session, you will understand the significance of feedback loops in robotics and how logging can be leveraged for continuous improvement. +By the end of this session you will have experience with the difficulties of configuring your own assistant, and in what ways fine-tuning can change the assistants behavior. ## Congratulations on Completing the Course! You’ve successfully completed the hands-on course on cognition-enabled robotics, gaining valuable insights into each step of the robot's tasks. ## Further Reading/Exercises -- Explore techniques for analyzing various scenarios from logged data to refine your understanding. -- Investigate machine learning methods that can optimize task performance based on analysis of logs. +- TBD Example Videos --- @@ -81,4 +89,17 @@ Example Videos Profile Vanessa Hassouna + +
+ +
+ +
+

Arthur Niedzwiecki

+ Tel: +49 421 218 64033
+ Mail: aniedz@cs.uni-bremen.de
+ + Profile Arthur Niedzwiecki + +