Skip to content

Commit

Permalink
Merge pull request #92 from artnie/fs-day5-llm
Browse files Browse the repository at this point in the history
Add content for FS Day5 LLMs RAG
  • Loading branch information
sunava authored Oct 30, 2024
2 parents 3f1af24 + a59b0ef commit f59f3ab
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 15 deletions.
Binary file added content/page/fallschool/Chapter5/img/arthur.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
51 changes: 36 additions & 15 deletions content/page/fallschool/Chapter5/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Chapter 05 - Logging and Analyzing Execution"
title: "Chapter 05 - Create your own LLM assistant"
date: 2023-15-05T10:35:35-06:00
subtitle: ""
tags: ["Research"]
Expand All @@ -12,8 +12,7 @@ resources:
src: "buttons.json"
---
<div class="hidde-after-preview">
On Chapter 5, you will learn how to log and analyze the execution of the milk delivery task.
This session emphasizes the importance of data logging and how it can be used to improve robotic performance in future tasks.
In Chapter 5, you will head into generative Large Language Models (LLMs) and how to fine-tune them. With Retreival Augmented Generation (RAG) you create a specialized assistant that serves as a companion for robot programming.

For Entering Chapter five click here:
<a class="btn btn-success" target="_blank" href="day5/"><b>Chapter 5!</b></a>
Expand All @@ -28,37 +27,46 @@ For Entering Chapter five click here:
</div>

<h1> Welcome to the fifth day of our hands-on course!</h1>
Today, you will focus on the critical aspect of logging and analyzing the robot's actions during the milk delivery task. This knowledge will help you identify areas for improvement and optimize future robotic tasks.
Today, you will head into generative Large Language Models (LLMs) and how to fine-tune them. With Retreival Augmented Generation (RAG) you create a specialized assistant that serves as a companion for robot programming. The software used is completely open-source and can be installed on your personal machines. For this course we offer the webservice <a href="https://ragflow.io/">RAGflow</a>, utilizing models from Ollama to digest and extend its knowledge.

**Goal**: By the end of the session, you will be able to set up logging to capture execution data and analyze this data to enhance task performance.
**Goal**: By the end of this session, you will know how to fine-tune an existing LLM with a knowledgebase in the fashion of RAG, and define the assistants behavior through (initial) prompt-engineering.

## Prerequisites
- ?
- Laptop with internet connection

(Optional) If run on your own machine:
- 16GB RAM
- GPU recommended

## Theoretical Background
- We will discuss logging frameworks and how they capture important execution data.
- You’ll learn how analysis of logs can reveal inefficiencies and inform better planning algorithms.
- Build your LLM knowledgebase with the content of the past weeks lectures.
- Prompt-engineer to constraint and form an assistants behavior.
- Refine your model to improve the assistent.

## Step-by-Step Hands-On Exercises
1. **Set Up Logging**: Configure logging to capture the robot's actions during the milk delivery task.
2. **Analyze Logs**: Examine the collected logs to identify any inefficiencies or issues during task execution.
3. **Discussion**: Discuss how the insights gained from log analysis can lead to improvements in robotic planning and execution.
4. **Code Examples**: Work with scripts for logging and analyzing critical data points.
1. **Scope and Recap**: We will get an overview of useful lecture material, presented over the past week, to refresh your memory and collect that as training data.
2. **Introduction to RAGflow**: What is Retreival Augmented Generation and how can you set it up for any kind of application you need?
3. **Discuss first impressions**: Gather our first ideas on strength and weakness of generative LLMs and RAG.
4. **Refine your assistant**: Exceed boundaries and try to break the system, explore creative ways of forming your assistant.
5. **Share experiences**: Condense the experience we made by sharing them with your peers.

Access to RAGFlow
---
TBD

Interactive Actions and/or Examples
---

{{<action_forms data="ActionButtons">}}

## Summary
By the end of this session, you will understand the significance of feedback loops in robotics and how logging can be leveraged for continuous improvement.
By the end of this session you will have experience with the difficulties of configuring your own assistant, and in what ways fine-tuning can change the assistants behavior.

## Congratulations on Completing the Course!
You’ve successfully completed the hands-on course on cognition-enabled robotics, gaining valuable insights into each step of the robot's tasks.

## Further Reading/Exercises
- Explore techniques for analyzing various scenarios from logged data to refine your understanding.
- Investigate machine learning methods that can optimize task performance based on analysis of logs.
- TBD

Example Videos
---
Expand All @@ -81,4 +89,17 @@ Example Videos
<span style="font-size: 15px;">Profile Vanessa Hassouna</span>
</a>
</div>

<div style="flex:30%;">
<img src="img/arthur.jpg" style="clip-path: circle(35%);">
</div>

<div style="flex:70%;">
<h3> Arthur Niedzwiecki</h3>
Tel: +49 421 218 64033 <br>
Mail: <a href="mailto:[email protected]">[email protected]</a> <br>
<a style="color:red" href="https://ai.uni-bremen.de/team/arthur_niedzwiecki">
<span style="font-size: 15px;">Profile Arthur Niedzwiecki</span>
</a>
</div>
</div>

0 comments on commit f59f3ab

Please sign in to comment.