How to create youtube video summarizer with new Agents feature

[s-KaiNet]

Well, first of all that's not a question, that's a real how-to, I've tried some other tools in the past to summarize youtube videos, but they didn't work quite well for me, with LibreChat Agents I was able to implement it as I want without any coding. Maybe my experience will be helpful, so I'm sharing it in the discussions.

First of, we need some tool for youtube scraping, I found Apify service works good for that purpose. There is a YT scraper streamers/youtube-scraper or just 'YouTube Scraper'. For 5$ included in free plan you have 1000 scraper runs, which is more then enough for personal usage. You should configure the scraper by setup different settings, like enable subtitles, video url input, etc. Then copy the resulting payload JSON from the interface. On the start page for the scraper in the top right corner there is also a drop-down "API", from where you should copy OpenAPI specification. This is how it looks:

Now it's time to go to the LibreChat. In the models menu select Agents, then on the right panel Agent Builder. Give it a name, select a model, something cheap should be enough, like gpt-4o-mini. Then you should add Actions, but Actions button is not available until you save the Agent. After saving, click Add Actions and paste YT scraper OpenAPI spec. You will see something like this:

method streamers_youtube-scraper is recognized, which is good. Click Save everywhere and we'r almost good to go.

The last part is a system instruction. I found the below pattern works, but you can tweak it as you want (put it into the "Instructions" input for the Agent):

You are the video summarizing expert. Your task will be to read and provide a nice and detailed summary for a YouTube video based on it's subtitles. As an input, you will receive the video YouTube URL "<yt_url>" as the first line and the language code "<lang_code>" as the second line.
execute streamers_youtube-scraper action with below JSON payload, replacing <yt_url> with the real YouTube video URL, replacing <lang_code> with the real language code. Do not modify the rest of the payload, send it as-is. After receiving results, find subtitles in language: <lang_code>, subtitles type: "user_generated", if there are no any subtitles with type "user_generated", then use type "auto_generated" and language <lang_code>. Carefully read the subtitles and give me the nice and detailed summary. The "\n" inside subtitles body is a new line delimiter. Always answer in language, defined in <lang_code>. Extract thumbnailUrl and use it as a preview to the answer.
Do not provide information about video length, likes, view count.

The JSON payload:

{
"downloadSubtitles": true,
"hasCC": false,
"hasLocation": false,
"hasSubtitles": false,
"is360": false,
"is3D": false,
"is4K": false,
"isBought": false,
"isHD": false,
"isHDR": false,
"isLive": false,
"isVR180": false,
"maxResultStreams": 0,
"maxResults": 10,
"maxResultsShorts": 0,
"preferAutoGeneratedSubtitles": false,
"saveSubsToKVS": false,
"startUrls": [{
"url": "<yt_url>",
"method": "GET"
}
],
"subtitlesFormat": "plaintext",
"subtitlesLanguage": "<lang_code>"
}

In a nutshell, we tell LLM to execute the action with specific parameters. The actual url and the language is received from a user message. It's important, that video has this language in subtitles (it can be even auto-generated, available subtitles can be checked on YT interface). The subtitles format I selected is plain text, because it produces less output. Save the Agent and it's ready.

Here is a short demo how it works (with gpt-4o-mini):

2024-12-23_23-16-41.mp4

As demonstrated, in the chat, you can ask further questions about the video. The video was slightly cut, because the actual call to Apify and scraper takes ~20-30sec.

That's just one example, you can customize it to fit your needs, you can add more actions, etc. But I love the Agents feature a lot, so many interesting things could be implemented. Huge thanks to the authors!
The discussion is open, for any further questions or potential enhancements for the solution.

[danny-avila]

Thanks for sharing!