How to avoid reaching maximum context length with repetitive task

[profnimrod]

I am using Crew AI to draft a short booklet on various subjects. The first task generates a list of essay titles (or book layout) given the title entered by the user. The second task then goes through each essay title (book section) and writes a 750-word essay for each. About halfway through I'm getting the error below. I think this is happening because the task that is writing the next essay for each section is including all the previous essays in the context when writing the next one. Assuming I have interpreted this correctly, how might I limit the context for this task?

`Exception has occurred: BadRequestError
Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens. However, your messages resulted in 8327 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api.openai.com/v1/chat/completions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

During handling of the above exception, another exception occurred:

File "E:\EmergentPublications\WellnessPress\CrewAI\main.py", line 71, in
result = crew.kickoff()
^^^^^^^^^^^^^^
openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens. However, your messages resulted in 8327 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}`

[Prayforhanluo]

some problem

[gururise]

Would be great if CrewAI could integrate MemGPT or some other type of method to manage context window without exceeding maximum.

[joaomdmoura]

There should be a guardrails for that already so I'll look into it and why it's not working. Some version of MemGPT integration is coming btw 😎👉👉

[gururise]

I have found the Nous Hermes2 Yi works very well, but is unusable for most tasks because it constantly exceeds its max context length of 4096.

[jfjensen]

I keep running into this issue as the max context length of my local LLM is 2048.

[robshox]

Any solution for this. Having the same problem with GPT4

[joaomdmoura]

Hey folks, we are working on a more native fix for this, probably a rolling context window, we needed to add the memory first before we could support it, but now that we have it we should be able to improve on this.

Sometime this could also indicated that you agent is trying to do too much and could be a good candidate to breaking into smaller agents / tasks.

That is not always de case tho.

[Abbie2020]

I came across this issue when running the sample code in Getting Started so you might want to include a simpler example that works.

[twelsh37]

Any update on this issue?

[rs8893]

The "Create agents to research and write an article" lesson in the Multi-agent Systems with crewAI Tutorial on deeplearning.ai Shorts works just fine with "HuggingFaceH4/zephyr-7b-beta". The markdown output has 2280 characters. But when I try the exact same notebook on Google Colab Pro and on my local Mac Mini M1 the output truncates at 485 characters without the markdown. Any suggestion for a fix? Thanks in advance.

[dhirajnair]

Any update on this @joaomdmoura . If there is native support to handle this , then an example would be helpful.
Appreciate the good work on crewAI

[LeoAnt02]

Any solution for this ?

[lorenzejay]

We are actively developing this issue now. Will share the progress with everyone within the following week

[codebrain001]

Still looking forward to a resolution

[luandro]

Why the change to discussion, is this not being worked, or has been solved?

[theCyberTech]

It has been moved to the Feature Request category that is all, I am currently going through all issues and FRs to understand whether they are still viable. I hadn't got to this one yet but I have marked it in my calendar to have a read through to understand what the status is

[luandro]

It's a pretty crucial issue, and last notice was:

We are actively developing this issue now. Will share the progress with everyone within the following week

There has been no followup, and the sudden change to discussion brings hopes down.

[joaomdmoura]

Context window feature is done and will be available on the next version, it will ask you if you want to summarize the work before proceeding once it it's maximum for the model.
We are still testing to make it better but it's a move in the right direction.

I think the main thing for the original creator of this would be going mulit-crew, it's what I ahve used internally for things like, planning chapters and then writing each chapters, you can now use a mix of kickoff and kickoff_for_each, but we are also launching a new feature in this new version (Coming out tomorrow) called pipelines, it's early, but it's a way for you to connect multiple crews together. There are already docs on it (https://docs.crewai.com/core-concepts/Pipeline/) but keep in mind this is coming out as early feature tomorrow, so we might find some bugs and follow up with patches :)

[joebwd]

Would be amazing to include a flag to auto accept (so no manual input needed)! Thank you!

[mesmalif]

Lowering default of max_iter=25, stops the script sooner with Agent stopped due to iteration limit or time limit. I thought it should continue with any knowledge it gathered to that point instead of exiting! Maybe I am missing something?

note: result = my_crew.kickoff(inputs= inputs_array) returns Agent stopped due to iteration limit or time limit.