Can we reduce the prompt size of entity_extraction.txt, it takes most of tokens consumption #460
KylinMountain
started this conversation in
General
Replies: 2 comments 5 replies
-
Hi @KylinMountain |
Beta Was this translation helpful? Give feedback.
1 reply
-
Beta Was this translation helpful? Give feedback.
4 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
It is too large, it has 10835 words, about 2224 tokens.
I have txt with 26000 words, chunk it with 300 overlap 100, there's about 260 chunk to be processed with this entity_extraction.
It will consume about 260*(2224+300)=634240 tokens....
I think this is the most part of tokens consumption.
Beta Was this translation helpful? Give feedback.
All reactions