Keeps getting inputs too long error

[wenzhao98]

changed embedding to hkunlp/instructor-large and using HuggingFace instead of OpenA. Keeps getting the following error:

ValueError: Error raised by inference API: Input validation error: inputs must have less than 1024 tokens. Given: 1062

I think it is due to the memory.

[costabm]

Check my pull request (#45) where the "chunk_size" and "max_new_tokens", have been updated, among other improvements.