Summary handler token limit retry

[jasnoo]

Description

Currently, feedback that is received via the Feedback Widget is stored in a Google Sheet. There are specific tabs that only contain feedback comments related to one conceptual webpage tab. For example, there is a maternity-timeline tab corresponding to all feedback received on the maternity timeline tool.

Previously, the summary handler would receive a http request that includes some URL. The handler determines what sheet to grab info from, pulls a specific hardcoded amount of rows from a tab google sheet, and attempt to summarize the comments from that data via chatGPT. The limitation here is that there is a max token for the model, so if the comment content exceeded the model's maximum context length, it would throw an error.

Approach

This has been improved in 3 ways:

1. Estimating the tokens in the comments via reduceCommentToNotExceedTokenLimit. A token can be estimated as ~4 characters, so there is a check to see if the comments exceed this. If so, the comments are reduced to be under the estimated limit. Note that the max tokens for the entire transaction is 16384, with 2000 being dedicated to the summary output (so 14384 for the input).
2. Individual comments are capped at 500 characters before being passed to the getSummary function. A very small percentage of comments exceed this - for instance in claim-status tab, it was about ~8/990 that exceeded this. Doing this helps allow for more comments to be considered and summarized
3. A retry mechanism has been added in getSummary so that an error is thrown due to exceeding the max context length/token count, the comment array will slice 50 comments on and retry again - up to 5 times.

Note: The summary handler is only utilized in an internal tool currently which I plan to update as well.

Steps to Test

This handler does not have complete tests, just some tests for the libs. I have added/updated them for now. npm test for tests.

I have also run example HTTP calls intentionally using comment content that would exceed the character limits, context limits, and also max out the retries and can see that it works as expected