Refactoring of the local metrics

[liwii]

Refactored (mainly) local metrics code. The changes include:

• For reference_free metrics with Detoxify / AutoModelForSequenceClassification, created base classes and made logics such as tqdm progress bar common. For these metrics I also added an argument local_overflow_strategy, with which you can handle the input longer than the model input window in 3 different ways (truncate, nullify, raise)
• For semantic similarity (both local and openai), made a base class and put most of the logic into that class.
• For non-english factual consisntency, long inputs always caused error, so avoided it by adding truncation=True argument. (I would like to add the overflow handling to the factual consietency metrics too, but it should be huge enough to develop in another PR)
• Moved loading logics of many models into model_manager. check_model_availability became flaky as a result, so I disabled these checks for now.

[liwii]

@yosukehigashi
The PR became much bigger than I thought, but I would be happy if you could take a look when you have time!!

[yosukehigashi]

Wow what a beautiful refactor 🤩 Left a bunch of comments but generally LGTM!

Let's do a sanity check with LangCheckChat as well since this is a pretty big change. Could you test out English and Japanese, and I'll do Chinese and German?

[yosukehigashi]

Can you add a docstring here? E.g. we should explain the allowed overflow strategies and what they do

[liwii]

Added a simple docstring there

[liwii]

Thanks for the review! Let me resolve the comments first, then we can do the sanity-check together as you say!

[liwii]

@yosukehigashi
Does the change look good to you? If so, we can start testing the new version.

[yosukehigashi]

@yosukehigashi Does the change look good to you? If so, we can start testing the new version.

yeah looks fine to me! can you start testing English and Japanese then? I need to make some code changes to test German / Chinese

[liwii]

Tried some examples of Japanese & English with references, there were no errors at least.

[yosukehigashi]

Chinese and German work fine too. LGTM!