litellm support

[cyyeh]

Currently we use Haystack to provide various LLM providers; however, we found it's still too hard for the community to contribute to various LLMs. We plan to use litellm instead; it provides more functionalities such as fallback, and it provides openai api compatible interface for all llms. We think it's easier for the community's contribution

[cyyeh]

This task is scheduled to assign to the contributor Gomaa

[cyyeh]

@MGomaa435

implementation plan doc

1. We plan to replace implementation of LLMProvider and EmbedderProvider using litellm. However, we think we can focus on replacing LLMProvider first.
2. For the get_generator method in each LLMProvider, we can implement a higher-order function that accepts model params and returns a function. when we invoke the returned function, it executes the llm generation api, which mean acompletion in litellm
3. For all llm api calls, we need to use async version
4. We can finish openai, openai-compatible, and ollama first?

[MGomaa435]

Hi Jimmy, my pleasure to support in this