Semantic kernel model adapter #4851
Conversation
|
I can see it is going to be useful for integrating with non-openai APIs such as Gemini and Mistral, using SK's large ecosystem of connectors. Could you show a few examples of such in the API documentation? This way users can take away a clear picture of why this model adapter is useful. |
lspinheiro
force-pushed
the
lpinheiro/feat/add-sk-model-adapter
branch
from
f77cd39 to
3fe641e
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4851 +/- ##
==========================================
+ Coverage 68.53% 68.64% +0.11%
==========================================
Files 156 160 +4
Lines 10140 10334 +194
==========================================
+ Hits 6949 7094 +145
- Misses 3191 3240 +49
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
lspinheiro
changed the title
| def sk_client() -> AzureChatCompletion: | ||
| deployment_name = os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME") | ||
| endpoint = os.getenv("AZURE_OPENAI_ENDPOINT") | ||
| api_key = os.getenv("AZURE_OPENAI_API_KEY") |
There was a problem hiding this comment.
If the keys are in the environment, it will actually run the tests, otherwise it uses a mock. I set it like this so I could easily swap between real test and testing with a mock. Let me know if you think this could cause any issues (e.g., unexpected API calls in CI)
| semantic-kernel-all = [ | ||
| "semantic-kernel[google,hugging_face,mistralai,ollama,onnx,anthropic,usearch,pandas,aws,dapr]>=1.17.1", | ||
| ] | ||
|
|
There was a problem hiding this comment.
I added all the model extras. There were some extras for search plugins that had dependencies in pre-release azure library versions. When we decide to fully enable semantic kernel we will need to decide if we update the CI or limit functionality until its dependencies are fully released.
| OutputT = TypeVar("OutputT", bound=BaseModel) | ||
|
|
||
|
|
||
| class KernelFunctionFromTool(KernelFunction): |
There was a problem hiding this comment.
I don't have separate tests for the tool adapter yet but it has good coverage in the model adapter test file. I think it will be useful to add a few dedicated tests when we implement the agent so I can write tests for different semantic kernel plugins, I hope that is OK.
| return max_tokens - used_tokens | ||
|
|
||
| @property | ||
| def model_info(self) -> ModelInfo: |
There was a problem hiding this comment.
One details here that I almost forgot. The model info is generally not exposed in a direct way by the semantic kernel ChatCompletionClientBase. I think this is something that is not often exposed by all providers. And I think the only reliable way to provide accurate information would be to keep a hardcoded map from model to capabilities. Have we thought about this, how to maintain model info for different providers?
Why are these changes needed?
Related issue number
Checks