Azure OpenAI support in Marvin v2

[lostmygithubaccount]

First check

• I added a descriptive title to this issue.
• I used the GitHub search to try to find a similar issue and didn't find one.
• I searched the Marvin documentation for this issue.

Bug summary

this seems to be blocked on the error: AttributeError: 'NoneType' object has no attribute 'get_secret_value'

I've tried a few times to use Marvin v2 with Azure OpenAI, following the most recent docs for each. repro below

Reproduction

from dotenv import load_dotenv

load_dotenv()

from openai import AzureOpenAI

# gets the API Key from environment variable AZURE_OPENAI_API_KEY
client = AzureOpenAI(
    # https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning
    api_version="2023-07-01-preview",
    # https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
    azure_endpoint="https://birdbrain-eh.openai.azure.com",
)

# just checking Azure OpenAI works, and it does
completion = client.chat.completions.create(
    model="gpt-4-turbo",  # e.g. gpt-35-instant
    messages=[
        {
            "role": "user",
            "content": "How do I output all files in a directory using Python?",
        },
    ],
)
print(completion.model_dump_json(indent=2))


# now trying marvin
from marvin import ai_model
from pydantic import BaseModel, Field

@ai_model(client = client)
class Location(BaseModel):
    city: str
    state_abbreviation: str = Field(
        ..., 
        description="The two-letter state abbreviation"
    )

# or
from marvin import ai_classifier
from enum import Enum

class AppRoute(Enum):
    """Represents distinct routes command bar for a different application"""

    USER_PROFILE = "/user-profile"
    SEARCH = "/search"
    NOTIFICATIONS = "/notifications"
    SETTINGS = "/settings"
    HELP = "/help"
    CHAT = "/chat"
    DOCS = "/docs"
    PROJECTS = "/projects"
    WORKSPACES = "/workspaces"

@ai_classifier(client = client)
def classify_intent(text: str) -> AppRoute:
    '''Classifies user's intent into most useful route'''

Error

True
{
  "id": "chatcmpl-8bsfOeP0Du5lXL9cAtZYPpAvnEchV",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "You can list all files in a directory using Python with the `os` or `pathlib` module. Here are examples of both:\n\n### Using the `os` Module\n\n\nimport os\n\n# Define the directory you want to list files from\ndirectory = \"/path/to/your/directory\"\n\n# Use os.listdir to list all entries in the directory\nentries = os.listdir(directory)\n\n# Filter out directories from the list (optional)\nfiles = [entry for entry in entries if os.path.isfile(os.path.join(directory, entry))]\n\n# Print all files\nfor file in files:\n    print(file)\n\n\nIf you only want to print the file names and ignore directories, make sure to check if the entry is a file using `os.path.isfile`.\n\n### Using the `pathlib` Module\n\n```python\nfrom pathlib import Path\n\n# Define the directory you want to list files from\ndirectory = Path(\"/path/to/your/directory\")\n\n# Use the .glob method to match all files\nfiles = directory.glob('*')\n\n# Iterating through the files and print them\nfor file in files:\n    if file.is_file():  # Check if it's a file and not a directory\n        print(file.name)  # Print only file name; use `file` to print the full path\n```\n\nUsing `pathlib` is generally considered more modern and idiomatic in Python 3.x, as `pathlib` provides an object-oriented interface for handling filesystem paths.\n\nRemember to replace `\"/path/to/your/directory\"` with the actual file path to the directory you want to list files from. If you want to list files from the current working directory, you can use `'.'` or simply `os.getcwd()` with the `os` module or `Path.cwd()` with the `pathlib` module as the directory path.",
        "role": "assistant",
        "function_call": null,
        "tool_calls": null
      },
      "content_filter_results": {
        "hate": {
          "filtered": false,
          "severity": "safe"
        },
        "self_harm": {
          "filtered": false,
          "severity": "safe"
        },
        "sexual": {
          "filtered": false,
          "severity": "safe"
...
      }
    }
  ]
}
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 4
      1 from marvin import ai_model
      2 from pydantic import BaseModel, Field
----> 4 @ai_model(client = client)
      5 class Location(BaseModel):
      6     city: str
      7     state_abbreviation: str = Field(
      8         ..., 
      9         description="The two-letter state abbreviation"
     10     )

File ~/repos/marvin/src/marvin/components/ai_model.py:66, in ai_model(_type, **kwargs)
     63         return _type
     65     extract.__annotations__["return"] = _type
---> 66     return ai_fn(
     67         fn=extract,
     68         **AIModelKwargsDefaults(**kwargs).model_dump(exclude_none=True),
     69     )
     71 return partial(
     72     ai_model, **AIModelKwargsDefaults(**kwargs).model_dump(exclude_none=True)
     73 )

File ~/repos/marvin/src/marvin/components/ai_function.py:253, in ai_fn(fn, **kwargs)
    242 def ai_fn(
    243     fn: Optional[Callable[P, Union[T, Coroutine[Any, Any, T]]]] = None,
    244     **kwargs: Unpack[AIFunctionKwargs],
   (...)
    250     Callable[P, Union[T, Coroutine[Any, Any, T]]],
    251 ]:
    252     if fn is not None:
--> 253         return AIFunction[P, T].as_decorator(
    254             fn=fn, **AIFunctionKwargsDefaults(**kwargs).model_dump(exclude_none=True)
    255         )
    257     def decorator(
    258         func: Callable[P, Union[T, Coroutine[Any, Any, T]]],
    259     ) -> Callable[P, Union[T, Coroutine[Any, Any, T]]]:
    260         return AIFunction[P, T].as_decorator(
    261             fn=func,
    262             **AIFunctionKwargsDefaults(**kwargs).model_dump(exclude_none=True),
    263         )

File ~/repos/marvin/src/marvin/components/ai_function.py:222, in AIFunction.as_decorator(cls, fn, **kwargs)
    216     return cls(
    217         fn=func,
    218         **AIFunctionKwargsDefaults(**kwargs).model_dump(exclude_none=True),
    219     )
    221 if fn is not None:
--> 222     return decorator(fn)
    224 return decorator

File ~/repos/marvin/src/marvin/components/ai_function.py:216, in AIFunction.as_decorator.<locals>.decorator(func)
    215 def decorator(func: Callable[P, Union[T, Coroutine[Any, Any, T]]]) -> Self:
--> 216     return cls(
    217         fn=func,
    218         **AIFunctionKwargsDefaults(**kwargs).model_dump(exclude_none=True),
    219     )

    [... skipping hidden 1 frame]

File ~/repos/marvin/src/marvin/components/ai_function.py:98, in AIFunction.<lambda>()
     96 temperature: Optional[float] = None
     97 client: Client = Field(default_factory=lambda: MarvinClient().client)
---> 98 aclient: AsyncClient = Field(default_factory=lambda: AsyncMarvinClient().client)
    100 @property
    101 def logger(self):
    102     return get_logger(self.__class__.__name__)

    [... skipping hidden 1 frame]

File ~/repos/marvin/src/marvin/client/openai.py:177, in AsyncMarvinClient.<lambda>()
    167 class AsyncMarvinClient(pydantic.BaseModel):
    168     model_config = pydantic.ConfigDict(
    169         arbitrary_types_allowed=True, protected_namespaces=()
    170     )
    172     client: AsyncClient = pydantic.Field(
    173         default_factory=lambda: AsyncClient(
    174             **settings.openai.model_dump(
    175                 exclude={"chat", "images", "audio", "assistants", "api_key"}
    176             )
--> 177             | dict(api_key=settings.openai.api_key.get_secret_value())
    178         )
    179     )
    181     @classmethod
    182     def wrap(cls, client: AsyncClient) -> "AsyncClient":
    183         client.chat.completions.create = partial(
    184             cls(client=client).chat, completion=client.chat.completions.create
    185         )  # type: ignore #noqa

AttributeError: 'NoneType' object has no attribute 'get_secret_value'

Versions

│ No such command 'version'.                                                                                                                                                                                                                         


I installed from main

Additional context

No response

[sdeep27]

Using prerelease v2.0.1a1, got this same error with regular OpenAI (not Azure) and using this copied example from docs. :
@ai_fn(client=client)
def sentiment_list(texts: list[str]) -> list[float]:
"""
Given a list of texts, returns a list of numbers between 1 (positive) and
-1 (negative) indicating their respective sentiment scores.
"""

sentiment_list(
[
"That was surprisingly easy!",
"Oh no, not again.",
]
)

[zzstoatzz]

@lostmygithubaccount do you get the same error if you set OPENAI_API_KEY instead of AZURE_OPENAI_API_KEY?

@sdeep27 - how have you set your api key?

[lostmygithubaccount]

yes same error -- tested by just copying AZURE_OPENAI_API_KEY as OPENAI_API_KEY in an additional line in my .env file and re-running

[zzstoatzz]

@lostmygithubaccount - ah so we look for dotenv stuff in ~/.marvin/.env, e.g.

(marvin) pad-2 :: src/open-source/marvin ‹main›
» cat ~/.marvin/.env | rg OPENAI
MARVIN_OPENAI_API_KEY=sk-xxx
MARVIN_OPENAI_ORGANIZATION=org-xxx

also my bad, the way the pydantic settings work, I would prefix the env var with MARVIN_

if you set OPENAI_API_KEY somewhere global like your ~/.zshrc , it should get picked up as well

[lostmygithubaccount]

I got a step farther by copying my .env file to ~/.marvin and prefixing each line with MARVIN_, then hit this (another error I think I saw in a different attempt):

NotFoundError: Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}

I'm assuming I need to set a separate deployment environment variable from the client instantiation? regardless of what's needed right now, would appreciate a clear setup guide for getting Azure OpenAI to work w/ v2! would love to upgrade, have been stuck on v1.3.0

[zzstoatzz]

@lostmygithubaccount that's a good callout, thanks - we'll get an azure setup guide added in soon as we can

[sdeep27]

@lostmygithubaccount do you get the same error if you set OPENAI_API_KEY instead of AZURE_OPENAI_API_KEY?

@sdeep27 - how have you set your api key?

@zzstoatzz
I explicitly set it while testing, something like this
client = OpenAI(api_key = credentials['open_ai'])

[sdeep27]

Here's the full code:

from marvin import ai_model, ai_fn, ai_classifier
from pydantic import BaseModel, Field
from openai import OpenAI
oai_client = OpenAI(api_key=SECRETS['openai1'])
@ai_fn(client=oai_client)
def sentiment_list(texts: list[str]) -> list[float]:
    """
    Given a list of `texts`, returns a list of numbers between 1 (positive) and
    -1 (negative) indicating their respective sentiment scores.
    """


sentiment_list(
    [
        "That was surprisingly easy!",
        "Oh no, not again.",
    ]
)

---

returns:
AttributeError: 'NoneType' object has no attribute 'get_secret_value'

[zzstoatzz]

@sdeep27 if you're not using azure, you should just be able to set MARVIN_OPENAI_API_KEY in ~/.marvin/.env and not have to pass a client to ai_fn

although it does seem we should fail more gracefully here so I've opened #707

[sdeep27]

@zzstoatzz thanks for the quick responses! it is working now. There is some confusion because the website docs seem to be using the v2 syntax with no mention that it is not the default version installed on pip. Then to compound, the website docs have us passing the openAI client with the key rather than mentioning setting the marvin env variable.

[zzstoatzz]

@sdeep27 sorry yep that's on us, right now the docs are showing marvin 2.x which is confusing because only the pre-release is available on pip - full release coming soon.

I've just updated the quickstart / settings docs in that PR to make it clearer - thanks for the callout!

[lostmygithubaccount]

@zzstoatzz can you confirm Azure is supported right now in v2 and which environment variables need to be set? I'm not seeing much with "Azure" in the codebase and can't figure it out, but things seemed to have changed from v1

if I can get that working I'd be happy to help PR a setup guide for Azure OpenAI -- for now going to play around w/ v2 on a regular OpenAI account

[zzstoatzz]

hi @lostmygithubaccount - forgive me I feel like we've chatted on different issues at this point about this, but do you think we can close this issue?

i've been able to test against azure myself recently and I'm not having any troubles using these env vars

[lostmygithubaccount]

yep! I'll open more specific issues with Azure as I run into them