Pydantic Settings env_prefix is applying to secrets filenames

[ensignavenger]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

The documentation for Pydantic Settings states the env_prefix applies to environment variables. However, it is also applying to secret filenames. See https://docs.pydantic.dev/latest/concepts/pydantic_settings/#validation-of-default-values

I think it makes more sense to have env_prefix only applies to environment vars, as is reflected in the documentation, so I would think this is a code bug, but if the behavior is intended, the documentation should reflect that.

Example Code

>>> from pydantic_settings import BaseSettings, SettingsConfigDict
>>> class Settings(BaseSettings):
...     model_config = SettingsConfigDict(secrets_dir="/run/secrets")
...     secret_key: str
>>> a = Settings()
>>> a.secret_key
'mysupersecretkeyshhhh'

>>> class Settings(BaseSettings):
...     model_config = SettingsConfigDict(secrets_dir="/run/secrets", env_prefix="PB_")
...     secret_key: str
...     
>>> a = Settings()
Traceback (most recent call last):
  File "<python-input-8>", line 1, in <module>
    a = Settings()
  File "/home/workspace/.venv/lib/python3.13/site-packages/pydantic_settings/main.py", line 171, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        **__pydantic_self__._settings_build_values(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<25 lines>...
        )
        ^
    )
    ^
  File "/home/workspace/.venv/lib/python3.13/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings
secret_key
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing

Python, Pydantic & OS Version

Python 3.13.1, Debian 12 (Docker Container), Pydantic 2.23.4, Pydantic Settings 2.7


 python -c "import pydantic.version; print(pydantic.version.version_info())"
             pydantic version: 2.9.2
        pydantic-core version: 2.23.4
          pydantic-core build: profile=release pgo=false
                 install path: /home/workspace/.venv/lib/python3.13/site-packages/pydantic
               python version: 3.13.1 (main, Dec  4 2024, 20:40:24) [GCC 12.2.0]
                     platform: Linux-6.1.0-22-amd64-x86_64-with-glibc2.36
             related packages: typing_extensions-4.12.2 pydantic-settings-2.7.0
                       commit: unknown

[hramezani]

This is an intended behavior. nev_prefix will be applied to all settings sources. So, this is not a bug.

I agree that the env_prefix name is not a good name for a config used by all the setting sources. file_prefix or prefix are probably better names. but this is something that we can't change in this version because it will introduce a breaking change.

[ensignavenger]

Cool, for now then can I suggest updating the https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names section in the docs to something along the lines of "Setting Name Prefixes" and update the text?

Then, consider if it would be worth it to create a new name and alias the old one to it, then deprecate the old one?

The key for me is that I read the documentation and was surprised by the actual behavior, so if the documentation is updated, I don't think it is hard to work around it. I just wasn't sure it was intentional or not :)

Oh, and thank you @hramezani for your contributions to Django and Pydantic, I use both of these tools extensively! Your contributions are very appreciated!

[hramezani]

Cool, for now then can I suggest updating the https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names section in the docs to something along the lines of "Setting Name Prefixes" and update the text?

Agree, we need to update the document here.

Then, consider if it would be worth it to create a new name and alias the old one to it, then deprecate the old one?

Would be a good idea. probably I work on it later.

The key for me is that I read the documentation and was surprised by the actual behavior, so if the documentation is updated, I don't think it is hard to work around it. I just wasn't sure it was intentional or not :)

Oh, and thank you @hramezani for your contributions to Django and Pydantic, I use both of these tools extensively! Your contributions are very appreciated!

Thanks for your message. it worth a lot to me ❤️

[makukha]

@ensignavenger I faced the same problem and ended up writing a separate settings source for secrets: https://github.com/makukha/pydantic-file-secrets

[hramezani]

I merged a PR to improve on env_prefix doc.