Skip to content

Add experimental support for PEP 764 inline TypedDicts #18889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/source/command_line.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1085,7 +1085,7 @@ format into the specified directory.
Enabling incomplete/experimental features
*****************************************

.. option:: --enable-incomplete-feature {PreciseTupleTypes, InlineTypedDict}
.. option:: --enable-incomplete-feature {PreciseTupleTypes, NewInlineTypedDict, InlineTypedDict}

Some features may require several mypy releases to implement, for example
due to their complexity, potential for backwards incompatibility, or
Expand Down Expand Up @@ -1132,6 +1132,14 @@ List of currently incomplete/experimental features:
# Without PreciseTupleTypes: tuple[int, ...]
# With PreciseTupleTypes: tuple[()] | tuple[int] | tuple[int, int]

* ``NewInlineTypedDict``: this feature enables :pep:`764` syntax for inline
:ref:`TypedDicts <typeddict>`, for example:

.. code-block:: python

def test_values() -> TypedDict[{"int": int, "str": str}]:
return {"int": 42, "str": "test"}

* ``InlineTypedDict``: this feature enables non-standard syntax for inline
:ref:`TypedDicts <typeddict>`, for example:

Expand Down
38 changes: 24 additions & 14 deletions docs/source/typed_dict.rst
Original file line number Diff line number Diff line change
Expand Up @@ -294,35 +294,45 @@ Inline TypedDict types

.. note::

This is an experimental (non-standard) feature. Use
``--enable-incomplete-feature=InlineTypedDict`` to enable.
This is an experimental feature proposed by :pep:`764`. Use
``--enable-incomplete-feature=NewInlineTypedDict`` to enable.

Sometimes you may want to define a complex nested JSON schema, or annotate
a one-off function that returns a TypedDict. In such cases it may be convenient
to use inline TypedDict syntax. For example:

.. code-block:: python

def test_values() -> {"int": int, "str": str}:
def test_values() -> TypedDict[{"int": int, "str": str}]:
return {"int": 42, "str": "test"}

class Response(TypedDict):
status: int
msg: str
# Using inline syntax here avoids defining two additional TypedDicts.
content: {"items": list[{"key": str, "value": str}]}
content: TypedDict[{"items": list[TypedDict[{"key": str, "value": str}]]}]

Inline TypedDicts can also by used as targets of type aliases, but due to
ambiguity with a regular variables it is only allowed for (newer) explicit
type alias forms:
.. note::

.. code-block:: python
Mypy also supports a legacy syntax for inline TypedDicts that pre-dates :pep:`764`:

.. code-block:: python

def test_values() -> {"int": int, "str": str}:
return {"int": 42, "str": "test"}

This legacy syntax can be enabled using ``--enable-incomplete-feature=InlineTypedDict``.
Due to ambiguity with a regular variables, the legacy syntax may only be used in
type aliases when using (newer) explicit type alias forms:

.. code-block:: python

from typing import TypeAlias
from typing import TypeAlias

X = {"a": int, "b": int} # creates a variable with type dict[str, type[int]]
Y: TypeAlias = {"a": int, "b": int} # creates a type alias
type Z = {"a": int, "b": int} # same as above (Python 3.12+ only)
X = {"a": int, "b": int} # creates a variable with type dict[str, type[int]]
Y: TypeAlias = {"a": int, "b": int} # creates a type alias
type Z = {"a": int, "b": int} # same as above (Python 3.12+ only)

Also, due to incompatibility with runtime type-checking it is strongly recommended
to *not* use inline syntax in union types.
This restriction does not apply to the :pep:`764` syntax.
Also, due to incompatibility with runtime type-checking, it is strongly recommended
to *not* use legacy inline syntax in union types.
2 changes: 0 additions & 2 deletions mypy/fastparse.py
Original file line number Diff line number Diff line change
Expand Up @@ -2077,8 +2077,6 @@ def visit_Tuple(self, n: ast3.Tuple) -> Type:
)

def visit_Dict(self, n: ast3.Dict) -> Type:
if not n.keys:
return self.invalid_type(n)
items: dict[str, Type] = {}
extra_items_from = []
for item_name, value in zip(n.keys, n.values):
Expand Down
5 changes: 4 additions & 1 deletion mypy/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,10 @@ class BuildType:
PRECISE_TUPLE_TYPES: Final = "PreciseTupleTypes"
NEW_GENERIC_SYNTAX: Final = "NewGenericSyntax"
INLINE_TYPEDDICT: Final = "InlineTypedDict"
INCOMPLETE_FEATURES: Final = frozenset((PRECISE_TUPLE_TYPES, INLINE_TYPEDDICT))
NEW_INLINE_TYPEDDICT: Final = "NewInlineTypedDict"
INCOMPLETE_FEATURES: Final = frozenset(
(PRECISE_TUPLE_TYPES, INLINE_TYPEDDICT, NEW_INLINE_TYPEDDICT)
)
COMPLETE_FEATURES: Final = frozenset((TYPE_VAR_TUPLE, UNPACK, NEW_GENERIC_SYNTAX))


Expand Down
43 changes: 39 additions & 4 deletions mypy/typeanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
check_arg_names,
get_nongen_builtins,
)
from mypy.options import INLINE_TYPEDDICT, Options
from mypy.options import INLINE_TYPEDDICT, NEW_INLINE_TYPEDDICT, Options
from mypy.plugin import AnalyzeTypeContext, Plugin, TypeAnalyzerPluginInterface
from mypy.semanal_shared import (
SemanticAnalyzerCoreInterface,
Expand All @@ -66,6 +66,7 @@
FINAL_TYPE_NAMES,
LITERAL_TYPE_NAMES,
NEVER_NAMES,
TPDICT_NAMES,
TYPE_ALIAS_NAMES,
UNPACK_TYPE_NAMES,
AnyType,
Expand Down Expand Up @@ -125,6 +126,7 @@
"typing.Union",
*LITERAL_TYPE_NAMES,
*ANNOTATED_TYPE_NAMES,
*TPDICT_NAMES,
}

ARG_KINDS_BY_CONSTRUCTOR: Final = {
Expand Down Expand Up @@ -810,6 +812,22 @@ def try_analyze_special_unbound_type(self, t: UnboundType, fullname: str) -> Typ
# TODO: verify this is unreachable and replace with an assert?
self.fail("Unexpected Self type", t)
return AnyType(TypeOfAny.from_error)
elif fullname in TPDICT_NAMES:
if len(t.args) != 1:
self.fail(
"TypedDict[] must have exactly one type argument", t, code=codes.VALID_TYPE
)
return AnyType(TypeOfAny.from_error)
item = t.args[0]
if not isinstance(item, TypedDictType): # type: ignore[misc]
self.fail(
"Argument to TypedDict[] must be a literal dictionary mapping item names to types",
t,
code=codes.VALID_TYPE,
)
return AnyType(TypeOfAny.from_error)
item.is_pep764 = True
return self.anal_type(item, allow_typed_dict_special_forms=True)
return None

def get_omitted_any(self, typ: Type, fullname: str | None = None) -> AnyType:
Expand Down Expand Up @@ -1336,14 +1354,26 @@ def visit_typeddict_type(self, t: TypedDictType) -> Type:
analyzed = analyzed.item
items[item_name] = analyzed
if t.fallback.type is MISSING_FALLBACK: # anonymous/inline TypedDict
if INLINE_TYPEDDICT not in self.options.enable_incomplete_feature:
if not t.is_pep764 and INLINE_TYPEDDICT not in self.options.enable_incomplete_feature:
self.fail(
"Inline TypedDict is experimental,"
" must be enabled with --enable-incomplete-feature=InlineTypedDict",
"Legacy inline TypedDict is experimental,"
f" must be enabled with --enable-incomplete-feature={INLINE_TYPEDDICT}",
t,
)
self.note("Did you mean TypedDict[...]?", t)
if t.is_pep764 and NEW_INLINE_TYPEDDICT not in self.options.enable_incomplete_feature:
self.fail(
"PEP 764 inline TypedDict is experimental,"
f" must be enabled with --enable-incomplete-feature={NEW_INLINE_TYPEDDICT}",
t,
)
required_keys = req_keys
fallback = self.named_type("typing._TypedDict")
if t.is_pep764 and t.extra_items_from:
self.fail(
"PEP 764 inline TypedDict does not support merge-in", t, code=codes.VALID_TYPE
)
return AnyType(TypeOfAny.from_error)
for typ in t.extra_items_from:
analyzed = self.analyze_type(typ)
p_analyzed = get_proper_type(analyzed)
Expand All @@ -1363,6 +1393,11 @@ def visit_typeddict_type(self, t: TypedDictType) -> Type:
else:
required_keys = t.required_keys
fallback = t.fallback
if not t.is_pep764 and not t.items:
self.fail(
"Legacy inline TypedDict must have at least one item", t, code=codes.VALID_TYPE
)
return AnyType(TypeOfAny.from_error)
return TypedDictType(items, required_keys, readonly_keys, fallback, t.line, t.column)

def visit_raw_expression_type(self, t: RawExpressionType) -> Type:
Expand Down
3 changes: 3 additions & 0 deletions mypy/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -2575,6 +2575,7 @@ class TypedDictType(ProperType):
"fallback",
"extra_items_from",
"to_be_mutated",
"is_pep764",
)

items: dict[str, Type] # item_name -> item_type
Expand All @@ -2584,6 +2585,7 @@ class TypedDictType(ProperType):

extra_items_from: list[ProperType] # only used during semantic analysis
to_be_mutated: bool # only used in a plugin for `.update`, `|=`, etc
is_pep764: bool

def __init__(
self,
Expand All @@ -2603,6 +2605,7 @@ def __init__(
self.can_be_false = len(self.required_keys) == 0
self.extra_items_from = []
self.to_be_mutated = False
self.is_pep764 = False

def accept(self, visitor: TypeVisitor[T]) -> T:
return visitor.visit_typeddict_type(self)
Expand Down
3 changes: 2 additions & 1 deletion test-data/unit/check-literal.test
Original file line number Diff line number Diff line change
Expand Up @@ -615,7 +615,8 @@ e: Literal[dummy()] # E: Invalid type: Literal[...] cannot contain a
from typing import Literal
a: Literal[{"a": 1, "b": 2}] # E: Parameter 1 of Literal[...] is invalid
b: Literal[{1, 2, 3}] # E: Invalid type: Literal[...] cannot contain arbitrary expressions
c: {"a": 1, "b": 2} # E: Inline TypedDict is experimental, must be enabled with --enable-incomplete-feature=InlineTypedDict \
c: {"a": 1, "b": 2} # E: Legacy inline TypedDict is experimental, must be enabled with --enable-incomplete-feature=InlineTypedDict \
# N: Did you mean TypedDict[...]? \
# E: Invalid type: try using Literal[1] instead? \
# E: Invalid type: try using Literal[2] instead?
d: {1, 2, 3} # E: Invalid type comment or annotation
Expand Down
11 changes: 11 additions & 0 deletions test-data/unit/check-python312.test
Original file line number Diff line number Diff line change
Expand Up @@ -1733,6 +1733,17 @@ type Y[T] = {"item": T, **Y[T]} # E: Overwriting TypedDict field "item" while m
[builtins fixtures/dict.pyi]
[typing fixtures/typing-full.pyi]

[case testTypedDictInlinePEP764YesNewStyleAlias]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
type X[T] = TypedDict[{"item": T, "other": X[T] | None}]
x: X[str]
reveal_type(x) # N: Revealed type is "TypedDict({'item': builtins.str, 'other': Union[..., None]})"
if x["other"] is not None:
reveal_type(x["other"]["item"]) # N: Revealed type is "builtins.str"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-full.pyi]

[case testPEP695UsingIncorrectExpressionsInTypeVariableBound]
type X[T: (yield 1)] = Any # E: Yield expression cannot be used as a type variable bound
type Y[T: (yield from [])] = Any # E: Yield expression cannot be used as a type variable bound
Expand Down
130 changes: 129 additions & 1 deletion test-data/unit/check-typeddict.test
Original file line number Diff line number Diff line change
Expand Up @@ -3659,7 +3659,7 @@ reveal_type(x) # N: # N: Revealed type is "TypedDict({'int': builtins.int, 'st

[case testTypedDictInlineNoEmpty]
# flags: --enable-incomplete-feature=InlineTypedDict
x: {} # E: Invalid type comment or annotation
x: {} # E: Legacy inline TypedDict must have at least one item
reveal_type(x) # N: Revealed type is "Any"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]
Expand Down Expand Up @@ -3707,6 +3707,134 @@ reveal_type(x) # N: Revealed type is "TypedDict({'a': builtins.int, 'b': builti
[builtins fixtures/dict.pyi]
[typing fixtures/typing-full.pyi]

[case testTypedDictInlinePEP764NotEnabled]
from typing import TypedDict
x: TypedDict[{"a": int}] # E: PEP 764 inline TypedDict is experimental, must be enabled with --enable-incomplete-feature=NewInlineTypedDict
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764OldStyleAlias]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
X = TypedDict[{"int": int, "str": str}]
x: X
reveal_type(x) # N: Revealed type is "TypedDict({'int': builtins.int, 'str': builtins.str})"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764MidStyleAlias]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
from typing_extensions import TypeAlias
X: TypeAlias = TypedDict[{"int": int, "str": str}]
x: X
reveal_type(x) # N: Revealed type is "TypedDict({'int': builtins.int, 'str': builtins.str})"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764OYesEmpty]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
x: TypedDict[{}]
reveal_type(x) # N: Revealed type is "TypedDict({})"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]


[case testTypedDictInlinePEP764NotRequired]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import NotRequired, TypedDict

x: TypedDict[{"one": int, "other": NotRequired[int]}]
x = {"one": 1} # OK
y: TypedDict[{"one": int, "other": int}]
y = {"one": 1} # E: Expected TypedDict keys ("one", "other") but found only key "one"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764ReadOnly]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import ReadOnly, TypedDict

x: TypedDict[{"one": int, "other": ReadOnly[int]}]
x["one"] = 1 # ok
x["other"] = 1 # E: ReadOnly TypedDict key "other" TypedDict is mutated
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764NestedSchema]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
def nested() -> TypedDict[{"one": str, "other": TypedDict[{"a": int, "b": int}]}]:
if bool():
return {"one": "yes", "other": {"a": 1, "b": 2}} # OK
else:
return {"one": "no", "other": {"a": 1, "b": "2"}} # E: Incompatible types (expression has type "str", TypedDict item "b" has type "int")
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764MergeAnother]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypeVar, TypedDict
from typing_extensions import TypeAlias

T = TypeVar("T")
X: TypeAlias = TypedDict[{"item": T}]
x: TypedDict[{"a": int, **X[str], "b": int}] # E: PEP 764 inline TypedDict does not support merge-in
reveal_type(x) # N: Revealed type is "Any"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-full.pyi]

[case testTypedDictInlinePEP764BadArg]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
x: TypedDict[int] # E: Argument to TypedDict[] must be a literal dictionary mapping item names to types
reveal_type(x) # N: Revealed type is "Any"
y: TypedDict[{1: str}] # E: Argument to TypedDict[] must be a literal dictionary mapping item names to types
reveal_type(y) # N: Revealed type is "Any"
z: TypedDict[{"a": 1}] # E: Invalid type: try using Literal[1] instead?
reveal_type(z) # N: Revealed type is "TypedDict({'a': Any})"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764TooManyArgs]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
x: TypedDict[{"a": int}, "foo"] # E: TypedDict[] must have exactly one type argument
reveal_type(x) # N: Revealed type is "Any"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764TypeVarValid]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a test with PEP 695 type aliases?

Copy link
Collaborator Author

@brianschubert brianschubert Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's one in check-python312.test - does that cover things well enough?

# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import Generic, TypeVar, TypedDict
T = TypeVar("T")
class X(Generic[T]):
attr: TypedDict[{'name': T}]
def f(arg: T) -> TypedDict[{'name': T}]: ...
Y = TypedDict[{'name': T}]
reveal_type(X[int]().attr['name']) # N: Revealed type is "builtins.int"
reveal_type(f('a')['name']) # N: Revealed type is "builtins.str"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

[case testTypedDictInlinePEP764TypeVarInvalid]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypeVar, TypedDict
T = TypeVar('T')
def f():
X = TypedDict[{'name': T}] # TODO: emit error - this is invalid per PEP-764
[builtins fixtures/dict.pyi]
[typing fixtures/typing-full.pyi]

[case testTypedDictInlinePEP764UsesExtensionSyntax]
# flags: --enable-incomplete-feature=NewInlineTypedDict
from typing import TypedDict
x: TypedDict[{'name': str, 'production': {'location': str}}] # E: Legacy inline TypedDict is experimental, must be enabled with --enable-incomplete-feature=InlineTypedDict \
# N: Did you mean TypedDict[...]?
reveal_type(x) # N: Revealed type is "TypedDict({'name': builtins.str, 'production': TypedDict({'location': builtins.str})})"
[builtins fixtures/dict.pyi]
[typing fixtures/typing-typeddict.pyi]

# ReadOnly
# See: https://peps.python.org/pep-0705
Expand Down