Variable types not enforced/autosensed properly

[karolyi]

Hello,

not sure I'm at the right place with this, didn't want to pollute the issues: I've recently bumped into a problem that I don't seem to be able to correct the way I want to. I've put together a minimal rudimentary example with the error, perhaps you could advise as to what I'm thinking/doing wrong:

Code sample in pyright playground

from ipaddress import IPv4Network, IPv6Network, ip_network
from typing import NamedTuple


class _WhitelistType(NamedTuple):
    v4: dict[IPv4Network, bool]
    v6: dict[IPv6Network, bool]


net_str = ['127.0.0.0/24', '192.168.0.0/16']
nets = [ip_network(address=x) for x in net_str]
wl = _WhitelistType(
    v4={IPv4Network(address='127.0.0.0/24'): True},
    v6={IPv6Network(address='127.0.0./32'): True},
)


def test(net: IPv4Network | IPv6Network):
    searched_wl = wl.v4 if isinstance(net, IPv4Network) else wl.v6
    for ipnet_item in searched_wl:
        if net.subnet_of(other=ipnet_item):
            print('Yes')

The problem is with the second to last line (see the linked example). Pyright complains,

Argument of type "IPv4Network | IPv6Network" cannot be assigned to parameter "other" of type "IPv4Network" in function "subnet_of"
  Type "IPv4Network | IPv6Network" is not assignable to type "IPv4Network"
    "IPv6Network" is not assignable to "IPv4Network"  (reportArgumentType)
Argument of type "IPv4Network | IPv6Network" cannot be assigned to parameter "other" of type "IPv6Network" in function "subnet_of"
  Type "IPv4Network | IPv6Network" is not assignable to type "IPv6Network"
    "IPv4Network" is not assignable to "IPv6Network"  (reportArgumentType)

I would think that the variable type of ipnet_item should be detected properly.

[erictraut]

Static type checkers like pyright apply a technique called "type narrowing". You can read more about this here.

In your example above, the type of the expression net cannot be narrowed by the isinstance check; prior to the check, its type is IPv4Network | IPv6Network and after the check, its type remains the same. The type of ipnet_item is also a union and cannot be narrowed further by the in operator.

It is theoretically possible to narrow types conditionally — for example, "the type of searched_wl is IPv4Network if the type of net is also IPv4Network but it IPv6Network otherwise. However, tracking such conditional types is not computationally feasible in practice. I don't know of any static type checker or compiler in any language that does this. Even if this were computationally feasible, it would be very confusing for users; describing such conditional types in hover text or error messages would be very confusing.

You'll see that if you duplicate your code, the type error disappears. In this version, type narrowing does apply. The type of net is narrowed separately in each of the two branches of the if statement.

def test(net: IPv4Network | IPv6Network):
    if isinstance(net, IPv4Network):
        searched_wl = wl.v4
        for ipnet_item in searched_wl:
            if net.subnet_of(other=ipnet_item):
                print("Yes")
    else:
        searched_wl = wl.v6
        for ipnet_item in searched_wl:
            if net.subnet_of(other=ipnet_item):
                print("Yes")

Duplicating code like this is not something I'd recommend, so you may want to consider different ways of accomplishing your goal through restructuring. If you don't want to change your code, you could suppress the type error using a # pyright: ignore comment.

[karolyi]

Hey,

thanks for the swift response! I've actually arrived to my pasted code after having it deduplicated. It looked the same like you've expanded now. The error came after the deduplication, since I noticed it's the same logic, only with different variable types.

I've refactored it already to have only one type of the variable passed from the outside into the function, and the function signature uses a

_NetType = TypeVar('_NetType', IPv4Network, IPv6Network)

variable.

This way it works, however it doesn't look elegant to my eyes. The full code is at https://git.ksol.io/karolyi/abuseipdb-fetcher/src/branch/master/src/abuseipdb_fetcher/iplist_merge.py.

One would think that the type narrowing would work as it's only logical to have those variables narrowed to their respective types. I've linked the code for more context, since I forgot the iteration in the example, that calls the test() function. That cycle just iterates trough of the elements of a nets-like variable, from which it would be self-explanatory that the function should only get similar inputs with regard to variable types.

Thanks for the help again!