Pydantic v2 migration issue #99218
Comments
|
As also reported on Discord. But replying here as well, as I was tagged. I have no intention of supporting v2 of Pydantic in my libraries. Instead, I might be re-evaluating the use of Pydantic overall. ../Frenck |
|
@frenck if you do find a good alternative please do share it here. Others may want to choose the same option. I have v2 semi working but since all the internals have changed and are IMO more complicated I haven't been interested much in investing time into it |
|
A good reminder: one can retain the V1 standards while upgrading to the V2 library. This is an excellent way to unstick this PR without committing yourself to a complete upgrade (assuming they don't get rid of this path at some point). https://docs.pydantic.dev/latest/migration/#continue-using-pydantic-v1-features |
|
As far as I am aware, it doesn't support everything backward. As said, I have no plans of supporting V2 at this point, and I am not willing to spend any second on it until I have explored other options. ../Frenck |
|
|
|
|
|
I've been checking all the boxes :) |
|
Right, but those don't take part in the migration process. It simply aren't using Pydantic at all anymore. |
|
|
|
|
|
|
|
Man, what a liberating feeling! I should have learned about other options before using Pydantic in the first place. It would have saved me from many headaches now and in the past. |
|
@frenck Are you using |
|
I do not plan on adding support for Pydantic 2. As stated before. Sorry. If anything I will move away from it, as I have done with all other packages. Hopefully, I can find some time to do so soon. |
@jeeftor as far as I can tell not, if your tests run pydantic 2 then it should integrate fine.
@frenck Of course I understand your point, but what about the proposal in my comment, what do you think? |
|
I also want to add, this list has been outdated. I have been keeping an eye out for new integrations using the shims. But I'm sure I missed some |
I think replacing it with just dataclasses isn’t the right approach. |
|
Just to update, here is the updated list of dependencies using the new dependencies that seem problematic:
|
|
One more small update, this time for pykoplenti, the upgrade via shims has been done and the new version is already updated on HA. Confirming just to be sure: $ pip install pykoplenti
Collecting pykoplenti
Downloading pykoplenti-1.3.0-py3-none-any.whl.metadata (5.8 kB)
...
Downloading pydantic-2.9.2-py3-none-any.whl (434 kB)--
|
|
One more update! This time weheat has been made compatible with pydantic via shims. The dependency has already been upgraded on HA. Thanks for the quick responses and the HA upgrade @jesperraemaekers --
|
|
This all seems very close... I just reviewed the list and I believe this may be all that is left?
Anything else missing? |
|
It looks like |
According to this source, Livisi shut down at the end of 2020, but I believe some of their devices still work offline, hence the reason why there is a fork. As I mentioned in my comment above, I would propose for this library that we do a class patch in order for it to work (from my analysis, this should make the code fully compatible): Code proposalfrom aiolivisi.websocket import (
Websocket as WebsocketDeprecated,
EVENT_STATE_CHANGED,
ON_STATE,
VALUE,
LivisiEvent,
SET_POINT_TEMPERATURE,
POINT_TEMPERATURE,
TEMPERATURE,
HUMIDITY,
LUMINANCE,
IS_REACHABLE,
IS_OPEN,
EVENT_BUTTON_PRESSED,
KEY_INDEX,
KEY_PRESS_LONG,
KEY_PRESS_TYPE,
)
from pydantic import ValidationError
from typing import Callable
class Websocket(WebsocketDeprecated):
async def consumer_handler(self, websocket, on_data: Callable):
"""Used when data is transmited using the websocket."""
async for message in websocket:
try:
event_data = LivisiEvent.model_validate(message)
except ValidationError:
continue
if "device" in event_data.source:
event_data.source = event_data.source.replace("/device/", "")
if event_data.properties is None:
continue
if event_data.type == EVENT_STATE_CHANGED:
if ON_STATE in event_data.properties.keys():
event_data.onState = event_data.properties.get(ON_STATE)
elif VALUE in event_data.properties.keys() and isinstance(event_data.properties.get(VALUE), bool):
event_data.onState = event_data.properties.get(VALUE)
if SET_POINT_TEMPERATURE in event_data.properties.keys():
event_data.vrccData = event_data.properties.get(SET_POINT_TEMPERATURE)
elif POINT_TEMPERATURE in event_data.properties.keys():
event_data.vrccData = event_data.properties.get(POINT_TEMPERATURE)
elif TEMPERATURE in event_data.properties.keys():
event_data.vrccData = event_data.properties.get(TEMPERATURE)
elif HUMIDITY in event_data.properties.keys():
event_data.vrccData = event_data.properties.get(HUMIDITY)
if LUMINANCE in event_data.properties.keys():
event_data.luminance = event_data.properties.get(LUMINANCE)
if IS_REACHABLE in event_data.properties.keys():
event_data.isReachable = event_data.properties.get(IS_REACHABLE)
if IS_OPEN in event_data.properties.keys():
event_data.isOpen = event_data.properties.get(IS_OPEN)
elif event_data.type == EVENT_BUTTON_PRESSED:
if KEY_INDEX in event_data.properties.keys():
event_data.keyIndex = event_data.properties.get(KEY_INDEX)
event_data.isLongKeyPress = KEY_PRESS_LONG == event_data.properties.get(KEY_PRESS_TYPE)
on_data(event_data)
As of now just the demetrik update. @frenck if you have some I would really appreciate if you could review all the PRs I've placed. I've split all unrelated changes into separate PRs described in frenck/python-demetriek#740 (comment). If those get merged then all the workflows and tests should pass on CI for the mashumaro upgrade. |
It would be a lot cleaner to fork the library, modify it and republish it under a new name |
The integration apparently is local, and HA is now one of the ways to keep controlling it. So I don't really feel like dropping it, also seeing that there are 237 active installations |
|
I think |
|
Picking it up this weekend. Planned it last weekend, but unfortunately have experienced an NVMe going belly up on my dev machine. ../Frenck |
|
Nice, thanks to everyone who put work into this migration! 🎉 |
and others
The problem
Pydantic released v2 of their library. Some libraries want to utilize the new features of Pydantic v2 so HA should update to Pydantic v2. To not burden all libraries to rework, Pydantic is providing v1 in a separate package of v2. This has been implemented in several libraries already made by some HA community members (https://github.com/AngellusMortis/pyunifiprotect/pull/297). This way the library supports both v1 and v2, this way of working is preferred to not burden the v2 upgrade with a lot of libraries to update at once.
After installing all dependencies and using
pipdeptreewe see that the current libraries use pydantic.This issue will be the progress tracker of this migration.
❌ aiolivisi
Library: [aiolivisi](https://github.com/StefanIacobLivisi/aiolivisi) Integrations using this library: [Livisi](https://github.com/home-assistant/core/tree/dev/homeassistant/components/livisi) Code owners: @StefanIacobLivisi @planbnetRemarks:
✔️ aionotion
Library: aionotion
Integrations using this library: Notion
Code owners: @bachya
Remarks:
There seems to be no version that supports both v1 and v2, so this library should be directly bumped with the v2 update.
✔️ aioopenexchangerates
--
Library: aioopenexchangerates
Integrations using this library: OpenExchangeRates
Code owners: @MartinHjelmare
Remarks:
❌ aiopurpleair
--
Library: aiopurpleair
Integrations using this library: PurpleAir
Code owners: @bachya
Remarks:
There seems to be no version that supports both v1 and v2, so this library should be directly bumped with the v2 update.
✔️ aiowaqi
--
Library: aiowaqi
Integrations using this library: WAQI
Code owners: @joostlek
Remarks:
❌ demetriek
--
Library: demetriek
Integrations using this library: lametric
Code owners: @robbiet480 @frenck @bachya
Remarks:
✔️ elgato
--
Library: elgato
Integrations using this library: elgato
Code owners: @frenck
Remarks:
✔️ gcal-sync
--
Library: gcal-sync
Integrations using this library: Google Calendar
Code owners: @allenporter
Remarks:
Supports shims. Needs a release so we can bump the dependency beforehand. Latest version also has ical 5.0.1, and that has pydantic v2 support since 5.0.0.
✔️ google-nest-sdm
--
Library: google-nest-sdm
Integrations using this library: Nest
Code owners: @allenporter
Remarks:
✔️ ical
--
Library: ical
Integrations using this library: Local calendar
Code owners: @allenporter
Remarks:
❌ inflect
--
Library: inflect
Integrations using this library: Abode
Code owners: @shred86
Remarks:
This seems to be a deeper problem. I need to dive in to this to know the status.
❌ intellifire4py
--
Library: intellifire4py
Integrations using this library: IntelliFire
Code owners: @jeeftor
Remarks:
❌ lacrosse-view
--
Library: lacrosse-view
Integrations using this library: LaCrosse View
Code owners: @IceBotYT
Remarks:
✔️ open-meteo
--
Library: open-meteo
Integrations using this library: Open Meteo
Code owners: @frenck
Remarks:
To verify: There has been no code change to support it, but in the lock file is 2.0.3 so I am expecting this works.
If this is as expected, it still needs a release and to be bumped.
❌ peco
--
Library: peco
Integrations using this library: PECO Outage Counter
Code owners: @IceBotYT
Remarks:
There has been no explicit PR to fix support for v2. This might work but needs to be verified.
✔️ pvo
--
Library: pvo
Integrations using this library: PVOutput
Code owners: @frenck
Remarks:
❌ pyaussiebb
--
Library: pyaussiebb
Integrations using this library: Aussie Broadband
Code owners: @nickw444 @Bre77
Remarks:
Library does not support v1 shims. Should be bumped with the v2 bump.
cc @yaleman
✔️ pyrainbird
--
Library: pyrainbird
Integrations using this library: Rain Bird
Code owners: @konikvranik @allenporter
Remarks:
Still needs to update ical to =>5.0.0 as ical relies on pydantic as well.
✔️ python-bsblan
--
Library: python-bsblan
Integrations using this library: BSB-Lan
Code owners: @liudger
Remarks:
✔️ python-kasa
--
Library: python-kasa
Integrations using this library: TP-Link Kasa Smart
Code owners: @rytilahti @TheGardenMonkey
Remarks:
✔️ python-opensky
--
Library: python-opensky
Integrations using this library: OpenSky Network
Code owners: @joostlek
Remarks:
✔️ pytraccar
--
Library: pytraccar
Integrations using this library: Traccar
Code owners: @ludeeus
Remarks:
✔️ pyunifiprotect
--
Library: pyunifiprotect
Integrations using this library: asd
Code owners: @AngellusMortis @bdraco
Remarks:
✔️ radios
--
Library: radios
Integrations using this library: Radio Browser
Code owners: @frenck
Remarks:
✔️ sfrbox-api
--
Library: sfrbox-api
Integrations using this library: SFR Box
Code owners: @epenet
Remarks:
❌ systembridgeconnector
--
Library: systembridgeconnector
Integrations using this library: System Bridge
Code owners: @timmo001
Remarks:
✔️ tailscale
--
Library: tailscale
Integrations using this library: asd
Code owners: @frenck
Remarks:
To verify: There has been no code change to support it, but in the lock file is 2.0.3 so I am expecting this works.
If this is as expected, it still needs a release and to be bumped.
✔️ vehicle
--
Library: vehicle
Integrations using this library: RDW
Code owners: @frenck @joostlek
Remarks:
✔️ withings-api
--
Library: withings-api
Integrations using this library: Withings
Code owners: @vangorra
Remarks:
Is now/will be replaced with a new lib without pydantic
❌ xbox-webapi
--
Library: xbox-webapi
Integrations using this library: Xbox
Code owners: @hunterjm
Remarks:
✔️ yolink-api
--
Library: yolink-api
Integrations using this library: YoLink
Code owners: @matrixd2
Remarks:
✔️ youtubeaio
--
Library: youtubeaio
Integrations using this library: YouTube
Code owners: @joostlek
Remarks:
✔️ zwave-js-server-python
--
Library: zwave-js-server-python
Integrations using this library: Z-Wave
Code owners: @home-assistant/z-wave
Remarks:
--
For the core members, feel free to update where needed. For others, let us know if something has updated in a comment and we will update it.
cc @cdce8p
The text was updated successfully, but these errors were encountered: