Skip to content

Commit b6f30e6

Browse files
Future-OutlierFuture-Outlier
authored
[Flyte Decks] support ydata-profiling in python 3.12 (flyteorg#2766)
* [Flyte Decks] support ydata-profiling in python 3.12 Signed-off-by: Future-Outlier <[email protected]> * remove exclude deck standard python3.12 ci Signed-off-by: Future-Outlier <[email protected]> * make plugin soft dependencies Signed-off-by: Future-Outlier <[email protected]> * add dev-requirements.in Signed-off-by: Future-Outlier <[email protected]> * nit Signed-off-by: Future-Outlier <[email protected]> * better README with dependenc Signed-off-by: Future-Outlier <[email protected]> * add other dependency in dev-requirements.in, this will help setup-global-uv Signed-off-by: Future-Outlier <[email protected]> * Trigger CI Signed-off-by: Future-Outlier <[email protected]> * Trigger CI Signed-off-by: Future-Outlier <[email protected]> * Update dependenct Signed-off-by: Future-Outlier <[email protected]> * new dockerfile dev Signed-off-by: Future-Outlier <[email protected]> * new dockerfile Signed-off-by: Future-Outlier <[email protected]> * new dockerfile Signed-off-by: Future-Outlier <[email protected]> * revert back Signed-off-by: Future-Outlier <[email protected]> * new dev image Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]>
1 parent 9a9f5b2 commit b6f30e6

File tree

9 files changed

+191
-31
lines changed

9 files changed

+191
-31
lines changed

‎.github/workflows/pythonbuild.yml

-3
Original file line numberDiff line numberDiff line change
@@ -396,9 +396,6 @@ jobs:
396396
# apache-beam, one of flytekit-airflow dependencies, does not support python 3.12: https://github.com/apache/beam/issues/29149
397397
- python-version: 3.12
398398
plugin-names: "flytekit-airflow"
399-
# ydata-profiling, a dependency of flytekit-deck-standard, does not support python 3.12: https://github.com/ydataai/ydata-profiling/issues/1510
400-
- python-version: 3.12
401-
plugin-names: "flytekit-deck-standard"
402399
# Tensorflow is a dependency of flytekit-mlflow tests and that is not supported yet: https://github.com/tensorflow/tensorflow/issues/62003
403400
- python-version: 3.12
404401
plugin-names: "flytekit-mlflow"

‎Dockerfile.dev

+6
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,13 @@ RUN SETUPTOOLS_SCM_PRETEND_VERSION_FOR_FLYTEKIT=$PSEUDO_VERSION \
4040
-e /flytekit \
4141
-e /flytekit/plugins/flytekit-deck-standard \
4242
-e /flytekit/plugins/flytekit-flyteinteractive \
43+
markdown \
44+
pandas \
45+
pillow \
46+
plotly \
47+
pygments \
4348
scikit-learn \
49+
ydata-profiling \
4450
&& apt-get clean autoclean \
4551
&& apt-get autoremove --yes \
4652
&& rm -rf /var/lib/{apt,dpkg,cache,log}/ \

‎dev-requirements.in

+4
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,15 @@ types-decorator
4848
types-mock
4949
autoflake
5050

51+
markdown
5152
pillow
5253
numpy
5354
pandas
55+
plotly
5456
pyarrow
57+
pygments
5558
scikit-learn
59+
ydata-profiling
5660
types-requests
5761
prometheus-client
5862

‎dev-requirements.txt

+114-10
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ aioitertools==0.11.0
2020
# via aiobotocore
2121
aiosignal==1.3.1
2222
# via aiohttp
23+
annotated-types==0.7.0
24+
# via pydantic
2325
asn1crypto==1.5.1
2426
# via snowflake-connector-python
2527
asttokens==2.4.1
@@ -29,6 +31,7 @@ attrs==23.2.0
2931
# aiohttp
3032
# hypothesis
3133
# jsonlines
34+
# visions
3235
autoflake==2.3.1
3336
# via -r dev-requirements.in
3437
azure-core==1.30.1
@@ -70,6 +73,8 @@ cloudpickle==3.0.0
7073
# via flytekit
7174
codespell==2.3.0
7275
# via -r dev-requirements.in
76+
contourpy==1.3.0
77+
# via matplotlib
7378
coverage[toml]==7.5.3
7479
# via
7580
# -r dev-requirements.in
@@ -83,8 +88,11 @@ cryptography==43.0.1
8388
# msal
8489
# pyjwt
8590
# pyopenssl
86-
# secretstorage
8791
# snowflake-connector-python
92+
cycler==0.12.1
93+
# via matplotlib
94+
dacite==1.8.1
95+
# via ydata-profiling
8896
dataclasses-json==0.5.9
8997
# via flytekit
9098
decorator==5.1.1
@@ -111,6 +119,8 @@ flyteidl @ git+https://github.com/flyteorg/flyte.git@master#subdirectory=flyteid
111119
# via
112120
# -r dev-requirements.in
113121
# flytekit
122+
fonttools==4.54.1
123+
# via matplotlib
114124
frozenlist==1.4.1
115125
# via
116126
# aiohttp
@@ -175,6 +185,8 @@ grpcio-status==1.62.2
175185
# via
176186
# flytekit
177187
# google-api-core
188+
htmlmin==0.1.12
189+
# via ydata-profiling
178190
hypothesis==6.103.0
179191
# via -r dev-requirements.in
180192
icdiff==2.0.7
@@ -186,6 +198,10 @@ idna==3.7
186198
# requests
187199
# snowflake-connector-python
188200
# yarl
201+
imagehash==4.3.1
202+
# via
203+
# visions
204+
# ydata-profiling
189205
importlib-metadata==7.1.0
190206
# via flytekit
191207
iniconfig==2.0.0
@@ -206,16 +222,15 @@ jaraco-functools==4.0.1
206222
# via keyring
207223
jedi==0.19.1
208224
# via ipython
209-
jeepney==0.8.0
210-
# via
211-
# keyring
212-
# secretstorage
225+
jinja2==3.1.4
226+
# via ydata-profiling
213227
jmespath==1.0.1
214228
# via botocore
215229
joblib==1.4.2
216230
# via
217231
# -r dev-requirements.in
218232
# flytekit
233+
# phik
219234
# scikit-learn
220235
jsonlines==4.0.0
221236
# via flytekit
@@ -225,12 +240,20 @@ keyring==25.2.1
225240
# via flytekit
226241
keyrings-alt==5.0.1
227242
# via -r dev-requirements.in
243+
kiwisolver==1.4.7
244+
# via matplotlib
228245
kubernetes==29.0.0
229246
# via -r dev-requirements.in
247+
llvmlite==0.43.0
248+
# via numba
249+
markdown==3.7
250+
# via -r dev-requirements.in
230251
markdown-it-py==3.0.0
231252
# via
232253
# flytekit
233254
# rich
255+
markupsafe==2.1.5
256+
# via jinja2
234257
marshmallow==3.21.2
235258
# via
236259
# dataclasses-json
@@ -244,6 +267,12 @@ marshmallow-jsonschema==0.13.0
244267
# via flytekit
245268
mashumaro==3.13
246269
# via flytekit
270+
matplotlib==3.9.2
271+
# via
272+
# phik
273+
# seaborn
274+
# wordcloud
275+
# ydata-profiling
247276
matplotlib-inline==0.1.7
248277
# via ipython
249278
mdurl==0.1.2
@@ -265,21 +294,41 @@ multidict==6.0.5
265294
# via
266295
# aiohttp
267296
# yarl
297+
multimethod==1.12
298+
# via
299+
# visions
300+
# ydata-profiling
268301
mypy==1.6.1
269302
# via -r dev-requirements.in
270303
mypy-extensions==1.0.0
271304
# via
272305
# mypy
273306
# typing-inspect
307+
networkx==3.3
308+
# via visions
274309
nodeenv==1.9.0
275310
# via pre-commit
311+
numba==0.60.0
312+
# via ydata-profiling
276313
numpy==1.26.4
277314
# via
278315
# -r dev-requirements.in
316+
# contourpy
317+
# imagehash
318+
# matplotlib
319+
# numba
279320
# pandas
321+
# patsy
322+
# phik
280323
# pyarrow
324+
# pywavelets
281325
# scikit-learn
282326
# scipy
327+
# seaborn
328+
# statsmodels
329+
# visions
330+
# wordcloud
331+
# ydata-profiling
283332
oauthlib==3.2.2
284333
# via
285334
# kubernetes
@@ -291,22 +340,42 @@ packaging==24.0
291340
# docker
292341
# google-cloud-bigquery
293342
# marshmallow
343+
# matplotlib
294344
# msal-extensions
345+
# plotly
295346
# pytest
296347
# setuptools-scm
297348
# snowflake-connector-python
349+
# statsmodels
298350
pandas==2.2.2
299-
# via -r dev-requirements.in
351+
# via
352+
# -r dev-requirements.in
353+
# phik
354+
# seaborn
355+
# statsmodels
356+
# visions
357+
# ydata-profiling
300358
parso==0.8.4
301359
# via jedi
360+
patsy==0.5.6
361+
# via statsmodels
302362
pexpect==4.9.0
303363
# via ipython
364+
phik==0.12.4
365+
# via ydata-profiling
304366
pillow==10.3.0
305-
# via -r dev-requirements.in
367+
# via
368+
# -r dev-requirements.in
369+
# imagehash
370+
# matplotlib
371+
# visions
372+
# wordcloud
306373
platformdirs==4.2.2
307374
# via
308375
# snowflake-connector-python
309376
# virtualenv
377+
plotly==5.24.1
378+
# via -r dev-requirements.in
310379
pluggy==1.5.0
311380
# via pytest
312381
portalocker==2.8.2
@@ -350,10 +419,15 @@ pyasn1-modules==0.4.0
350419
# via google-auth
351420
pycparser==2.22
352421
# via cffi
422+
pydantic==2.9.2
423+
# via ydata-profiling
424+
pydantic-core==2.23.4
425+
# via pydantic
353426
pyflakes==3.2.0
354427
# via autoflake
355428
pygments==2.18.0
356429
# via
430+
# -r dev-requirements.in
357431
# flytekit
358432
# ipython
359433
# rich
@@ -363,6 +437,8 @@ pyjwt[crypto]==2.8.0
363437
# snowflake-connector-python
364438
pyopenssl==24.2.1
365439
# via snowflake-connector-python
440+
pyparsing==3.1.4
441+
# via matplotlib
366442
pytest==8.2.1
367443
# via
368444
# -r dev-requirements.in
@@ -390,6 +466,7 @@ python-dateutil==2.9.0.post0
390466
# croniter
391467
# google-cloud-bigquery
392468
# kubernetes
469+
# matplotlib
393470
# pandas
394471
python-json-logger==2.0.7
395472
# via flytekit
@@ -402,11 +479,14 @@ pytz==2024.1
402479
# croniter
403480
# pandas
404481
# snowflake-connector-python
482+
pywavelets==1.7.0
483+
# via imagehash
405484
pyyaml==6.0.1
406485
# via
407486
# flytekit
408487
# kubernetes
409488
# pre-commit
489+
# ydata-profiling
410490
requests==2.32.3
411491
# via
412492
# azure-core
@@ -421,6 +501,7 @@ requests==2.32.3
421501
# msal
422502
# requests-oauthlib
423503
# snowflake-connector-python
504+
# ydata-profiling
424505
requests-oauthlib==2.0.0
425506
# via
426507
# google-auth-oauthlib
@@ -438,9 +519,14 @@ s3fs==2024.5.0
438519
scikit-learn==1.5.0
439520
# via -r dev-requirements.in
440521
scipy==1.13.1
441-
# via scikit-learn
442-
secretstorage==3.3.3
443-
# via keyring
522+
# via
523+
# imagehash
524+
# phik
525+
# scikit-learn
526+
# statsmodels
527+
# ydata-profiling
528+
seaborn==0.13.2
529+
# via ydata-profiling
444530
setuptools-scm==8.1.0
445531
# via -r dev-requirements.in
446532
six==1.16.0
@@ -449,6 +535,7 @@ six==1.16.0
449535
# azure-core
450536
# isodate
451537
# kubernetes
538+
# patsy
452539
# python-dateutil
453540
snowflake-connector-python==3.12.1
454541
# via -r dev-requirements.in
@@ -460,14 +547,22 @@ stack-data==0.6.3
460547
# via ipython
461548
statsd==3.3.0
462549
# via flytekit
550+
statsmodels==0.14.3
551+
# via ydata-profiling
552+
tenacity==9.0.0
553+
# via plotly
463554
threadpoolctl==3.5.0
464555
# via scikit-learn
465556
tomlkit==0.13.2
466557
# via snowflake-connector-python
558+
tqdm==4.66.5
559+
# via ydata-profiling
467560
traitlets==5.14.3
468561
# via
469562
# ipython
470563
# matplotlib-inline
564+
typeguard==4.3.0
565+
# via ydata-profiling
471566
types-croniter==2.0.0.20240423
472567
# via -r dev-requirements.in
473568
types-decorator==5.1.8.20240310
@@ -485,8 +580,11 @@ typing-extensions==4.12.0
485580
# flytekit
486581
# mashumaro
487582
# mypy
583+
# pydantic
584+
# pydantic-core
488585
# rich-click
489586
# snowflake-connector-python
587+
# typeguard
490588
# typing-inspect
491589
typing-inspect==0.9.0
492590
# via dataclasses-json
@@ -502,16 +600,22 @@ urllib3==2.2.1
502600
# types-requests
503601
virtualenv==20.26.2
504602
# via pre-commit
603+
visions[type-image-path]==0.7.6
604+
# via ydata-profiling
505605
wcwidth==0.2.13
506606
# via prompt-toolkit
507607
websocket-client==1.8.0
508608
# via
509609
# docker
510610
# kubernetes
611+
wordcloud==1.9.3
612+
# via ydata-profiling
511613
wrapt==1.16.0
512614
# via aiobotocore
513615
yarl==1.9.4
514616
# via aiohttp
617+
ydata-profiling==4.10.0
618+
# via -r dev-requirements.in
515619
zipp==3.19.1
516620
# via importlib-metadata
517621

0 commit comments

Comments
 (0)