Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ciso8601 as an optional dependency #447

Closed
keul opened this issue Aug 4, 2022 · 12 comments · Fixed by #448
Closed

ciso8601 as an optional dependency #447

keul opened this issue Aug 4, 2022 · 12 comments · Fixed by #448

Comments

@keul
Copy link
Contributor

keul commented Aug 4, 2022

We have currently an issue with stac_fastapi 2.4 in Python 3.10 conda environment due to the recent introduction on ciso8601 as a dependency: see #368

What about having this dependency optional? ciso8601 is a C dependency, so a bit trickier to be installed and it's only an optimisation.
@vincentsarago
Copy link
Member

ref #204 (comment)

@geospatial-jeff
Copy link
Collaborator

What type of issue are you seeing? Including traceback would be helpful.

@geospatial-jeff
Copy link
Collaborator

I'm unable to replicate this on my machine (using miniconda):

conda create -n py310 python=3.10
conda activate py310
pip install stac-fastapi.sqlalchemy

Output:

Collecting stac-fastapi.sqlalchemy
  Downloading stac_fastapi.sqlalchemy-2.4.0-py3-none-any.whl (34 kB)
Collecting stac-fastapi.extensions
  Downloading stac_fastapi.extensions-2.4.0-py3-none-any.whl (16 kB)
Collecting attrs
  Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting geoalchemy2<0.8.0
  Downloading GeoAlchemy2-0.7.0-py2.py3-none-any.whl (21 kB)
Collecting stac-fastapi.api
  Downloading stac_fastapi.api-2.4.0-py3-none-any.whl (12 kB)
Collecting stac-pydantic>=2.0.3
  Using cached stac_pydantic-2.0.3-py3-none-any.whl (17 kB)
Collecting sqlalchemy==1.3.23
  Downloading SQLAlchemy-1.3.23.tar.gz (6.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 3.8 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting alembic
  Downloading alembic-1.8.1-py3-none-any.whl (209 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 kB 3.2 MB/s eta 0:00:00
Collecting shapely
  Using cached Shapely-1.8.2-cp310-cp310-win_amd64.whl (1.3 MB)
Collecting pydantic[dotenv]
  Using cached pydantic-1.9.1-cp310-cp310-win_amd64.whl (2.0 MB)
Collecting psycopg2-binary
  Downloading psycopg2_binary-2.9.3-cp310-cp310-win_amd64.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 3.4 MB/s eta 0:00:00
Collecting stac-fastapi.types
  Downloading stac_fastapi.types-2.4.0-py3-none-any.whl (13 kB)
Collecting sqlakeyset
  Downloading sqlakeyset-1.0.1659142803-py3-none-any.whl (20 kB)
Collecting fastapi-utils
  Downloading fastapi_utils-0.2.1-py3-none-any.whl (16 kB)
Collecting click
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting geojson-pydantic
  Using cached geojson_pydantic-0.4.3-py3-none-any.whl (7.9 kB)
Collecting Mako
  Downloading Mako-1.2.1-py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.6/78.6 kB 4.3 MB/s eta 0:00:00
Collecting fastapi
  Downloading fastapi-0.79.0-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.6/54.6 kB ? eta 0:00:00
Collecting typing-extensions>=3.7.4.3
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Collecting python-dotenv>=0.10.4
  Using cached python_dotenv-0.20.0-py3-none-any.whl (17 kB)
Collecting packaging>=20.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting brotli-asgi
  Using cached brotli_asgi-1.1.0-py3-none-any.whl (4.3 kB)
Collecting ciso8601~=2.2.0
  Using cached ciso8601-2.2.0.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pystac==1.*
  Downloading pystac-1.6.1-py3-none-any.whl (146 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 146.0/146.0 kB 2.9 MB/s eta 0:00:00
Collecting starlette==0.19.1
  Downloading starlette-0.19.1-py3-none-any.whl (63 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.3/63.3 kB 3.3 MB/s eta 0:00:00
Collecting anyio<5,>=3.4.0
  Using cached anyio-3.6.1-py3-none-any.whl (80 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting brotli>=1.0.7
  Downloading Brotli-1.0.9-cp310-cp310-win_amd64.whl (383 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 383.3/383.3 kB 3.4 MB/s eta 0:00:00
Collecting colorama
  Using cached colorama-0.4.5-py2.py3-none-any.whl (16 kB)
Collecting MarkupSafe>=0.9.2
  Using cached MarkupSafe-2.1.1-cp310-cp310-win_amd64.whl (17 kB)
Collecting idna>=2.8
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sniffio>=1.1
  Using cached sniffio-1.2.0-py3-none-any.whl (10 kB)
Building wheels for collected packages: sqlalchemy, ciso8601
  Building wheel for sqlalchemy (setup.py) ... done
  Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.23-cp310-cp310-win_amd64.whl size=1221724 sha256=6229034aba102b3725b9bb3cbf4cfd65e2aa74912a54f75661d6ac121cb18232
  Stored in directory: c:\users\jeffa\appdata\local\pip\cache\wheels\6b\bd\5f\b63dc9976fc54a34ac2019fdbef4eb6a15aef4958298ce4b55
  Building wheel for ciso8601 (pyproject.toml) ... done
  Created wheel for ciso8601: filename=ciso8601-2.2.0-cp310-cp310-win_amd64.whl size=14986 sha256=77476234792bda11e7514618bec4ca23824f131f41824fa6771489fcbbff5fba
  Stored in directory: c:\users\jeffa\appdata\local\pip\cache\wheels\02\1f\3b\a3fc12fc79c99e80e030beb98f07c92cbf1e43e00a7abccfcb
Successfully built sqlalchemy ciso8601
Installing collected packages: ciso8601, brotli, typing-extensions, sqlalchemy, sniffio, six, shapely, python-dotenv, pyparsing, psycopg2-binary, MarkupSafe, idna, colorama, attrs, python-dateutil, pydantic, packaging, Mako, geoalchemy2, click, anyio, starlette, sqlakeyset, pystac, geojson-pydantic, alembic, stac-pydantic, fastapi, brotli-asgi, stac-fastapi.types, fastapi-utils, stac-fastapi.api, stac-fastapi.extensions, stac-fastapi.sqlalchemy
Successfully installed Mako-1.2.1 MarkupSafe-2.1.1 alembic-1.8.1 anyio-3.6.1 attrs-22.1.0 brotli-1.0.9 brotli-asgi-1.1.0 ciso8601-2.2.0 click-8.1.3 colorama-0.4.5 fastapi-0.79.0 fastapi-utils-0.2.1 geoalchemy2-0.7.0 geojson-pydantic-0.4.3 idna-3.3 packaging-21.3 psycopg2-binary-2.9.3 pydantic-1.9.1 pyparsing-3.0.9 pystac-1.6.1 python-dateutil-2.8.2 python-dotenv-0.20.0 shapely-1.8.2 six-1.16.0 sniffio-1.2.0 sqlakeyset-1.0.1659142803 sqlalchemy-1.3.23 stac-fastapi.api-2.4.0 stac-fastapi.extensions-2.4.0 stac-fastapi.sqlalchemy-2.4.0 stac-fastapi.types-2.4.0 stac-pydantic-2.0.3 starlette-0.19.1 typing-extensions-4.3.0

@keul
Copy link
Contributor Author

keul commented Aug 4, 2022

I missed another information: everything is inside a docker image based on continuumio/miniconda3

@geospatial-jeff

#13 98.81 Installing pip dependencies: ...working... Pip subprocess error:
#13 108.2   error: subprocess-exited-with-error
#13 108.2
#13 108.2   × Building wheel for ciso8601 (pyproject.toml) did not run successfully.
#13 108.2   │ exit code: 1
#13 108.2   ╰─> [13 lines of output]
#13 108.2       running bdist_wheel
#13 108.2       running build
#13 108.2       running build_py
#13 108.2       creating build
#13 108.2       creating build/lib.linux-x86_64-cpython-310
#13 108.2       creating build/lib.linux-x86_64-cpython-310/ciso8601
#13 108.2       copying ciso8601/__init__.pyi -> build/lib.linux-x86_64-cpython-310/ciso8601
#13 108.2       copying ciso8601/py.typed -> build/lib.linux-x86_64-cpython-310/ciso8601
#13 108.2       running build_ext
#13 108.2       building 'ciso8601' extension
#13 108.2       creating build/temp.linux-x86_64-cpython-310
#13 108.2       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -DCISO8601_VERSION=2.2.0 -DCISO8601_CACHING_ENABLED=1 -I/opt/conda/include/python3.10 -c module.c -o build/temp.linux-x86_64-cpython-310/module.o
#13 108.2       error: command 'gcc' failed: No such file or directory
#13 108.2       [end of output]
#13 108.2
#13 108.2   note: This error originates from a subprocess, and is likely not a problem with pip.
#13 108.2   ERROR: Failed building wheel for ciso8601
#13 108.2 ERROR: Could not build wheels for ciso8601, which is required to install pyproject.toml-based projects
#13 108.2

@vincentsarago
Copy link
Member

error: command 'gcc' failed: No such file or directory

install gcc on your image 🤷‍♂️

@geospatial-jeff
Copy link
Collaborator

geospatial-jeff commented Aug 4, 2022 •
edited
Loading

It looks like that image doesn't have gcc so its unable to build the wheel, can you try installing build-essential?

sudo apt-get install build-essential

https://github.com/stac-utils/stac-fastapi/blob/master/Dockerfile#L5-L9

@duckontheweb
Copy link
Contributor

I still get an error after installing build-essential. This seems like a known issue with building ciso8601 in some environments (see closeio/ciso8601#123(comment).

I'll see if I can figure out a workaround for this.

@geospatial-jeff
Copy link
Collaborator

geospatial-jeff commented Aug 4, 2022 •
edited
Loading

Looks like python 3.10 support is added in ciso8601==2.2.1 which was PRd but never merged closeio/ciso8601#115. This release also adds wheels that @kylebarron made to make it easier to install (closeio/ciso8601#109).

@geospatial-jeff
Copy link
Collaborator

geospatial-jeff commented Aug 4, 2022 •
edited
Loading

So at this point I'd like to wait to see if the new version of ciso8601 is released. We have the option of switching to pyiso8601 but its not a complete RFC3339 parser (while ciso8601 is).

https://github.com/radiantearth/stac-api-spec/blob/main/implementation.md#datetime-parameter-handling

@duckontheweb
Copy link
Contributor

I still get an error after installing build-essential. This seems like a known issue with building ciso8601 in some environments (see closeio/ciso8601#123(comment).

I'll see if I can figure out a workaround for this.

This particular error seems to be specific to Python 3.9 in the continuumio/miniconda3:latest image and it seems like it stems from the fact that that version of Python sets the -n1 compile flag when building C extensions, which gcc doesn't like:

(base) root@635c876df361:/opt/conda# python -c "from sysconfig import get_config_var; print(get_config_var(\"CFLAGS\"));"
-Wno-unused-result -Wsign-compare -DNDEBUG  -O2 -Wall  -fPIC   -O2  -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include -I/opt/conda/include  -fPIC   -O2  -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include

Creating a Python 3.8 or 3.10 environment allows me to install the library without a problem:

(test) root@635c876df361:/opt/conda# conda create --name test-310 python=3.10
(test) root@635c876df361:/opt/conda# conda activate test-310
(test-310) root@635c876df361:/opt/conda# python -c "from sysconfig import get_config_var; print(get_config_var(\"CFLAGS\"));"
-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC   -O3  -isystem /opt/conda/envs/test-310/include -fPIC   -O3  -isystem /opt/conda/envs/test-310/include
(test-310) root@635c876df361:/opt/conda# pip install ciso8601
Collecting ciso8601
  Using cached ciso8601-2.2.0.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ciso8601
  Building wheel for ciso8601 (pyproject.toml) ... done
  Created wheel for ciso8601: filename=ciso8601-2.2.0-cp310-cp310-linux_aarch64.whl size=29928 sha256=040a6185a7d8e1857e9edd3bef856225a647ee39f8c604ba75bca2867229837e
  Stored in directory: /root/.cache/pip/wheels/02/1f/3b/a3fc12fc79c99e80e030beb98f07c92cbf1e43e00a7abccfcb
Successfully built ciso8601
Installing collected packages: ciso8601
Successfully installed ciso8601-2.2.0

@keul
Copy link
Contributor Author

keul commented Aug 5, 2022

There's also ciso8601 on conda. I'll try that before adding gcc

@TomAugspurger TomAugspurger mentioned this issue Aug 5, 2022
Merged
@geospatial-jeff geospatial-jeff mentioned this issue Aug 5, 2022
Merged
4 tasks
@geospatial-jeff
Copy link
Collaborator

@keul The ciso8601 dependency is removed in 2.4.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

replace ciso8601 with pyiso8601 stac-utils/stac-fastapi
4 participants
@keul @duckontheweb @vincentsarago @geospatial-jeff