Build fails on Raspbian 11

[mwweinberg]

I've been able to get this running in a few other environments, but am having some problems installing it on a clean version of Raspbian 11. Docker is installed and working. After cloning the repo and moving into the opentopodata directory, I run sudo make build. That works for a bit, until I get the error. Here's the full output:

docker build --tag opentopodata:`cat VERSION` --file docker/Dockerfile .
[+] Building 276.0s (11/14)                                      docker:default
 => [internal] load build definition from Dockerfile                       0.6s
 => => transferring dockerfile: 1.57kB                                     0.0s
 => [internal] load .dockerignore                                          0.8s
 => => transferring context: 113B                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.9.16-slim-bul  1.4s
 => [builder 1/3] FROM docker.io/library/python:3.9.16-slim-bullseye@sha  28.2s
 => => resolve docker.io/library/python:3.9.16-slim-bullseye@sha256:5cde4  0.1s
 => => sha256:5cde4e147c4165ad8dbf8a4df9631863766eeb0b79b 1.86kB / 1.86kB  0.0s
 => => sha256:532d38204a77f64141b806c5901b70bb4c8afaad713 1.37kB / 1.37kB  0.0s
 => => sha256:95ffefcf5046bc50f8e2f44f051330432e884e97ce8 6.83kB / 6.83kB  0.0s
 => => sha256:a27027e97f260d9b7aac9bae941b44639374700d 26.56MB / 26.56MB  18.6s
 => => sha256:9c18fa1eb965a9cd27f27d4f8153653a65b50942439 1.04MB / 1.04MB  0.9s
 => => sha256:c40b00875c85bb3eae5a8f99dfec4f209daecd543 10.44MB / 10.44MB  4.8s
 => => sha256:8f5389f7afcc196284369c8fcdb64cd6e849f7941b4070d 245B / 245B  1.2s
 => => sha256:df127cfcda927e4c0fcb852f882cc5805f8a09aeeb1 3.20MB / 3.20MB  5.4s
 => => extracting sha256:a27027e97f260d9b7aac9bae941b44639374700dc4c32cc2  4.3s
 => => extracting sha256:9c18fa1eb965a9cd27f27d4f8153653a65b50942439f8c14  0.3s
 => => extracting sha256:c40b00875c85bb3eae5a8f99dfec4f209daecd543cf08e24  1.4s
 => => extracting sha256:8f5389f7afcc196284369c8fcdb64cd6e849f7941b4070d9  0.0s
 => => extracting sha256:df127cfcda927e4c0fcb852f882cc5805f8a09aeeb1c87e2  0.8s
 => [internal] load build context                                          1.7s
 => => transferring context: 32.95MB                                       1.6s
 => [builder 2/3] RUN set -e &&     apt-get update &&     apt-get insta  147.8s
 => [stage-1 2/8] RUN set -e &&     apt-get update &&     apt-get insta  167.6s
 => [builder 3/3] RUN pip config set global.disable-pip-version-check tr  84.6s
 => [stage-1 3/8] COPY --from=builder /root/wheels /root/wheels            1.0s
 => [stage-1 4/8] COPY requirements.txt /app/requirements.txt              0.2s
 => ERROR [stage-1 5/8] RUN pip install         --no-index         --no-  10.9s
------
 > [stage-1 5/8] RUN pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt:
7.712 Looking in links: /root/wheels
7.744 ERROR: Could not find a version that satisfies the requirement uwsgi (from versions: none)
7.746 ERROR: No matching distribution found for uwsgi
------
Dockerfile:25
--------------------
  24 |     COPY requirements.txt /app/requirements.txt
  25 | >>> RUN pip install \
  26 | >>>         --no-index \
  27 | >>>         --no-cache-dir \
  28 | >>>         --disable-pip-version-check \
  29 | >>>         --find-links=/root/wheels \
  30 | >>>         uwsgi regex && \
  31 | >>>     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt && \
  32 | >>>         rm -rf /root/.cache/pip/* && \
  33 | >>>         rm root/wheels/* && \
  34 | >>>         rm /app/requirements.txt
  35 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt" did not complete successfully: exit code: 1
make: *** [Makefile:5: build] Error 1

I would love any suggestions.

thanks!

[ajnisbet]

Raspberry PIs are ARM right? In that case you'll need make build-m1 (despite the name it should work for all ARM systems).

If that doesn't work, I have been having some issues with the docker images on master that should be fixed on the dev branch. Try git checkout dev; make build-m1

[mwweinberg]

Thank you for the quick response!

Pis are ARM. I tried make build-m1in master and got this:

docker build --tag opentopodata:`cat VERSION` --file docker/apple-silicon.Dockerfile .
[+] Building 0.8s (3/3) FINISHED                                 docker:default
 => [internal] load build definition from apple-silicon.Dockerfile         0.1s
 => => transferring dockerfile: 1.38kB                                     0.0s
 => [internal] load .dockerignore                                          0.1s
 => => transferring context: 113B                                          0.0s
 => ERROR [internal] load metadata for docker.io/osgeo/gdal:ubuntu-full-3  0.6s
------
 > [internal] load metadata for docker.io/osgeo/gdal:ubuntu-full-3.5.2:
------
apple-silicon.Dockerfile:8
--------------------
   6 |     # build.
   7 |     
   8 | >>> FROM osgeo/gdal:ubuntu-full-3.5.2
   9 |     RUN set -e && \
  10 |         apt-get update && \
--------------------
ERROR: failed to solve: osgeo/gdal:ubuntu-full-3.5.2: no match for platform in manifest sha256:de03a3f94743fbbeb9fc7860125cb343587f8fd8c385eeac13e8aa7b72f3628f: not found
make: *** [Makefile:8: build-m1] Error 1

switching to dev got me:

docker build --tag opentopodata:`cat VERSION` --file docker/apple-silicon.Dockerfile .
[+] Building 0.8s (3/3) FINISHED                                 docker:default
 => [internal] load build definition from apple-silicon.Dockerfile         0.0s
 => => transferring dockerfile: 1.41kB                                     0.0s
 => [internal] load .dockerignore                                          0.1s
 => => transferring context: 113B                                          0.0s
 => ERROR [internal] load metadata for ghcr.io/osgeo/gdal:ubuntu-full-3.6  0.6s
------
 > [internal] load metadata for ghcr.io/osgeo/gdal:ubuntu-full-3.6.4:
------
apple-silicon.Dockerfile:8
--------------------
   6 |     # build.
   7 |     
   8 | >>> FROM  ghcr.io/osgeo/gdal:ubuntu-full-3.6.4
   9 |     RUN python --version
  10 |     RUN set -e && \
--------------------
ERROR: failed to solve: ghcr.io/osgeo/gdal:ubuntu-full-3.6.4: no match for platform in manifest sha256:5e9897d6cf4874c718ef99a76083b485ff4332e66cc8ab2a726f7aa046acf63a: not found
make: *** [Makefile:8: build-m1] Error 1

[ajnisbet]

Thanks for the detailed debugging output!

Unfortunately I don't have access to a raspberry pi for testing, and the build works on my ARM macbook.

If you can find a osgeo/gdal docker image that works on ARM then I'd be happy to update the docker file to use that: perhaps replacing ghcr.io/osgeo/gdal:ubuntu-full-3.6.4 with ghcr.io/osgeo/gdal or osgeo/gdal will pull in the latest image that works with raspian?

[mwweinberg]

Thanks! I'll dig into this and report back. I'm not super familiar with docker, so I'll start poking around, see if I can find a fix, and report back.

[mwweinberg]

I tried making both of those edits to the docker/apple-silicon.Dockerfile file, but it didn't seem to change the error. However, this PR:

OSGeo/gdal#2959

led me to this list:

https://hub.docker.com/r/osgeo/gdal/tags?page=1&name=arm

After playing around a bit and changing the file to osgeo/gdal:ubuntu-full-latest-arm64

I'm now getting this error message:

docker build --tag opentopodata:`cat VERSION` --file docker/apple-silicon.Dockerfile .
[+] Building 646.4s (6/12)                                       docker:default
 => [internal] load .dockerignore                                          0.2s
 => => transferring context: 113B                                          0.0s
 => [internal] load build definition from apple-silicon.Dockerfile         0.2s
 => => transferring dockerfile: 1.45kB                                     0.0s
 => [internal] load metadata for docker.io/osgeo/gdal:ubuntu-full-latest-  0.6s
 => [1/8] FROM docker.io/osgeo/gdal:ubuntu-full-latest-arm64@sha256:3a2  635.6s
 => => resolve docker.io/osgeo/gdal:ubuntu-full-latest-arm64@sha256:3a290  0.1s
 => => sha256:8b150fd943bcd54ef788cece17523d19031f745b 27.34MB / 27.34MB  11.5s
 => => sha256:3a290c1b39e2badf4e354870c87bffac18c8c87a029 3.48kB / 3.48kB  0.0s
 => => sha256:710ff6b80b18e53db04cdd68a4c2302921de80 248.67MB / 248.67MB  87.6s
 => => sha256:18d66c19cef1408d41dccdf129765a576af1e79dba3 7.66kB / 7.66kB  0.0s
 => => sha256:cdb003aab22f5c1d26eb7fd1938a959d216e2a8f 30.25MB / 30.25MB  19.6s
 => => sha256:8d6283bf8caaa510465b02b5acf669de3aed543b54 8.09MB / 8.09MB  25.8s
 => => extracting sha256:8b150fd943bcd54ef788cece17523d19031f745b099a798d  6.7s
 => => sha256:05f54c121ef82db019e25a47ad0fa16c0776b 710.10MB / 710.10MB  355.2s
 => => extracting sha256:cdb003aab22f5c1d26eb7fd1938a959d216e2a8f4f7c172  26.8s
 => => sha256:bbe155741b2f7b46021f470013e50a3939e0ee9fb 1.73MB / 1.73MB  156.6s
 => => sha256:55cef1f7687dc8483364b7b52d153f53719cd73 84.05kB / 84.05kB  169.9s
 => => extracting sha256:710ff6b80b18e53db04cdd68a4c2302921de801a01fb1d  199.9s
 => => sha256:1aea7848060b7ef77fbfdabec703862ba4778 298.93kB / 298.93kB  180.2s
 => => sha256:5ad4e0fda653e704061a04ecc4d6662e2fb0a34 23.22MB / 23.22MB  318.2s
 => => sha256:2393b1b33ba3b881e2864472c311448da96db99c6 2.54MB / 2.54MB  326.7s
 => => sha256:6a2905ff38b3a2e34d022a5c801fa739274a6 347.56kB / 347.56kB  360.9s
 => => sha256:c96db87622d4bdf2193b63020bab9ce28b908a41b 3.42MB / 3.42MB  367.8s
 => => extracting sha256:8d6283bf8caaa510465b02b5acf669de3aed543b5421a84b  1.8s
 => => sha256:4d6ec36e379dd54f21a7604ee216e224a12d4 125.15MB / 125.15MB  437.2s
 => => sha256:97ea871f361dcd542c4711105ca27467223ae 236.59kB / 236.59kB  369.9s
 => => extracting sha256:05f54c121ef82db019e25a47ad0fa16c0776be08091868  215.7s
 => => sha256:433bba3f55d8f73ea2e4e286decd03641fefc4a81 8.96kB / 8.96kB  375.7s
 => => extracting sha256:bbe155741b2f7b46021f470013e50a3939e0ee9fb3af4b99  0.2s
 => => extracting sha256:55cef1f7687dc8483364b7b52d153f53719cd731c90f1019  0.0s
 => => extracting sha256:1aea7848060b7ef77fbfdabec703862ba477893bd6666968  0.0s
 => => extracting sha256:5ad4e0fda653e704061a04ecc4d6662e2fb0a34e7db044b4  0.8s
 => => extracting sha256:2393b1b33ba3b881e2864472c311448da96db99c65c13793  0.2s
 => => extracting sha256:6a2905ff38b3a2e34d022a5c801fa739274a65b1eda139f7  0.1s
 => => extracting sha256:97ea871f361dcd542c4711105ca27467223ae37b8820a403  0.1s
 => => extracting sha256:c96db87622d4bdf2193b63020bab9ce28b908a41b51a5444  0.3s
 => => extracting sha256:4d6ec36e379dd54f21a7604ee216e224a12d4da446371df  43.2s
 => => extracting sha256:433bba3f55d8f73ea2e4e286decd03641fefc4a814c55707  0.0s
 => [internal] load build context                                          0.1s
 => => transferring context: 7.12kB                                        0.0s
 => ERROR [2/8] RUN python --version                                       9.8s
------
 > [2/8] RUN python --version:
------
apple-silicon.Dockerfile:11
--------------------
   9 |     FROM  osgeo/gdal:ubuntu-full-latest-arm64
  10 |     
  11 | >>> RUN python --version
  12 |     RUN set -e && \
  13 |         apt-get update && \
--------------------
ERROR: failed to solve: process "/bin/sh -c python --version" did not complete successfully: exit code: 159
make: *** [Makefile:8: build-m1] Error 1

Is there any chance that suggests any obvious next steps?

[arnesetzer]

1. I can't test it right now, but it is a common problem that the python command is not linked to the python2 nor python3 binary, so it might be worth a shot to change RUN python --version to RUN python3 --version
2. Did you install the 64bit version of raspian11? According to some websites exit code 159 in combination with docker and raspian may be an indicator, that 64bit code was attempted to be executed on a 32bit OS.
3. (May switch to the github package registry. According to the docker hub they switched the publishing platform)
   So instead of FROM osgeo/gdal:ubuntu-full-latest-arm64 use FROM ghcr.io/osgeo/gdal:ubuntu-small-latest-arm64 to get the latest version
4. Just in case: Your raspberry is newer than model zero? Otherwise the architecture is not capable of executing the code (32bit architecture with 64bit code).

[mwweinberg]

Thank you for the additional suggestions!

I can confirm that I'm running the 64 bit version on a model 4B.

Also, to confirm that I'm doing what I should be doing, I'm running the make sudo make build-m1 command and editing the docker/apple-silicon.Dockerfile.

When I update the RUN python --version to RUN python3 --version, I get the following error very early in the build:

=> ERROR [2/8] RUN python3 --version                                      1.0s
------
 > [2/8] RUN python3 --version:
------
apple-silicon.Dockerfile:11
--------------------
   9 |     FROM  osgeo/gdal:ubuntu-full-latest-arm64
  10 |     
  11 | >>> RUN python3 --version
  12 |     RUN set -e && \
  13 |         apt-get update && \
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 --version" did not complete successfully: exit code: 159
make: *** [Makefile:8: build-m1] Error 1

When I reverted that line back and then updated FROM osgeo/gdal:ubuntu-full-latest-arm64 to FROM ghcr.io/osgeo/gdal:ubuntu-small-latest-arm64, I got the following error much later in the build:

=> ERROR [2/8] RUN python --version                                      12.3s
------
 > [2/8] RUN python --version:
------
apple-silicon.Dockerfile:12
--------------------
  10 |     FROM ghcr.io/osgeo/gdal:ubuntu-small-latest-arm64
  11 |     
  12 | >>> RUN python --version
  13 |     RUN set -e && \
  14 |         apt-get update && \
--------------------
ERROR: failed to solve: process "/bin/sh -c python --version" did not complete successfully: exit code: 159
make: *** [Makefile:8: build-m1] Error 1

Keeping the updated URL and replacing RUN python --version to RUN python3 --version (so making two changes) gives me this error early in the build:

 => ERROR [2/8] RUN python3 --version                                      1.1s
------
 > [2/8] RUN python3 --version:
------
apple-silicon.Dockerfile:12
--------------------
  10 |     FROM ghcr.io/osgeo/gdal:ubuntu-small-latest-arm64
  11 |     
  12 | >>> RUN python3 --version
  13 |     RUN set -e && \
  14 |         apt-get update && \
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 --version" did not complete successfully: exit code: 159
make: *** [Makefile:8: build-m1] Error 1

I also just double checked and python 3.9.2 is installed on the machine.

[arkonsolutions]

@mwweinberg , Have you found a working build solution for raspberry?
I tryed to use FROM arm32v7/python:3.10.10-slim-buster, builds started, but crached after several minutes

[ajnisbet]

I made a new release a few weeks ago, @arkonsolutions did that version not work for you?

Unfortunately the build-m1 command works for me on my arm macbook so I can’t replicate the problem

[arkonsolutions]

@ajnisbet thank you for great product generally! Unfortunately, I haven't been able to get it to assemble for raspberry yet.
Errors related to NumPy and its internal dependencies pop up during the build.

[mwweinberg]

@arkonsolutions unfortunately I have not found a solution yet either

[Kandik]

I had the same problem on RPi5 running Raspbian 12, but managed to get it running with

sudo apt-get install libgdal-dev gdal-bin

and then using make build-m1