Title: hf-xet Installation Fails in Termux Due to python source path Error; Local Install from Source Required

[Manamama]

See: huggingface/huggingface_hub#3067 (comment) - I should have pasted it here, I know.

Only after the git clone it works in pure Termux:

~/.../xet-core/hf_xet $ pip install -U . -v
Using pip 25.1.1 from /data/data/com.termux/files/usr/lib/python3.12/site-packages/pip (python 3.12)
Looking in indexes: https://pypi.org/simple, https://termux-user-repository.github.io/pypi/
Processing /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet
  Running command pip subprocess to install build dependencies
  Using pip 25.1.1 from /data/data/com.termux/files/usr/lib/python3.12/site-packages/pip (python 3.12)
  Looking in indexes: https://pypi.org/simple, https://termux-user-repository.github.io/pypi/, https://termux-user-repository.github.io/pypi/
  Collecting maturin<2.0,>=1.7
    Using cached maturin-1.8.4-cp312-cp312-linux_aarch64.whl
  Installing collected packages: maturin
  Successfully installed maturin-1.8.4
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  ⚠️ Warning: You specified the python source as /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet/python, but the python module at /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet/python/hf_xet is missing. No python module will be included.
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features from pyproject.toml
  hf_xet-1.1.0.dist-info
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /data/data/com.termux/files/usr/tmp/pip-modern-metadata-liaezy7i --interpreter /data/data/com.termux/files/usr/bin/python3.12`
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: hf-xet
  Running command Building wheel for hf-xet (pyproject.toml)
  Running `maturin pep517 build-wheel -i /data/data/com.termux/files/usr/bin/python3.12 --compatibility off`
  ⚠️ Warning: You specified the python source as /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet/python, but the python module at /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet/python/hf_xet is missing. No python module will be included.
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features from pyproject.toml
     Compiling hf_xet v1.1.0 (/data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet)
      Finished `release` profile [optimized + debuginfo] target(s) in 48.53s
  📦 Built wheel for abi3 Python ≥ 3.7 to /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet/target/wheels/hf_xet-1.1.0-cp37-abi3-linux_aarch64.whl
  /data/data/com.termux/files/home/downloads/whisperX/xet-core/hf_xet/target/wheels/hf_xet-1.1.0-cp37-abi3-linux_aarch64.whl
  Building wheel for hf-xet (pyproject.toml) ... done
  Created wheel for hf-xet: filename=hf_xet-1.1.0-cp37-abi3-linux_aarch64.whl size=46949239 sha256=4c07a9f94d41c1a7330512b8fd35cec5ecd2049364fc7a84bb4d6ab92c623c1c
  Stored in directory: /data/data/com.termux/files/usr/tmp/pip-ephem-wheel-cache-2oqmdbd8/wheels/28/13/99/2447c082766c054c7ba73ae456d1309816a89391b7107ff5b6
Successfully built hf-xet
Installing collected packages: hf-xet
Successfully installed hf-xet-1.1.0
~/.../xet-core/hf_xet $ 

[rajatarya]

Same as #255 and #304.

As a workaround while we see how to fix this, does this work for you?

This way you can build the same tag that we released to PyPI as 1.1.0 while building it locally.

pip install git+https://github.com/huggingface/[email protected]#subdirectory=hf_xet

[Manamama]

Quick confirmation that pip install git+https://github.com/huggingface/[email protected]#subdirectory=hf_xet reinstallation (sic) works too as it replicates the above hand git cloning. The problem is with downstream, e.g. below, as they try to pip install hf_xet (only) after a recent update. (All this works on prooted Debian under Termux, out of the box).

Ref, downstream that throws this:

.../0/Movies $ pip show huggingface-hub
Name: huggingface-hub
Version: 0.31.1
Summary: Client library to download and publish models, datasets and other repos on the huggingface.co hub
Home-page: https://github.com/huggingface/huggingface_hub
Author: Hugging Face, Inc.
Author-email: [email protected]
License: Apache
Location: /data/data/com.termux/files/usr/lib/python3.12/site-packages
Requires: filelock, fsspec, hf-xet, packaging, pyyaml, requests, tqdm, typing-extensions
Required-by: accelerate, asteroid, datasets, diffusers, docling, docling-ibm-models, faster-whisper, garak, llama-index-embeddings-huggingface, llama-index-llms-huggingface-api, optimum-quanto, peft, pyannote.audio, sentence-transformers, speechbrain, timm, tokenizers, transformers
.../0/Movies $ uname -a
Linux localhost 4.14.186+ #1 SMP PREEMPT Thu Mar 17 16:28:22 CST 2022 aarch64 Android
.../0/Movies $