Debug

This page regroups some problems encountered during the different operations and their fix. For issues arising during run operations, the solutions are probably tracked down in the ELOG.

Insatllation of ndlar_flow

ERROR: Failed building wheel for pylandau

Error message

(ndlar_flow.venv) [nsallin@neutrino01 ndlar_flow]$ pip install -e .
Obtaining file:///terabig/nsallin/ndlar_flow
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: h5py>=2.10 in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from ndlar_flow==1.0.1) (3.13.0)
Requirement already satisfied: pytest in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from ndlar_flow==1.0.1) (8.3.5)
Collecting scipy (from ndlar_flow==1.0.1)
  Downloading scipy-1.13.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Collecting scikit-image (from ndlar_flow==1.0.1)
  Downloading scikit_image-0.24.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (14 kB)
Collecting scikit-learn>=1.3.0 (from ndlar_flow==1.0.1)
  Downloading scikit_learn-1.6.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Requirement already satisfied: h5flow>=0.2.0 in /terabig/nsallin/h5flow (from ndlar_flow==1.0.1) (0.2.4)
Collecting pylandau@ git+https://github.com/cuddandr/pylandau.git#egg=pylandau (from ndlar_flow==1.0.1)
  Cloning https://github.com/cuddandr/pylandau.git to /tmp/pip-install-a32r5txi/pylandau_d8f676d31cae45139ef22c068c1411a7
  Running command git clone --filter=blob:none --quiet https://github.com/cuddandr/pylandau.git /tmp/pip-install-a32r5txi/pylandau_d8f676d31cae45139ef22c068c1411a7
  Resolved https://github.com/cuddandr/pylandau.git to commit a50571d566ec64bd3d6bbc77e9e55eeadfdb830f
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting adc64format@ git+https://github.com/larpix/adc64format.git@v0.1.3#egg=adc64format (from ndlar_flow==1.0.1)
  Cloning https://github.com/larpix/adc64format.git (to revision v0.1.3) to /tmp/pip-install-a32r5txi/adc64format_dba4f0ede7d64c018a1266a8728659d2
  Running command git clone --filter=blob:none --quiet https://github.com/larpix/adc64format.git /tmp/pip-install-a32r5txi/adc64format_dba4f0ede7d64c018a1266a8728659d2
  Resolved https://github.com/larpix/adc64format.git to commit 33f94a60cb654b24723bfeb9dd6f63e097fd4a61
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from adc64format@ git+https://github.com/larpix/adc64format.git@v0.1.3#egg=adc64format->ndlar_flow==1.0.1) (2.0.2)
Requirement already satisfied: tqdm in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from adc64format@ git+https://github.com/larpix/adc64format.git@v0.1.3#egg=adc64format->ndlar_flow==1.0.1) (4.67.1)
Collecting cython==0.29.34 (from pylandau@ git+https://github.com/cuddandr/pylandau.git#egg=pylandau->ndlar_flow==1.0.1)
  Using cached Cython-0.29.34-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (3.1 kB)
Requirement already satisfied: PyYAML in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from h5flow>=0.2.0->ndlar_flow==1.0.1) (6.0.2)
Requirement already satisfied: pyyaml-include<2.0 in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from h5flow>=0.2.0->ndlar_flow==1.0.1) (1.4.1)
Collecting joblib>=1.2.0 (from scikit-learn>=1.3.0->ndlar_flow==1.0.1)
  Downloading joblib-1.5.0-py3-none-any.whl.metadata (5.6 kB)
Collecting threadpoolctl>=3.1.0 (from scikit-learn>=1.3.0->ndlar_flow==1.0.1)
  Downloading threadpoolctl-3.6.0-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from pytest->ndlar_flow==1.0.1) (1.2.2)
Requirement already satisfied: iniconfig in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from pytest->ndlar_flow==1.0.1) (2.1.0)
Requirement already satisfied: packaging in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from pytest->ndlar_flow==1.0.1) (25.0)
Requirement already satisfied: pluggy<2,>=1.5 in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from pytest->ndlar_flow==1.0.1) (1.5.0)
Requirement already satisfied: tomli>=1 in /terabig/nsallin/ndlar_flow.venv/lib/python3.9/site-packages (from pytest->ndlar_flow==1.0.1) (2.2.1)
Collecting networkx>=2.8 (from scikit-image->ndlar_flow==1.0.1)
  Downloading networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pillow>=9.1 (from scikit-image->ndlar_flow==1.0.1)
  Downloading pillow-11.2.1-cp39-cp39-manylinux_2_28_x86_64.whl.metadata (8.9 kB)
Collecting imageio>=2.33 (from scikit-image->ndlar_flow==1.0.1)
  Downloading imageio-2.37.0-py3-none-any.whl.metadata (5.2 kB)
Collecting tifffile>=2022.8.12 (from scikit-image->ndlar_flow==1.0.1)
  Downloading tifffile-2024.8.30-py3-none-any.whl.metadata (31 kB)
Collecting lazy-loader>=0.4 (from scikit-image->ndlar_flow==1.0.1)
  Downloading lazy_loader-0.4-py3-none-any.whl.metadata (7.6 kB)
Using cached Cython-0.29.34-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (2.0 MB)
Downloading scikit_learn-1.6.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.5/13.5 MB 50.1 MB/s eta 0:00:00
Downloading joblib-1.5.0-py3-none-any.whl (307 kB)
Downloading scipy-1.13.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (38.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.6/38.6 MB 54.3 MB/s eta 0:00:00
Downloading threadpoolctl-3.6.0-py3-none-any.whl (18 kB)
Downloading scikit_image-0.24.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.9/14.9 MB 52.0 MB/s eta 0:00:00
Downloading imageio-2.37.0-py3-none-any.whl (315 kB)
Downloading lazy_loader-0.4-py3-none-any.whl (12 kB)
Downloading networkx-3.2.1-py3-none-any.whl (1.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 36.1 MB/s eta 0:00:00
Downloading pillow-11.2.1-cp39-cp39-manylinux_2_28_x86_64.whl (4.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 44.9 MB/s eta 0:00:00
Downloading tifffile-2024.8.30-py3-none-any.whl (227 kB)
Building wheels for collected packages: ndlar_flow, adc64format, pylandau
  Building editable for ndlar_flow (pyproject.toml) ... done
  Created wheel for ndlar_flow: filename=ndlar_flow-1.0.1-0.editable-py3-none-any.whl size=7829 sha256=98929ffdd58bd65a31a73ea604bf68cdc95a17280fc5f1a2c0d27c8076e8349a
  Stored in directory: /tmp/pip-ephem-wheel-cache-3aj383d4/wheels/b0/6b/8f/daa407c51bf2b21a3a79bdcd2058ecdac5f0b12434ee962a33
  Building wheel for adc64format (pyproject.toml) ... done
  Created wheel for adc64format: filename=adc64format-0.1.2-py3-none-any.whl size=10830 sha256=31e47604250a8ae6d6b2e85c549d899c530d3d3bfb8629259aa56b0d5349a808
  Stored in directory: /tmp/pip-ephem-wheel-cache-3aj383d4/wheels/21/4d/e6/c1081e07e37fd78e22e5cb71be9cc0a7ee6634010dfc94f915
  Building wheel for pylandau (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for pylandau (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [92 lines of output]
      /tmp/pip-build-env-hq4fm6bt/overlay/lib/python3.9/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
      !!
      
              ********************************************************************************
              Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).
      
              By 2026-Feb-18, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
      
              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************
      
      !!
        corresp(dist, value, root_dir)
      /tmp/pip-build-env-hq4fm6bt/overlay/lib/python3.9/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsWarning: `install_requires` overwritten in `pyproject.toml` (dependencies)
        corresp(dist, value, root_dir)
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-39/pyLandau
      copying pyLandau/__init__.py -> build/lib.linux-x86_64-cpython-39/pyLandau
      creating build/lib.linux-x86_64-cpython-39/tests
      copying tests/__init__.py -> build/lib.linux-x86_64-cpython-39/tests
      copying tests/constraints.py -> build/lib.linux-x86_64-cpython-39/tests
      copying tests/test_data_types.py -> build/lib.linux-x86_64-cpython-39/tests
      copying tests/test_exceptions.py -> build/lib.linux-x86_64-cpython-39/tests
      copying tests/test_fuzzing.py -> build/lib.linux-x86_64-cpython-39/tests
      copying tests/test_properties.py -> build/lib.linux-x86_64-cpython-39/tests
      running egg_info
      writing pylandau.egg-info/PKG-INFO
      writing dependency_links to pylandau.egg-info/dependency_links.txt
      writing requirements to pylandau.egg-info/requires.txt
      writing top-level names to pylandau.egg-info/top_level.txt
      reading manifest file 'pylandau.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching '*' under directory 'docs'
      adding license file 'LICENSE'
      writing manifest file 'pylandau.egg-info/SOURCES.txt'
      /tmp/pip-build-env-hq4fm6bt/overlay/lib/python3.9/site-packages/setuptools/command/build_py.py:212: _Warning: Package 'pyLandau.cpp' is absent from the `packages` configuration.
      !!
      
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'pyLandau.cpp' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
      
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'pyLandau.cpp' is explicitly added
              to the `packages` configuration field.
      
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
      
              You can read more about "package discovery" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
      
              If you don't want 'pyLandau.cpp' to be distributed and are
              already explicitly excluding 'pyLandau.cpp' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
      
              You can read more about "package data files" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
      
      
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
      
      !!
        check.warn(importable)
      creating build/lib.linux-x86_64-cpython-39/pyLandau/cpp
      copying pyLandau/cpp/pylandau.cpp -> build/lib.linux-x86_64-cpython-39/pyLandau/cpp
      copying pyLandau/cpp/pylandau.pyx -> build/lib.linux-x86_64-cpython-39/pyLandau/cpp
      copying pyLandau/cpp/pylandau_src.cpp -> build/lib.linux-x86_64-cpython-39/pyLandau/cpp
      running build_ext
      building 'pylandau' extension
      creating build/temp.linux-x86_64-cpython-39/pyLandau/cpp
      g++ -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DCYTHON_TRACE=1 -IpyLandau/cpp -I/terabig/nsallin/ndlar_flow.venv/include -I/usr/include/python3.9 -I/tmp/pip-build-env-hq4fm6bt/overlay/lib64/python3.9/site-packages/numpy/_core/include -c pyLandau/cpp/pylandau.cpp -o build/temp.linux-x86_64-cpython-39/pyLandau/cpp/pylandau.o
      pyLandau/cpp/pylandau.cpp:31:10: fatal error: Python.h: No such file or directory
         31 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/g++' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pylandau
Successfully built ndlar_flow adc64format
Failed to build pylandau
ERROR: Failed to build installable wheels for some pyproject.toml based projects (pylandau)

Solution

The solution was to install pylandau first and then rerun the ndlar_flow install command:

pip install pylandau

and then (in .../ndlar_flow)

pip install -e .