Yes, that's a good call out. I don't think pinning Horizon will be sufficient because of that pip behaviour. The error I am seeing at the moment looks like this (sorry for the long paste, but I want to show the whole chain):

Step 9/9 : RUN ln -s horizon-source/* horizon     &&
    sed -e "/^horizon===/d" -i requirements/upper-constraints.txt     &&
    python3 -m pip --no-cache-dir install --upgrade -c /requirements/upper-constraints.txt /horizon "setuptools<82.0.0"     &&
    mkdir -p /etc/openstack-dashboard     &&
    cp -r /horizon/openstack_dashboard/conf/* /etc/openstack-dashboard/     &&
    cp /horizon/openstack_dashboard/local/local_settings.py.example /etc/openstack-dashboard/local_settings.py     &&
    local_settings=$(python -c 'import os;import openstack_dashboard;settings=os.path.dirname(openstack_dashboard.__file__) + "/local/local_settings.py";print(settings);')     &&
    rm -f $local_settings     &&
    ln -s /etc/openstack-dashboard/local_settings.py $local_settings     &&
    mkdir -p /etc/openstack-dashboard/local_settings.d     &&
    local_settings_d=$(python -c 'import os;import openstack_dashboard;settings_d=os.path.dirname(openstack_dashboard.__file__) + "/local/local_settings.d";print(settings_d);')     &&
    rm -rf $local_settings_d     &&
    ln -s /etc/openstack-dashboard/local_settings.d $local_settings_d     &&
    cp /horizon/manage.py /var/lib/kolla/venv/bin/manage.py     &&
    if [ "$(ls /plugins)" ]; then            python3 -m pip --no-cache-dir install --upgrade -c /requirements/upper-constraints.txt /plugins/* "setuptools<82.0.0";        fi     &&
    for locale in  /var/lib/kolla/venv/lib/python3/site-packages/*/locale; do            (cd ${locale%/*} &&
    /var/lib/kolla/venv/bin/django-admin compilemessages)        done     &&
    chmod 644 /usr/local/bin/kolla_extend_start

---> Running in 895340bd989f
Processing ./horizon
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Feb 09 11:01:03 master Requirement already satisfied: setuptools<82.0.0 in ./var/lib/kolla/venv/lib/python3.13/site-packages (81.0.0)
Requirement already satisfied: pbr>=5.5.0 in ./var/lib/kolla/venv/lib/python3.13/site-packages (from horizon==25.6.0.dev84) (7.0.3)
Feb 09 11:01:03 master Requirement already satisfied: Babel>=2.6.0 in ./var/lib/kolla/venv/lib/python3.13/site-packages (from horizon==25.6.0.dev84) (2.17.0)
Collecting Django<4.3,>=4.2 (from horizon==25.6.0.dev84)
Downloading django-4.2.25-py3-none-any.whl.metadata (4.2 kB)
Collecting django-compressor>=4.4 (from horizon==25.6.0.dev84)
Downloading django_compressor-4.5.1-py2.py3-none-any.whl.metadata (5.0 kB)
Collecting django-debreach>=1.4.2 (from horizon==25.6.0.dev84)
Downloading django_debreach-2.1.0-py3-none-any.whl.metadata (3.7 kB)
│ exit code: 1
╰─> [26 lines of output]
    Traceback (most recent call last):
      File "/var/lib/kolla/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
        main()
        ~~~~^^
      File "/var/lib/kolla/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
        json_out["return_val"] = hook(**hook_input["kwargs"])
                                 ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
      File "/var/lib/kolla/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
        return hook(config_settings)
      File "/tmp/pip-build-env-n_96tptu/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 333, in get_requires_for_build_wheel
        return self._get_build_requires(config_settings, requirements=[])
               ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-n_96tptu/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
        self.run_setup()
        ~~~~~~~~~~~~~~^^
      File "/tmp/pip-build-env-n_96tptu/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 520, in run_setup
        super().run_setup(setup_script=setup_script)
        ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/pip-build-env-n_96tptu/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 317, in run_setup
        exec(code, locals())
        ~~~~^^^^^^^^^^^^^^^^
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-rgkzbtcj/xstatic-angular-schema-form_9a119b9343f1471a806215e0133ced88/xstatic/__init__.py", line 1, in <module>
        __import__('pkg_resources').declare_namespace(__name__)
        ~~~~~~~~~~^^^^^^^^^^^^^^^^^
    ModuleNotFoundError: No module named 'pkg_resources'
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed to build 'XStatic-Angular-Schema-Form' when getting requirements to build wheel

That paste has been lightly edited for clarity and contains a setuptools pin in the docker command RUN line for my attempts to resolve this.

So I think somewhere in the dependency chain of 'XStatic-Angular-Schema-Form' is a problematic usage of setuptools.pkg_resources, I think in XStatic-Angular-Schema-Form itself? Interestingly, that's an Openstack project (one of the pypi maintainers is "openstack-ci") with exactly one release on pypi... From 2016.

Michael