Open Stack

On 2023-03-10 10:39:53 -0800 (-0800), Jay Faulkner wrote:
[...]
> To be explicit, that testing platform means we require that
> OpenStack and its dependencies are installable *in a virtualenv*,
> not using distro python.

Well, "sort of." Let's unpack those assertions:

1. "[we're testing] that OpenStack and its dependencies are
   installable in a virtualenv"

Here I think you mean specifically its Python library dependencies,
OpenStack has lots of (in many cases versioned) dependencies on
non-Python components of a system and that's a big part of what
we're checking by testing on multiple platforms. Where Python
libraries are concerned, some may include non-Python "binary"
extensions so it's not purely Python in the Python dependencies
either. We also install some things like Javascript libraries from
places other than the distribution (e.g. NPM) on those platforms.

But we don't only test in virtualenvs/venvs either. Today at least,
DevStack/Grenade jobs install a vast majority of those dependencies
into the base system environment. That probably won't be the case in
the future thanks to increasing adoption of the PEP 668
EXTERNALLY-MANAGED flag, but for now at least we're comingling
pip-installed and distro-installed Python libraries in those jobs.

2. "[we're testing] OpenStack and its dependencies [...] not using
   distro python"

This is definitely not true, but is maybe not what you meant. We
definitely test with the exact build of Python interpreter and
stdlib that these platforms ship, backported fixes and all. That's
also a major reason why we test on multiple platforms, so that we
can know our software will work with the Python that's provided on
those platforms, which is how many of our users will be running our
software too.

> So while this is a small imitation for this case (we cannot use
> cryptography that might require newer tooling to build than that
> ubuntu LTS would), the real motivation behind holding onto a
> lower-constraint for a longer time is in partnership with stable
> distros that ship OpenStack, who we don't want to alienate by
> giving them extra work of using newer releases than they are ready
> to.

Yes, more precisely these distros are not likely to backport an
entire new Rust toolchain just so that new versions of OpenStack can
be installed. Instead, they're going to patch the new versions of
OpenStack so that they work with older libraries that don't require
an entire new Rust toolchain, because that's the easier of the two
options. If we can avoid requiring them to do either of those
things, obviously it's even nicer for them. If they're going to do
the second thing anyway, we could just say "hey send us those
patches and we'll consider them bug fixes, because we want people to
be able to easily use OpenStack on stable server platforms."

> I am OK with this approach; but there is a trade-off: we may be
> leading OpenStack consumers who don't use distro packaging that
> it's OK to use an older cryptography which may not have security
> support.

I'd argue that we already do this, because our stable branches use
frozen-in-time constraints lists specifying obsolete versions of
dependencies which are not receiving upstream security support. The
intent is that we only use those to test backported fixes without
destabilizing our CI jobs, but some of our deployment projects even
still build container images or deploy packages from PyPI based on
what's in those frozen constraints lists, much to my dismay.

> It'd be interesting if we could ensure backwards compatibility
> through testing while also ensuring that anything installed with
> pip gets a new enough version... but frankly, I don't know offhand
> what approach would be best to do that, and I don't have the time
> to pursue it myself so the status quo wins :D.
[...]

My preference would be to just keep testing with the latest version
of PYCA/Cryptography from PyPI in our master branch jobs, and tell
people who want to use OpenStack on stable server distributions that
we'll accept bug reports and review their fixes if we start doing
something that the python3-cryptography package on those
distributions doesn't support. That's just one approach though,
there are certainly other ways to go about it.
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230310/1033959e/attachment-0001.sig>