Hi Jeremy,
It's still unclear to me why we're doing this at all. Our stable constraints lists are supposed to be a snapshot in time from when we released, modulo stable point release updates of the libraries we're maintaining. Agreeing to bump random dependencies on stable branches because of security vulnerabilities in them is a slippery slope toward our users expecting the project to be on top of vulnerability announcements for every one of the ~600 packages in our constraints list.
I think this is combining two different viewpoints in one: "snapshot in time" and "user expects it to be updated asap on security vulnerabilities". We are already updating upper-constraints on bugfixes for projects that openstack maintains, and we do so similarly for security fixes for packages that are part of openstack. Also, distribution vendor provided (non-pip installed bindeps) are maintained and updated by security fixes.
Deployment projects already should not depend on our requirements team tracking security vulnerabilities, so need to have a mechanism to override constraints entries anyway if they're making such guarantees to their users (and I would also caution against doing that too).
for traditional openstack deployment projects that might be well the case. However several virtualenv / container / dockerbuild based projects are using upperconstraints to generate a stable, coherent and tested container. Without adjustments those will not be including security fixes though.
Distributions are far better equipped than our project to handle such tracking, as they generally get advance notice of vulnerabilities and selectively backport fixes for them.
Agreed, still OpenStack chose to use pip for managing its dependencies, so I think it is preferable to find a solution within that ecosystem. Not every dependency with security issues is so offensive as "requests" is which does not maintain any stable branch but asks you to update to the latest version instead. Most others do have backports available on micro versions as pip installable project.
accomplish the same with a mix of old and new dependency versions in our increasingly aging stable and extended maintenance branches seems like a disaster waiting to happen.
I agree it is a difficult exercise and we need to document a clear policy. For me documenting that it upper-constraints is maintained with security fix version updates as a best effort/risk bases is good enough, fwiw. Greetings, Dirk