<div dir="ltr">I'd noticed other-requirements.txt around, but figured it needed a bunch of custom tooling to actually make it useful.<div><br></div><div>And... it's a subprocess wrapper to a handful of package management tools (surprised to see emerge and pacmac - Kudos!) and a custom format for describing package requirements...<div><br></div><div>... but ... it doesn't have a --install option? Do you know if that is strictly out-of-scope or roadmap or ... ?</div><div><br></div><div>-Clay<br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 12, 2016 at 10:31 AM, Andreas Jaeger <span dir="ltr"><<a href="mailto:aj@suse.com" target="_blank">aj@suse.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">TL;DR: Projects can use bindep.txt to document in a programmatic way<br>
their binary dependencies<br>
<br>
Python developers record their dependencies on other Python packages in<br>
requirements.txt and test-requirements.txt. But some packages<br>
havedependencies outside of python and we should document<br>
thesedependencies as well so that operators, developers, and CI systems<br>
know what needs to be available for their programs.<br>
<br>
Bindep is a solution to this, it allows a repo to document<br>
binarydependencies in a single file. It even enablies specification of<br>
which distribution the package belongs to - Debian, Fedora, Gentoo,<br>
openSUSE, RHEL, SLES and Ubuntu have different package names - and<br>
allows profiles, like a test profile.<br>
<br>
Bindep is one of the tools the OpenStack Infrastructure team has written<br>
and maintains. It is in use by already over 130 repositories.<br>
<br>
For better bindep adoption, in the just released bindep 2.1.0 we have<br>
changed the name of the default file used by bindep from<br>
other-requirements.txt to bindep.txt and have pushed changes [3] to<br>
master branches of repositories for this.<br>
<br>
Projects are encouraged to create their own bindep files. Besides<br>
documenting what is required, it also gives a speedup in running tests<br>
since you install only what you need and not all packages that some<br>
other project might need and are installed by default. Each test system<br>
comes with a basic installation and then we either add the repo defined<br>
package list or the large default list.<br>
<br>
In the OpenStack CI infrastructure, we use the "test" profile for<br>
installation of packages. This allows projects to document their run<br>
time dependencies - the default packages - and the additional packages<br>
needed for testing.<br>
<br>
Be aware that bindep is not used by devstack based tests, those have<br>
their own way to document dependencies.<br>
<br>
A side effect is that your tests run faster, they have less packages to<br>
install. A Ubuntu Xenial test node installs 140 packages and that can<br>
take between 2 and 5 minutes. With a smaller bindep file, this can change.<br>
<br>
Let's look at the log file for a normal installation with using the<br>
default dependencies:<br>
2 upgraded, 139 newly installed, 0 to remove and 41 not upgraded<br>
Need to get 148 MB of archives.<br>
After this operation, 665 MB of additional disk space will be used.<br>
<br>
Compare this with the openstack-manuals repostiry that uses bindep -<br>
this example was 20 seconds and not minutes:<br>
0 upgraded, 17 newly installed, 0 to remove and 43 not upgraded.<br>
Need to get 35.8 MB of archives.<br>
After this operation, 128 MB of additional disk space will be used.<br>
<br>
If you want to learn more about bindep, read the Infra Manual on package<br>
requirements [1] or the bindep manual [2].<br>
<br>
If you have further questions about bindep, feel free to ask the Infra<br>
team on #openstack-infra.<br>
<br>
Thanks to Anita for reviewing and improving this blog post and to the<br>
OpenStack Infra team that maintains bindep, especially to Jeremy Stanley<br>
and Robert Collins.<br>
<br>
Note I'm sending this out while not all our test clouds have images that<br>
know about bindep.txt (they only handle other-requirements.txt). The<br>
infra team is in the process of ensuring updated images in all our test<br>
clouds for later today. Thanks, Paul!<br>
<br>
Andreas<br>
<br>
<br>
References:<br>
[1] <a href="http://docs.openstack.org/infra/manual/drivers.html#package-requirements" rel="noreferrer" target="_blank">http://docs.openstack.org/<wbr>infra/manual/drivers.html#<wbr>package-requirements</a><br>
[2] <a href="http://docs.openstack.org/infra/bindep/" rel="noreferrer" target="_blank">http://docs.openstack.org/<wbr>infra/bindep/</a><br>
[3] <a href="https://review.openstack.org/#/q/branch:master+topic:bindep-mv" rel="noreferrer" target="_blank">https://review.openstack.org/#<wbr>/q/branch:master+topic:bindep-<wbr>mv</a><br>
<span class="HOEnZb"><font color="#888888">--<br>
Andreas Jaeger aj@{<a href="http://suse.com" rel="noreferrer" target="_blank">suse.com</a>,<a href="http://opensuse.org" rel="noreferrer" target="_blank">opensuse.org</a>} Twitter/Identica: jaegerandi<br>
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany<br>
GF: Felix Imendörffer, Jane Smithard, Graham Norton,<br>
HRB 21284 (AG Nürnberg)<br>
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126<br>
<br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
</font></span></blockquote></div><br></div></div></div></div>