Open Stack

Paul and myself discussed this morning how to move forward with binary 
package installation - the bindep-fallback.txt/other-requirements.txt story.

I wrote up what we discussed and hope I didn't forgot anything here. 
Please review and comment - and sign up if you want to help.

Andreas - who will board his plane to Washington in a few minutes

Goals:
1. bindep-fallback.txt should be minimal and handle packages that the
    majority of projects need.
2. Projects can use other-requirements to document their binary
    requirements.
3. There are no apt-get/yum install commands in jenkins/jobs/

Proposal:

1. Resolve with lifeless how the other-requirements.txt file should
    look like, see:
    nova: https://review.openstack.org/#/c/200672/
    swift: https://review.openstack.org/#/c/298313/

2. Document how to use other-requirements.txt.

3. Review list of packages in bindep-fallback.txt and write up
    packages that are not in the minimal set. Review apt-get/yum
    install lists.

    First iteration for apt-get/yum install (just jobs/macros):
    * npm-install
    * xvfb-install
    * firefox-install
    * chrome-install
    * diskimage-builder-install
    * puppet-prepare-node - this might be tricky since it removes
      packages as well, so might be the only exception.
    * chef-bundler-prep
    * elasticsearch-install
    * gate-anvil-rpms-dsvm-{node}
    * periodic-package-afs-centos7 - might need to stay as is.

    First iteration of extra packages to remove for
    bindep-fallback.txt (let's not discuss specific packages here yet
    but instead the broader principle (see goal 1 and how to refine that
    one using this first proposal):
    * asciidoc
    * docbook-style-xsl [platform:rpm]
    * docbook-xml [platform:dpkg]
    * docbook-xsl [platform:dpkg]
    * docbook5-schemas [platform:rpm]
    * docbook5-style-xsl [platform:rpm]
    * docbook5-xml [platform:dpkg]
    * firefox [!platform:debian]
    * fonts-nanum [platform:dpkg]
    * fonts-takao [platform:dpkg]
    * iceweasel [platform:debian]
    * liberasurecode-dev [platform:dpkg]
    * liberasurecode-devel [platform:rpm]
    * libevent-dev [platform:dpkg]
    * libevent-devel [platform:rpm]
    * libffi-dev [platform:dpkg]
    * libffi-devel [platform:rpm]
    * libjerasure-dev [platform:ubuntu-trusty]
    * libjpeg-dev [platform:dpkg]
    * libjpeg-turbo-devel [platform:rpm]
    * libldap2-dev [platform:dpkg]
    * libpcap-dev [platform:dpkg]
    * libpcap-devel [platform:rpm]
    * libpq-dev [platform:dpkg]
    * librrd-dev [platform:dpkg]
    * libvirt-dev [platform:dpkg]
    * libvirt-devel [platform:rpm]
    * libvirt-python [platform:rpm]
    * libxml2-dev [platform:dpkg]
    * libxml2-devel [platform:rpm]
    * libxml2-utils [platform:dpkg]
    * libxslt-devel [platform:rpm]
    * libxslt1-dev [platform:dpkg]
    * mariadb [platform:rpm]
    * mariadb-devel [platform:rpm]
    * mariadb-server [platform:rpm]
    * memcached
    * mongodb [platform:dpkg]
    * mongodb-server [platform:rpm]
    * mysql-server [platform:dpkg]
    * php-cli [platform:rpm]
    * php5-cli [platform:dpkg]
    * php5-mcrypt [platform:dpkg]
    * postgresql
    * postgresql-devel [platform:rpm]
    * postgresql-server [platform:rpm]
    * python-zmq
    * redis [platform:rpm]
    * redis-server [platform:dpkg]
    * rrdtool-devel [platform:rpm]
    * swig
    * xorg-x11-server-Xvfb [platform:rpm]
    * xsltproc [platform:dpkg]
    * xvfb [platform:dpkg]
    * zookeeperd [platform:dpkg]


4. Send email to openstack-dev to inform projects that
    other-requirements.txt can be used - and in which cases it has to
    be used. Explain:
    * that we will remove all apt-get/yum install usage from
      jenkins/jobs by specific date X.
    * infra will send other-requirements.txt files to projects with
      current default package list to projects that currently use
      apt-get/yum install. Projects should merge these and then remove
      packages that they do not need. If this is not merged by day X,
      the projects will break.
    * which packages will be removed from bindep-fallback.txt

5. Review extra package installs. Send other-requirements.txt changes
    for projects who's jobs are affected.

6. Review how integration jobs can be done. For example
    gate-bandit-integration-{target-project} in jenkins/jobs/bandit.yaml
    could be changed to:
    1) checkout bandit and the integration repository
    2) Install other-requirements/bindep-fallback for *both*
       repositories. In this case installing packages for bandit is
       enough since tox is run from bandit. But for others it might be
       different.
    3) Run tests

7. By day X: Remove all extra package installs from jenkins/ and
    remove these packages from bindep-fallback.txt.

8. Party!

-- 
  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
    GF: Felix Imendörffer, Jane Smithard, Graham Norton,
        HRB 21284 (AG Nürnberg)
     GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126