Open Stack

Hello,

We're at the point of implementing the tarballs.openstack.org
publishing changes from [1], and I would just like to propose some
low-level plans for feedback that exceeded the detail in the spec.

We currently have tarballs.opendev.org which publishes content from
/afs/openstack.org/project/opendev.org/tarballs.  This is hosted on
the physical server files02.openstack.org and managed by puppet [2].

 1) I propose we move tarballs.opendev.org to be served by
    static01.opendev.org and configured via ansible

Because:

 * it's one less thing running on a Xenial host with puppet we don't
   want to maintain.
 * it will be alongside tarballs.openstack.org per below

The /afs/openstack.org/project/opendev.org directory is currently a
single AFS volume "projet.opendev" and contains subdirectories:

 docs tarballs www

opendev.org jobs currently write their tarball content into the AFS
location, which is periodically "vos released" by [3].

 2) I propose we make a separate volume, with separate quota, and
    mount it at /afs/openstack.org/project/tarballs.opendev.org.  We
    copy the current data to that location and modify the opendev.org
    tarball publishing jobs to use that location, and setup the same
    periodic release.

Because:

 * Although currently the volume is tiny (<100mb), it will become
   quite large when combined with ~140gb of openstack repos
 * this seems distincly separate from docs and www data
 * we have content for other hosts at /afs/openstack.org/project like
   this, it fits logically.

The next steps are described in the spec; with this in place, we copy
the current openstack tarballs from
static.openstack.org:/srv/static/tarballs to
/afs/openstack.org/project/tarballs.opendev.org/openstack/

We then update the openstack tarball publishing jobs to publish to
this new location via AFS (we should be able to make this happen in
parallel, initially).

Finally, we need to serve these files.

 3) I propose we make tarballs.openstack.org a vhost on
    static.opendev.org that serves the
    /afs/openstack.org/project/tarballs.opendev.org/openstack/
    directory.

Because

 * This is transparent for tarballs.openstack.org; all URLs work with
   no redirection, etc.
 * anyone hitting tarballs.opendev.org will see top-level project
   directories (openstack, zuul, airship, etc.) which makes sense.

I think this will get us where we want to be.

Any feedback welcome, thanks.  We will keep track of things in [4].

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html
[2] https://opendev.org/opendev/system-config/src/branch/master/manifests/site.pp#L441
[3] https://opendev.org/opendev/system-config/src/branch/master/modules/openstack_project/files/openafs/release-volumes.py
[4] https://storyboard.openstack.org/#!/story/2006598