Open Stack

On 21 July 2016 at 23:04, Paul Belanger <pabelanger at redhat.com> wrote:
> Greetings,
>
> I write today to see how I can remove this server from tripleo-test-cloud-rh2. I
> have an open patch[1] currently to migrate tripleo-ci to use our AFS mirrors for
> centos and epel.  However, I'm still struggling to see what else you are using
> the local mirror for.
>
> From what I see, there appears to be some puppet modules in the mirror?
>
> The reason I am doing this work, is to help bring tripleo inline with
> openstack-infra tooling.  There shouldn't be the need for a project to maintain
> its own infrastructure outside of openstack-infra.  If so, I see that as some
> sort of a failure between the project and openstack-infra.   And with that in
> mind, I am here to help fix that.
>
> For the most part, I think we have everything currently in place to migrate away
> from your locally mirror. I just need some help figuring what else is left and
> then delete it.

Hi Paul,
    The mirror server hosts 3 sets of data used in CI long with a cron
a job aimed at promoting trunk repositories,
The first you've already mentioned, there is a list of puppet modules
hosted here, we soon hope to move to packaged puppet modules so the
need for this will go away.

The second is a mirror of the centos cloud images, these are updated
hourly by the centos-cloud-images cronjob[1], I guess these could be
easily replaced with the AFS server

Then we come to the parts where it will probably be more tricky to
move away from our own server

o cached images - our nightly periodic jobs run tripleo ci with
master/HEAD for all openstack projects (using the most recent rdo
trunk repository), if the jobs pass then we upload the overcloud-full
and ipa images to the mirror server along with logging what jobs
passed, this happens at the end of toci_instack.sh[2], nothing else
happens at this point the files are just uploaded nothing starts using
them yet.

o promote script - hourly we then run the promote script[3], this
script is whats responsible for the promotion of the master rdo
repository that is used by tripleo ci (and devs), it checks to see if
images have been updated to the mirror server by the periodic jobs,
and if all of the jobs we care about (currently
periodic-tripleo-ci-centos-7-ovb-ha
periodic-tripleo-ci-centos-7-ovb-nonha[4]) passed then it does 2
things
  1. updates the current-tripleo link on the mirror server[5]
  2. updates the current-tripleo link on the rdo trunk server[6]
By doing this we ensure that the the current-tripleo link on the rdo
trunk server is always pointing to something that has passed tripleo
ci jobs, and that tripleo ci is using cached images that were built
using this repository

We've had to run this promote script on the mirror server as the
individual jobs run independently and in oder to make the promote
decision we needed somewhere that is aware of the status of all the
jobs

Hope this answers your questions,
Derek.

[1] - http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n40
[2] - http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n198
[3] - http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/promote.sh
[4] - http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n51
[5] - http://8.43.87.241/builds/current-tripleo/
[6] - http://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/

>
> [1] https://review.openstack.org/#/c/326143/
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev