[openstack-dev] [tripleo] Pin some puppet dependencies on git clone

Jaume Devesa devvesa at gmail.com
Mon Dec 21 09:48:43 UTC 2015


My bet is first solve the problem, and then discuss how to improve the
solution.

Specifying the DIB_REPOREF variables by hand on the puppet-module element
in t-p-e will be an immediate
improvement in stability terms.

Then, it is fair to discuss how to automate the process to syncrhonise with
puppet-openstack dependencies.

Regards,


On 16 December 2015 at 16:40, Jiří Stránský <jistr at redhat.com> wrote:

> On 15.12.2015 19:12, Emilien Macchi wrote:
>
>>
>>
>> On 12/15/2015 12:23 PM, Jiří Stránský wrote:
>>
>>> On 15.12.2015 17:46, Emilien Macchi wrote:
>>>
>>>> For information, Puppet OpenStack CI is consistent for unit & functional
>>>> tests, we use a single (versionned) Puppetfile:
>>>>
>>>> https://github.com/openstack/puppet-openstack-integration/blob/master/Puppetfile
>>>>
>>>>
>>>> TripleO folks might want to have a look at this to follow the
>>>> dependencies actually supported by upstream OR if you prefer surfing on
>>>> the edge and risk to break CI every morning.
>>>>
>>>> Let me know if you're interested to support that in TripleO Puppet
>>>> elements, I can help with that.
>>>>
>>>
>>> Syncing tripleo-puppet-elements with puppet-openstack-integration is a
>>> good idea i think, to prevent breakages like the puppet-mysql one
>>> mentioned before.
>>>
>>> One thing to keep in mind is that the module sets in t-p-e and p-o-i are
>>> not the same. E.g. recently we added the timezone module to t-p-e, and
>>> it's not in the p-o-i Puppetfile.
>>>
>>> Also, sometimes we do have to go to non-openstack puppet modules to fix
>>> things for TripleO (i don't recall a particular example but i think we
>>> did a couple of fixes in non-openstack modules to allow us to deploy HA
>>> with Pacemaker). In cases like this it would be helpful if we still had
>>> the possibility to pin to something different than what's in
>>> puppet-openstack-integration perhaps.
>>>
>>>
>>> Considering the above, if we could figure out a way to have t-p-e behave
>>> like this:
>>>
>>> * install the module set listed in t-p-e, not p-o-i.
>>>
>>> * if there's a ref/branch specified directly in t-p-e, use that
>>>
>>> * if t-p-e doesn't have a ref/branch specified, use ref/branch from p-o-i
>>>
>>> * if t-p-e doesn't have a ref/branch specified, and the module is not
>>> present in p-o-i, use master
>>>
>>> * still honor DIB_REPOREF_* variables to pin individual puppet modules
>>> to whatever wanted at time of building the image -- very useful for
>>> temporary workarounds done either manually or in tripleo.sh.
>>>
>>> ...then i think this would be very useful. Not sure at the moment what
>>> would be the best way to meet these points though, these are just some
>>> immediate thoughts on the matter.
>>>
>>
>> I think we shout not use puppet-openstack-integration per-se, it was
>> just an example.
>>
>> Though we can take this project as reference to build a tool that
>> prepare Puppet modules in TripleO CI.
>>
>> If you look at puppet-openstack-integration, we have some scripts that
>> allow or not to use zuul-cloner with r10k, that's nice because it allows
>> us to:
>> * use depends-on puppet patches
>> * if the end-user does not have zuul, it will git-clone, in tripleo case
>> I think if DIB_REPOREF_* is set, let's use it
>> * otherwise use git clone master.
>>
>> I would suggest also TripleO CI having a Puppetfile that would be gated
>> (maybe in tripleo-ci repo?).
>>
>
> We should probably put the pins somewhere else than tripleo-ci, because
> we'd want dev environments to use the pinned versions too. Perhaps t-p-e is
> the right place.
>
> The more i think about this the more i like the approach in Dan's patch --
> an extra element which will pin modules the DIB way. What we're lacking
> here is a tool which could take a Puppetfile (specifically the Puppetfile
> from puppet-openstack-integration) and produce the DIB_REPOREF variables
> (perhaps ignoring all :ref => 'master' ones), so that we don't have to
> track and update them by hand.
>
> I'm not sure if we absolutely need a Puppetfile for TripleO. The value
> added is more in the pins themselves, not so much in syntax (Puppetfile vs.
> DIB-style-file). We could use Puppetfile format too, but since we'll not be
> able to use the one from puppet-openstack-integration directly (it's a
> different set of modules), i don't see much value in switching over.
>
> Jirka
>
>
>
>> What do you think?
>>
>>
>>> Jirka
>>>
>>>
>>>> On 12/14/2015 02:25 PM, Dan Prince wrote:
>>>>
>>>>> On Fri, 2015-12-11 at 21:50 +0100, Jaume Devesa wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Today TripleO CI jobs failed because a new commit introduced on
>>>>>> puppetlabs-mysql[1].
>>>>>> Mr. Jiri Stransky solved it as a temporally fix by pinning the puppet
>>>>>> module clone to a previous
>>>>>> commit in the tripleo-common project[2].
>>>>>>
>>>>>> source-repositories puppet element[3] allows you to pin the puppet
>>>>>> module clone as well by
>>>>>> adding a reference commit in the source-repository-<element-name>
>>>>>> file. In this case,
>>>>>> I am talking about the source-repository-puppet-modules[4].
>>>>>>
>>>>>> I know you TripleO guys are brave people that live dangerously in the
>>>>>> cutting edge, but I think
>>>>>> the dependencies to puppet modules not managed by the OpenStack
>>>>>> community should be
>>>>>> pinned to last repo tag for the sake of stability.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>
>>>>> I've previously considered added a stable puppet modules element for
>>>>> just this case:
>>>>>
>>>>> https://review.openstack.org/#/c/184844/
>>>>>
>>>>> Using stable branches of things like MySQL, Rabbit, etc might make
>>>>> sense. However I would want to consider following what the upstream
>>>>> Puppet community does as well specifically because we do want to
>>>>> continue using upstream openstack/puppet-* modules as well. At least
>>>>> for our upstream CI.
>>>>>
>>>>> We also want to make sure our stable TripleO jobs use the stable
>>>>> branches of openstack/puppet-* so we might need to be careful about
>>>>> pinning those things too.
>>>>>
>>>>> Dan
>>>>>
>>>>>
>>>>>    I can take care of this.
>>>>>>
>>>>>> [1]: https://github.com/puppetlabs/puppetlabs-mysql/commit/bdf4d0f52d
>>>>>> fc244d10bbd5b67efb791a39520ed2
>>>>>> [2]: https://review.openstack.org/#/c/256572/
>>>>>> [3]: https://github.com/openstack/diskimage-builder/tree/master/eleme
>>>>>> nts/source-repositories
>>>>>> [4]: https://github.com/openstack/tripleo-puppet-elements/blob/master
>>>>>> /elements/puppet-modules/source-repository-puppet-modules
>>>>>>
>>>>>> --
>>>>>> Jaume Devesa
>>>>>> Software Engineer at Midokura
>>>>>> _____________________________________________________________________
>>>>>> _____
>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
>>>>>> cribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>>
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>>
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Jaume Devesa
Software Engineer at Midokura
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151221/d9de3e03/attachment.html>


More information about the OpenStack-dev mailing list