Open Stack

On 05/28/2015 08:24 PM, Zane Bitter wrote:
> On 28/05/15 03:35, Jan Provaznik wrote:
>> On 05/28/2015 01:10 AM, Steve Baker wrote:
>>> On 28/05/15 10:54, Richard Raseley wrote:
>>>> Zane Bitter wrote:
>>>>> Steve is working on a patch to allow package-based updates of
>>>>> overcloud
>>>>> nodes[1] using the distro's package manager (yum in the case of RDO,
>>>>> but
>>>>> conceivable apt in others). Note we're talking exclusively about minor
>>>>> updates, not version-to-version upgrades here.
>>>>>
>>>>> Dan mentioned at the summit that this approach fails to take into
>>>>> account the complex ballet of service restarts required to update
>>>>> OpenStack services. (/me shakes fist at OpenStack services.) And
>>>>> furthermore, that the Puppet manifests already encode the necessary
>>>>> relationships to do this properly. (Thanks Puppeteers!) Indeed we'd be
>>>>> doing the Wrong Thing by Puppet if we changed this stuff from under
>>>>> it.
>>>>>
>>>>> The problem of course is that neither Puppet nor yum/apt has a view of
>>>>> the entire system. Yum doesn't know about the relationships between
>>>>> services and Puppet doesn't know about all of the _other_ packages
>>>>> that
>>>>> they depend on.
>>>>>
>>>>> One solution proposed was to do a yum update first but specifically
>>>>> exclude any packages that Puppet knows about (the --excludes flag
>>>>> appears sufficient for this); then follow that up with another Puppet
>>>>> run using ensure -> latest.
>>>>>
>>> My only concern with this approach is how do we collect and maintain the
>>> excludes list. Other than that it sounds reasonable.
>>
>> Why not swap the order? First run puppet using ensure=>latest which
>> updates/restarts everything Openstack depends on, then run yum/apt
>> update to update any remaining bits. You wouldn't need exclude list then.
>
> Will ensure=>latest update all packages that the given one is dependent
> on, even if it doesn't require new versions? I assumed that it wouldn't,
> so by doing Puppet first we would just ensure that we are even less
> likely to pick up library changes by restarting services after the
> libraries are updated.
>

We could take advantage of this only when both a service and a 
dependency library are part of the upgrade. Other services depending on 
the lib would have to be restarted out-of-puppet in post yum-update 
phase anyway. I wonder if it is worth to try generate list of exclude 
packages (to be able to run yum first) given quite limited benefit it 
provides, but if it's simple enough then +1 :).

>>>>> A problem with that approach is that it still fails to restart
>>>>> services
>>>>> which have had libraries updated but have not themselves been updated.
>>>>> That's no worse than the pure yum approach though. We might need an
>>>>> additional way to just manually trigger a restart of services.
>>>
>>> Maybe this could be handled at the packaging stage by reving the package
>>> version when there is a known fix in a low-level library, thus
>>> triggering a service restart in the puppet phase.
>>>
>>
>> My concern is that then e.g. libc update would mean repackaging (bumping
>> version) of zillion of other packages, also zillion of packages would be
>> downloaded/upgraded on each system because of a new package version.
>
> Yes, no distro ever created works like this - for good reason - and it
> is waaaaaaaaay out of scope for us to try to change that :)
>
>> I think that providing user a manual (script) way to restart services
>> after update would be sufficient solution (though not so sophisticated).
>
> Maybe there's a way we can poke Puppet to do it.
>
> cheers,
> Zane.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev