Open Stack

On 04/10/2014 12:01 PM, Eoghan Glynn wrote:
> 
>>> Sean Dague wrote:
>>>> I've proposed another add to the expectations post graduation to the
>>>> governance repo - https://review.openstack.org/#/c/86162/
>>>>
>>>> This was in the original QA requirements proposal, however when that got
>>>> restructured, it got lost somewhere. Also happy to discuss it if there
>>>> are any concerns.
>>>>
>>>> As a scorecard today the following projects participate in upgrade
>>>> testing:
>>>>
>>>>  * Nova
>>>>  * Swift
>>>>  * Keystone
>>>>  * Glance
>>>>  * Cinder
>>>>  * Horizon
>>>>  * Neutron (new as of this week)
>>>>
>>>> Projects that don't (yet)
>>>>  * Heat
>>>>  * Ceilometer
> 
> Hi Sean,
> 
> Thanks for raising this.
> 
> Can I clarify one related point, whether grenade is intended
> primarily to catch intrinsic or extrinsic upgrade issues?
> 
> Or both?
> 
> Intrinsic in this context would for example be service A failing
> post-upgrade due to a bad DB schema migration.
> 
> Whereas extrinsic would for example be service A failing post-
> upgrade because some dependency/assumption with respect to
> service B is now broken.
> 
> I ask because ceilometer when used with a "schema free" storage
> driver may be more likely to suffer from extrinsic issues on
> upgrade as opposed to intrinsic (e.g. nova changes a notification
> payload between versions, as opposed to a failed db-sync as that
> wouldn't even arise).
> 
> But surely such integration issues should have been caught in
> advance via the normal Tempest testing of master-versus-master,
> and wouldn't necessarily require the big-bang stable-to-master
> approach of grenade to smoke out?
> 
> Or am I missing something obvious here?

I will in no way say we catch all upgrade issues in grenade, but let me
tell you what we expect to work, and are testing against.

In grenade we bring up stable/last version of OpenStack, exercise it
with Tempest (smoke subset), and then specifically create some
additional resources we expect to survive the upgrade. We then upgrade
the *code* to master. The configuration is left at the stable/last
version, as we should have 1 cycle deprecation on configuration. We run
a series of specific upgrade scripts (which in the base case do
db-sync), then check for the expected resources to still be there, and
complete with another Tempest smoke run.

This should catch config deprecation issues, it should catch bad db
upgrades (as we've got content in the db), and it represents some rough
state of running an openstack cloud that includes data from an old state
of openstack.

There are plenty of things that could be improved, but it's managed to
catch a lot of basic issues that are huge pains for deployers rolling
openstack forward.

> 
> Thanks,
> Eoghan
> 
>>>>  * Trove
>>>>  * Sahara
>>>
>>> Icehouse is the first release with Trove (and Juno will be the first
>>> release with Sahara) so I think it's fine if they aren't included in
>>> upgrade testing yet.
>>>
>>> The fact that Ceilometer and Heat haven't caught up with upgrade testing
>>> during their N+1 integrated cycle is more of a problem, so I think it's
>>> a good idea to document it in the post-graduation requirements.
>>>
>>> --
>>> Thierry Carrez (ttx)
>>>
>>> _______________________________________________
>>> OpenStack-TC mailing list
>>> OpenStack-TC at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
> 
> _______________________________________________
> OpenStack-TC mailing list
> OpenStack-TC at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
> 


-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-tc/attachments/20140410/f5eeb4ed/attachment.pgp>