Open Stack

> > Thanks for raising this.
> > 
> > Can I clarify one related point, whether grenade is intended
> > primarily to catch intrinsic or extrinsic upgrade issues?
> > 
> > Or both?
> > 
> > Intrinsic in this context would for example be service A failing
> > post-upgrade due to a bad DB schema migration.
> > 
> > Whereas extrinsic would for example be service A failing post-
> > upgrade because some dependency/assumption with respect to
> > service B is now broken.
> > 
> > I ask because ceilometer when used with a "schema free" storage
> > driver may be more likely to suffer from extrinsic issues on
> > upgrade as opposed to intrinsic (e.g. nova changes a notification
> > payload between versions, as opposed to a failed db-sync as that
> > wouldn't even arise).
> > 
> > But surely such integration issues should have been caught in
> > advance via the normal Tempest testing of master-versus-master,
> > and wouldn't necessarily require the big-bang stable-to-master
> > approach of grenade to smoke out?
> > 
> > Or am I missing something obvious here?
> 
> I will in no way say we catch all upgrade issues in grenade, but let me
> tell you what we expect to work, and are testing against.
> 
> In grenade we bring up stable/last version of OpenStack, exercise it
> with Tempest (smoke subset), and then specifically create some
> additional resources we expect to survive the upgrade. We then upgrade
> the *code* to master. The configuration is left at the stable/last
> version, as we should have 1 cycle deprecation on configuration. We run
> a series of specific upgrade scripts (which in the base case do
> db-sync), then check for the expected resources to still be there, and
> complete with another Tempest smoke run.
> 
> This should catch config deprecation issues, it should catch bad db
> upgrades (as we've got content in the db), and it represents some rough
> state of running an openstack cloud that includes data from an old state
> of openstack.

Thanks for the clarification, IIUC sounds like grenade is best suited to
three particular potential upgrade issues:

1. forward compatibility of configuration

2. schema migrations applied to a non-empty DB

3. survivability of pre-existing resources

So ceilometer will benefit fully from #1 like any other service,
may benefit from #2 depending on the choice of storage backend,
and will partially benefit from #3 (in the sense of an upgraded
ceilometer being capable of continuing to gather data on those
pre-existing resources, but not directly being responsible for
the resources surviving the upgrade).

That basically addresses my uncertainty around whether ceilometer
would benefit from participating in grenade-based upgrade testing.

Cheers,
Eoghan
 
> There are plenty of things that could be improved, but it's managed to
> catch a lot of basic issues that are huge pains for deployers rolling
> openstack forward.