Open Stack

We're using the v2 API at the moment.

The overall approach we've taken is to do the migration with identical function as the first step. Additional features can then be tested independently as smaller steps with clearer rollback. The concurrent version API features help greatly with this approach so we don't have simultaneous client/server upgrades.

We're testing the v3 API at the moment and will start to use it in the near future. The domains features in particular are of great interest for us.

Tim

> -----Original Message-----
> From: Jay Pipes [mailto:jaypipes at gmail.com]
> Sent: 24 January 2014 01:25
> To: openstack-operators at lists.openstack.org
> Subject: Re: [Openstack-operators] Keystone Grizzly to Havana upgrade
> 
> Hi Tim, sorry for top-posting, but I'm curious whether you are using the
> v2 Keystone API or the v3 Keystone API (and whether you migrated API versions during your G -> H migration.
> 
> Best,
> -jay
> 
> On Thu, 2014-01-23 at 21:09 +0000, Tim Bell wrote:
> >
> >
> > Following on the sequence of quick notes for those doing Grizzly to
> > Havana upgrades.
> >
> >
> >
> > At CERN, we’ve just migrated Keystone from Grizzly to Havana today. It
> > actually happened much faster than we had originally intended!
> >
> >
> >
> > The overall approach taken was to try an online upgrade. We have a
> > keystone in each of our cells (so we can talk to each of them
> > independently of the top level cell API nodes) so these were good
> > candidates to upgrade first. We use Active Directory for the user
> > credentials so the Keystone service itself has a limited amount of
> > state only (related to EC2 credentials and token management).
> >
> >
> >
> > For the main keystone instance, we were aiming to benefit from the
> > load balancing layer and that there were minimal database changes in
> > Keystone. The largest problem foreseen was the EC2 credentials since
> > these, in Grizzly, are stored in a different column to those in Havana
> > (just Credentials). These columns had to be kept in sync with a script
> > during the phase where both versions were running.
> >
> >
> >
> > In practice, we performed the upgrade more rapidly than originally
> > planned as some of the clients gave errors when they had received a
> > token from a Grizzly keystone and were authenticating against a Havana
> > one. Thus, we upgraded all the keystone servers as soon as the final
> > functional tests in the production environment were completed on the
> > first keystone.
> >
> >
> >
> > The detailed error message of the error on a Grizzly client was
> >
> >
> >
> > 2014-01-23 09:10:02    ERROR [root] 'token'
> >
> > Traceback (most recent call last):
> >
> >   File "/usr/lib/python2.6/site-packages/keystone/common/wsgi.py",
> > line 265, in __call__
> >
> >     result = method(context, **params)
> >
> >   File
> > "/usr/lib/python2.6/site-packages/keystone/token/controllers.py", line
> > 541, in validate_token
> >
> >     self._assert_default_domain(context, token_ref)
> >
> >   File
> > "/usr/lib/python2.6/site-packages/keystone/token/controllers.py", line
> > 482, in _assert_default_domain
> >
> >     if (token_ref['token_data']['token']['user']['domain']['id'] !=
> >
> > KeyError: 'token'
> >
> >
> >
> > The solution was to complete the upgrade to Keystone on all the
> > servers.
> >
> >
> >
> > Other than that, things are running fine and now we can start the work
> > to take advantage of the new features.
> >
> >
> >
> > Tim
> >
> >
> >
> > From: Tim Bell
> > Sent: 15 January 2014 20:23
> > To: openstack-operators at lists.openstack.org
> > Subject: Glance Grizzly to Havana upgrade
> >
> >
> >
> >
> >
> >
> > Quick warning to those doing the Grizzly to Havana migration,
> >
> >
> >
> > Our upgrade went smoothly (around 45 minutes to complete).
> >
> >
> >
> > Following the migration, we found that nova image-list and glance
> > image-list were showing different results.
> >
> >
> >
> > The root cause was related to the bug
> > https://bugs.launchpad.net/glance/+bug/1152716. The fix for the
> > problem was to add the policy statement for the parameter
> > context_is_admin which is needed to limit access to private images for
> > projects.
> >
> >
> >
> > Tim
> >
> >
> >
> > --
> >
> >
> >
> >
> > _______________________________________________
> > OpenStack-operators mailing list
> > OpenStack-operators at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operator
> > s
> 
> 
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators