Open Stack

Hi,

I did some investigation last week, why this was happening before I file
the bug. Here are my observations and I would like to work on it. Any
guidance is appreciated.


Upgrade procedure:
In my setup I have openstack controller on one node, neutron on another
node and rest of the nodes are computes. Assume they are running juno. As
the first step,
a) Bringup up a new openstack node in kilo
b) Migrate database from juno to kilo.
c) Run xxx manage db sync commands.
d) Set upgrade levels to juno
e) Migrate neutron and computes to the newer controller.

second step.
a) Upgrade neutron, and then start compute upgrades one at a time.

third step,
a) Finalize the upgrade by clearing upgrade levels and restarting services.
b) Issue migrate flavor data command.

This procedure is openstack side by side upgraded and neutron and computes
inplace upgraded.

Issue:
VMs spawned after step1 and during step2 are not flavor migrated. So,
migration to liberty was failing.

Investigation:
flavor migrate command filters instance uuids in nova.instance_extra table
whose 'flavor' column is NULL and fill in with the necessary flavor
information and delete those respective rows from the
nova.instance_system_metadata associated with that VM.
For the VMs spawned after the first step and before step2, this flavor
information in nova.instance_extra and also respective flavor information
in nova.instance_system_metadata table.
Since the filter looks for 'flavor ' column as NULL, these instances are
not caught to delete entries in nova.instance_system_metadata. And presence
of this data is failing nova db sync while migrating to liberty.

Possible Solutions:
I feel we should broaden the filter so that instances spawned after step1
and during step 2 are also accounted for data translation. As a quick
verification, I tried this by having no filer and it worked. I know filter
give efficiency, but would it matter in this context or should we broaden
the filter?

Please let me know if I am going in the right direction.

thanks




On Thu, Sep 1, 2016 at 10:28 AM, Matt Riedemann <mriedem at linux.vnet.ibm.com>
wrote:

> On 9/1/2016 12:05 PM, Suresh Vinapamula wrote:
>
>> On 8/31/2016 4:17 PM, Dan Smith wrote:
>>
>>         Thanks Dan for your response. While I do run that before I start
>> my
>>         move to liberty, what I see is that it doesn't seem to flavor
>> migrate
>>         meta data for the VMs that are spawned after controller upgrade
>> from
>>         juno to kilo and before all computes upgraded from juno to kilo.
>> The
>>         current work around is to delete those VMs that are spawned after
>>         controller upgrade and before all computes upgrade, and then
>> initiate
>>         liberty upgrade. Then it works fine.
>>
>>     I can't think of any reason why that would be, or why it would be a
>>     problem. Instances created after the controllers are upgraded should
>> not
>>     have old-style flavor info, so they need not be touched by the
>> migration
>>     code.
>>
>>     Maybe filing a bug is in order describing what you see?
>>
>>     --Dan
>>
>>     ____________________________________________________________
>> ______________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe: openstack-dev-requ... at lists.op
>> enstack.org?subject:unsubscribe
>>     <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> Also, are you running with the latest kilo patch update? There were
>> some bug fixes backported after the release from what I remember.
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>> Hi Matt,
>>
>> Thanks for the response. I am currently using 2015.1.2. From the release
>> notes of 2015.1.4 https://wiki.openstack.org/wiki/ReleaseNotes/2015.1.4,
>> I can't correlate any fixes with this issue. Am I missing something?
>>
>> thanks
>>
>> Suresh
>>
>>
> The things I was looking for look like they are in 2015.1.2 so you should
> be OK there for at least known issues.
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161003/5df443fc/attachment.html>