[openstack-dev] [nova][cinder] how to handle AZ bug 1496235?

Matt Riedemann mriedem at linux.vnet.ibm.com
Wed Sep 23 20:29:56 UTC 2015



On 9/23/2015 2:57 PM, Sylvain Bauza wrote:
>
>
> Le 23/09/2015 21:45, John Griffith a écrit :
>>
>>
>> On Wed, Sep 23, 2015 at 1:34 PM, Matt Riedemann
>> <mriedem at linux.vnet.ibm.com <mailto:mriedem at linux.vnet.ibm.com>> wrote:
>>
>>
>>
>>     On 9/23/2015 2:15 PM, Matt Riedemann wrote:
>>
>>
>>
>>         On 9/23/2015 1:46 PM, Ivan Kolodyazhny wrote:
>>
>>             Hi Matt,
>>
>>             In Liberty, we introduced allow_availability_zone_fallback
>>             [1] option in
>>             Cinder config as fix for bug [2]. If you set this option,
>>             Cinder will
>>             create volume in a default AZ instead of set volume into
>>             the error state
>>
>>             [1]
>>             https://github.com/openstack/cinder/commit/b85d2812a8256ff82934d150dbc4909e041d8b31
>>
>>             [2] https://bugs.launchpad.net/cinder/+bug/1489575
>>
>>             Regards,
>>             Ivan Kolodyazhny
>>
>>             On Wed, Sep 23, 2015 at 9:00 PM, Matt Riedemann
>>             <mriedem at linux.vnet.ibm.com
>>             <mailto:mriedem at linux.vnet.ibm.com>
>>             <mailto:mriedem at linux.vnet.ibm.com
>>             <mailto:mriedem at linux.vnet.ibm.com>>> wrote:
>>
>>                 I came across bug 1496235 [1] today.  In this case the
>>             user is
>>                 booting an instance from a volume using source=image,
>>             so nova
>>                 actually does the volume create call to the volume
>>             API.  They are
>>                 booting the instance into a valid nova availability
>>             zone, but that
>>                 same AZ isn't defined in Cinder, so the volume create
>>             request fails
>>                 (since nova passes the instance AZ to cinder [2]).
>>
>>                 I marked this as invalid given how the code works.
>>
>>                 I'm posting here since I'm wondering if there are
>>             alternatives worth
>>                 pursuing.  For example, nova could get the list of AZs
>>             from the
>>                 volume API and if the nova AZ isn't in that list,
>>             don't provide it
>>                 on the volume create request.  That's essentially the
>>             same as first
>>                 creating the volume outside of nova and not specifying
>>             an AZ, then
>>                 when doing the boot from volume, provide the volume_id
>>             as the source.
>>
>>                 The question is, is it worth doing that?  I'm not
>>             familiar enough
>>                 with how availability zones are meant to work between
>>             nova and
>>                 cinder so it's hard for me to have much of an opinion
>>             here.
>>
>>                 [1] https://bugs.launchpad.net/nova/+bug/1496235
>>                 [2]
>>
>>             https://github.com/openstack/nova/blob/master/nova/virt/block_device.py#L381-L383
>>
>>
>>                 --
>>
>>                 Thanks,
>>
>>                 Matt Riedemann
>>
>>
>>
>>             __________________________________________________________________________
>>
>>                 OpenStack Development Mailing List (not for usage
>>             questions)
>>                 Unsubscribe:
>>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>>             __________________________________________________________________________
>>
>>             OpenStack Development Mailing List (not for usage questions)
>>             Unsubscribe:
>>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>         Sorry but that seems like a hack.
>>
>>         I'm trying to figure out the relationship between AZs in nova
>>         and cinder
>>         and so far no one seems to really know.  In the cinder IRC
>>         channel I was
>>         told there isn't one, which would mean we shouldn't even try
>>         creating
>>         the volume using the server instance AZ.
>>
>>         Also, if there is no relationship, I was trying to figure out
>>         why there
>>         is the cinder.cross_az_attach config option.  That was added
>>         in grizzly
>>         [1].  I was thinking maybe it was a legacy artifact from
>>         nova-volume,
>>         but that was dropped in grizzly.
>>
>>         So is cinder.cross_az_attach even useful?
>>
>>         [1] https://review.openstack.org/#/c/21672/
>>
>>
>>     The plot thickens.
>>
>>     I was checking to see what change was made to start passing the
>>     server instance az on the volume create call during boot from
>>     volume, and that was [1] which was added in kilo to fix a bug
>>     where boot from volume into a nova az will fail if
>>     cinder.cross_az_attach=False and storage_availability_zone is set
>>     in cinder.conf.
>>
>>     So I guess we can't just stop passing the instance az to the
>>     volume create call.
>>
>>     But what I'd really like to know is how this is all used between
>>     cinder and nova, or was this all some work done as part of a
>>     larger effort that was never completed? Basically, can we
>>     deprecate the cinder.cross_az_attach config option in nova and
>>     start decoupling this code?
>>
>>     [1] https://review.openstack.org/#/c/157041/
>>
>>
>>     --
>>
>>     Thanks,
>>
>>     Matt Riedemann
>>
>>
>>     __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe:
>>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> ​To be honest this is probably my fault, AZ's were pulled in as part
>> of the nova-volume migration to Cinder and just sort of died. Quite
>> frankly I wasn't sure "what" to do with them but brought over the
>> concept and the zones that existing in Nova-Volume.  It's been an
>> issue since day 1 of Cinder, and as you note there are little hacks
>> here and there over the years to do different things.
>>
>> I think your question about whether they should be there at all or not
>> is a good one.  We have had some interest from folks lately that want
>> to couple Nova and Cinder AZ's (I'm really not sure of any details or
>> use-cases here).
>>
>> My opinion would be until somebody proposes a clear use case and need
>> that actually works that we consider deprecating it.
>>
>
> Given what are currently AZs in Nova (explicit user-visible aggregates
> of compute nodes, see [1]), I tend to say that there is no reason to
> provide the AZ information to Cinder because AZs are not failure
> domains, just very specific placement information used for scheduling
> instances.
>
> Based on some conversation we had on IRC, MHO is that we should at least
> deprecate the config flag that Matt discussed above and we should also
> plan to remove the AZ information from the Cinder call we do in Nova
> when creating a volume.
>
> I leave Cinder community to discuss the plans about having AZs or not,
> but I also think that any reboot of the idea could necessarly need a
> renaming and no longer use the "availability zone" wording.
>
>
> [1]
> http://docs.openstack.org/developer/nova/aggregates.html#availability-zones-azs
>
>> While we're on the subject (kinda) I've never been a very fond of
>> having Nova create the volume during boot process either; there's a
>> number of things that go wrong here (timeouts almost guaranteed for a
>> "real" image) and some things that are missing last I looked like type
>> selection etc.
>>
>> We do have a proposal to talk about this at the Summit, so maybe we'll
>> have a descent primer before we get there :)
>>
>> Thanks,
>>
>> John
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Here is the change to deprecate the cinder.cross_az_attach option:

https://review.openstack.org/#/c/226977/

-- 

Thanks,

Matt Riedemann




More information about the OpenStack-dev mailing list