[openstack-dev] [neutron] - availability zone performance regression and discussion about added network field

Hirofumi Ichihara ichihara.hirofumi at lab.ntt.co.jp
Mon Dec 21 09:48:11 UTC 2015



On 2015/12/16 17:16, Kevin Benton wrote:
> What will the availability zones API tell the user about the zones? 
> Are they just opaque strings that the user doesn't really understand?
They shows available zones in the time.

>
> What I'm a little worried about is that it seems like we are having 
> the user doing the work of the scheduler.
I understand your worry. However, I think that AZ feature includes a use 
case that users can specify zone which their resource is scheduled.

>
> Is this is for getting affinity or anti-affinity with resources for 
> another network? If so, why not just have the user explicitly say in 
> the API request 'anti-affinity=network_id' or 'affinity=network_id'. 
> Then the scheduler would use the zones info to either place resources 
> on a different zone or the same zone, depending on which was requested.
>
I like it. But we may have other issues, for example,

1. We have NW1(with anti-affinity=NW2) and NW2(with anti-affinity=NW1)
2. We delete NW1 and then create NW3(with anti-affinity=NW2) instead of NW1
3. NW2(with anti-affinity=NW1) is rescheduled because of some reasons
4. Neutron cannot find NW1 in anti-affinit of NW2. How does neutron also 
schedule NW2 to a zone which doesn't have NW3?

Of course, we can find a way of solving this issue itself. But the 
similar issue may happen.

I think that we must remove the filed if it always happens performance 
issue.
However, we should find out another solution for the issue as long as 
there are use cases that are needed by operators and users.

> On Sun, Dec 13, 2015 at 11:25 PM, Hirofumi Ichihara 
> <ichihara.hirofumi at lab.ntt.co.jp 
> <mailto:ichihara.hirofumi at lab.ntt.co.jp>> wrote:
>
>
>
>     On 2015/12/14 15:58, Kevin Benton wrote:
>>     What decision would lead the user to request AZ1 and AZ2 in the
>>     first place? Especially since when it fails to get AZ2, they just
>>     request again with AZ1 and AZ3 instead.
>     I expected that user gets AZ1 and AZ2 (and AZ3) via GET
>     Availability zones API first. There is a gap between the time user
>     threw and the time his resource is scheduled. After user threw API
>     request with AZ1 and AZ2, if all agents of AZ2 are dead before
>     scheduling, the resource is scheduled in AZ1 only.
>
>
>
>>
>>     On Sun, Dec 13, 2015 at 10:31 PM, Hirofumi Ichihara
>>     <ichihara.hirofumi at lab.ntt.co.jp
>>     <mailto:ichihara.hirofumi at lab.ntt.co.jp>> wrote:
>>
>>
>>
>>         On 2015/12/14 14:52, Kevin Benton wrote:
>>>         I see, so regular users are supposed to use this information
>>>         as well. But how are they supposed to use it? For example,
>>>         if they see that their network has availability zones 1 and
>>>         4, but their instance is hosted in zone 3, what are they
>>>         supposed to do?
>>         I don't think that there is what they should do in the case
>>         because Neutron AZ is different from Nova AZ. For example,
>>         there may be a case like the following.
>>
>>         1. User throws POST Network API and Subnet API with
>>         availability_zone_hints [AZ1, AZ2]
>>         2. Neutron server tries to schedule the resource on both AZ1
>>         and AZ2 but the resource are scheduled on AZ1 only by some
>>         reasons
>>         3. User confirms via GET Network API where his resource is
>>         hosted and he knows it's AZ1 only
>>         4. User also can know AZ is ready via GET Availability zones
>>         API: AZ1, AZ3
>>         5. User deletes previous resource and he recreates his
>>         resource with availability_zone_hints [AZ1, AZ3]
>>
>>
>>>
>>>         On Sun, Dec 13, 2015 at 8:57 PM, Hirofumi Ichihara
>>>         <ichihara.hirofumi at lab.ntt.co.jp
>>>         <mailto:ichihara.hirofumi at lab.ntt.co.jp>> wrote:
>>>
>>>             Hi Kevin,
>>>
>>>             On 2015/12/14 11:10, Kevin Benton wrote:
>>>>             Hi all,
>>>>
>>>>             The availability zone code added a new field to the
>>>>             network API that shows the availability zones of a
>>>>             network. This caused a pretty big performance impact to
>>>>             get_networks calls because it resulted in a database
>>>>             lookup for every network.[1]
>>>>
>>>>             I already put a patch up to join the information ahead
>>>>             of time in the network model.[2]
>>>             I agree with your suggestion. I believe that the patch
>>>             can solve the performance issue.
>>>
>>>>             However, before we go forward with that, I think we
>>>>             should consider the removal of that field from the API.
>>>>
>>>>             Having to always join to the DHCP agents table to
>>>>             lookup which zones a network has DHCP agents on is
>>>>             expensive and is duplicating information available with
>>>>             other API calls.
>>>>
>>>>             Additionally, the field is just called
>>>>             'availability_zones' but it's being derived solely from
>>>>             AZ definitions in DHCP agent bindings for that network.
>>>>             To me that doesn't represent where the network is
>>>>             available, it just says which zones its scheduled DHCP
>>>>             instances live in. If that's the purpose, then we
>>>>             should just be using the DHCP agent API for this info
>>>>             and not impact the network API.
>>>             I don't think so. I have three points.
>>>
>>>             1. Availability zone is implemented in just a case with
>>>             Agent now, but it's reference implementation. For
>>>             example, we should expect that availability zone will be
>>>             used by plugin without agent.
>>>
>>>             2. In users view, availability zone is related to
>>>             network resource. On the other hand, users doesn't need
>>>             to consider Agent or operators doesn't like to enable
>>>             users to do in the first place. So I don't agree with
>>>             using Agent API.
>>>
>>>             3. We should consider whether users want to know the
>>>             field. Originally, the field doesn't exist in Spec[3]
>>>             but I added it according with reviewer's opinion(maybe
>>>             Akihiro?). This is about discussion of use case. After
>>>             users create resources via API with
>>>             availability_zone_hints so that they achieve HA for
>>>             their service, they want to know which zones are their
>>>             resources hosted on because their resources might not be
>>>             distributed on multiple availability zones by any
>>>             reasons. In the case, they need to know
>>>             "availability_zones" for the resources via Network API.
>>>
>>>             Thanks,
>>>             Hirofumi
>>>
>>>             [3]: https://review.openstack.org/#/c/169612/31
>>>
>>>>
>>>>             Thoughts?
>>>>
>>>>             1. https://bugs.launchpad.net/neutron/+bug/1525740
>>>>             2. https://review.openstack.org/#/c/257086/
>>>>
>>>>             -- 
>>>>             Kevin Benton
>>>>
>>>>
>>>>             __________________________________________________________________________
>>>>             OpenStack Development Mailing List (not for usage questions)
>>>>             Unsubscribe:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>             <mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>             __________________________________________________________________________
>>>             OpenStack Development Mailing List (not for usage questions)
>>>             Unsubscribe:
>>>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>>         -- 
>>>         Kevin Benton
>>>
>>>
>>>         __________________________________________________________________________
>>>         OpenStack Development Mailing List (not for usage questions)
>>>         Unsubscribe:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>         __________________________________________________________________________
>>         OpenStack Development Mailing List (not for usage questions)
>>         Unsubscribe:
>>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>         <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>>     -- 
>>     Kevin Benton
>>
>>
>>     __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> -- 
> Kevin Benton
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151221/5fcdf449/attachment.html>


More information about the OpenStack-dev mailing list