[openstack-dev] [magnum]problems for horizontal scale

王华 wanghua.humble at gmail.com
Thu Aug 13 07:28:18 UTC 2015


Hi Kai Qiang Wu,

I have some comments in line.

On Thu, Aug 13, 2015 at 1:32 PM, Kai Qiang Wu <wkqwu at cn.ibm.com> wrote:

> Hi Hua,
>
> I have some comments about this:
>
> A>
>  remove heat poller can be a way, but some of its logic needs to make sure
> it work and performance not burden.
> 1) for old heat poller it is quick loop, with fixed interval, to make sure
> stack status update quickly can be reflected in bay status
> 2) for periodic task running, it seems dynamic loop, and period is long,
> it was added for some stacks creation timeout, 1) loop exit, this 2) loop
> can help update the stack and also conductor crash issue
>
 It is not necessary to remove heat poller, so we can keep it.

>
>
>
> It would be ideal to put in one place for looping over the stacks, but
> periodic tasks need to consider if it really just need to loop
> IN_PROGRESS status stack ? And what's the interval for loop that ? (60s or
> short, loop performance)
>
It is necessary to loop IN_PROGRESS status stack for conductor crash issue.

>
>
> Does heat have other status transition  path, like delete_failed -->
> (status reset) --> become OK.  etc.
>
 It needs to be made sure.

>
>
>
>
> B> For remove db operation in bay_update case. I did not understand your
> suggestion.
> bay_update include update_stack and poll_and_check(it is in heat poller),
> if you removed heat poller to periodic task(as you said in your 3). It
> still needs db operations.
>
> Race conditions occur in periodic tasks too.  If we save the stack params
such as node_count in bay_update and race condition occurs, then the
node_count in db is wrong and the status is UPDATE_COMPLETE. And there is
no way to correct it.
If we save stack params in periodic tasks and race condition occurs, the
node_count in db is still wrong and status is UPDATE_COMPLETE. We can
correct it in the next periodic task if race condition does not occur. The
solution I proposed can not promise the data in db is always right.

>
>
> C> For allow admin user to show stacks in other tenant, it seems OK. Does
> other projects try this before? Is it reasonable case for customer ?
>
> Nova allow admin user to show instances in other tenant. Neutron allow
> admin user to show ports in other tenant, nova uses it to sync up network
> info for instance from neutron.
>
> Thanks
>
>
> Best Wishes,
>
> --------------------------------------------------------------------------------
> Kai Qiang Wu (吴开强  Kennan)
> IBM China System and Technology Lab, Beijing
>
> E-mail: wkqwu at cn.ibm.com
> Tel: 86-10-82451647
> Address: Building 28(Ring Building), ZhongGuanCun Software Park,
>         No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China
> 100193
>
> --------------------------------------------------------------------------------
> Follow your heart. You are miracle!
>
> [image: Inactive hide details for 王华 ---08/13/2015 11:31:53 AM---any
> comments on this? On Wed, Aug 12, 2015 at 2:50 PM, 王华 <wan]王华
> ---08/13/2015 11:31:53 AM---any comments on this? On Wed, Aug 12, 2015 at
> 2:50 PM, 王华 <wanghua.humble at gmail.com> wrote:
>
> From: 王华 <wanghua.humble at gmail.com>
> To: openstack-dev at lists.openstack.org
> Date: 08/13/2015 11:31 AM
> Subject: Re: [openstack-dev] [magnum]problems for horizontal scale
> ------------------------------
>
>
>
> any comments on this?
>
> On Wed, Aug 12, 2015 at 2:50 PM, 王华 <*wanghua.humble at gmail.com*
> <wanghua.humble at gmail.com>> wrote:
>
>    Hi All,
>
>    In order to prevent race conditions due to multiple conductors, my
>    solution is as blew:
>    1. remove the db operation in bay_update to prevent race
>    conditions.Stack operation is atomic. Db operation is atomic. But the two
>    operations together are not atomic.So the data in the db may be wrong.
>    2. sync up stack status and stack parameters(now only node_count) from
>    heat by periodic tasks. bay_update can change stack parameters, so we need
>    to sync up them.
>    3. remove heat poller, because we have periodic tasks.
>
>    To sync up stack parameters from heat, we need to show stacks using
>    admin_context. But heat don't allow to show stacks in other tenant. If we
>    want to show stacks in other tenant, we need to store auth context for
>    every bay. That is a problem. Even if we store the auth context, there is a
>    timeout for token. The best way I think is to let heat allow admin user to
>    show stacks in other tenant.
>
>    Do you have a better solution or any improvement for my solution?
>
>    Regards,
>    Wanghua
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150813/0223d9a3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150813/0223d9a3/attachment.gif>


More information about the OpenStack-dev mailing list