[openstack-dev] [Magnum] API service won't work if conductor down?

王华 wanghua.humble at gmail.com
Thu Feb 4 07:42:29 UTC 2016


I think we should allow magnum-api to access DB directly like nova-api.

As describe in [1], nova may have many compute nodes and it may take an
hour or a month to upgrade. But the number of magnum-api and
magnum-conductor is limited, the upgrade of them is fast. They don't
benefit from the method. We should upgrade them like the control services
in nova and upgrade them together.

In this step, you will upgrade everything but the compute nodes. This means
nova-api, nova-scheduler, nova-conductor, nova-consoleauth, nova-network,
and nova-cert. In reality, this needs to be done fairly atomically. So,
shut down all of the affected services, roll the new code, and start them
back up. This will result in some downtime for your API, but in reality, it
should be easy to quickly perform the swap. In later releases, we’ll reduce
the pain felt here by eliminating the need for the control services to go
together.

[1]
http://www.danplanet.com/blog/2015/06/26/upgrading-nova-to-kilo-with-minimal-downtime/


On Thu, Feb 4, 2016 at 4:59 AM, Hongbin Lu <hongbin.lu at huawei.com> wrote:

> I can clarify Eli’s question further.
>
>
>
> 1) is this by designed that we don't allow magnum-api to access DB
> directly ?
>
> Yes, that is what it is. Actually, The magnum-api was allowed to access DB
> directly in before. After the indirection API patch landed [1], magnum-api
> starts using magnum-conductor as a proxy to access DB. According to the
> inputs from oslo team, this design allows operators to take down either
> magnum-api or magnum-conductor to upgrade. This is not the same as
> nova-api, because nova-api, nova-scheduler, and nova-conductor are assumed
> to be shutdown all together as an atomic unit.
>
>
>
> I think we should make our own decision here. If we can pair magnum-api
> with magnum-conductor as a unit, we can remove the indirection API and
> allow both binaries to access DB. This could mitigate the potential
> performance bottleneck of message queue. On the other hand, if we stay with
> the current design, we would allow magnum-api and magnum-conductor to scale
> independently. Thoughts?
>
>
>
> [1] https://review.openstack.org/#/c/184791/
>
>
>
> Best regards,
>
> Hongbin
>
>
>
> *From:* Kumari, Madhuri [mailto:madhuri.kumari at intel.com]
> *Sent:* February-03-16 10:57 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [Magnum] API service won't work if
> conductor down?
>
>
>
> Corey the one you are talking about has changed to coe-service-*.
>
>
>
> Eli, IMO we should display proper error message. M-api service should only
> have read permission.
>
>
>
> Regards,
>
> Madhuri
>
>
>
> *From:* Corey O'Brien [mailto:coreypobrien at gmail.com
> <coreypobrien at gmail.com>]
> *Sent:* Wednesday, February 3, 2016 6:50 PM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev at lists.openstack.org>
> *Subject:* Re: [openstack-dev] [Magnum] API service won't work if
> conductor down?
>
>
>
> The service-* commands aren't related to the magnum services (e.g.
> magnum-conductor). The service-* commands are for services on the bay that
> the user creates and deletes.
>
>
>
> Corey
>
>
>
> On Wed, Feb 3, 2016 at 2:25 AM Eli Qiao <liyong.qiao at intel.com> wrote:
>
> hi
> Whey I try to run magnum service-list to list all services (seems now we
> only have m-cond service), it m-cond is down(which means no conductor at
> all),
> API won't response and will return a timeout error.
>
> taget at taget-ThinkStation-P300:~/devstack$ magnum service-list
> ERROR: Timed out waiting for a reply to message ID
> fd1e9529f60f42bf8db903bbf75bbade (HTTP 500)
>
> And I debug more and compared with nova service-list, nova will give
> response and will tell the conductor is down.
>
> and deeper I get this in magnum-api boot up:
>
>
> *     # Enable object backporting via the conductor
> base.MagnumObject.indirection_api = base.MagnumObjectIndirectionAPI()*
>
> so in magnum_service api code
>
>         return objects.MagnumService.list(context, limit, marker, sort_key,
>                                           sort_dir)
>
> will require to use magnum-conductor to access DB, but no magnum-conductor
> at all, then we get a 500 error.
> (nova-api doesn't specify *indirection_api so nova-api can access DB*)
>
> My question is:
>
> 1) is this by designed that we don't allow magnum-api to access DB
> directly ?
> 2) if 1) is by designed, then `magnum service-list` won't work, and the
> error message should be improved such as "magnum service is down , please
> check magnum conductor is alive"
>
> What do you think?
>
> P.S. I tested comment this line:
> *# base.MagnumObject.indirection_api = base.MagnumObjectIndirectionAPI()*
> magnum-api will response but failed to create bay(), which means api
> service have read access but can not write it at all since(all db write
> happened in conductor layer).
>
> --
>
> Best Regards, Eli(Li Yong)Qiao
>
> Intel OTC China
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160204/2f2b525c/attachment-0001.html>


More information about the OpenStack-dev mailing list