[openstack-dev] Openstack-Zun Service Appears down

Hongbin Lu hongbin034 at gmail.com
Thu Jun 21 08:14:11 UTC 2018


HI  Muhammad,

Here is the code (run in controller node) that decides whether a service is
up https://github.com/openstack/zun/blob/master/zun/api/servicegroup.py .
There are several possibilities to cause a service to be 'down':
1. The service was being 'force_down' via API (e.g. explicitly issued a
command like "appcontainer service forcedown")
2. The zun compute process is not doing the heartbeat for a certain period
of time (CONF.service_down_time).
3. The zun compute process is doing the heartbeat properly but the time
between controller node and compute node is out of sync.

In before, #3 is the common pitfall that people ran into. If it is not #3,
you might want to check if the zun compute process is doing the heartbeat
properly. Each zun compute process is running a periodic task to update its
state in DB:
https://github.com/openstack/zun/blob/master/zun/servicegroup/zun_service_periodic.py
. The call of ' report_state_up ' will record this service is up in DB at
current time. You might want to check if this periodic task is running
properly, or if the current state is updated in the DB.

Above is my best guess. Please feel free to follow it up with me or other
team members if you need further assistant for this issue.

Best regards,
Hongbin

On Wed, Jun 20, 2018 at 9:14 AM Usman Awais <usman.awais at gmail.com> wrote:

> Dear Zuners,
>
> I have installed RDO pike. I stopped openstack-nova-compute service on one
> of the hosts, and installed a zun-compute service. Although all the
> services seems to be running ok on both controller and compute but when I
> do
>
>  openstack appcontainer service list
>
> It gives me following
>
>
> +----+--------------+-------------+-------+----------+-----------------+---------------------+-------------------+
> | Id | Host         | Binary      | State | Disabled | Disabled Reason |
> Updated At          | Availability Zone |
>
> +----+--------------+-------------+-------+----------+-----------------+---------------------+-------------------+
> |  1 | node1.os.lab | zun-compute | down  | False    | None            |
> 2018-06-20 13:14:31 | nova              |
>
> +----+--------------+-------------+-------+----------+-----------------+---------------------+-------------------+
>
> I have checked all ports in both directions they are open, including etcd
> ports and others. All services are running, only docker service has the
> warning message saying "failed to retrieve docker-runc version: exec:
> \"docker-runc\": executable file not found in $PATH". There is also a
> message at zun-compute
> "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:161:
> SAWarning: The IN-predicate on "container.uuid" was invoked with an empty
> sequence. This results in a contradiction, which nonetheless can be
> expensive to evaluate.  Consider alternative strategies for improved
> performance."
>
> Please guide...
>
> Regards,
> Muhammad Usman Awais
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180621/2313b28b/attachment.html>


More information about the OpenStack-dev mailing list