kubernetes cluster stuck in CREATE_IN_PROGRESS but stack list shows CREATE_COMPLETE

Ammad Syed syedammad83 at gmail.com
Thu Sep 16 05:42:06 UTC 2021


Hello Tony,

I am suspecting that you are hitting the same bug as mine. You can see full
details of bug here.

https://storyboard.openstack.org/#!/story/2009141

Usually what happens it magnum-conductor service got crash randomly. You
can confirm this by executing the command:

openstack coe service list

You will see that conductor service is down, once you restart conductor
service, the cluster state will be updated to CREATE_COMPLETE.

Ammad

On Thu, Sep 16, 2021 at 10:02 AM Karera Tony <tonykarera at gmail.com> wrote:

> Hello Feilong,
>
> The Clusters are now Ok and they are all in CEATE_COMPLETE after
> restarting the heat and magnum containers.
>
> Thanks
>
> Regards
>
> Tony Karera
>
>
>
>
> On Wed, Sep 15, 2021 at 8:48 PM feilong <feilong at catalyst.net.nz> wrote:
>
>> Hi Tony,
>>
>> Good to see your cluster can be created "successfully" ;) Technically,
>> Magnum doesn't maintain the cluster status, the status is synced from Heat.
>> That said, if the stack is created successfully, but the magnum cluster is
>> in progress, that means something wrong when syncing the status. The issue
>> happened before, I just cannot remember the exact bug ID, you should be
>> able to find it by google. And here is the code,
>> https://github.com/openstack/magnum/blob/master/magnum/service/periodic.py#L159
>> you can just add some breakpoints to debug it, it shouldn't be hard.
>>
>>
>> On 15/09/21 8:54 pm, Karera Tony wrote:
>>
>> Dear Team,
>>
>> I have configured Magnum and I am able to kubernetes clusters and when I
>> log in to the nodes everything is OK.
>>
>> But Minus the first cluster I created which completed well as per the
>> openstack coe cluster list command and the output on Horizon, the rest of
>> the clusters get stack in CREATE_IN_PROGRESS .
>>
>> Please check the below commands output and advise.
>>
>> (kolla-openstack) stack at deployment:~$ openstack stack list
>>
>> +--------------------------------------+---------------------------+----------------------------------+-----------------+----------------------+--------------+
>> | ID                                   | Stack Name                |
>> Project                          | Stack Status    | Creation Time        |
>> Updated Time |
>>
>> +--------------------------------------+---------------------------+----------------------------------+-----------------+----------------------+--------------+
>> | e9bfad7c-c82e-4fc6-9f05-793b31631761 | k8s-cluster2-zsgfkrobjlw6 |
>> 456586218b26442ebeff03643684faed | CREATE_COMPLETE | 2021-09-15T08:01:50Z |
>> None         |
>> | 9dde1665-8e50-4482-a986-810eb2a9cbaa | k8s-cluster1-eogcos7ee3ub |
>> 456586218b26442ebeff03643684faed | CREATE_COMPLETE | 2021-09-10T07:27:41Z |
>> None         |
>>
>> +--------------------------------------+---------------------------+----------------------------------+-----------------+----------------------+--------------+
>> (kolla-openstack) stack at deployment:~$ openstack coe cluster list
>>
>> +--------------------------------------+--------------+---------+------------+--------------+--------------------+---------------+
>> | uuid                                 | name         | keypair |
>> node_count | master_count | status             | health_status |
>>
>> +--------------------------------------+--------------+---------+------------+--------------+--------------------+---------------+
>> | 277e0047-4ae1-4006-869e-a824cc602bcc | k8s-cluster1 | Newkey  |
>>  2 |            1 | CREATE_COMPLETE    | HEALTHY       |
>> | 4336f9d9-9bea-477f-bc13-c16ff0f7e141 | k8s-cluster2 | Newkey  |
>>  2 |            1 | CREATE_IN_PROGRESS | None          |
>>
>> +--------------------------------------+--------------+---------+------------+--------------+--------------------+---------------+
>> Regards
>>
>> Tony Karera
>>
>>
>> --
>> Cheers & Best regards,
>> ------------------------------------------------------------------------------
>> Feilong Wang (王飞龙) (he/him)
>> Head of Research & Development
>>
>> Catalyst Cloud
>> Aotearoa's own
>>
>> Mob: +64 21 0832 6348 | www.catalystcloud.nz
>> Level 6, 150 Willis Street, Wellington 6011, New Zealand
>>
>> CONFIDENTIALITY NOTICE: This email is intended for the named recipients only.
>> It may contain privileged, confidential or copyright information. If you are
>> not the named recipient, any use, reliance upon, disclosure or copying of this
>> email or its attachments is unauthorised. If you have received this email in
>> error, please reply via email or call +64 21 0832 6348.
>> ------------------------------------------------------------------------------
>>
>>

-- 
Regards,


Syed Ammad Ali
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210916/067420c3/attachment.htm>


More information about the openstack-discuss mailing list