[Magnum][kolla-ansible][kayobe] Information gathering for 2 blocking issues
feilong
feilong at catalyst.net.nz
Fri Oct 9 17:27:56 UTC 2020
Hi Tony,
Firstly, please reminder me what's your Magnum version. I would suggest
use stable/victoria or at least stable/train. As for the output you posted:
| eaccb23e-226b-4a36-a2a5-5b9d9bbb4fea | delta-23-cluster-p4khbhat4sgp-node-2 | ACTIVE | private=10.0.0.250, 192.168.20.71 | Fedora-Atomic-27 | kube_node_10gb |
| c7d42436-7c27-4a11-897e-d09eb716e9b9 | delta-23-cluster-p4khbhat4sgp-node-0 | ACTIVE | private=10.0.0.88, 192.168.20.52 | Fedora-Atomic-27 | kube_node_10gb |
Given you can still see the image name from the OpenStack instance list
command, that means the config "boot_volume_size = 10" is not being used
for some reasons. In other words, I think I'm confident that the config
should resolve your local disk consuming issue. But there is another
issue which is preventing it works.
On 16/09/20 12:58 am, Tony Pearce wrote:
> Hi Ionut, thank you for your reply. Do you know if this configuration
> prevents consuming of local disk on the compute node for instance
> storage eg OS or swap etc?
>
> Kind regards
>
> On Tue, 15 Sep 2020, 20:53 Ionut Biru, <ionut at fleio.com
> <mailto:ionut at fleio.com>> wrote:
>
> Hi,
>
> To boot minions or master from volume, I use the following labels:
>
> boot_volume_size = 20
> boot_volume_type = ssd
> availability_zone = nova
>
> volume type and zone might differ on your setup.
>
>
>
> On Tue, Sep 15, 2020 at 11:23 AM Tony Pearce <tonyppe at gmail.com
> <mailto:tonyppe at gmail.com>> wrote:
>
> Hi Feilong, I hope you are keeping well.
>
> Thank you for sticking with me on this issue to try and help
> me here. I really appreciate it!
>
> I tried creating a new flavour like you suggested and using
> 10GB for root volume [1]. The cluster does start to be created
> (no error about 0mb disk) but while being created, I can check
> the compute node and see that the instance disk is being
> provisioned on the compute node [2]. I assume that this is the
> 10GB root volume that is specified in the flavour.
>
> When I list the volumes which have been created, I do not see
> the 10GB disk allocated on the compute node, but I do see the
> iSCSI network volume that has been created and attached to the
> instance (eg master) [3]. This is 15GB volume and this 15GB is
> coming from the kubernetes cluster template, under "Docker
> Volume Size (GB)" in the "node spec" section. There is very
> little data written to this volume at the time of master
> instance booted.
>
> Eventually, kube cluster failed to create with error "Status
> Create_Failed: Resource CREATE failed: Error:
> resources.kube_minions.resources[0].resources.node_config_deployment:
> Deployment to server failed: deploy_status_code: Deployment
> exited with non-zero status code: 1". I'll try and find the
> root cause of this later.
>
> What are your thoughts on this outcome? Is it possible to
> avoid consuming compute node disk? I require it because it
> cannot scale.
>
> [1] http://paste.openstack.org/show/797862/
> [2] http://paste.openstack.org/show/797865/
> [3] http://paste.openstack.org/show/797863/
>
> Kind regards,
> Tony
>
> Tony Pearce
>
>
>
> On Mon, 14 Sep 2020 at 17:44, feilong <feilong at catalyst.net.nz
> <mailto:feilong at catalyst.net.nz>> wrote:
>
> Hi Tony,
>
> Does your Magnum support this config
> https://github.com/openstack/magnum/blob/master/magnum/conf/cinder.py#L47
> can you try to change it from 0 to 10? 10 means the root
> disk volume size for the k8s node. By default the 0 means
> the node will be based on image instead of volume.
>
>
> On 14/09/20 9:37 pm, Tony Pearce wrote:
>> Hi Feilong, sure. The flavour I used has 2 CPU and 2GB
>> memory. All other values either unset or 0mb.
>> I also used the same fedora 27 image that is being used
>> for the kubernetes cluster.
>>
>> Thank you
>> Tony
>>
>> On Mon, 14 Sep 2020, 17:20 feilong,
>> <feilong at catalyst.net.nz
>> <mailto:feilong at catalyst.net.nz>> wrote:
>>
>> Hi Tony,
>>
>> Could you please let me know your flavor details? I
>> would like to test it in my devstack environment
>> (based on LVM). Thanks.
>>
>>
>> On 14/09/20 8:27 pm, Tony Pearce wrote:
>>> Hi feilong, hope you are keeping well. Thank you for
>>> the info!
>>>
>>> For issue 1. Maybe this should be with the
>>> kayobe/kolla-ansible team. Thanks for the insight :)
>>>
>>> For the 2nd one, I was able to run the HOT template
>>> in your link. There's no issues at all running that
>>> multiple times concurrently while using the 0MB disk
>>> flavour. I tried four times with the last three
>>> executing one after the other so that they ran
>>> parallelly. All were successful and completed and
>>> did not complain about the 0MB disk issue.
>>>
>>> Does this conclude that the error and create-failed
>>> issue relates to Magnum or could you suggest other
>>> steps to test on my side?
>>>
>>> Best regards,
>>>
>>> Tony Pearce
>>>
>>>
>>>
>>>
>>> On Thu, 10 Sep 2020 at 16:01, feilong
>>> <feilong at catalyst.net.nz
>>> <mailto:feilong at catalyst.net.nz>> wrote:
>>>
>>> Hi Tony,
>>>
>>> Sorry for the late response for your thread.
>>>
>>> For you HTTPS issue, we (Catalyst Cloud) are
>>> using Magnum with HTTPS and it works.
>>>
>>> For the 2nd issue, I think we were
>>> misunderstanding the nodes disk capacity. I was
>>> assuming you're talking about the k8s nodes, but
>>> seems you're talking about the physical compute
>>> host. I still don't think it's a Magnum issue
>>> because a k8s master/worker nodes are just
>>> normal Nova instances and managed by Heat. So I
>>> would suggest you use a simple HOT to test it,
>>> you can use this
>>> https://gist.github.com/openstacker/26e31c9715d52cc502397b65d3cebab6
>>>
>>> Most of the cloud providers or organizations who
>>> have adopted Magnum are using Ceph as far as I
>>> know, just FYI.
>>>
>>>
>>> On 10/09/20 4:35 pm, Tony Pearce wrote:
>>>> Hi all, hope you are all keeping safe and well.
>>>> I am looking for information on the following
>>>> two issues that I have which surrounds Magnum
>>>> project:
>>>>
>>>> 1. Magnum does not support Openstack API with HTTPS
>>>> 2. Magnum forces compute nodes to consume disk
>>>> capacity for instance data
>>>>
>>>> My environment: Openstack Train deployed using
>>>> Kayobe (Kolla-ansible).
>>>>
>>>> With regards to the HTTPS issue, Magnum stops
>>>> working after enabling HTTPS because the
>>>> certificate / CA certificate is not trusted by
>>>> Magnum. The certificate which I am using is one
>>>> that was purchased from GoDaddy and is trusted
>>>> in web browsers (and is valid), just not
>>>> trusted by the Magnum component.
>>>>
>>>> Regarding compute node disk consumption issue -
>>>> I'm at a loss with regards to this and so I'm
>>>> looking for more information about why this is
>>>> being done and is there any way that I could
>>>> avoid it? I have storage provided by a Cinder
>>>> integration and so the consumption of compute
>>>> node disk for instance data I need to avoid.
>>>>
>>>> Any information the community could provide to
>>>> me with regards to the above would be much
>>>> appreciated. I would very much like to use the
>>>> Magnum project in this deployment for
>>>> Kubernetes deployment within projects.
>>>>
>>>> Thanks in advance,
>>>>
>>>> Regards,
>>>>
>>>> Tony
>>>
>>> --
>>> Cheers & Best regards,
>>> Feilong Wang (王飞龙)
>>> ------------------------------------------------------
>>> Senior Cloud Software Engineer
>>> Tel: +64-48032246
>>> Email: flwang at catalyst.net.nz <mailto:flwang at catalyst.net.nz>
>>> Catalyst IT Limited
>>> Level 6, Catalyst House, 150 Willis Street, Wellington
>>> ------------------------------------------------------
>>>
>> --
>> Cheers & Best regards,
>> Feilong Wang (王飞龙)
>> ------------------------------------------------------
>> Senior Cloud Software Engineer
>> Tel: +64-48032246
>> Email: flwang at catalyst.net.nz <mailto:flwang at catalyst.net.nz>
>> Catalyst IT Limited
>> Level 6, Catalyst House, 150 Willis Street, Wellington
>> ------------------------------------------------------
>>
> --
> Cheers & Best regards,
> Feilong Wang (王飞龙)
> ------------------------------------------------------
> Senior Cloud Software Engineer
> Tel: +64-48032246
> Email: flwang at catalyst.net.nz <mailto:flwang at catalyst.net.nz>
> Catalyst IT Limited
> Level 6, Catalyst House, 150 Willis Street, Wellington
> ------------------------------------------------------
>
>
>
> --
> Ionut Biru - https://fleio.com
>
--
Cheers & Best regards,
Feilong Wang (王飞龙)
------------------------------------------------------
Senior Cloud Software Engineer
Tel: +64-48032246
Email: flwang at catalyst.net.nz
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20201010/509ac007/attachment-0001.html>
More information about the openstack-discuss
mailing list