April 2020 - openstack-discuss - lists.openstack.org

[neutron] CI meeting cancelled
by Slawek Kaplonski 29 Apr '20

29 Apr '20

Hi, As 1st of May is Labor Day and public holiday in Poland, I will not be available to chair drivers meeting. Lets cancel it this week. See You all on the meeting next week. Have a great (long) weekend :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[TC] Ussuri Virtual Celebration!
by Kendall Nelson 29 Apr '20

29 Apr '20

Hello :) So! Since we won't be able to see each other in June and celebrate the recent release together, is there interest in doing a virtual cheers as a community? We could do something fun like.. trivia or a baking contest or something? I am reaching out to the TC specifically because we would probably want to do more than one of these and spread across all timezones so we could have a TC member or two 'host' the cheers. Can't wait to hear what you all think! -Kendall (diablo_rojo)

9 12

[openstack-helm-infra][elasticsearch]Error deploying chart in k8s cluster with local storage
by Sanjay K 29 Apr '20

29 Apr '20

Hello everyone, I have been trying to deploy elasticsearch helm chart in my k8s cluster consisting of 4 nodes (1 k8s master, 1 controller and 2 computes) after deploying fluentbit and fluentd helm charts which are deployed with success. I have plan to make the EFK (Elasticsearch-Fluentd-Kibana) stack running in my k8s cluster setup for logs management. However I face issues while deploying elasticsearch helm chart using the below command : $ helm install elasticsearch local/elasticsearch --values values_overrides/local-storage.yaml -n openstack Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(StatefulSet.spec): unknown field "strategy" in io.k8s.api.apps.v1.StatefulSetSpec Can anyone point me if I missing anything? What are the prerequisites before deploying elasticsearch chart which intended to use local storage/pv in my case avoiding the default S3 storage as per the chart? Thanks for your pointers and suggestions. ' best regards, SanjayK

2 7

[requirements][all] - requirements is now frozen for non-critical updates
by Matthew Thode 29 Apr '20

29 Apr '20

After the Ussuri-3 milestone, only critical requirements and constraints changes will be allowed. Freezing our requirements list gives packagers downstream an opportunity to catch up and prepare packages for everything necessary for distributions of the upcoming release. The requirements remain frozen until the stable branches are created, with the release candidates. All future changes not previously approved will need a FFE request set to the list, please prefix it with [FFE][requirements][PROJECTNAME]. -- Matthew Thode

1 1

Re: [TripleO][train][rdo] installation of undercloud fails during Run container-puppet tasks step1
by Ruslanas Gžibovskis 29 Apr '20

29 Apr '20

During installation, I can see them running fro 25sec max. not sure if it is as expected, Can I check somehow, how to get them up? if I do podman run -it Image_ID will it be enough to test if images are good? On Wed, 29 Apr 2020 at 12:01, Wu, Heng <wuh.fnst(a)cn.fujitsu.com> wrote: > > I am new to containers, can I somehow transfer these images between, as > I understand, it might help? > > > > The image will be downloaded automatically with the installation, you > don't need to do it manually. > > The problem seems to be that the container failed to start, not the errors > in the images themselves. > > You may need to check the configuration of undercloud.conf and proxy > settings (if they exist). > > > > *From:* Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > *Sent:* 2020年4月29日 15:48 > *To:* Alex Schultz <aschultz(a)redhat.com> > *Cc:* Openstack-discuss <openstack-discuss(a)lists.openstack.org>; > kent.martin(a)ericsson.com > *Subject:* Re: [TripleO][train][rdo] installation of undercloud fails > during Run container-puppet tasks step1 > > > > On the other installation I have, I see a bit more images: > > > > > I am new to containers, can I somehow transfer these images between, as I > understand, it might help? > > > > On Wed, 29 Apr 2020 at 09:45, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > I just now realized, that I have seen in some log, messages, missing some > puppet-dependencies... Cannot find log again... > > > > On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > podman ps -a = clean, no containers at all. > > I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df -h" > > > > paunch.log is empty. (I did several reinstallations). > > > > > > > Later I saw connectionpool retrying, but I have not seen " > tripleo_common.image.image_uploader" with same. > > > > > > > On Tue, 28 Apr 2020 at 20:10, Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > > > Hi all, > > > > I am running a fresh install of rdo train, on centos7 > > I almost a week I am facing error at this step: > > TASK [Run container-puppet tasks (generate config) during step 1] > > > > So I have ansible.log attached, I cannot find anything, where it is > failing. > > According to some understanding in ansible, it fails if it finds stderr > output. > > I cannot find error/fail or smth, I see Notices and Warnings, but I > believe it is not stderr? > > > > I see containers running and removed after some time. > > (as it should be I think)... > > > > Could you help me, where to dig? > > > > 2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for > step 1 using paunch] > > ***************************************************************************************************************************** > 2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020 > 22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********? > 2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u] > 2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for > task: Start containers for step 1] > > ******************************************************************************************************************* > 2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020 > 22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********? > 2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! => { > "failed_when_result": true,? > "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | > default([]))": [] > > Check /var/log/paunch.log. It probably has additional information as > to why the containers didn't start. You might also check the output > of 'sudo podman ps -a' to see if any containers exited with errors. > > > -- > > Ruslanas Gžibovskis > > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > -- Ruslanas Gžibovskis +370 6030 7030

1 1

[neutron] CI meeting
by Slawek Kaplonski 29 Apr '20

29 Apr '20

Hi, Sorry for late notice but I totally forgot about it. I need to cancel today's Neutron CI meeting. There is RH Summit going on and I have some sessions which I want to attend during our CI meeting. I checked grafana, and I don't see any urgent issues related to our CI so lets cancel this meeting and see You all next week :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[Neutron] Cancel neutron L3 meeting
by LIU Yulong 29 Apr '20

29 Apr '20

Hi there, Because I have to some local workload, I will not be able to run the L3 meeting tonight. And I'm going to take a vacation next week. So the next L3 meeting will be set on 13th May 2020. See you guys online then. Regards, LIU Yulong

1 0

Re: [TripleO][train][rdo] installation of undercloud fails during Run container-puppet tasks step1
by Ruslanas Gžibovskis 29 Apr '20

29 Apr '20

Moving output to pastebin I have found it very interesting, that I was able to download images by running some pull commands (below), podman images -a show it, but the installation still gets 401 errors. http://paste.debian.net/1143698/ > > On Wed, 29 Apr 2020 at 09:47, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > >> On the other installation I have, I see a bit more images: >> >> I am new to containers, can I somehow transfer these images between, as I >> understand, it might help? >> >> On Wed, 29 Apr 2020 at 09:45, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >> wrote: >> >>> I just now realized, that I have seen in some log, messages, missing >>> some puppet-dependencies... Cannot find log again... >>> >>> On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>> wrote: >>> >>>> podman ps -a = clean, no containers at all. >>>> I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df >>>> -h" >>>> >>>> paunch.log is empty. (I did several reinstallations). >>>> >>>> I found in image logs: >>>> 2020-04-29 08:52:49,854 140572 DEBUG urllib3.connectionpool [ ] >>>> https://registry-1.docker.io:443 "GET /v2/ HTTP/1.1" 401 87 >>>> 2020-04-29 08:52:49,855 140572 DEBUG >>>> tripleo_common.image.image_uploader [ ] >>>> https://registry-1.docker.io/v2/ status code 401 >>>> 2020-04-29 08:52:49,855 140572 DEBUG >>>> tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': >>>> 'repository:tripleotrain/centos-binary-zaqar-wsgi:pull', 'service': ' >>>> registry.docker.io'} >>>> >>>> Later I saw connectionpool retrying, but I have not seen " >>>> tripleo_common.image.image_uploader" with same. >>>> >>>> On Tue, 28 Apr 2020 at 20:10, Alex Schultz <aschultz(a)redhat.com> wrote: >>>> >>>>> On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>>>> wrote: >>>>> > >>>>> > Hi all, >>>>> > >>>>> > I am running a fresh install of rdo train, on centos7 >>>>> > I almost a week I am facing error at this step: >>>>> > TASK [Run container-puppet tasks (generate config) during step 1] >>>>> > >>>>> > So I have ansible.log attached, I cannot find anything, where it is >>>>> failing. >>>>> > According to some understanding in ansible, it fails if it finds >>>>> stderr output. >>>>> > I cannot find error/fail or smth, I see Notices and Warnings, but I >>>>> believe it is not stderr? >>>>> > >>>>> > I see containers running and removed after some time. >>>>> > (as it should be I think)... >>>>> > >>>>> > Could you help me, where to dig? >>>>> > >>>>> >>>>> 2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for >>>>> step 1 using paunch] >>>>> >>>>> ***************************************************************************************************************************** >>>>> 2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020 >>>>> 22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********? >>>>> 2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u] >>>>> 2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for >>>>> task: Start containers for step 1] >>>>> >>>>> ******************************************************************************************************************* >>>>> 2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020 >>>>> 22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********? >>>>> 2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! >>>>> => { >>>>> "failed_when_result": true,? >>>>> "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | >>>>> default([]))": [] >>>>> >>>>> Check /var/log/paunch.log. It probably has additional information as >>>>> to why the containers didn't start. You might also check the output >>>>> of 'sudo podman ps -a' to see if any containers exited with errors. >>>>> >>>>> -- Ruslanas Gžibovskis +370 6030 7030

1 0

[all][TC][PTL][release] Ussuri Community Meeting Proposal
by Allison Price 28 Apr '20

28 Apr '20

Hi everyone, As we prepare for the upcoming Ussuri release on May 13, I wanted to see if there was interest among the community to do a community meeting with folks presenting release highlights / answering questions that people may have. We were thinking that atleast one member from the TC or release management team could moderate the discussion and call on various PTLs to present what their teams accomplished during this cycle. If possible, we could do one that is around 1400 UTC to hit Europe and the United States and potentially another one around 0200 UTC for APAC, so we may need two volunteers. We would also like to promote Ussuri demos in the release comms, including the landing page and promotional materials. If you are planning to do a demo or would like to, please let me know and we can include it on the Ussuri landing page. Let me know what y’all think about doing a community meeting and if particular folks would like to volunteer to moderate and / or present. Looking forward to hearing feedback, and, in particular, if anyone is interested in volunteering to help moderate or participate in the Ussuri focused community meeting. Thanks! Allison Allison Price OpenStack Foundation allison(a)openstack.org <mailto:allison@openstack.org>

7 12

[magnum] issues with cloud-init and user-data on fedora-atomic image
by Jean-Philippe Méthot 28 Apr '20

28 Apr '20

Hi, I’ve been testing out magnum (rocky) in my staging environment and I’ve been running into an issue I can’t really explain. Essentially, Magnum is able to provision all the VMs it needs according to the template. However, when it comes time to install kubernetes through cloud-init, cloud-init is unable to fetch the user-data. I must specify that ONLY the user-data is affected. Everything else that’s read from the metadata gets through without any issue. I can also do 'curl http://169.254.169.254/2009-04-04/' <http://169.254.169.254/2009-04-04/'> from the VM without any issue. When I try to curl http://169.254.169.254/2009-04-04/user-data <http://169.254.169.254/2009-04-04/user-data> , curl just hangs without any data showing up : sudo curl -v http://169.254.169.254/2009-04-04/user-data * Trying 169.254.169.254... * TCP_NODELAY set * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) > GET /2009-04-04/user-data HTTP/1.1 > Host: 169.254.169.254 > User-Agent: curl/7.59.0 > Accept: */* Tcpdump -vvv looks like this : sudo tcpdump -vvv -i eth0 host 169.254.169.254 tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes 20:36:22.284195 IP (tos 0x0, ttl 64, id 11248, offset 0, flags [DF], proto TCP (6), length 60) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [S], cksum 0x5e34 (incorrect -> 0x51a5), seq 2359174217, win 26730, options [mss 8910,sackOK,TS val 583688334 ecr 0,nop,wscale 7], length 0 20:36:22.284793 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [S.], cksum 0x9ed1 (correct), seq 2099498427, ack 2359174218, win 26694, options [mss 8910,sackOK,TS val 4068504964 ecr 583688334,nop,wscale 9], length 0 20:36:22.284875 IP (tos 0x0, ttl 64, id 11249, offset 0, flags [DF], proto TCP (6), length 52) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [.], cksum 0x5e2c (incorrect -> 0x522e), seq 1, ack 1, win 209, options [nop,nop,TS val 583688335 ecr 4068504964], length 0 20:36:22.284962 IP (tos 0x0, ttl 64, id 11250, offset 0, flags [DF], proto TCP (6), length 151) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [P.], cksum 0x5e8f (incorrect -> 0xb692), seq 1:100, ack 1, win 209, options [nop,nop,TS val 583688335 ecr 4068504964], length 99: HTTP, length: 99 GET /2009-04-04/user-data HTTP/1.1 Host: 169.254.169.254 User-Agent: curl/7.59.0 Accept: */* 20:36:22.325212 IP (tos 0x0, ttl 64, id 47160, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [.], cksum 0x523e (correct), seq 1, ack 100, win 53, options [nop,nop,TS val 4068505005 ecr 583688335], length 0 20:36:33.546523 IP (tos 0x0, ttl 64, id 11251, offset 0, flags [DF], proto TCP (6), length 52) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [F.], cksum 0x5e2c (incorrect -> 0x25a4), seq 100, ack 1, win 209, options [nop,nop,TS val 583699596 ecr 4068505005], length 0 20:36:33.586154 IP (tos 0x0, ttl 64, id 47169, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [.], cksum 0x91fc (correct), seq 26695, ack 101, win 53, options [nop,nop,TS val 4068516266 ecr 583699596], length 0 So the instance does receive packets as a reply to the curl, but they’re empty. I have tested on both Fedora-Atomic-27-20180212.2.x86_64.qcow2 and Fedora-AtomicHost-28-20180425.0.x86_64.qcow2 images and reproduced the issue both times. Nova-api and neutron-metadata logs show http 200 for each of my requests to nova. What could be blocking the user-data from reaching the instance in this case? Is there a way to set magnum-created intances to use configdrive instead? I must also specify that user-data works on all regular instances. It’s only affecting instances created by magnum. Best regards, Jean-Philippe Méthot Senior Openstack system administrator Administrateur système Openstack sénior PlanetHoster inc.

1 0