From aj at suse.com Sun Mar 1 17:05:30 2020 From: aj at suse.com (Andreas Jaeger) Date: Sun, 1 Mar 2020 18:05:30 +0100 Subject: Retiring openstack/faafo repository Message-ID: <6f829fe2-5dc6-2748-9744-baaaf49446bf@suse.com> This repo was part of the first-app document that was retired last year as part of the api-site retirement work. Thus, I'm retiring the openstack/faafo repository now. Patches are pushed with topic:retire-faafo, starting at https://review.opendev.org/710652 Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From yamamoto at midokura.com Mon Mar 2 02:33:57 2020 From: yamamoto at midokura.com (Takashi Yamamoto) Date: Mon, 2 Mar 2020 11:33:57 +0900 Subject: [neutron] bug deputy report for 2020-02-24 week Message-ID: 2020-02-24 week Someone familiar with these topics need to investigate DHCP agent: https://bugs.launchpad.net/neutron/+bug/1864711 DHCP port rescheduling causes ports to grow, internal DNS to be broken WSGI: https://bugs.launchpad.net/neutron/+bug/1864418 has wrong with use apache to start neutron api in docker container L3 agent: https://bugs.launchpad.net/neutron/+bug/1864963 loosing connectivity to instance with FloatingIP randomly API server performance: https://bugs.launchpad.net/neutron/+bug/1865223 [scale issue] regression for security group list between Newton and Rocky+ RFE https://bugs.launchpad.net/neutron/+bug/1864841 Neutron -> Designate integration does not consider the dns pool for zone Medium https://bugs.launchpad.net/neutron/+bug/1864822 Openvswitch Agent - Connexion openvswitch DB Broken Low https://bugs.launchpad.net/neutron/+bug/1864374 ml2 ovs does not flush iptables switching to FW ovs OVN-specific stuff High https://bugs.launchpad.net/neutron/+bug/1864620 [OVN] neutron_tempest_plugin.scenario.test_security_groups.NetworkSecGroupTest.test_multiple_ports_portrange_remote often fails Medium https://bugs.launchpad.net/neutron/+bug/1864833 [OVN] Functional tests start with OVSDB binary 2.9 instead 2.12 https://bugs.launchpad.net/neutron/+bug/1864639 [OVN] UpdateLRouterPortCommand and AddLRouterPortCommand needs to specify network Wishlist https://bugs.launchpad.net/neutron/+bug/1864640 [Ussuri] Neutron API writes to the Southbound DB From eandersson at blizzard.com Mon Mar 2 03:03:51 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Mon, 2 Mar 2020 03:03:51 +0000 Subject: [neutron] security group list regression In-Reply-To: References: , Message-ID: When we went from Mitaka to Rocky in August last year and we saw an exponential increase in api times for listing security group rules. I think I last commented on this bug https://bugs.launchpad.net/neutron/+bug/1810563, but I have brought it up on a few other occasions as well. Bug #1810563 “adding rules to security groups is slow” : Bugs : neutron Sometime between liberty and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review with fixes is incoming. You can repro with a vanilla devstack install on master, and this script: #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') export OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost make_rules() { iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file <<EOF {... bugs.launchpad.net ________________________________ From: Slawek Kaplonski Sent: Saturday, February 29, 2020 12:44 AM To: James Denton Cc: openstack-discuss Subject: Re: [neutron] security group list regression Hi, I just replied in Your bug report. Can You try to apply patch https://urldefense.com/v3/__https://review.opendev.org/*/c/708695/__;Iw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Ul-OlUqQ$ to see if that will help with this problem? > On 29 Feb 2020, at 02:41, James Denton wrote: > > Hello all, > > We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty severe regression in the time it takes the API to return the list of security groups. This environment has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack security group list’ command to complete. I don’t have actual data from the same environment running Newton, but was able to replicate this behavior with the following lab environments running a mix of virtual and baremetal machines: > > Newton (VM) > Rocky (BM) > Stein (VM) > Train (BM) > > Number of sec grps vs time in seconds: > > # Newton Rocky Stein Train > 200 4.1 3.7 5.4 5.2 > 500 5.3 7 11 9.4 > 1000 7.2 12.4 19.2 16 > 2000 9.2 24.2 35.3 30.7 > 3000 12.1 36.5 52 44 > 4000 16.1 47.2 73 58.9 > 5000 18.4 55 90 69 > > As you can see (hopefully), the response time increased significantly between Newton and Rocky, and has grown slightly ever since. We don't know, yet, if this behavior can be seen with other 'list' commands or is limited to secgroups. We're currently verifying on some intermediate releases to see where things went wonky. > > There are some similar recent reports out in the wild with little feedback: > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1788749__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Vx5jGlrA$ > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1721273__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8U9NbN_LA$ > > I opened a bug here, too: > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/1865223__;Kw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8UtMQ2-Dw$ > > Bottom line: Has anyone else experienced similar regressions in recent releases? If so, were you able to address them with any sort of tuning? > > Thanks in advance, > James > — Slawek Kaplonski Senior software engineer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From akekane at redhat.com Mon Mar 2 06:26:34 2020 From: akekane at redhat.com (Abhishek Kekane) Date: Mon, 2 Mar 2020 11:56:34 +0530 Subject: [glance] Different checksum between CLI and curl In-Reply-To: <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: Hi Gaëtan, Glance team doesn't recommend to use OSC anymore. I will recommend you to check the same behaviour using python-glanceclient. Thanks & Best Regards, Abhishek Kekane On Sat, Feb 29, 2020 at 3:54 AM Monty Taylor wrote: > > > > On Feb 28, 2020, at 4:15 PM, gaetan.trellu at incloudus.com wrote: > > > > Hey Monty, > > > > If I download the image via the CLI, the checksum of the file matches > the checksum from the image details. > > If I download the image via "curl", the "Content-Md5" header matches the > image details but the file checksum doesn't. > > > > The files have the same size, this is really weird. > > WOW. > > I still don’t know the issue - but my unfounded hunch is that the curl > command is likely not doing something it should be. If OSC is producing a > file that matches the image details, that seems like the right choice for > now. > > Seriously fascinating though. > > > Gaëtan > > > > On 2020-02-28 17:00, Monty Taylor wrote: > >>> On Feb 28, 2020, at 2:29 PM, gaetan.trellu at incloudus.com wrote: > >>> Hi guys, > >>> Does anyone know why the md5 checksum is different between the > "openstack image save" CLI and "curl" commands? > >>> During the image creation a checksum is computed to check the image > integrity, using the "openstack" CLI match the checksum generated but when > "curl" is used by following the API documentation[1] the checksum change at > every "download". > >>> Any idea? > >> That seems strange. I don’t know off the top of my head. I do know > >> Artem has patches up to switch OSC to using SDK for image operations. > >> https://review.opendev.org/#/c/699416/ > >> That said, I’d still expect current OSC checksums to be solid. Perhaps > >> there is some filtering/processing being done cloud-side in your > >> glance? If you download the image to a file and run a checksum on it - > >> does it match the checksum given by OSC on upload? Or the checksum > >> given by glance API on download? > >>> Thanks, > >>> Gaëtan > >>> [1] > https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-binary-image-data-detail#download-binary-image-data > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Mon Mar 2 08:03:53 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 2 Mar 2020 09:03:53 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: Hi Abhishek, Abhishek Kekane wrote: > Glance team doesn't recommend to use OSC anymore. > I will recommend you to check the same behaviour using python-glanceclient. Whoa, so OSC suddenly fell out of support? When did that happen? I thought OSC is the future, not the other way around... -yoctozepto From mark at stackhpc.com Mon Mar 2 09:54:03 2020 From: mark at stackhpc.com (Mark Goddard) Date: Mon, 2 Mar 2020 09:54:03 +0000 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > Hi Gaëtan, > > Glance team doesn't recommend to use OSC anymore. > I will recommend you to check the same behaviour using python-glanceclient. That's not cool - everyone has switched to OSC. It's also the first time I've heard of it. > > Thanks & Best Regards, > > Abhishek Kekane > > > On Sat, Feb 29, 2020 at 3:54 AM Monty Taylor wrote: >> >> >> >> > On Feb 28, 2020, at 4:15 PM, gaetan.trellu at incloudus.com wrote: >> > >> > Hey Monty, >> > >> > If I download the image via the CLI, the checksum of the file matches the checksum from the image details. >> > If I download the image via "curl", the "Content-Md5" header matches the image details but the file checksum doesn't. >> > >> > The files have the same size, this is really weird. >> >> WOW. >> >> I still don’t know the issue - but my unfounded hunch is that the curl command is likely not doing something it should be. If OSC is producing a file that matches the image details, that seems like the right choice for now. >> >> Seriously fascinating though. >> >> > Gaëtan >> > >> > On 2020-02-28 17:00, Monty Taylor wrote: >> >>> On Feb 28, 2020, at 2:29 PM, gaetan.trellu at incloudus.com wrote: >> >>> Hi guys, >> >>> Does anyone know why the md5 checksum is different between the "openstack image save" CLI and "curl" commands? >> >>> During the image creation a checksum is computed to check the image integrity, using the "openstack" CLI match the checksum generated but when "curl" is used by following the API documentation[1] the checksum change at every "download". >> >>> Any idea? >> >> That seems strange. I don’t know off the top of my head. I do know >> >> Artem has patches up to switch OSC to using SDK for image operations. >> >> https://review.opendev.org/#/c/699416/ >> >> That said, I’d still expect current OSC checksums to be solid. Perhaps >> >> there is some filtering/processing being done cloud-side in your >> >> glance? If you download the image to a file and run a checksum on it - >> >> does it match the checksum given by OSC on upload? Or the checksum >> >> given by glance API on download? >> >>> Thanks, >> >>> Gaëtan >> >>> [1] https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-binary-image-data-detail#download-binary-image-data >> > >> >> From ralonsoh at redhat.com Mon Mar 2 11:26:33 2020 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Mon, 02 Mar 2020 11:26:33 +0000 Subject: [neutron] security group list regression In-Reply-To: References: , Message-ID: Hello James: Just to make a quick summary of the status of the commented bugs/regressions: 1) https://bugs.launchpad.net/neutron/+bug/1810563: adding rules to security groups is slow That was addressed in https://review.opendev.org/#/c/633145/ and https://review.opendev.org/#/c/637407/, removing the O^2 check and using lazy loading. 2) https://bugzilla.redhat.com/show_bug.cgi?id=1788749: Neutron List networks API regression The last reply was marked as private. I've undone this and you can read now c#2. Testing with a similar scenario, I don't see any performance degradation between Queens and Train. 3) https://bugzilla.redhat.com/show_bug.cgi?id=1721273: Neutron API List Ports Performance regression That problem was solved in https://review.opendev.org/#/c/667981/ and https://review.opendev.org/#/c/667998/, by refactoring how the port QoS extension was reading and applying the QoS info in the port dict. 4) https://bugs.launchpad.net/neutron/+bug/1865223: regression for security group list between Newton and Rocky+ This is similar to https://bugs.launchpad.net/neutron/+bug/1863201. In this case, the regression was detected from R to S. The performance dropped from 3 secs to 110 secs (36x). That issue was addressed by https://review.opendev.org/#/c/708695/. But while 1865223 is talking about *SG list*, 1863201 is related to *SG rule list*. I would like to make this differentiation, because both retrieval commands are not related. In this bug (1863201), the performance degradation multiplies by x3 (N->Q) the initial time. This could be caused by the OVO integration (O->P: https://review.opendev.org/#/c/284738/). Instead of using the DB object now we make this call using the OVO object containing the DB register (something like a DB view). That's something I still need to check. Just to make a concretion: the patch 708695 improves the *SG rule* retrieval, not the SG list command. Another punctualization is that this patch will help in the case of having a balance between SG rules and SG. This patch will help to retrieve from the DB only those SG rules belonging to the project. If, as you state in https://bugs.launchpad.net/neutron/+bug/1865223/comments/4, most of those SG rules belong to the same project, there is little improvement there. As commented, I'm still looking at improving the SG OVO performance. Regards On Mon, 2020-03-02 at 03:03 +0000, Erik Olof Gunnar Andersson wrote: > When we went from Mitaka to Rocky in August last year and we saw an exponential increase in api > times for listing security group rules. > > I think I last commented on this bug https://bugs.launchpad.net/neutron/+bug/1810563, but I have > brought it up on a few other occasions as well. > Bug #1810563 “adding rules to security groups is slow” : Bugs : neutron Sometime between liberty > and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review with > fixes is incoming. You can repro with a vanilla devstack install on master, and this script: > #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') export > OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost make_rules() { > iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file <<EOF > {... bugs.launchpad.net > > > From: Slawek Kaplonski > Sent: Saturday, February 29, 2020 12:44 AM > To: James Denton > Cc: openstack-discuss > Subject: Re: [neutron] security group list regression > > Hi, > > I just replied in Your bug report. Can You try to apply patch > https://urldefense.com/v3/__https://review.opendev.org/*/c/708695/__;Iw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Ul-OlUqQ$ > to see if that will help with this problem? > > > On 29 Feb 2020, at 02:41, James Denton wrote: > > > > Hello all, > > > > We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty severe > regression in the time it takes the API to return the list of security groups. This environment > has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack security > group list’ command to complete. I don’t have actual data from the same environment running > Newton, but was able to replicate this behavior with the following lab environments running a mix > of virtual and baremetal machines: > > > > Newton (VM) > > Rocky (BM) > > Stein (VM) > > Train (BM) > > > > Number of sec grps vs time in seconds: > > > > # Newton Rocky Stein Train > > 200 4.1 3.7 5.4 5.2 > > 500 5.3 7 11 9.4 > > 1000 7.2 12.4 19.2 16 > > 2000 9.2 24.2 35.3 30.7 > > 3000 12.1 36.5 52 44 > > 4000 16.1 47.2 73 58.9 > > 5000 18.4 55 90 69 > > > > As you can see (hopefully), the response time increased significantly between Newton and Rocky, > and has grown slightly ever since. We don't know, yet, if this behavior can be seen with other > 'list' commands or is limited to secgroups. We're currently verifying on some intermediate > releases to see where things went wonky. > > > > There are some similar recent reports out in the wild with little feedback: > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1788749__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Vx5jGlrA$ > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1721273__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8U9NbN_LA$ > > > > > I opened a bug here, too: > > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/1865223__;Kw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8UtMQ2-Dw$ > > > > > Bottom line: Has anyone else experienced similar regressions in recent releases? If so, were you > able to address them with any sort of tuning? > > > > Thanks in advance, > > James > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From james.denton at rackspace.com Mon Mar 2 14:27:49 2020 From: james.denton at rackspace.com (James Denton) Date: Mon, 2 Mar 2020 14:27:49 +0000 Subject: [neutron] security group list regression In-Reply-To: References: Message-ID: Thanks, Rodolfo. I'll take a look at each of these after coffee and clarify my position (if needed). James On 3/2/20, 6:27 AM, "Rodolfo Alonso" wrote: CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello James: Just to make a quick summary of the status of the commented bugs/regressions: 1) https://bugs.launchpad.net/neutron/+bug/1810563: adding rules to security groups is slow That was addressed in https://review.opendev.org/#/c/633145/ and https://review.opendev.org/#/c/637407/, removing the O^2 check and using lazy loading. 2) https://bugzilla.redhat.com/show_bug.cgi?id=1788749: Neutron List networks API regression The last reply was marked as private. I've undone this and you can read now c#2. Testing with a similar scenario, I don't see any performance degradation between Queens and Train. 3) https://bugzilla.redhat.com/show_bug.cgi?id=1721273: Neutron API List Ports Performance regression That problem was solved in https://review.opendev.org/#/c/667981/ and https://review.opendev.org/#/c/667998/, by refactoring how the port QoS extension was reading and applying the QoS info in the port dict. 4) https://bugs.launchpad.net/neutron/+bug/1865223: regression for security group list between Newton and Rocky+ This is similar to https://bugs.launchpad.net/neutron/+bug/1863201. In this case, the regression was detected from R to S. The performance dropped from 3 secs to 110 secs (36x). That issue was addressed by https://review.opendev.org/#/c/708695/. But while 1865223 is talking about *SG list*, 1863201 is related to *SG rule list*. I would like to make this differentiation, because both retrieval commands are not related. In this bug (1863201), the performance degradation multiplies by x3 (N->Q) the initial time. This could be caused by the OVO integration (O->P: https://review.opendev.org/#/c/284738/). Instead of using the DB object now we make this call using the OVO object containing the DB register (something like a DB view). That's something I still need to check. Just to make a concretion: the patch 708695 improves the *SG rule* retrieval, not the SG list command. Another punctualization is that this patch will help in the case of having a balance between SG rules and SG. This patch will help to retrieve from the DB only those SG rules belonging to the project. If, as you state in https://bugs.launchpad.net/neutron/+bug/1865223/comments/4, most of those SG rules belong to the same project, there is little improvement there. As commented, I'm still looking at improving the SG OVO performance. Regards On Mon, 2020-03-02 at 03:03 +0000, Erik Olof Gunnar Andersson wrote: > When we went from Mitaka to Rocky in August last year and we saw an exponential increase in api > times for listing security group rules. > > I think I last commented on this bug https://bugs.launchpad.net/neutron/+bug/1810563, but I have > brought it up on a few other occasions as well. > Bug #1810563 “adding rules to security groups is slow” : Bugs : neutron Sometime between liberty > and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review with > fixes is incoming. You can repro with a vanilla devstack install on master, and this script: > #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') export > OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost make_rules() { > iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file <<EOF > {... bugs.launchpad.net > > > From: Slawek Kaplonski > Sent: Saturday, February 29, 2020 12:44 AM > To: James Denton > Cc: openstack-discuss > Subject: Re: [neutron] security group list regression > > Hi, > > I just replied in Your bug report. Can You try to apply patch > https://urldefense.com/v3/__https://review.opendev.org/*/c/708695/__;Iw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Ul-OlUqQ$ > to see if that will help with this problem? > > > On 29 Feb 2020, at 02:41, James Denton wrote: > > > > Hello all, > > > > We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty severe > regression in the time it takes the API to return the list of security groups. This environment > has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack security > group list’ command to complete. I don’t have actual data from the same environment running > Newton, but was able to replicate this behavior with the following lab environments running a mix > of virtual and baremetal machines: > > > > Newton (VM) > > Rocky (BM) > > Stein (VM) > > Train (BM) > > > > Number of sec grps vs time in seconds: > > > > # Newton Rocky Stein Train > > 200 4.1 3.7 5.4 5.2 > > 500 5.3 7 11 9.4 > > 1000 7.2 12.4 19.2 16 > > 2000 9.2 24.2 35.3 30.7 > > 3000 12.1 36.5 52 44 > > 4000 16.1 47.2 73 58.9 > > 5000 18.4 55 90 69 > > > > As you can see (hopefully), the response time increased significantly between Newton and Rocky, > and has grown slightly ever since. We don't know, yet, if this behavior can be seen with other > 'list' commands or is limited to secgroups. We're currently verifying on some intermediate > releases to see where things went wonky. > > > > There are some similar recent reports out in the wild with little feedback: > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1788749__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Vx5jGlrA$ > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1721273__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8U9NbN_LA$ > > > > > I opened a bug here, too: > > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/1865223__;Kw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8UtMQ2-Dw$ > > > > > Bottom line: Has anyone else experienced similar regressions in recent releases? If so, were you > able to address them with any sort of tuning? > > > > Thanks in advance, > > James > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From gaetan.trellu at incloudus.com Mon Mar 2 15:11:16 2020 From: gaetan.trellu at incloudus.com (gaetan.trellu at incloudus.com) Date: Mon, 02 Mar 2020 10:11:16 -0500 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: Abhishek, Thansk for your answer, I tried both CLIs (Train release) and the issue still the same. Paste of the "curl" command: http://paste.openstack.org/show/790197/ Result of the "md5sum" on the file created by the "curl": $ md5sum /tmp/kernel.glance c3726de8e03158305453f328d85e9957 /tmp/kernel.glance As Mark and Radoslaw, I'm quite surprised about OSC been deprecated. Do you have any release note about this? Thanks for your help. Gaëtan curl -g -i -X GET http://10.0.0.11:9292/v2/images/de39fc9c-b943-47e3-82c4-bd6d577a9577/file -H "Content-Type: application/octet-stream" -H "User-Agent: python-glanceclient" -H "X-Auth-Token: $token" --output /tmp/kernel.glance -v Note: Unnecessary use of -X or --request, GET is already inferred. * Expire in 0 ms for 6 (transfer 0x557679b1de80) * Trying 10.0.0.11... * TCP_NODELAY set * Expire in 200 ms for 4 (transfer 0x557679b1de80) % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 10.0.0.11 (10.0.0.11) port 9292 (#0) > GET /v2/images/de39fc9c-b943-47e3-82c4-bd6d577a9577/file HTTP/1.1 > Host: 10.0.0.11:9292 > Accept: */* > Content-Type: application/octet-stream > User-Agent: python-glanceclient > X-Auth-Token: > gAAAAABeXRzKVS3uQIIv9t-wV7njIV-T9HIvcwFqcHNivrpyBlesDtgAj1kpWk59a20EJLUo8IeHpTdKgVFwhnAVvbSWHY-HQvxu5dwSFsK4A-7CzAOwdp3svSqxB-FdwWhsY_PElftMX4geA-y_YFZJamefZapiAv6g1gSm-BSv5GYQ0hj3yGY > 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0< HTTP/1.1 200 OK < Content-Type: application/octet-stream < Content-Md5: 26c6d5c3d8ba9fd4bc4d1ee5959a827c < Content-Length: 5631728 < X-Openstack-Request-Id: req-e7ba2455-780f-48a8-b6a2-1c6741d0e368 < Date: Mon, 02 Mar 2020 15:03:53 GMT < { [32768 bytes data] 100 5499k 100 5499k 0 0 4269k 0 0:00:01 0:00:01 --:--:-- 4269k * Connection #0 to host 10.0.0.11 left intact On 2020-03-02 04:54, Mark Goddard wrote: > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane > wrote: >> >> Hi Gaëtan, >> >> Glance team doesn't recommend to use OSC anymore. >> I will recommend you to check the same behaviour using >> python-glanceclient. > > That's not cool - everyone has switched to OSC. It's also the first > time I've heard of it. > >> >> Thanks & Best Regards, >> >> Abhishek Kekane >> >> >> On Sat, Feb 29, 2020 at 3:54 AM Monty Taylor >> wrote: >>> >>> >>> >>> > On Feb 28, 2020, at 4:15 PM, gaetan.trellu at incloudus.com wrote: >>> > >>> > Hey Monty, >>> > >>> > If I download the image via the CLI, the checksum of the file matches the checksum from the image details. >>> > If I download the image via "curl", the "Content-Md5" header matches the image details but the file checksum doesn't. >>> > >>> > The files have the same size, this is really weird. >>> >>> WOW. >>> >>> I still don’t know the issue - but my unfounded hunch is that the >>> curl command is likely not doing something it should be. If OSC is >>> producing a file that matches the image details, that seems like the >>> right choice for now. >>> >>> Seriously fascinating though. >>> >>> > Gaëtan >>> > >>> > On 2020-02-28 17:00, Monty Taylor wrote: >>> >>> On Feb 28, 2020, at 2:29 PM, gaetan.trellu at incloudus.com wrote: >>> >>> Hi guys, >>> >>> Does anyone know why the md5 checksum is different between the "openstack image save" CLI and "curl" commands? >>> >>> During the image creation a checksum is computed to check the image integrity, using the "openstack" CLI match the checksum generated but when "curl" is used by following the API documentation[1] the checksum change at every "download". >>> >>> Any idea? >>> >> That seems strange. I don’t know off the top of my head. I do know >>> >> Artem has patches up to switch OSC to using SDK for image operations. >>> >> https://review.opendev.org/#/c/699416/ >>> >> That said, I’d still expect current OSC checksums to be solid. Perhaps >>> >> there is some filtering/processing being done cloud-side in your >>> >> glance? If you download the image to a file and run a checksum on it - >>> >> does it match the checksum given by OSC on upload? Or the checksum >>> >> given by glance API on download? >>> >>> Thanks, >>> >>> Gaëtan >>> >>> [1] https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-binary-image-data-detail#download-binary-image-data >>> > >>> >>> From ltoscano at redhat.com Mon Mar 2 15:25:02 2020 From: ltoscano at redhat.com (Luigi Toscano) Date: Mon, 02 Mar 2020 16:25:02 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: Message-ID: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > Hi Gaëtan, > > > > Glance team doesn't recommend to use OSC anymore. > > I will recommend you to check the same behaviour using > > python-glanceclient. > > That's not cool - everyone has switched to OSC. It's also the first > time I've heard of it. > Do we have proper microversion support then? This is a blocker for cinder. More generally I observed a disconection between the needs of a few teams (Cinder and Glance for sure) and OSC, with a real split on the community and no apparent interest in trying to bridge the gap, which is very sad. -- Luigi From flux.adam at gmail.com Mon Mar 2 15:27:08 2020 From: flux.adam at gmail.com (Adam Harwell) Date: Tue, 3 Mar 2020 00:27:08 +0900 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: I've heard this from members of the glance team for the past few (maybe 3?) summits at least, and every time I try to correct them, but it feels like talking to a brick wall. OSC is the future direction, it should be supported, and there's not even any ambiguity that I'm aware of, other than some strange refusal to accept reality on behalf of a few people... Yes, my tone here is definitely a little aggressive, but this has been an ongoing frustration of mine as I've been hearing this for a while and it's actively harmful and misleading, so I won't apologize for it. Some folks need to wake up. <_< --Adam On Mon, Mar 2, 2020, 19:00 Mark Goddard wrote: > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > > Hi Gaëtan, > > > > Glance team doesn't recommend to use OSC anymore. > > I will recommend you to check the same behaviour using > python-glanceclient. > > That's not cool - everyone has switched to OSC. It's also the first > time I've heard of it. > > > > > Thanks & Best Regards, > > > > Abhishek Kekane > > > > > > On Sat, Feb 29, 2020 at 3:54 AM Monty Taylor > wrote: > >> > >> > >> > >> > On Feb 28, 2020, at 4:15 PM, gaetan.trellu at incloudus.com wrote: > >> > > >> > Hey Monty, > >> > > >> > If I download the image via the CLI, the checksum of the file matches > the checksum from the image details. > >> > If I download the image via "curl", the "Content-Md5" header matches > the image details but the file checksum doesn't. > >> > > >> > The files have the same size, this is really weird. > >> > >> WOW. > >> > >> I still don’t know the issue - but my unfounded hunch is that the curl > command is likely not doing something it should be. If OSC is producing a > file that matches the image details, that seems like the right choice for > now. > >> > >> Seriously fascinating though. > >> > >> > Gaëtan > >> > > >> > On 2020-02-28 17:00, Monty Taylor wrote: > >> >>> On Feb 28, 2020, at 2:29 PM, gaetan.trellu at incloudus.com wrote: > >> >>> Hi guys, > >> >>> Does anyone know why the md5 checksum is different between the > "openstack image save" CLI and "curl" commands? > >> >>> During the image creation a checksum is computed to check the image > integrity, using the "openstack" CLI match the checksum generated but when > "curl" is used by following the API documentation[1] the checksum change at > every "download". > >> >>> Any idea? > >> >> That seems strange. I don’t know off the top of my head. I do know > >> >> Artem has patches up to switch OSC to using SDK for image operations. > >> >> https://review.opendev.org/#/c/699416/ > >> >> That said, I’d still expect current OSC checksums to be solid. > Perhaps > >> >> there is some filtering/processing being done cloud-side in your > >> >> glance? If you download the image to a file and run a checksum on it > - > >> >> does it match the checksum given by OSC on upload? Or the checksum > >> >> given by glance API on download? > >> >>> Thanks, > >> >>> Gaëtan > >> >>> [1] > https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-binary-image-data-detail#download-binary-image-data > >> > > >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mordred at inaugust.com Mon Mar 2 15:41:53 2020 From: mordred at inaugust.com (Monty Taylor) Date: Mon, 2 Mar 2020 09:41:53 -0600 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: > On Mar 2, 2020, at 9:27 AM, Adam Harwell wrote: > > I've heard this from members of the glance team for the past few (maybe 3?) summits at least, and every time I try to correct them, but it feels like talking to a brick wall. OSC is the future direction, it should be supported, and there's not even any ambiguity that I'm aware of, other than some strange refusal to accept reality on behalf of a few people... > > Yes, my tone here is definitely a little aggressive, but this has been an ongoing frustration of mine as I've been hearing this for a while and it's actively harmful and misleading, so I won't apologize for it. Some folks need to wake up. <_ I fully agree. Users should not use python-glanceclient, they should use OSC. If there are bugs in OSC, we will fix them. > --Adam > > On Mon, Mar 2, 2020, 19:00 Mark Goddard wrote: > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > > Hi Gaëtan, > > > > Glance team doesn't recommend to use OSC anymore. > > I will recommend you to check the same behaviour using python-glanceclient. > > That's not cool - everyone has switched to OSC. It's also the first > time I've heard of it. > > > > > Thanks & Best Regards, > > > > Abhishek Kekane > > > > > > On Sat, Feb 29, 2020 at 3:54 AM Monty Taylor wrote: > >> > >> > >> > >> > On Feb 28, 2020, at 4:15 PM, gaetan.trellu at incloudus.com wrote: > >> > > >> > Hey Monty, > >> > > >> > If I download the image via the CLI, the checksum of the file matches the checksum from the image details. > >> > If I download the image via "curl", the "Content-Md5" header matches the image details but the file checksum doesn't. > >> > > >> > The files have the same size, this is really weird. > >> > >> WOW. > >> > >> I still don’t know the issue - but my unfounded hunch is that the curl command is likely not doing something it should be. If OSC is producing a file that matches the image details, that seems like the right choice for now. > >> > >> Seriously fascinating though. > >> > >> > Gaëtan > >> > > >> > On 2020-02-28 17:00, Monty Taylor wrote: > >> >>> On Feb 28, 2020, at 2:29 PM, gaetan.trellu at incloudus.com wrote: > >> >>> Hi guys, > >> >>> Does anyone know why the md5 checksum is different between the "openstack image save" CLI and "curl" commands? > >> >>> During the image creation a checksum is computed to check the image integrity, using the "openstack" CLI match the checksum generated but when "curl" is used by following the API documentation[1] the checksum change at every "download". > >> >>> Any idea? > >> >> That seems strange. I don’t know off the top of my head. I do know > >> >> Artem has patches up to switch OSC to using SDK for image operations. > >> >> https://review.opendev.org/#/c/699416/ > >> >> That said, I’d still expect current OSC checksums to be solid. Perhaps > >> >> there is some filtering/processing being done cloud-side in your > >> >> glance? If you download the image to a file and run a checksum on it - > >> >> does it match the checksum given by OSC on upload? Or the checksum > >> >> given by glance API on download? > >> >>> Thanks, > >> >>> Gaëtan > >> >>> [1] https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-binary-image-data-detail#download-binary-image-data > >> > > >> > >> > From smooney at redhat.com Mon Mar 2 15:47:37 2020 From: smooney at redhat.com (Sean Mooney) Date: Mon, 02 Mar 2020 15:47:37 +0000 Subject: [glance] Different checksum between CLI and curl In-Reply-To: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> Message-ID: On Mon, 2020-03-02 at 16:25 +0100, Luigi Toscano wrote: > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > Hi Gaëtan, > > > > > > Glance team doesn't recommend to use OSC anymore. > > > I will recommend you to check the same behaviour using > > > python-glanceclient. > > > > That's not cool - everyone has switched to OSC. It's also the first > > time I've heard of it. > > > > Do we have proper microversion support then? This is a blocker for cinder. osc support microverions but not the auto negociation. the microverion support is fully integrated with the help text generageion and you can specify the desired microverion with --os--api-version=X option when using osc. > > More generally I observed a disconection between the needs of a few teams > (Cinder and Glance for sure) and OSC, with a real split on the community and > no apparent interest in trying to bridge the gap, which is very sad. i know that it was hoped that using the sdk in osc would allow osc to inherit the auto negotiation capability but even without that it does fully support micorverions it just does not have the same behavior as the legacy project clients. i actully tought there was a cross project goal/resolution to move all project to osc a few releases ago so if glance are considering deprection there osc support i think that is problematic and need a much stonger justification then automatic micorverions support. i know there has been an ongoing effort form multiple cycles to stop using the project clients in all documentaiton where its possibel to use osc instead so it realy feels like this would be a regression. > From dtantsur at redhat.com Mon Mar 2 15:49:35 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Mon, 2 Mar 2020 16:49:35 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> Message-ID: Hi, On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > Hi Gaëtan, > > > > > > Glance team doesn't recommend to use OSC anymore. > > > I will recommend you to check the same behaviour using > > > python-glanceclient. > > > > That's not cool - everyone has switched to OSC. It's also the first > > time I've heard of it. > > > > Do we have proper microversion support then? This is a blocker for cinder. > The ironic team has been successfully hacking around the absence of a native microversion support for a while. We use ironicclient instead of openstacksdk, which makes things harder. If you use openstacksdk, it's easier to teach it microversions. In any case, I can provide some guidance if you'd like to. Dmitry > > More generally I observed a disconection between the needs of a few teams > (Cinder and Glance for sure) and OSC, with a real split on the community > and > no apparent interest in trying to bridge the gap, which is very sad. > > -- > Luigi > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Mon Mar 2 16:37:43 2020 From: smooney at redhat.com (Sean Mooney) Date: Mon, 02 Mar 2020 16:37:43 +0000 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> Message-ID: On Mon, 2020-03-02 at 16:49 +0100, Dmitry Tantsur wrote: > Hi, > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > > Hi Gaëtan, > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > I will recommend you to check the same behaviour using > > > > python-glanceclient. > > > > > > That's not cool - everyone has switched to OSC. It's also the first > > > time I've heard of it. > > > > > > > Do we have proper microversion support then? This is a blocker for cinder. > > > > The ironic team has been successfully hacking around the absence of a > native microversion support for a while. We use ironicclient instead of > openstacksdk, which makes things harder. If you use openstacksdk, it's > easier to teach it microversions. In any case, I can provide some guidance > if you'd like to. > > Dmitry that is also problematic. by harcking around it it gives the ironic command a different behavior to the rest of osc. osc does support microverions it just does not support automatic versin negociation which is what you are hacking in. i do agree that it would be nice to have support for version negociation where by you could do somehting like --os-compute-api-version=auto to opt in to it but automatic microverions detetion does make it harder to do help text generation unless you make "openstack --cloud=my-cloud --os-compute-api-version=auto help server create" call out to keystone get the nova endpoint and then lookup its max microversion when you render the help text. with that said if adding --os-image-api-version=auto was enough to get the glance team to fully adopt osc then i think that would be better then partioning the community between osc and legacy client. osc should behave consistently for all projects however so adding negocaiton for ironic and not for other services is not a good thing imo but i guess you were able to do that as ironic is integrated as a plugin correct? > > > > > > More generally I observed a disconection between the needs of a few teams > > (Cinder and Glance for sure) and OSC, with a real split on the community > > and > > no apparent interest in trying to bridge the gap, which is very sad. > > > > -- > > Luigi > > > > > > > > From tim.bell at cern.ch Mon Mar 2 16:59:24 2020 From: tim.bell at cern.ch (Tim Bell) Date: Mon, 2 Mar 2020 17:59:24 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> Message-ID: <17374BC0-3B5F-49F0-A747-B4D04ABD64C1@cern.ch> > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: > > Hi, > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano > wrote: > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane > wrote: > > > Hi Gaëtan, > > > > > > Glance team doesn't recommend to use OSC anymore. > > > I will recommend you to check the same behaviour using > > > python-glanceclient. > > > > That's not cool - everyone has switched to OSC. It's also the first > > time I've heard of it. > > > From the end user perspective, we’ve had positive feedback on the convergence to OSC from our cloud consumers. There has been great progress with Manila to get shares included (https://review.opendev.org/#/c/642222/26/ ) and it would be a pity if we’re asking our end users to understand all of the different project names and inconsistent options/arguments/syntax. We had hoped for a project goal to get everyone aligned on OSC but there was not consensus on this, I’d still encourage it to simplify the experience for OpenStack cloud consumers. Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Mon Mar 2 17:07:00 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 2 Mar 2020 18:07:00 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: <17374BC0-3B5F-49F0-A747-B4D04ABD64C1@cern.ch> References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> <17374BC0-3B5F-49F0-A747-B4D04ABD64C1@cern.ch> Message-ID: Folks, sorry to interrupt but I think we have diverged a bit too much from the subject. Only last Gaetan message is on topic here. Please switch to new subject to discuss OSC future. -yoctozepto pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): > > > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: > > Hi, > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: >> >> On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >> > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: >> > > Hi Gaëtan, >> > > >> > > Glance team doesn't recommend to use OSC anymore. >> > > I will recommend you to check the same behaviour using >> > > python-glanceclient. >> > >> > That's not cool - everyone has switched to OSC. It's also the first >> > time I've heard of it. >> > >> > > From the end user perspective, we’ve had positive feedback on the convergence to OSC from our cloud consumers. > > There has been great progress with Manila to get shares included (https://review.opendev.org/#/c/642222/26/) and it would be a pity if we’re asking our end users to understand all of the different project names and inconsistent options/arguments/syntax. > > We had hoped for a project goal to get everyone aligned on OSC but there was not consensus on this, I’d still encourage it to simplify the experience for OpenStack cloud consumers. > > Tim > > From cboylan at sapwetik.org Mon Mar 2 17:12:00 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 02 Mar 2020 09:12:00 -0800 Subject: Virtualenv (and Tox) broken when run under python<3.6 Message-ID: A recent release of importlib-resources (1.1.0) no longer works on python2.7 or python3.5. The issue is they import typing's ContextManager which didn't exist until python3.6 [0]. This means that python2 jobs and python3.5 jobs are currently unhappy if they need virtualenv. Unfortunately, many of our jobs use tox which uses virtualenv. One workaround being investigated [1] is to install importlib-resources==1.0.2 which does not try to use typing's ContextManager. If this is confirmed to work we will want to consider adding this change to the base job so that all jobs don't have to fix it separately. Note the version of python here is the one used to run virtualenv not the version of python being installed into the virtualenv. This means python3.6 running virtualenv to create a python2 virtualenv should be fine. But python3.5 running virtualenv to create a python3.6 env would not be fine. [0] https://gitlab.com/python-devs/importlib_resources/issues/83 [1] https://review.opendev.org/#/c/710729/ Clark From cboylan at sapwetik.org Mon Mar 2 17:18:58 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 02 Mar 2020 09:18:58 -0800 Subject: Virtualenv (and Tox) broken when run under python<3.6 In-Reply-To: References: Message-ID: On Mon, Mar 2, 2020, at 9:12 AM, Clark Boylan wrote: > A recent release of importlib-resources (1.1.0) no longer works on > python2.7 or python3.5. The issue is they import typing's > ContextManager which didn't exist until python3.6 [0]. This means that > python2 jobs and python3.5 jobs are currently unhappy if they need > virtualenv. Unfortunately, many of our jobs use tox which uses > virtualenv. I noticed after sending the first email that it wasn't clear why tox and virtualenv are effected by an importlib-resources release. Virtualenv depends on importlib-resources [2] and tox uses virtualenv by default to create venvs. [2] https://github.com/pypa/virtualenv/blob/20.0.7/setup.cfg#L48 From johnsomor at gmail.com Mon Mar 2 17:28:51 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Mon, 2 Mar 2020 09:28:51 -0800 Subject: [TaskFlow] running multiple engines on a shared thread In-Reply-To: References: Message-ID: Hi Sachin, Ok, I think I understand what you are trying better now. Do you have a link to your code for reference? The parallelism of tasks inside a flow are dictated by three things: 1. Are you using the parallel action engine? (i.e. setting engine='parallel' when loading) 2. Is the flow of type "unordered flow". 3. The executor defined for the parallel engine. Flows are graphs, so if the flow is defined as a linear flow, it will run each task in a serial manner. Likewise, if the engine is set to "serial" (the default), even the unordered flows will run sequentially. Michael On Thu, Feb 27, 2020 at 1:19 AM Sachin Laddha wrote: > > thanks Michael, > > I am aware that executor can be reused by different engines. > My query was regarding if multiple engines can share same thread for running the engines(and not the tasks of those engines). > > I tried run_iter which can be used to run multiple engines but the tasks of individual engines are run one after another. > Probably engine is holding on to the thread. > > This is limiting our ability run multiple workflows (i.e. engines) in parallel. > > My query is - is it possible to run multiple engines in parallel on same thread (using some asynchronous task execution) > > > On Thu, Feb 27, 2020 at 3:18 AM Michael Johnson wrote: >> >> Hi Sachin, >> >> I'm not 100% sure I understand your need, but I will attempt to answer >> and you can correct me if I am off base. >> >> Taskflow engines (you can create as many of these as you want) use an >> executor defined at flow load time. >> >> Here is a snippet from the Octavia code: >> self.executor = concurrent.futures.ThreadPoolExecutor( >> max_workers=CONF.task_flow.max_workers) >> eng = tf_engines.load( >> flow, >> engine=CONF.task_flow.engine, >> executor=self.executor, >> never_resolve=CONF.task_flow.disable_revert, >> **kwargs) >> >> The parts you are likely interested in are: >> >> 1. The executor. In this case we are using a >> concurrent.futures.ThreadPoolExecutor. We then set the max_workers >> setting to the number of threads we want in our taskflow engine thread >> pool. >> 2. During flow load, we define the engine to be 'parallel' (note: >> 'serial' is the default). This means that unordered flows will run in >> parallel as opposed to serially. >> 3. As noted in the documentation[1], You can share an executor between >> taskflow engines to share the thread pool. >> >> Finally, you want to use "unordered" flows or sub-flows to execute >> tasks concurrently. >> >> [1] https://docs.openstack.org/taskflow/latest/user/engines.html#parallel >> >> Michael >> >> On Wed, Feb 26, 2020 at 7:19 AM Sachin Laddha wrote: >> > >> > Hi, >> > >> > We are using taskflow to execute workflows. Each workflow is executed by a separate thread (using engine.run() method). This is limiting our capability to execute maximum number of workflows that can run in parallel. It is limited by the number of threads there in the thread pool. >> > >> > Most of the time, the workflow tasks are run by agents which could take some time to complete. Each engine is alive and runs on a dedicated thread. >> > >> > Is there any way to reuse or run multiple engines on one thread. The individual tasks of these engines can run in parallel. >> > >> > I came across iter_run method of the engine class. But not sure if that can be used for this purpose. >> > >> > Any help is highly appreciated. From Albert.Braden at synopsys.com Mon Mar 2 18:05:59 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Mon, 2 Mar 2020 18:05:59 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) Message-ID: As an openstack operator I was pretty ecstatic to hear that the assortment of clients would be replaced by a single client. I would be disappointed to find that a component would not integrate and would continue to use a separate client. This would be a step backward IMO. The discussion about microversions goes over my head, but I would hope to see the developers get together and solve the issue and continue working toward integration. -----Original Message----- From: Radosław Piliszek Sent: Monday, March 2, 2020 9:07 AM To: openstack-discuss Subject: Re: [glance] Different checksum between CLI and curl Folks, sorry to interrupt but I think we have diverged a bit too much from the subject. Only last Gaetan message is on topic here. Please switch to new subject to discuss OSC future. -yoctozepto pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): > > > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: > > Hi, > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: >> >> On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >> > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: >> > > Hi Gaëtan, >> > > >> > > Glance team doesn't recommend to use OSC anymore. >> > > I will recommend you to check the same behaviour using >> > > python-glanceclient. >> > >> > That's not cool - everyone has switched to OSC. It's also the first >> > time I've heard of it. >> > >> > > From the end user perspective, we’ve had positive feedback on the convergence to OSC from our cloud consumers. > > There has been great progress with Manila to get shares included (https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= ) and it would be a pity if we’re asking our end users to understand all of the different project names and inconsistent options/arguments/syntax. > > We had hoped for a project goal to get everyone aligned on OSC but there was not consensus on this, I’d still encourage it to simplify the experience for OpenStack cloud consumers. > > Tim > > From smooney at redhat.com Mon Mar 2 18:50:40 2020 From: smooney at redhat.com (Sean Mooney) Date: Mon, 02 Mar 2020 18:50:40 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: Message-ID: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: > As an openstack operator I was pretty ecstatic to hear that the assortment of clients would be replaced by a single > client. I would be disappointed to find that a component would not integrate and would continue to use a separate > client. This would be a step backward IMO. > > The discussion about microversions goes over my head, but I would hope to see the developers get together and solve > the issue and continue working toward integration. just to summerisie it in a non technical way. the project specific cli had a convention where the client would ask the api what the newest micoverion it supported and defualt to that if the clinet suported it. that meant that the same command executed against two different clouds with different versions of openstakc deploy could have different behavior and different responces. so from an interoperablity point of view that is not great but from a usablity point of view the fact enduser dont have to care about microverions and the client would try to do the right thing made some things much simpler. the unifeid client (osc) chose to priorities interoperablity by defaulting to the oldest micorverions, so for nova that would be 2.0/2.1 meaning that if you execute the same command on two different cloud with different version of nova it will behave the same but if you want to use a feature intoduced in a later micorverion you have to explcitly request that via --os-compute-api-version or set that as a env var or in you cloud.yaml so really the difference is that osc requires the end user to be explictl about what micoversion to use and therefor be explict about the behavior of the api they expect (this is what we expect application that use the the api should do) where as the project client tried to just work and use the latest microverion which mostly workd excpet where we remove a feature in a later micorverions. for example we removed the force option on some move operation in nova because allowing forcing caused many harder to fix issues. i dont thnk the nova clinet would cap at the latest micorvierion that allowed forcing. so the poject client genreally did not guarantee that a command would work without specifcing a new micorverison it just that we remove things a hell of a lot less often then we add them. so as an end user that is the main difference between using osc vs glance clinet other then the fact i belive there is a bunch of stuff you can do with glance client that is missing in osc. parity is a spereate disucssion but it is vaild concern. -----Original Message----- > From: Radosław Piliszek > Sent: Monday, March 2, 2020 9:07 AM > To: openstack-discuss > Subject: Re: [glance] Different checksum between CLI and curl > > Folks, > > sorry to interrupt but I think we have diverged a bit too much from the subject. > Only last Gaetan message is on topic here. > Please switch to new subject to discuss OSC future. > > -yoctozepto > > pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): > > > > > > > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: > > > > Hi, > > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: > > > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > > > Hi Gaëtan, > > > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > > I will recommend you to check the same behaviour using > > > > > python-glanceclient. > > > > > > > > That's not cool - everyone has switched to OSC. It's also the first > > > > time I've heard of it. > > > > > > > > From the end user perspective, we’ve had positive feedback on the convergence to OSC from our cloud consumers. > > > > There has been great progress with Manila to get shares included ( > > https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= > > ) and it would be a pity if we’re asking our end users to understand all of the different project names and > > inconsistent options/arguments/syntax. > > > > We had hoped for a project goal to get everyone aligned on OSC but there was not consensus on this, I’d still > > encourage it to simplify the experience for OpenStack cloud consumers. > > > > Tim > > > > > > From dtantsur at redhat.com Mon Mar 2 19:01:47 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Mon, 2 Mar 2020 20:01:47 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> Message-ID: On Mon, Mar 2, 2020 at 5:37 PM Sean Mooney wrote: > On Mon, 2020-03-02 at 16:49 +0100, Dmitry Tantsur wrote: > > Hi, > > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano > wrote: > > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane > wrote: > > > > > Hi Gaëtan, > > > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > > I will recommend you to check the same behaviour using > > > > > python-glanceclient. > > > > > > > > That's not cool - everyone has switched to OSC. It's also the first > > > > time I've heard of it. > > > > > > > > > > Do we have proper microversion support then? This is a blocker for > cinder. > > > > > > > The ironic team has been successfully hacking around the absence of a > > native microversion support for a while. We use ironicclient instead of > > openstacksdk, which makes things harder. If you use openstacksdk, it's > > easier to teach it microversions. In any case, I can provide some > guidance > > if you'd like to. > > > > Dmitry > that is also problematic. > by harcking around it it gives the ironic command a different behavior to > the rest of osc. > osc does support microverions it just does not support automatic versin > negociation which is > what you are hacking in. > Right, and it's a hard requirement for the CLI to be remotely usable. > > i do agree that it would be nice to have support for version negociation > where by you could do somehting like > --os-compute-api-version=auto to opt in to it but automatic microverions > detetion does make it harder to do help > text generation unless you make "openstack --cloud=my-cloud > --os-compute-api-version=auto help server create" call out > to keystone get the nova endpoint and then lookup its max microversion > when you render the help text. > The "auto" must be a default. This is what the users expect: the CLI just working. Defaulting to anything else does them a huge disservice (been there, done that). > > with that said if adding --os-image-api-version=auto was enough to get the > glance team to fully adopt osc > then i think that would be better then partioning the community between > osc and legacy client. > osc should behave consistently for all projects however so adding > negocaiton for ironic and not for other services > is not a good thing imo but i guess you were able to do that as ironic is > integrated as a plugin correct? > Yep. We could not wait for OSC to implement it because the CLI is borderline unusable without this negotiation in place. I don't recall what prevented us from updating OSC, but I think there was a reason, probably not entirely technical. Dmitry > > > > > > > > > > > > More generally I observed a disconection between the needs of a few > teams > > > (Cinder and Glance for sure) and OSC, with a real split on the > community > > > and > > > no apparent interest in trying to bridge the gap, which is very sad. > > > > > > -- > > > Luigi > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Mon Mar 2 19:42:45 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 02 Mar 2020 11:42:45 -0800 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> Message-ID: <70af048d-cf62-4f27-84fe-e6ed7b959837@www.fastmail.com> On Mon, Mar 2, 2020, at 11:01 AM, Dmitry Tantsur wrote: > > > On Mon, Mar 2, 2020 at 5:37 PM Sean Mooney wrote: > > On Mon, 2020-03-02 at 16:49 +0100, Dmitry Tantsur wrote: > > > Hi, > > > > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: > > > > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: > > > > > > Hi Gaëtan, > > > > > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > > > I will recommend you to check the same behaviour using > > > > > > python-glanceclient. > > > > > > > > > > That's not cool - everyone has switched to OSC. It's also the first > > > > > time I've heard of it. > > > > > > > > > > > > > Do we have proper microversion support then? This is a blocker for cinder. > > > > > > > > > > The ironic team has been successfully hacking around the absence of a > > > native microversion support for a while. We use ironicclient instead of > > > openstacksdk, which makes things harder. If you use openstacksdk, it's > > > easier to teach it microversions. In any case, I can provide some guidance > > > if you'd like to. > > > > > > Dmitry > > that is also problematic. > > by harcking around it it gives the ironic command a different behavior to the rest of osc. > > osc does support microverions it just does not support automatic versin negociation which is > > what you are hacking in. > > Right, and it's a hard requirement for the CLI to be remotely usable. > > > > i do agree that it would be nice to have support for version negociation where by you could do somehting like > > --os-compute-api-version=auto to opt in to it but automatic microverions detetion does make it harder to do help > > text generation unless you make "openstack --cloud=my-cloud --os-compute-api-version=auto help server create" call out > > to keystone get the nova endpoint and then lookup its max microversion when you render the help text. > > The "auto" must be a default. This is what the users expect: the CLI > just working. Defaulting to anything else does them a huge disservice > (been there, done that). As a user I strongly disagree. I don't want an API to magically start acting differently because the cloud side has upgraded. Those changes are opaque to me and I shouldn't need to know about them. Instead I should be able to opt into using new features when I know I need them. This is easily achieved by setting the desired microversion when you know you need it. > > > > with that said if adding --os-image-api-version=auto was enough to get the glance team to fully adopt osc > > then i think that would be better then partioning the community between osc and legacy client. > > osc should behave consistently for all projects however so adding negocaiton for ironic and not for other services > > is not a good thing imo but i guess you were able to do that as ironic is integrated as a plugin correct? > > Yep. We could not wait for OSC to implement it because the CLI is > borderline unusable without this negotiation in place. I don't recall > what prevented us from updating OSC, but I think there was a reason, > probably not entirely technical. > > Dmitry From cboylan at sapwetik.org Mon Mar 2 21:09:56 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 02 Mar 2020 13:09:56 -0800 Subject: Virtualenv (and Tox) broken when run under python<3.6 In-Reply-To: References: Message-ID: <475d299c-ed47-44a3-a5b7-c286e7730134@www.fastmail.com> On Mon, Mar 2, 2020, at 9:12 AM, Clark Boylan wrote: > A recent release of importlib-resources (1.1.0) no longer works on > python2.7 or python3.5. The issue is they import typing's > ContextManager which didn't exist until python3.6 [0]. This means that > python2 jobs and python3.5 jobs are currently unhappy if they need > virtualenv. Unfortunately, many of our jobs use tox which uses > virtualenv. > > One workaround being investigated [1] is to install > importlib-resources==1.0.2 which does not try to use typing's > ContextManager. If this is confirmed to work we will want to consider > adding this change to the base job so that all jobs don't have to fix > it separately. We've landed a version of this workaround, https://review.opendev.org/710851, to the base job in opendev/base-jobs. By default this is the base job that all zuul jobs inherit from. This appears to fix use of virtualenv and tox within jobs that use the globally installed versions of these tools. If you run a nested version of the tools (DIB chroot, containers, etc) you'll need to address this issue within that separate context. > > Note the version of python here is the one used to run virtualenv not > the version of python being installed into the virtualenv. This means > python3.6 running virtualenv to create a python2 virtualenv should be > fine. But python3.5 running virtualenv to create a python3.6 env would > not be fine. > > [0] https://gitlab.com/python-devs/importlib_resources/issues/83 > [1] https://review.opendev.org/#/c/710729/ > > Clark > > From mnaser at vexxhost.com Mon Mar 2 21:45:47 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Mon, 2 Mar 2020 16:45:47 -0500 Subject: [all][tc] Moving PTL role to "Maintainers" Message-ID: Hi everyone: We're now in a spot where we have an increasing amount of projects that don't end up with a volunteer as PTL, even if the project has contributors .. no one wants to hold that responsibility alone for many reasons. With time, the PTL role has become far more overloaded with many extra responsibilities than what we define in our charter: https://governance.openstack.org/tc/reference/charter.html#project-team-leads I think it's time to re-evaluate the project leadership model that we have. I am thinking that perhaps it would make a lot of sense to move from a single PTL model to multiple maintainers. This would leave it up to the maintainers to decide how they want to sort the different requirements/liaisons/contact persons between them. The above is just a very basic idea, I don't intend to diving much more in depth for now as I'd like to hear about what the rest of the community thinks. Thanks, Mohammed From james.denton at rackspace.com Mon Mar 2 22:25:12 2020 From: james.denton at rackspace.com (James Denton) Date: Mon, 2 Mar 2020 22:25:12 +0000 Subject: [neutron] security group list regression In-Reply-To: References: Message-ID: <7DD0691D-19A3-4CDB-B377-F67829A86AD7@rackspace.com> Rodolfo, Thanks for continuing to push this on the ML and in the bug report. Happy to report that the client and SDK patches you provided have drastically reduced the SG list time from ~90-120s to ~12-14s within Stein and Train lab environments. One last thing... when you perform an 'openstack security group delete ', the initial lookup by name fails. In Train, the client falls back to using the 'name' parameter (/security-groups?name=). This lookup is quick and the security group is found and deleted. However, on Rocky/Stein (e.g. client 3.18.1), instead of searching by parameter, the client appears to perform a GET /security-groups without limiting the fields and takes a long time. 'openstack security group list' with patch: REQ: curl -g -i -X GET "http://10.0.236.150:9696/v2.0/security-groups?fields=set%28%5B%27description%27%2C+%27project_id%27%2C+%27id%27%2C+%27tags%27%2C+%27name%27%5D%29" -H "Accept: application/json" -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H "X-Auth-Token: {SHA256}3e747da939e8c4befe72d5ca7105971508bd56cdf36208ba6b960d1aee6d19b6" 'openstack security group delete ': Train (notice the name param): REQ: curl -g -i -X GET http://10.20.0.11:9696/v2.0/security-groups/train-test-1755 -H "User-Agent: openstacksdk/0.36.0 keystoneauth1/3.17.1 python-requests/2.22.0 CPython/3.6.7" -H "X-Auth-Token: {SHA256}bf291d5f12903876fc69151db37d295da961ba684a575e77fb6f4829b55df1bf" http://10.20.0.11:9696 "GET /v2.0/security-groups/train-test-1755 HTTP/1.1" 404 125 REQ: curl -g -i -X GET "http://10.20.0.11:9696/v2.0/security-groups?name=train-test-1755" -H "Accept: application/json" -H "User-Agent: openstacksdk/0.36.0 keystoneauth1/3.17.1 python-requests/2.22.0 CPython/3.6.7" -H "X-Auth-Token: {SHA256}bf291d5f12903876fc69151db37d295da961ba684a575e77fb6f4829b55df1bf" http://10.20.0.11:9696 "GET /v2.0/security-groups?name=train-test-1755 HTTP/1.1" 200 1365 Stein & below (notice lack of fields): REQ: curl -g -i -X GET http://10.0.236.150:9696/v2.0/security-groups/stein-test-5189 -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H "X-Auth-Token: {SHA256}e9f87afe851ff5380d8402ee81199c466be9c84fe67ed0302e8b178f33aa1fc2" http://10.0.236.150:9696 "GET /v2.0/security-groups/stein-test-5189 HTTP/1.1" 404 125 REQ: curl -g -i -X GET http://10.0.236.150:9696/v2.0/security-groups -H "Accept: application/json" -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H "X-Auth-Token: {SHA256}e9f87afe851ff5380d8402ee81199c466be9c84fe67ed0302e8b178f33aa1fc2" Haven't quite figured out where fields can be used to speed up the delete process on the older client, or if the newer client would be backwards-compatible (and how far back). Thanks, James On 3/2/20, 9:31 AM, "James Denton" wrote: CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Thanks, Rodolfo. I'll take a look at each of these after coffee and clarify my position (if needed). James On 3/2/20, 6:27 AM, "Rodolfo Alonso" wrote: CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello James: Just to make a quick summary of the status of the commented bugs/regressions: 1) https://bugs.launchpad.net/neutron/+bug/1810563: adding rules to security groups is slow That was addressed in https://review.opendev.org/#/c/633145/ and https://review.opendev.org/#/c/637407/, removing the O^2 check and using lazy loading. 2) https://bugzilla.redhat.com/show_bug.cgi?id=1788749: Neutron List networks API regression The last reply was marked as private. I've undone this and you can read now c#2. Testing with a similar scenario, I don't see any performance degradation between Queens and Train. 3) https://bugzilla.redhat.com/show_bug.cgi?id=1721273: Neutron API List Ports Performance regression That problem was solved in https://review.opendev.org/#/c/667981/ and https://review.opendev.org/#/c/667998/, by refactoring how the port QoS extension was reading and applying the QoS info in the port dict. 4) https://bugs.launchpad.net/neutron/+bug/1865223: regression for security group list between Newton and Rocky+ This is similar to https://bugs.launchpad.net/neutron/+bug/1863201. In this case, the regression was detected from R to S. The performance dropped from 3 secs to 110 secs (36x). That issue was addressed by https://review.opendev.org/#/c/708695/. But while 1865223 is talking about *SG list*, 1863201 is related to *SG rule list*. I would like to make this differentiation, because both retrieval commands are not related. In this bug (1863201), the performance degradation multiplies by x3 (N->Q) the initial time. This could be caused by the OVO integration (O->P: https://review.opendev.org/#/c/284738/). Instead of using the DB object now we make this call using the OVO object containing the DB register (something like a DB view). That's something I still need to check. Just to make a concretion: the patch 708695 improves the *SG rule* retrieval, not the SG list command. Another punctualization is that this patch will help in the case of having a balance between SG rules and SG. This patch will help to retrieve from the DB only those SG rules belonging to the project. If, as you state in https://bugs.launchpad.net/neutron/+bug/1865223/comments/4, most of those SG rules belong to the same project, there is little improvement there. As commented, I'm still looking at improving the SG OVO performance. Regards On Mon, 2020-03-02 at 03:03 +0000, Erik Olof Gunnar Andersson wrote: > When we went from Mitaka to Rocky in August last year and we saw an exponential increase in api > times for listing security group rules. > > I think I last commented on this bug https://bugs.launchpad.net/neutron/+bug/1810563, but I have > brought it up on a few other occasions as well. > Bug #1810563 “adding rules to security groups is slow” : Bugs : neutron Sometime between liberty > and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review with > fixes is incoming. You can repro with a vanilla devstack install on master, and this script: > #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') export > OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost make_rules() { > iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file <<EOF > {... bugs.launchpad.net > > > From: Slawek Kaplonski > Sent: Saturday, February 29, 2020 12:44 AM > To: James Denton > Cc: openstack-discuss > Subject: Re: [neutron] security group list regression > > Hi, > > I just replied in Your bug report. Can You try to apply patch > https://urldefense.com/v3/__https://review.opendev.org/*/c/708695/__;Iw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Ul-OlUqQ$ > to see if that will help with this problem? > > > On 29 Feb 2020, at 02:41, James Denton wrote: > > > > Hello all, > > > > We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty severe > regression in the time it takes the API to return the list of security groups. This environment > has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack security > group list’ command to complete. I don’t have actual data from the same environment running > Newton, but was able to replicate this behavior with the following lab environments running a mix > of virtual and baremetal machines: > > > > Newton (VM) > > Rocky (BM) > > Stein (VM) > > Train (BM) > > > > Number of sec grps vs time in seconds: > > > > # Newton Rocky Stein Train > > 200 4.1 3.7 5.4 5.2 > > 500 5.3 7 11 9.4 > > 1000 7.2 12.4 19.2 16 > > 2000 9.2 24.2 35.3 30.7 > > 3000 12.1 36.5 52 44 > > 4000 16.1 47.2 73 58.9 > > 5000 18.4 55 90 69 > > > > As you can see (hopefully), the response time increased significantly between Newton and Rocky, > and has grown slightly ever since. We don't know, yet, if this behavior can be seen with other > 'list' commands or is limited to secgroups. We're currently verifying on some intermediate > releases to see where things went wonky. > > > > There are some similar recent reports out in the wild with little feedback: > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1788749__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Vx5jGlrA$ > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1721273__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8U9NbN_LA$ > > > > > I opened a bug here, too: > > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/1865223__;Kw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8UtMQ2-Dw$ > > > > > Bottom line: Has anyone else experienced similar regressions in recent releases? If so, were you > able to address them with any sort of tuning? > > > > Thanks in advance, > > James > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From whayutin at redhat.com Tue Mar 3 00:07:54 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Mon, 2 Mar 2020 17:07:54 -0700 Subject: [tripleo] centos-8 status Message-ID: Greetings everyone! First off, apologies for using hackmd in this case for our status. We will be moving to a fully non-hosted open source version asap. We have been making significant progress with CentOS-8 based jobs in TripleO Ussuri, our full status is here [1]. You will see tripleo-ci-centos-8 standalone jobs hitting your reviews. If you find any issues please just treat the same as any other bug and open a launchpad w/ "alert" set to the tag. Thanks for everybody's hard work and patience as we get the rest of the jobs running smoothly in the upstream. [1] https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Tue Mar 3 01:49:42 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 02 Mar 2020 19:49:42 -0600 Subject: [goals][Drop Python 2.7 Support] Week R-10 Update Message-ID: <1709e159fda.12941c762339759.7997748923601929729@ghanshyammann.com> Hello Everyone, Below is the progress on "Drop Python 2.7 Support" at end of R-10 week. Schedule: https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html#schedule Highlights: ======== * We already passed the deadline but still work is not completed on this. * Few tempest plugins are failing. * I request projects again to merge the passing patches asap. Project wise status and need reviews: ============================ Phase-1 status: All the OpenStack services have dropped the python2.7. Phase-2 status: * Few Tempest plugins are still not merged. I am debugging a few failing plugins with the project team. ** tempest plugins are passing and ready to merge. *** barbican-tempest-plugin: https://review.opendev.org/#/c/704083/ *** cyborg-tempest-plugin: https://review.opendev.org/#/c/704076/ *** magnum-tempest-plugin: https://review.opendev.org/#/c/704069/ *** congress-tempest-plugin: https://review.opendev.org/#/c/694437/ *** heat-tempest-plugin: https://review.opendev.org/#/c/704225/ *** kuryr-tempest-plugin: https://review.opendev.org/#/c/704072/ ** Failing plugins which need debugging and more work: *** trove-tempest-plugin: https://review.opendev.org/#/c/692041/ *** ironic-tempest-plugin: https://review.opendev.org/#/c/704093/ * Started pushing the required updates on deployment projects. ** Completed or no updates required: *** Openstack-Chef - not required *** Packaging-Rpm - Done *** Puppet Openstack- Done ** In progress: *** Openstack Charms *** Openstackansible - In-progress. centos7 jobs are failing on few projects. ** Waiting from projects team to know the status: *** Openstack-Helm (Helm charts for OpenStack services) *** Tripleo (Deployment service) * Open review: https://review.opendev.org/#/q/topic:drop-py27-support+status:open Phase-3 status: This is audit and requirement repo work which is not started yet. I will start this once all the phase-2 work mentioned above is completed. How you can help: ============== - Review the patches. Push the patches if I missed any repo. -gmann From akekane at redhat.com Tue Mar 3 05:00:45 2020 From: akekane at redhat.com (Abhishek Kekane) Date: Tue, 3 Mar 2020 10:30:45 +0530 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: Hi All, Thank you for making this different thread, OSC is not up to date with the current glance features and neither it has shown any interest in doing so. >From glance prospective we also didn't have any bandwidth to work on adding these support to OSC. There is some major feature gap between current OSC and Glance and that's the reason why glance does not recommend to use OSC. 1. Support for new image import workflow 2. Support for hidden images 3. Support for multihash 4. Support for multiple stores If anyone is interested to take up this work it will be great. Thanks & Best Regards, Abhishek Kekane On Tue, Mar 3, 2020 at 12:24 AM Sean Mooney wrote: > On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: > > As an openstack operator I was pretty ecstatic to hear that the > assortment of clients would be replaced by a single > > client. I would be disappointed to find that a component would not > integrate and would continue to use a separate > > client. This would be a step backward IMO. > > > > The discussion about microversions goes over my head, but I would hope > to see the developers get together and solve > > the issue and continue working toward integration. > just to summerisie it in a non technical way. > the project specific cli had a convention where the client would ask the > api what the newest micoverion it supported > and defualt to that if the clinet suported it. that meant that the same > command executed against two different clouds > with different versions of openstakc deploy could have different behavior > and different responces. so from an > interoperablity point of view that is not great but from a usablity point > of view the fact enduser dont have to care > about microverions and the client would try to do the right thing made > some things much simpler. > > the unifeid client (osc) chose to priorities interoperablity by defaulting > to the oldest micorverions, so for nova that > would be 2.0/2.1 meaning that if you execute the same command on two > different cloud with different version of nova it > will behave the same but if you want to use a feature intoduced in a later > micorverion you have to explcitly request > that via --os-compute-api-version or set that as a env var or in you > cloud.yaml > > so really the difference is that osc requires the end user to be explictl > about what micoversion to use and therefor be > explict about the behavior of the api they expect (this is what we expect > application that use the the api should do) > where as the project client tried to just work and use the latest > microverion which mostly workd excpet where we remove > a feature in a later micorverions. for example we removed the force option > on some move operation in nova because > allowing forcing caused many harder to fix issues. i dont thnk the nova > clinet would cap at the latest micorvierion that > allowed forcing. so the poject client genreally did not guarantee that a > command would work without specifcing a new > micorverison it just that we remove things a hell of a lot less often then > we add them. > > so as an end user that is the main difference between using osc vs glance > clinet other then the fact i belive there is a > bunch of stuff you can do with glance client that is missing in osc. > parity is a spereate disucssion but it is vaild > concern. > > -----Original Message----- > > From: Radosław Piliszek > > Sent: Monday, March 2, 2020 9:07 AM > > To: openstack-discuss > > Subject: Re: [glance] Different checksum between CLI and curl > > > > Folks, > > > > sorry to interrupt but I think we have diverged a bit too much from the > subject. > > Only last Gaetan message is on topic here. > > Please switch to new subject to discuss OSC future. > > > > -yoctozepto > > > > pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): > > > > > > > > > > > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: > > > > > > Hi, > > > > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano > wrote: > > > > > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane > wrote: > > > > > > Hi Gaëtan, > > > > > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > > > I will recommend you to check the same behaviour using > > > > > > python-glanceclient. > > > > > > > > > > That's not cool - everyone has switched to OSC. It's also the first > > > > > time I've heard of it. > > > > > > > > > > > From the end user perspective, we’ve had positive feedback on the > convergence to OSC from our cloud consumers. > > > > > > There has been great progress with Manila to get shares included ( > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= > > > ) and it would be a pity if we’re asking our end users to understand > all of the different project names and > > > inconsistent options/arguments/syntax. > > > > > > We had hoped for a project goal to get everyone aligned on OSC but > there was not consensus on this, I’d still > > > encourage it to simplify the experience for OpenStack cloud consumers. > > > > > > Tim > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Tue Mar 3 06:10:25 2020 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Tue, 3 Mar 2020 07:10:25 +0100 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: On Tue, 3 Mar 2020, 06:08 Abhishek Kekane, wrote: > Hi All, > > Thank you for making this different thread, > > OSC is not up to date with the current glance features and neither it has > shown any interest in doing so. > From glance prospective we also didn't have any bandwidth to work on > adding these support to OSC. > That's honestly not true this days > > There is some major feature gap between current OSC and Glance and that's > the reason why glance does not recommend to use OSC. > That's still not reason to say please don't use it anymore. 1. Support for new image import workflow > Partially implemented by me and I continue working on that 2. Support for hidden images > Implemented 3. Support for multihash > 4. Support for multiple stores > I am relying on OSC and especially for image service trying to bring it in a more useful state, thus fixing huge parts in SDK. > If anyone is interested to take up this work it will be great. > > Thanks & Best Regards, > > Abhishek Kekane > > > On Tue, Mar 3, 2020 at 12:24 AM Sean Mooney wrote: > >> On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: >> > As an openstack operator I was pretty ecstatic to hear that the >> assortment of clients would be replaced by a single >> > client. I would be disappointed to find that a component would not >> integrate and would continue to use a separate >> > client. This would be a step backward IMO. >> > >> > The discussion about microversions goes over my head, but I would hope >> to see the developers get together and solve >> > the issue and continue working toward integration. >> just to summerisie it in a non technical way. >> the project specific cli had a convention where the client would ask the >> api what the newest micoverion it supported >> and defualt to that if the clinet suported it. that meant that the same >> command executed against two different clouds >> with different versions of openstakc deploy could have different behavior >> and different responces. so from an >> interoperablity point of view that is not great but from a usablity point >> of view the fact enduser dont have to care >> about microverions and the client would try to do the right thing made >> some things much simpler. >> >> the unifeid client (osc) chose to priorities interoperablity by >> defaulting to the oldest micorverions, so for nova that >> would be 2.0/2.1 meaning that if you execute the same command on two >> different cloud with different version of nova it >> will behave the same but if you want to use a feature intoduced in a >> later micorverion you have to explcitly request >> that via --os-compute-api-version or set that as a env var or in you >> cloud.yaml >> >> so really the difference is that osc requires the end user to be explictl >> about what micoversion to use and therefor be >> explict about the behavior of the api they expect (this is what we expect >> application that use the the api should do) >> where as the project client tried to just work and use the latest >> microverion which mostly workd excpet where we remove >> a feature in a later micorverions. for example we removed the force >> option on some move operation in nova because >> allowing forcing caused many harder to fix issues. i dont thnk the nova >> clinet would cap at the latest micorvierion that >> allowed forcing. so the poject client genreally did not guarantee that a >> command would work without specifcing a new >> micorverison it just that we remove things a hell of a lot less often >> then we add them. >> >> so as an end user that is the main difference between using osc vs glance >> clinet other then the fact i belive there is a >> bunch of stuff you can do with glance client that is missing in osc. >> parity is a spereate disucssion but it is vaild >> concern. >> >> -----Original Message----- >> > From: Radosław Piliszek >> > Sent: Monday, March 2, 2020 9:07 AM >> > To: openstack-discuss >> > Subject: Re: [glance] Different checksum between CLI and curl >> > >> > Folks, >> > >> > sorry to interrupt but I think we have diverged a bit too much from the >> subject. >> > Only last Gaetan message is on topic here. >> > Please switch to new subject to discuss OSC future. >> > >> > -yoctozepto >> > >> > pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): >> > > >> > > >> > > >> > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: >> > > >> > > Hi, >> > > >> > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano >> wrote: >> > > > >> > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >> > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane >> wrote: >> > > > > > Hi Gaëtan, >> > > > > > >> > > > > > Glance team doesn't recommend to use OSC anymore. >> > > > > > I will recommend you to check the same behaviour using >> > > > > > python-glanceclient. >> > > > > >> > > > > That's not cool - everyone has switched to OSC. It's also the >> first >> > > > > time I've heard of it. >> > > > > >> > > >> > > From the end user perspective, we’ve had positive feedback on the >> convergence to OSC from our cloud consumers. >> > > >> > > There has been great progress with Manila to get shares included ( >> > > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= >> > > ) and it would be a pity if we’re asking our end users to understand >> all of the different project names and >> > > inconsistent options/arguments/syntax. >> > > >> > > We had hoped for a project goal to get everyone aligned on OSC but >> there was not consensus on this, I’d still >> > > encourage it to simplify the experience for OpenStack cloud consumers. >> > > >> > > Tim >> > > >> > > >> > >> > >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From akekane at redhat.com Tue Mar 3 06:17:32 2020 From: akekane at redhat.com (Abhishek Kekane) Date: Tue, 3 Mar 2020 11:47:32 +0530 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: Hi Artem, Thanks for sharing the update. The decision was collectively taken during last cycle by glance team, as we don't have enough people/resources to work on this front. I will be more than happy to change this if anyone comes forward and bridge the gaps. Thanks & Best Regards, Abhishek Kekane On Tue, Mar 3, 2020 at 11:40 AM Artem Goncharov wrote: > > > On Tue, 3 Mar 2020, 06:08 Abhishek Kekane, wrote: > >> Hi All, >> >> Thank you for making this different thread, >> >> OSC is not up to date with the current glance features and neither it has >> shown any interest in doing so. >> From glance prospective we also didn't have any bandwidth to work on >> adding these support to OSC. >> > > > That's honestly not true this days > >> >> There is some major feature gap between current OSC and Glance and that's >> the reason why glance does not recommend to use OSC. >> > > That's still not reason to say please don't use it anymore. > > 1. Support for new image import workflow >> > Partially implemented by me and I continue working on that > > 2. Support for hidden images >> > Implemented > > 3. Support for multihash >> > 4. Support for multiple stores >> > > I am relying on OSC and especially for image service trying to bring it in > a more useful state, thus fixing huge parts in SDK. > > >> If anyone is interested to take up this work it will be great. >> >> Thanks & Best Regards, >> >> Abhishek Kekane >> >> >> On Tue, Mar 3, 2020 at 12:24 AM Sean Mooney wrote: >> >>> On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: >>> > As an openstack operator I was pretty ecstatic to hear that the >>> assortment of clients would be replaced by a single >>> > client. I would be disappointed to find that a component would not >>> integrate and would continue to use a separate >>> > client. This would be a step backward IMO. >>> > >>> > The discussion about microversions goes over my head, but I would hope >>> to see the developers get together and solve >>> > the issue and continue working toward integration. >>> just to summerisie it in a non technical way. >>> the project specific cli had a convention where the client would ask the >>> api what the newest micoverion it supported >>> and defualt to that if the clinet suported it. that meant that the same >>> command executed against two different clouds >>> with different versions of openstakc deploy could have different >>> behavior and different responces. so from an >>> interoperablity point of view that is not great but from a usablity >>> point of view the fact enduser dont have to care >>> about microverions and the client would try to do the right thing made >>> some things much simpler. >>> >>> the unifeid client (osc) chose to priorities interoperablity by >>> defaulting to the oldest micorverions, so for nova that >>> would be 2.0/2.1 meaning that if you execute the same command on two >>> different cloud with different version of nova it >>> will behave the same but if you want to use a feature intoduced in a >>> later micorverion you have to explcitly request >>> that via --os-compute-api-version or set that as a env var or in you >>> cloud.yaml >>> >>> so really the difference is that osc requires the end user to be >>> explictl about what micoversion to use and therefor be >>> explict about the behavior of the api they expect (this is what we >>> expect application that use the the api should do) >>> where as the project client tried to just work and use the latest >>> microverion which mostly workd excpet where we remove >>> a feature in a later micorverions. for example we removed the force >>> option on some move operation in nova because >>> allowing forcing caused many harder to fix issues. i dont thnk the nova >>> clinet would cap at the latest micorvierion that >>> allowed forcing. so the poject client genreally did not guarantee that a >>> command would work without specifcing a new >>> micorverison it just that we remove things a hell of a lot less often >>> then we add them. >>> >>> so as an end user that is the main difference between using osc vs >>> glance clinet other then the fact i belive there is a >>> bunch of stuff you can do with glance client that is missing in osc. >>> parity is a spereate disucssion but it is vaild >>> concern. >>> >>> -----Original Message----- >>> > From: Radosław Piliszek >>> > Sent: Monday, March 2, 2020 9:07 AM >>> > To: openstack-discuss >>> > Subject: Re: [glance] Different checksum between CLI and curl >>> > >>> > Folks, >>> > >>> > sorry to interrupt but I think we have diverged a bit too much from >>> the subject. >>> > Only last Gaetan message is on topic here. >>> > Please switch to new subject to discuss OSC future. >>> > >>> > -yoctozepto >>> > >>> > pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): >>> > > >>> > > >>> > > >>> > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: >>> > > >>> > > Hi, >>> > > >>> > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano >>> wrote: >>> > > > >>> > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >>> > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane >>> wrote: >>> > > > > > Hi Gaëtan, >>> > > > > > >>> > > > > > Glance team doesn't recommend to use OSC anymore. >>> > > > > > I will recommend you to check the same behaviour using >>> > > > > > python-glanceclient. >>> > > > > >>> > > > > That's not cool - everyone has switched to OSC. It's also the >>> first >>> > > > > time I've heard of it. >>> > > > > >>> > > >>> > > From the end user perspective, we’ve had positive feedback on the >>> convergence to OSC from our cloud consumers. >>> > > >>> > > There has been great progress with Manila to get shares included ( >>> > > >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= >>> > > ) and it would be a pity if we’re asking our end users to >>> understand all of the different project names and >>> > > inconsistent options/arguments/syntax. >>> > > >>> > > We had hoped for a project goal to get everyone aligned on OSC but >>> there was not consensus on this, I’d still >>> > > encourage it to simplify the experience for OpenStack cloud >>> consumers. >>> > > >>> > > Tim >>> > > >>> > > >>> > >>> > >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralonsoh at redhat.com Tue Mar 3 09:48:57 2020 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Tue, 03 Mar 2020 09:48:57 +0000 Subject: [neutron] security group list regression In-Reply-To: <7DD0691D-19A3-4CDB-B377-F67829A86AD7@rackspace.com> References: <7DD0691D-19A3-4CDB-B377-F67829A86AD7@rackspace.com> Message-ID: <4740f4822e7b571b40aa5dc549e3c59a2ee659c4.camel@redhat.com> Hello James: Yes, this is a known issue in OSclient: most of the "objects" (networks, subnets, routers, etc) to be retrieved, can usually can be retrieved by ID and by name. OSclient tries first to use the ID because is unique and a DB key. Then, instead of asking the server for a unique register (filtered by the name), the client retrieves the whole list and filters the results. But this problem was resolved in Train: https://review.opendev.org/#/c/637238/. Can you check, in openstacksdk, that you have this patch? At least in T. According to [1] and [2], "name" should be used as filter in the OSsdk "find" call. Regards. [1]https://review.opendev.org/#/c/637238/20/openstack/resource.py [2]https://github.com/openstack/openstacksdk/blob/master/openstack/network/v2/security_group.py#L29 On Mon, 2020-03-02 at 22:25 +0000, James Denton wrote: > Rodolfo, > > Thanks for continuing to push this on the ML and in the bug report. > > Happy to report that the client and SDK patches you provided have drastically reduced the SG list > time from ~90-120s to ~12-14s within Stein and Train lab environments. > > One last thing... when you perform an 'openstack security group delete ', the initial lookup > by name fails. In Train, the client falls back to using the 'name' parameter (/security- > groups?name=). This lookup is quick and the security group is found and deleted. However, on > Rocky/Stein (e.g. client 3.18.1), instead of searching by parameter, the client appears to perform > a GET /security-groups without limiting the fields and takes a long time. > > 'openstack security group list' with patch: > REQ: curl -g -i -X GET " > http://10.0.236.150:9696/v2.0/security-groups?fields=set%28%5B%27description%27%2C+%27project_id%27%2C+%27id%27%2C+%27tags%27%2C+%27name%27%5D%29 > " -H "Accept: application/json" -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python- > requests/2.21.0 CPython/2.7.17" -H "X-Auth-Token: > {SHA256}3e747da939e8c4befe72d5ca7105971508bd56cdf36208ba6b960d1aee6d19b6" > > 'openstack security group delete ': > > Train (notice the name param): > REQ: curl -g -i -X GET http://10.20.0.11:9696/v2.0/security-groups/train-test-1755 -H "User-Agent: > openstacksdk/0.36.0 keystoneauth1/3.17.1 python-requests/2.22.0 CPython/3.6.7" -H "X-Auth-Token: > {SHA256}bf291d5f12903876fc69151db37d295da961ba684a575e77fb6f4829b55df1bf" > http://10.20.0.11:9696 "GET /v2.0/security-groups/train-test-1755 HTTP/1.1" 404 125 > REQ: curl -g -i -X GET "http://10.20.0.11:9696/v2.0/security-groups?name=train-test-1755" -H > "Accept: application/json" -H "User-Agent: openstacksdk/0.36.0 keystoneauth1/3.17.1 python- > requests/2.22.0 CPython/3.6.7" -H "X-Auth-Token: > {SHA256}bf291d5f12903876fc69151db37d295da961ba684a575e77fb6f4829b55df1bf" > http://10.20.0.11:9696 "GET /v2.0/security-groups?name=train-test-1755 HTTP/1.1" 200 1365 > > Stein & below (notice lack of fields): > REQ: curl -g -i -X GET http://10.0.236.150:9696/v2.0/security-groups/stein-test-5189 -H "User- > Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H "X-Auth- > Token: {SHA256}e9f87afe851ff5380d8402ee81199c466be9c84fe67ed0302e8b178f33aa1fc2" > http://10.0.236.150:9696 "GET /v2.0/security-groups/stein-test-5189 HTTP/1.1" 404 125 > REQ: curl -g -i -X GET http://10.0.236.150:9696/v2.0/security-groups -H "Accept: application/json" > -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H > "X-Auth-Token: {SHA256}e9f87afe851ff5380d8402ee81199c466be9c84fe67ed0302e8b178f33aa1fc2" > > > Haven't quite figured out where fields can be used to speed up the delete process on the older > client, or if the newer client would be backwards-compatible (and how far back). > > Thanks, > James > > On 3/2/20, 9:31 AM, "James Denton" wrote: > > CAUTION: This message originated externally, please use caution when clicking on links or > opening attachments! > > > Thanks, Rodolfo. I'll take a look at each of these after coffee and clarify my position (if > needed). > > James > > On 3/2/20, 6:27 AM, "Rodolfo Alonso" wrote: > > CAUTION: This message originated externally, please use caution when clicking on links or > opening attachments! > > > Hello James: > > Just to make a quick summary of the status of the commented bugs/regressions: > > 1) https://bugs.launchpad.net/neutron/+bug/1810563: adding rules to security groups is > slow > That was addressed in https://review.opendev.org/#/c/633145/ and > https://review.opendev.org/#/c/637407/, removing the O^2 check and using lazy loading. > > > 2) https://bugzilla.redhat.com/show_bug.cgi?id=1788749: Neutron List networks API > regression > The last reply was marked as private. I've undone this and you can read now c#2. Testing > with a > similar scenario, I don't see any performance degradation between Queens and Train. > > > 3) https://bugzilla.redhat.com/show_bug.cgi?id=1721273: Neutron API List Ports Performance > regression > That problem was solved in https://review.opendev.org/#/c/667981/ and > https://review.opendev.org/#/c/667998/, by refactoring how the port QoS extension was > reading and > applying the QoS info in the port dict. > > > 4) https://bugs.launchpad.net/neutron/+bug/1865223: regression for security group list > between > Newton and Rocky+ > > This is similar to https://bugs.launchpad.net/neutron/+bug/1863201. In this case, the > regression was > detected from R to S. The performance dropped from 3 secs to 110 secs (36x). That issue > was > addressed by https://review.opendev.org/#/c/708695/. > > But while 1865223 is talking about *SG list*, 1863201 is related to *SG rule list*. I > would like to > make this differentiation, because both retrieval commands are not related. > > In this bug (1863201), the performance degradation multiplies by x3 (N->Q) the initial > time. This > could be caused by the OVO integration (O->P: https://review.opendev.org/#/c/284738/). > Instead of > using the DB object now we make this call using the OVO object containing the DB register > (something > like a DB view). That's something I still need to check. > > Just to make a concretion: the patch 708695 improves the *SG rule* retrieval, not the SG > list > command. Another punctualization is that this patch will help in the case of having a > balance > between SG rules and SG. This patch will help to retrieve from the DB only those SG rules > belonging > to the project. If, as you state in > https://bugs.launchpad.net/neutron/+bug/1865223/comments/4, most > of those SG rules belong to the same project, there is little improvement there. > > As commented, I'm still looking at improving the SG OVO performance. > > Regards > > > On Mon, 2020-03-02 at 03:03 +0000, Erik Olof Gunnar Andersson wrote: > > When we went from Mitaka to Rocky in August last year and we saw an exponential increase > in api > > times for listing security group rules. > > > > I think I last commented on this bug https://bugs.launchpad.net/neutron/+bug/1810563, > but I have > > brought it up on a few other occasions as well. > > Bug #1810563 “adding rules to security groups is slow” : Bugs : neutron Sometime > between liberty > > and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review > with > > fixes is incoming. You can repro with a vanilla devstack install on master, and this > script: > > #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') > export > > OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost > make_rules() { > > iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file > <<EOF > > {... bugs.launchpad.net > > > > > > From: Slawek Kaplonski > > Sent: Saturday, February 29, 2020 12:44 AM > > To: James Denton > > Cc: openstack-discuss > > Subject: Re: [neutron] security group list regression > > > > Hi, > > > > I just replied in Your bug report. Can You try to apply patch > > > https://urldefense.com/v3/__https://review.opendev.org/*/c/708695/__;Iw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Ul-OlUqQ$ > > to see if that will help with this problem? > > > > > On 29 Feb 2020, at 02:41, James Denton wrote: > > > > > > Hello all, > > > > > > We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty > severe > > regression in the time it takes the API to return the list of security groups. This > environment > > has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack > security > > group list’ command to complete. I don’t have actual data from the same environment > running > > Newton, but was able to replicate this behavior with the following lab environments > running a mix > > of virtual and baremetal machines: > > > > > > Newton (VM) > > > Rocky (BM) > > > Stein (VM) > > > Train (BM) > > > > > > Number of sec grps vs time in seconds: > > > > > > # Newton Rocky Stein Train > > > 200 4.1 3.7 5.4 5.2 > > > 500 5.3 7 11 9.4 > > > 1000 7.2 12.4 19.2 16 > > > 2000 9.2 24.2 35.3 30.7 > > > 3000 12.1 36.5 52 44 > > > 4000 16.1 47.2 73 58.9 > > > 5000 18.4 55 90 69 > > > > > > As you can see (hopefully), the response time increased significantly between Newton > and Rocky, > > and has grown slightly ever since. We don't know, yet, if this behavior can be seen with > other > > 'list' commands or is limited to secgroups. We're currently verifying on some > intermediate > > releases to see where things went wonky. > > > > > > There are some similar recent reports out in the wild with little feedback: > > > > > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1788749__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Vx5jGlrA$ > > > > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1721273__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8U9NbN_LA$ > > > > > > > > I opened a bug here, too: > > > > > > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/1865223__;Kw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8UtMQ2-Dw$ > > > > > > > > Bottom line: Has anyone else experienced similar regressions in recent releases? If > so, were you > > able to address them with any sort of tuning? > > > > > > Thanks in advance, > > > James > > > > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > > > > > > From alfredo.deluca at gmail.com Tue Mar 3 09:50:41 2020 From: alfredo.deluca at gmail.com (Alfredo De Luca) Date: Tue, 3 Mar 2020 10:50:41 +0100 Subject: [CINDER] Snapshots export Message-ID: Hi all. We have our env with Openstack (Train) and cinder with CEPH (nautilus) backend. We are creating automatic volumes snapshots and now we'd like to export them as a backup/restore plan. After exporting the snapshots we will use Acronis as backup tool. I couldn't find the right steps/commands to exports the snapshots. Any info? Cheers -- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: From amotoki at gmail.com Tue Mar 3 10:11:04 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Tue, 3 Mar 2020 19:11:04 +0900 Subject: [release][tc][horizon] xstatic repositories marked as deprecated In-Reply-To: References: Message-ID: Thanks Thierry for the detail explanation. The horizon team will update the corresponding repos for new minor releases and follow the usual release process. One question: we have passed the milestone-2. Is it better to wait till Victoria dev cycle is open? Thanks, Akihiro On Fri, Feb 28, 2020 at 1:47 AM Thierry Carrez wrote: > > Thierry Carrez wrote: > > The way we've been handling this in the past was to ignore past releases > > (since they are not signed by the release team), and push a new one > > through the releases repository. It should replace the unofficial one in > > PyPI and make sure all is in order. > > Clarification with a practical example: > > xstatic-hogan 2.0.0.2 is on PyPI, but has no tag in the > openstack/xstatic-hogan repo, and no deliverable file in openstack/releases. > > Solution is to resync everything by proposing a 2.0.0.3 release that > will have tag, be in openstack/releases and have a matching upload on PyPI. > > This is done by: > > - bumping BUILD at > https://opendev.org/openstack/xstatic-hogan/src/branch/master/xstatic/pkg/hogan/__init__.py# > > - adding a deliverables/_independent/xstatic-hogan.yaml file in > openstack/releases defining a tag for 2.0.0.3 > > - removing the "deprecated" line from > https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml#L542 > > Repeat for every affected package :) > > -- > Thierry Carrez (ttx) > From ignaziocassano at gmail.com Tue Mar 3 10:11:37 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 3 Mar 2020 11:11:37 +0100 Subject: [queens][neutron][fwaas_v2] PENDING_UPDATE Message-ID: Hello All, I installed firewall v2 on queens based on centos 7. I create a firewall group policy and a firewall group rulle with that policy. The firewall results ACTIVE AND UP and INACTIVE When I ttry to apply the firewall group to an instance port : openstack firewall group set --port c7c8be58-35de-47fe-87db-39bbd681db8b fwg1 It does not works and goes in pending update status. L3 agent log reports: Could not load neutron_fwaas.services.firewall.drivers.linux.iptables_fwaas_v2.IptablesFwaasDriver But the file esists I am using firewall_driver = openvswitch Please, whai is wrong ? I read Supports L2 firewalling (VM ports) was planned for ocata and I am on queens. Please, help me. Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Tue Mar 3 12:33:29 2020 From: donny at fortnebula.com (Donny Davis) Date: Tue, 3 Mar 2020 07:33:29 -0500 Subject: [queens][neutron][fwaas_v2] PENDING_UPDATE In-Reply-To: References: Message-ID: I have never really used fwaas, but I do believe its targeted at routers. Security groups already do firewalling for the vm ports Donny Davis c: 805 814 6800 On Tue, Mar 3, 2020, 5:15 AM Ignazio Cassano wrote: > Hello All, I installed firewall v2 on queens based on centos 7. > I create a firewall group policy and a firewall group rulle with that > policy. > > The firewall results ACTIVE AND UP and INACTIVE > > When I ttry to apply the firewall group to an instance port : > openstack firewall group set --port c7c8be58-35de-47fe-87db-39bbd681db8b > fwg1 > > It does not works and goes in pending update status. > > > L3 agent log reports: > Could not load > neutron_fwaas.services.firewall.drivers.linux.iptables_fwaas_v2.IptablesFwaasDriver > > But the file esists > > I am using firewall_driver = openvswitch > > Please, whai is wrong ? > > I read Supports L2 firewalling (VM ports) was planned for ocata and I am > on queens. > > Please, help me. > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guimalufb at gmail.com Mon Mar 2 20:44:15 2020 From: guimalufb at gmail.com (Gui Maluf) Date: Mon, 2 Mar 2020 20:44:15 +0000 Subject: [Swift] Errno 13 Permission denied writing objects xattr while upgrading from Kilo to Queens Message-ID: Hi all, I'm struggling with something really wierd. 3 weeks ago I started upgrading my Keystone + Swift Ubuntu environment from Kilo to Queens. So I moved from Ubuntu 14.04 to 18.04. I can create new accounts and containers. But no objects. I think between Mitaka and Newton my storages started to throw Permission Denied error while writing object metadata. I saw that the piece of python code where I getting problem was changed in Rocky version and in hope of getting things fixed I've upgraded storage version. But the error persists. http://paste.openstack.org/show/790217/ I've check everything I could, user, permissions, mount options. But I still getting this error. I wrote a python script for creating files and writing metadata within the swift mount with swift user and everything works fines. Don't know what to do anymore. This is a "dev" environment with two storages only and a few disks. Since I'm planning to do in the production environment I'm quite scared if this happens again. Thanks in advance -- *guilherme* \n11 \t *maluf* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Tue Mar 3 12:41:11 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 3 Mar 2020 13:41:11 +0100 Subject: [queens][neutron][fwaas_v2] PENDING_UPDATE In-Reply-To: References: Message-ID: Hello Donny, please visit this link: it should work: https://superuser.openstack.org/articles/firewall-service-openstack/ Il giorno mar 3 mar 2020 alle ore 13:33 Donny Davis ha scritto: > I have never really used fwaas, but I do believe its targeted at routers. > > Security groups already do firewalling for the vm ports > > > > Donny Davis > c: 805 814 6800 > > On Tue, Mar 3, 2020, 5:15 AM Ignazio Cassano > wrote: > >> Hello All, I installed firewall v2 on queens based on centos 7. >> I create a firewall group policy and a firewall group rulle with that >> policy. >> >> The firewall results ACTIVE AND UP and INACTIVE >> >> When I ttry to apply the firewall group to an instance port : >> openstack firewall group set --port c7c8be58-35de-47fe-87db-39bbd681db8b >> fwg1 >> >> It does not works and goes in pending update status. >> >> >> L3 agent log reports: >> Could not load >> neutron_fwaas.services.firewall.drivers.linux.iptables_fwaas_v2.IptablesFwaasDriver >> >> But the file esists >> >> I am using firewall_driver = openvswitch >> >> Please, whai is wrong ? >> >> I read Supports L2 firewalling (VM ports) was planned for ocata and I am >> on queens. >> >> Please, help me. >> >> Ignazio >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Tue Mar 3 13:01:04 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Tue, 3 Mar 2020 13:01:04 +0000 Subject: [nova] [neutron] multiple fixed_ip Message-ID: <20200303130104.GA29109@sync> Hello all, I was doing some tests to create a server using nova API. My objective is to create a server with one port but multiples IPs (one IPv4 and one IPv6). If I understand well the neutron API, I can create a port using the fixed_ips array parameter [1] Unfortunately, on nova side, it seems to only accept a string with only one ip (fixed_ip) [2] Is it mandatory for me to create the port with neutron? Or is there any trick that I missed on nova API side? Thanks! [1] https://docs.openstack.org/api-ref/network/v2/?expanded=create-port-detail#ports [2] https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server -- Arnaud Morin From radoslaw.piliszek at gmail.com Tue Mar 3 13:23:52 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 3 Mar 2020 14:23:52 +0100 Subject: [nova] [neutron] multiple fixed_ip In-Reply-To: <20200303130104.GA29109@sync> References: <20200303130104.GA29109@sync> Message-ID: Hi Arnaud, Non-core here. Last time I checked you had to decide on one and then update with neutron (or first create the port with neutron and then give it to nova :-) ). Moreover, not sure if IPv6 goes through Nova directly or not (docs suggest still nah). -yoctozepto wt., 3 mar 2020 o 14:09 Arnaud Morin napisał(a): > > > Hello all, > > I was doing some tests to create a server using nova API. > My objective is to create a server with one port but multiples IPs (one > IPv4 and one IPv6). > > If I understand well the neutron API, I can create a port using the > fixed_ips array parameter [1] > > Unfortunately, on nova side, it seems to only accept a string with only > one ip (fixed_ip) [2] > > Is it mandatory for me to create the port with neutron? > Or is there any trick that I missed on nova API side? > > Thanks! > > > [1] https://docs.openstack.org/api-ref/network/v2/?expanded=create-port-detail#ports > [2] https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server > > > > -- > Arnaud Morin > > From dtantsur at redhat.com Tue Mar 3 13:35:28 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Tue, 3 Mar 2020 14:35:28 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: <70af048d-cf62-4f27-84fe-e6ed7b959837@www.fastmail.com> References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> <70af048d-cf62-4f27-84fe-e6ed7b959837@www.fastmail.com> Message-ID: On Mon, Mar 2, 2020 at 8:46 PM Clark Boylan wrote: > On Mon, Mar 2, 2020, at 11:01 AM, Dmitry Tantsur wrote: > > > > > > On Mon, Mar 2, 2020 at 5:37 PM Sean Mooney wrote: > > > On Mon, 2020-03-02 at 16:49 +0100, Dmitry Tantsur wrote: > > > > Hi, > > > > > > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano > wrote: > > > > > > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane < > akekane at redhat.com> wrote: > > > > > > > Hi Gaëtan, > > > > > > > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > > > > I will recommend you to check the same behaviour using > > > > > > > python-glanceclient. > > > > > > > > > > > > That's not cool - everyone has switched to OSC. It's also the > first > > > > > > time I've heard of it. > > > > > > > > > > > > > > > > Do we have proper microversion support then? This is a blocker > for cinder. > > > > > > > > > > > > > The ironic team has been successfully hacking around the absence of > a > > > > native microversion support for a while. We use ironicclient > instead of > > > > openstacksdk, which makes things harder. If you use openstacksdk, > it's > > > > easier to teach it microversions. In any case, I can provide some > guidance > > > > if you'd like to. > > > > > > > > Dmitry > > > that is also problematic. > > > by harcking around it it gives the ironic command a different > behavior to the rest of osc. > > > osc does support microverions it just does not support automatic > versin negociation which is > > > what you are hacking in. > > > > Right, and it's a hard requirement for the CLI to be remotely usable. > > > > > > i do agree that it would be nice to have support for version > negociation where by you could do somehting like > > > --os-compute-api-version=auto to opt in to it but automatic > microverions detetion does make it harder to do help > > > text generation unless you make "openstack --cloud=my-cloud > --os-compute-api-version=auto help server create" call out > > > to keystone get the nova endpoint and then lookup its max > microversion when you render the help text. > > > > The "auto" must be a default. This is what the users expect: the CLI > > just working. Defaulting to anything else does them a huge disservice > > (been there, done that). > > As a user I strongly disagree. I don't want an API to magically start > acting differently because the cloud side has upgraded. Those changes are > opaque to me and I shouldn't need to know about them. Instead I should be > able to opt into using new features when I know I need them. This is easily > achieved by setting the desired microversion when you know you need it. > We're talking about CLI, not API. I agree with you when it comes to calling code, but CLI must just work. This is how all CLI in the world work: you either get a behavior or you get a clear failure. It's the other way around: if you want to fix the feature set, and you know what you're doing, you can set a specific version in your environment. And, Clark, you and I are not mere users even if we use our CLI regularly. Draw the border here: a regular user is someone who doesn't know what a microversion even IS, to say nothing about a way to find the required microversion for a feature. These are the users I've dealt with and they have all been frustrated by using microversions explicitly. For a probably clearer explanation let me refer you to the API SIG specification that covers how to expose microversions: https://specs.openstack.org/openstack/api-sig/guidelines/sdk-exposing-microversions.html (see specifically about high-level SDKs). Dmitry > > > > > > > with that said if adding --os-image-api-version=auto was enough to > get the glance team to fully adopt osc > > > then i think that would be better then partioning the community > between osc and legacy client. > > > osc should behave consistently for all projects however so adding > negocaiton for ironic and not for other services > > > is not a good thing imo but i guess you were able to do that as > ironic is integrated as a plugin correct? > > > > Yep. We could not wait for OSC to implement it because the CLI is > > borderline unusable without this negotiation in place. I don't recall > > what prevented us from updating OSC, but I think there was a reason, > > probably not entirely technical. > > > > Dmitry > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Tue Mar 3 13:47:40 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 3 Mar 2020 14:47:40 +0100 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5594320.rlFBNPpyZN@whitebase.usersys.redhat.com> <70af048d-cf62-4f27-84fe-e6ed7b959837@www.fastmail.com> Message-ID: Folks, this is the *wrong* topic to discuss it. PS (basking in offtop): Thanks, Dmitry, I was looking for such a doc for jslib. -yoctozepto wt., 3 mar 2020 o 14:38 Dmitry Tantsur napisał(a): > > > > On Mon, Mar 2, 2020 at 8:46 PM Clark Boylan wrote: >> >> On Mon, Mar 2, 2020, at 11:01 AM, Dmitry Tantsur wrote: >> > >> > >> > On Mon, Mar 2, 2020 at 5:37 PM Sean Mooney wrote: >> > > On Mon, 2020-03-02 at 16:49 +0100, Dmitry Tantsur wrote: >> > > > Hi, >> > > > >> > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: >> > > > >> > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >> > > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: >> > > > > > > Hi Gaëtan, >> > > > > > > >> > > > > > > Glance team doesn't recommend to use OSC anymore. >> > > > > > > I will recommend you to check the same behaviour using >> > > > > > > python-glanceclient. >> > > > > > >> > > > > > That's not cool - everyone has switched to OSC. It's also the first >> > > > > > time I've heard of it. >> > > > > > >> > > > > >> > > > > Do we have proper microversion support then? This is a blocker for cinder. >> > > > > >> > > > >> > > > The ironic team has been successfully hacking around the absence of a >> > > > native microversion support for a while. We use ironicclient instead of >> > > > openstacksdk, which makes things harder. If you use openstacksdk, it's >> > > > easier to teach it microversions. In any case, I can provide some guidance >> > > > if you'd like to. >> > > > >> > > > Dmitry >> > > that is also problematic. >> > > by harcking around it it gives the ironic command a different behavior to the rest of osc. >> > > osc does support microverions it just does not support automatic versin negociation which is >> > > what you are hacking in. >> > >> > Right, and it's a hard requirement for the CLI to be remotely usable. >> > > >> > > i do agree that it would be nice to have support for version negociation where by you could do somehting like >> > > --os-compute-api-version=auto to opt in to it but automatic microverions detetion does make it harder to do help >> > > text generation unless you make "openstack --cloud=my-cloud --os-compute-api-version=auto help server create" call out >> > > to keystone get the nova endpoint and then lookup its max microversion when you render the help text. >> > >> > The "auto" must be a default. This is what the users expect: the CLI >> > just working. Defaulting to anything else does them a huge disservice >> > (been there, done that). >> >> As a user I strongly disagree. I don't want an API to magically start acting differently because the cloud side has upgraded. Those changes are opaque to me and I shouldn't need to know about them. Instead I should be able to opt into using new features when I know I need them. This is easily achieved by setting the desired microversion when you know you need it. > > > We're talking about CLI, not API. I agree with you when it comes to calling code, but CLI must just work. This is how all CLI in the world work: you either get a behavior or you get a clear failure. It's the other way around: if you want to fix the feature set, and you know what you're doing, you can set a specific version in your environment. > > And, Clark, you and I are not mere users even if we use our CLI regularly. Draw the border here: a regular user is someone who doesn't know what a microversion even IS, to say nothing about a way to find the required microversion for a feature. These are the users I've dealt with and they have all been frustrated by using microversions explicitly. > > For a probably clearer explanation let me refer you to the API SIG specification that covers how to expose microversions: https://specs.openstack.org/openstack/api-sig/guidelines/sdk-exposing-microversions.html (see specifically about high-level SDKs). > > Dmitry > >> >> >> > > >> > > with that said if adding --os-image-api-version=auto was enough to get the glance team to fully adopt osc >> > > then i think that would be better then partioning the community between osc and legacy client. >> > > osc should behave consistently for all projects however so adding negocaiton for ironic and not for other services >> > > is not a good thing imo but i guess you were able to do that as ironic is integrated as a plugin correct? >> > >> > Yep. We could not wait for OSC to implement it because the CLI is >> > borderline unusable without this negotiation in place. I don't recall >> > what prevented us from updating OSC, but I think there was a reason, >> > probably not entirely technical. >> > >> > Dmitry >> From sean.mcginnis at gmx.com Tue Mar 3 13:54:41 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 3 Mar 2020 07:54:41 -0600 Subject: [release][tc][horizon] xstatic repositories marked as deprecated In-Reply-To: References: Message-ID: On 3/3/20 4:11 AM, Akihiro Motoki wrote: > Thanks Thierry for the detail explanation. > The horizon team will update the corresponding repos for new minor > releases and follow the usual release process. > One question: we have passed the milestone-2. Is it better to wait > till Victoria dev cycle is open? > > Thanks, > Akihiro We are past the deadline for inclusion in ussuri. But that said, these are things that are currently being used by the team, so I think it's a little misleading in its current state. I think we should get these new releases done in this cycle if possible. Part of this is also the assumption that these will be cycle based. I wonder if this are more appropriate as independent deliverables? That means they are not tied to a specific release cycle and can be released whenever there is something to be released. At least something to think about. https://releases.openstack.org/reference/release_models.html#cycle-with-intermediary > On Fri, Feb 28, 2020 at 1:47 AM Thierry Carrez wrote: >> Thierry Carrez wrote: >>> The way we've been handling this in the past was to ignore past releases >>> (since they are not signed by the release team), and push a new one >>> through the releases repository. It should replace the unofficial one in >>> PyPI and make sure all is in order. >> Clarification with a practical example: >> >> xstatic-hogan 2.0.0.2 is on PyPI, but has no tag in the >> openstack/xstatic-hogan repo, and no deliverable file in openstack/releases. >> >> Solution is to resync everything by proposing a 2.0.0.3 release that >> will have tag, be in openstack/releases and have a matching upload on PyPI. >> >> This is done by: >> >> - bumping BUILD at >> https://opendev.org/openstack/xstatic-hogan/src/branch/master/xstatic/pkg/hogan/__init__.py# >> >> - adding a deliverables/_independent/xstatic-hogan.yaml file in >> openstack/releases defining a tag for 2.0.0.3 >> >> - removing the "deprecated" line from >> https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml#L542 >> >> Repeat for every affected package :) >> >> -- >> Thierry Carrez (ttx) >> From juliaashleykreger at gmail.com Tue Mar 3 15:06:54 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Tue, 3 Mar 2020 10:06:54 -0500 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: Thoughts below, thanks for bringing this up Mohammed! On Mon, Mar 2, 2020 at 4:47 PM Mohammed Naser wrote: > > Hi everyone: > > We're now in a spot where we have an increasing amount of projects > that don't end up with a volunteer as PTL, even if the project has > contributors .. no one wants to hold that responsibility alone for > many reasons. With time, the PTL role has become far more overloaded > with many extra responsibilities than what we define in our charter: > > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. > I think this is vital, however at the same time the projects need to reconsider what their commitments are. I feel like most of the liaison model was for us to handle community scale and relay information, and that essentially stopped being effective as teams began to scale back the pool of active contributors and time that can be focused on supporting projects. In other words, does it still make sense to have a release liaison? oslo liaison? etc. Can we not move to a collaborative model instead of putting single points of contact in place? See: https://wiki.openstack.org/wiki/CrossProjectLiaisons > The above is just a very basic idea, I don't intend to diving much > more in depth for now as I'd like to hear about what the rest of the > community thinks. > > Thanks, > Mohammed > Off hand, I feel like my initial mental response was "Noooo!". Upon thinking of this and talking to Mohammed some, I think it is a necessary evolutionary step. As a burned out PTL who cares, I wonder "who will step up after me" and carry what I perceive as the organizational and co-ordination overhead, standing on stage, and running meetings. Nothing prevents any contributor from running a community meeting, standing on a stage and giving a talk or project update! We are a community, and single points of contact just lead community members to burnout. Possibly what we are lacking is a "Time for a meeting!" bot. From skaplons at redhat.com Tue Mar 3 15:26:33 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 3 Mar 2020 16:26:33 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <1A2372A7-6A75-4C89-80A6-8146F82B5413@redhat.com> Hi, > On 2 Mar 2020, at 22:45, Mohammed Naser wrote: > > Hi everyone: > > We're now in a spot where we have an increasing amount of projects > that don't end up with a volunteer as PTL, even if the project has > contributors .. no one wants to hold that responsibility alone for > many reasons. With time, the PTL role has become far more overloaded > with many extra responsibilities than what we define in our charter: > > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. I’m afraid that in such maintainers group there will be still a need to have some kind of leader who will propose/ask others to be liaisons or take some other roles. So it will be still some kind of PTL but maybe with different name and/or elected in different way. Otherwise it may end up that everyone will look for others to do something. If responsibility for something is on many people then in fact nobody is responsible for that. > > The above is just a very basic idea, I don't intend to diving much > more in depth for now as I'd like to hear about what the rest of the > community thinks. > > Thanks, > Mohammed > — Slawek Kaplonski Senior software engineer Red Hat From jean-philippe at evrard.me Tue Mar 3 15:44:58 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Tue, 03 Mar 2020 16:44:58 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: > Off hand, I feel like my initial mental response was "Noooo!". Upon > thinking of this and talking to Mohammed some, I think it is a > necessary evolutionary step. As a burned out PTL who cares, I wonder > "who will step up after me" and carry what I perceive as the > organizational and co-ordination overhead, standing on stage, and > running meetings. Nothing prevents any contributor from running a > community meeting, standing on a stage and giving a talk or project > update! We are a community, and single points of contact just lead > community members to burnout. > > Possibly what we are lacking is a "Time for a meeting!" bot. > I am not sure to understand what you are proposing. Wasn't the liaison's system meant for avoiding burnout by delegating tasks, while staying clear on duties? It avoids the back and forth of communication to some maintainer, solving the question "who is handling that?". It still allows delegation. IMO, there was never a limitation of the amount of liaisons for a single "kind" of liaison. You could have 2 ppl working on the releases, 2 on the bugs, etc. Don't get me wrong: on the "drop of the PTL" story, I was more in the "we should drop this" clan. When I discussed it last time with Mohammed (and others, but it was loooooong ago), I didn't focus on the liaisons. But before side-tracking this thread, I would like to understand what are the pain points in the current model (explicitly! examples!), and how moving away from PTLs and liaisons will help the team of maintainers. At first sight, it looks like team duties will be vague. There are various levels of success on self-organizing teams. Regards, JP From corvus at inaugust.com Tue Mar 3 16:04:28 2020 From: corvus at inaugust.com (James E. Blair) Date: Tue, 03 Mar 2020 08:04:28 -0800 Subject: [kuryr] Job running open resolver Message-ID: <87zhcxpgqb.fsf@meyer.lemoncheese.net> Hi, The openstack-infra team received a report from one of our infrastructure donors that a gate job run by Kuryr is running a DNS resolver open to the Internet. This is dangerous as, if discovered, it can be used as part of DNS reflection attacks. The community and our infrastructure donors share an interest in avoiding misuse of our resources. Would you please look into whether this job is perhaps opening its iptables ports too liberally, and whether that can be avoided? The job is kuryr-kubernetes-tempest-containerized-ovn, and the build which triggered the alerting system is this one: https://zuul.opendev.org/t/openstack/build/166301f57b21402d8d8443bb1e17f970 Thanks, Jim From openstack at nemebean.com Tue Mar 3 16:12:54 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 3 Mar 2020 10:12:54 -0600 Subject: oslotest no longer pulls in stestr Message-ID: Hi all, We just released [0], which removes stestr as a requirement for oslotest. As a result, if you were relying on oslotest to pull in stestr you are most likely broken now. We did check for this situation before making the change, but it seems we missed at least one project so I'm sending this out in case anyone else is affected. The fix is to explicitly list stestr in your test-requirements (oslotest doesn't actually need stestr, so it's not the right place for the requirement to live). Reply here or ping us in #openstack-oslo with any questions. Thanks. -Ben 0: https://review.opendev.org/#/c/615826 From hberaud at redhat.com Tue Mar 3 16:47:20 2020 From: hberaud at redhat.com (Herve Beraud) Date: Tue, 3 Mar 2020 17:47:20 +0100 Subject: [idea] voting procedure and the choice's engineering Message-ID: Hello openstacker, I proposed a new openstack/idea to open discussion about how we should decide, when, and for what purpose, and the methods and tools to help us in this task. https://review.opendev.org/#/c/710107/ This document is not a "well finished document" but more a draft to open the debate and to help us to make the decisions better. I referenced some methods and tools. If some of you are interested by this topic then do not hesitate to leave comments and push changes on this document. Communities are driven by choices and decisions, Openstack is a community, let's decide how to choose on Openstack. -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Tue Mar 3 16:56:28 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?Q?Bal=E1zs_Gibizer?=) Date: Tue, 3 Mar 2020 16:56:28 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <1583254584.12170.11@est.tech> On Tue, Mar 3, 2020 at 16:44, Jean-Philippe Evrard wrote: >> Off hand, I feel like my initial mental response was "Noooo!". Upon >> thinking of this and talking to Mohammed some, I think it is a >> necessary evolutionary step. As a burned out PTL who cares, I >> wonder >> "who will step up after me" and carry what I perceive as the >> organizational and co-ordination overhead, standing on stage, and >> running meetings. Nothing prevents any contributor from running a >> community meeting, standing on a stage and giving a talk or project >> update! We are a community, and single points of contact just lead >> community members to burnout. >> >> Possibly what we are lacking is a "Time for a meeting!" bot. >> > > I am not sure to understand what you are proposing. > > Wasn't the liaison's system meant for avoiding burnout by delegating > tasks, while staying clear on duties? It avoids the back and forth of > communication to some maintainer, solving the question "who is > handling > that?". It still allows delegation. IMO, there was never a limitation > of the amount of liaisons for a single "kind" of liaison. You could > have 2 ppl working on the releases, 2 on the bugs, etc. > > Don't get me wrong: on the "drop of the PTL" story, I was more in the > "we should drop this" clan. When I discussed it last time with > Mohammed > (and others, but it was loooooong ago), I didn't focus on the > liaisons. > But before side-tracking this thread, I would like to understand what > are the pain points in the current model (explicitly! examples!), and > how moving away from PTLs and liaisons will help the team of > maintainers. At first sight, it looks like team duties will be vague. > There are various levels of success on self-organizing teams. My context: We have a shortage of PTL candidates in Nova but we still have a core team. I think the real problem is that contributors think that being a PTL is a huge extra burden. I haven't been a PTL yet but I share this view. I think being a Nova PTL is a sizable amount of work. E.g. the PLT is the liaison by default if nobody steps up. And in Nova, according to the wiki, most of the liaison spots are filled by people who already left the community. So a nova PTL has a lot of hats by default. It could be that those hats does not need real work to be fulfilled. Still the list is long. So for me a better solution would be to rationalize (review, clarify) the list of expectations on the project teams. Then let the project teams commit to it either in a single person (a PTL) or by the whole team sharing the responsibilities between each other some explicit way. I can even accept that the project team explicitly rejects some of the responsibilities due to shortage of bandwidth in the team. For me explicitly not doing something is better than simply ignoring that such responsibility exists. I think Mohammed's proposal helps in a sense that removes the need to _find a single person as PTL_ in a situation where nobody wants to be a PTL. Basically removes the Nova core team from the wait-for-a-PTL-candidate state where we are in now. And in the same time it allows the core team to start discussing how to fulfill every responsibilities as a team. Cheers, gibi > > > Regards, > JP > > From Albert.Braden at synopsys.com Tue Mar 3 17:07:55 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 3 Mar 2020 17:07:55 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: Sean, thank you for explaining this. I think I get it now. -----Original Message----- From: Sean Mooney Sent: Monday, March 2, 2020 10:51 AM To: Albert Braden ; openstack-discuss Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: > As an openstack operator I was pretty ecstatic to hear that the assortment of clients would be replaced by a single > client. I would be disappointed to find that a component would not integrate and would continue to use a separate > client. This would be a step backward IMO. > > The discussion about microversions goes over my head, but I would hope to see the developers get together and solve > the issue and continue working toward integration. just to summerisie it in a non technical way. the project specific cli had a convention where the client would ask the api what the newest micoverion it supported and defualt to that if the clinet suported it. that meant that the same command executed against two different clouds with different versions of openstakc deploy could have different behavior and different responces. so from an interoperablity point of view that is not great but from a usablity point of view the fact enduser dont have to care about microverions and the client would try to do the right thing made some things much simpler. the unifeid client (osc) chose to priorities interoperablity by defaulting to the oldest micorverions, so for nova that would be 2.0/2.1 meaning that if you execute the same command on two different cloud with different version of nova it will behave the same but if you want to use a feature intoduced in a later micorverion you have to explcitly request that via --os-compute-api-version or set that as a env var or in you cloud.yaml so really the difference is that osc requires the end user to be explictl about what micoversion to use and therefor be explict about the behavior of the api they expect (this is what we expect application that use the the api should do) where as the project client tried to just work and use the latest microverion which mostly workd excpet where we remove a feature in a later micorverions. for example we removed the force option on some move operation in nova because allowing forcing caused many harder to fix issues. i dont thnk the nova clinet would cap at the latest micorvierion that allowed forcing. so the poject client genreally did not guarantee that a command would work without specifcing a new micorverison it just that we remove things a hell of a lot less often then we add them. so as an end user that is the main difference between using osc vs glance clinet other then the fact i belive there is a bunch of stuff you can do with glance client that is missing in osc. parity is a spereate disucssion but it is vaild concern. From gr at ham.ie Tue Mar 3 17:11:03 2020 From: gr at ham.ie (Graham Hayes) Date: Tue, 3 Mar 2020 17:11:03 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <2ae9b18a-0366-af33-38bb-eb1e2c789928@ham.ie> On 02/03/2020 21:45, Mohammed Naser wrote: > Hi everyone: > > We're now in a spot where we have an increasing amount of projects > that don't end up with a volunteer as PTL, even if the project has > contributors .. no one wants to hold that responsibility alone for > many reasons. With time, the PTL role has become far more overloaded > with many extra responsibilities than what we define in our charter: > > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. > > The above is just a very basic idea, I don't intend to diving much > more in depth for now as I'd like to hear about what the rest of the > community thinks. > > Thanks, > Mohammed > Yeah, this is a tough spot. When we have talked about this in the past, we have theorized the role could be stripped back to "Project Liaison to the TC". As noted in other replies, the worry is that there is a lot of work that goes to the PTL by default currently. We should look at this work, and if is it not bringing value, just remove it. If it is bringing value, how do we ensure that someone does it? My consistent worry with the removal of the PTL single point of contact, is that without it, this work will get missed. From mdemaced at redhat.com Tue Mar 3 17:14:28 2020 From: mdemaced at redhat.com (Maysa De Macedo Souza) Date: Tue, 3 Mar 2020 18:14:28 +0100 Subject: [kuryr] Job running open resolver In-Reply-To: <87zhcxpgqb.fsf@meyer.lemoncheese.net> References: <87zhcxpgqb.fsf@meyer.lemoncheese.net> Message-ID: Hi James, Thank you for reporting it. We will take a look at it. Best, Maysa. On Tue, Mar 3, 2020 at 5:11 PM James E. Blair wrote: > Hi, > > The openstack-infra team received a report from one of our > infrastructure donors that a gate job run by Kuryr is running a DNS > resolver open to the Internet. This is dangerous as, if discovered, it > can be used as part of DNS reflection attacks. The community and our > infrastructure donors share an interest in avoiding misuse of our > resources. > > Would you please look into whether this job is perhaps opening its > iptables ports too liberally, and whether that can be avoided? > > The job is kuryr-kubernetes-tempest-containerized-ovn, and the build > which triggered the alerting system is this one: > > https://zuul.opendev.org/t/openstack/build/166301f57b21402d8d8443bb1e17f970 > > Thanks, > > Jim > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephenfin at redhat.com Tue Mar 3 15:05:26 2020 From: stephenfin at redhat.com (Stephen Finucane) Date: Tue, 03 Mar 2020 15:05:26 +0000 Subject: [all] oslotest 4.0 drops the moxstubout and functional modules, stestr dependency Message-ID: Per $subject, the 'oslotest.moxstubout' modules has been removed in oslotest 4.0 and we no longer include 'stestr' in our list of dependencies. I think I've resolved all issues in the 'openstack/' namespaced projects, but if not the remedies are to either switch to mock or explicitly included 'stestr' in your 'test-requirements.txt' file, respectively. We've also removed the 'oslotest.functional' module, but there do not appear to have been any users of this module. Stephen From hberaud at redhat.com Tue Mar 3 17:19:27 2020 From: hberaud at redhat.com (Herve Beraud) Date: Tue, 3 Mar 2020 18:19:27 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <2ae9b18a-0366-af33-38bb-eb1e2c789928@ham.ie> References: <2ae9b18a-0366-af33-38bb-eb1e2c789928@ham.ie> Message-ID: Le mar. 3 mars 2020 à 18:13, Graham Hayes a écrit : > On 02/03/2020 21:45, Mohammed Naser wrote: > > Hi everyone: > > > > We're now in a spot where we have an increasing amount of projects > > that don't end up with a volunteer as PTL, even if the project has > > contributors .. no one wants to hold that responsibility alone for > > many reasons. With time, the PTL role has become far more overloaded > > with many extra responsibilities than what we define in our charter: > > > > > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > > > I think it's time to re-evaluate the project leadership model that we > > have. I am thinking that perhaps it would make a lot of sense to move > > from a single PTL model to multiple maintainers. This would leave it > > up to the maintainers to decide how they want to sort the different > > requirements/liaisons/contact persons between them. > > > > The above is just a very basic idea, I don't intend to diving much > > more in depth for now as I'd like to hear about what the rest of the > > community thinks. > > > > Thanks, > > Mohammed > > > > Yeah, this is a tough spot. > > When we have talked about this in the past, we have theorized the role > could be stripped back to "Project Liaison to the TC". As noted in other > replies, the worry is that there is a lot of work that goes to the PTL > by default currently. > > We should look at this work, and if is it not bringing value, just > remove it. > > If it is bringing value, how do we ensure that someone does it? > > My consistent worry with the removal of the PTL single point > of contact, is that without it, this work will get missed. > I agree the best way to miss something is to spread responsibility between members, everybody thinks that others are watchful. -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From clay.gerrard at gmail.com Tue Mar 3 17:19:39 2020 From: clay.gerrard at gmail.com (Clay Gerrard) Date: Tue, 3 Mar 2020 11:19:39 -0600 Subject: [Swift] Errno 13 Permission denied writing objects xattr while upgrading from Kilo to Queens In-Reply-To: References: Message-ID: When we were debugging this issue in IRC it appeared that the issue was related to the introduction of O_TMPFILE support in Swift. Can you confirm that everything is still working properly when you force swift not to use o_tmpfile/linkat? The linkat detection has improved in subsequent releases of Swift, but it's still not clear the latest version would be properly workaround whatever issue your setup is having. When I google for O_TMPFILE and EPERM I see that there was bugs filed against glibc shortly after O_TMPFILE support was introduced to xfs. Can you share the output of "uname -a" and "ldd --version"? Perhaps we could discover or develop a base box image that can reproduce the error. On Tue, Mar 3, 2020 at 6:39 AM Gui Maluf wrote: > Hi all, > > I'm struggling with something really wierd. 3 weeks ago I started > upgrading my Keystone + Swift Ubuntu environment from Kilo to Queens. So I > moved from Ubuntu 14.04 to 18.04. > > I can create new accounts and containers. But no objects. I think between > Mitaka and Newton my storages started to throw Permission Denied error > while writing object metadata. > > I saw that the piece of python code where I getting problem was changed in > Rocky version and in hope of getting things fixed I've upgraded storage > version. But the error persists. > > http://paste.openstack.org/show/790217/ > > I've check everything I could, user, permissions, mount options. But I > still getting this error. > > I wrote a python script for creating files and writing metadata within the > swift mount with swift user and everything works fines. > > Don't know what to do anymore. This is a "dev" environment with two > storages only and a few disks. > Since I'm planning to do in the production environment I'm quite scared if > this happens again. > > Thanks in advance > > -- > *guilherme* \n11 > \t *maluf* > -- Clay Gerrard Wyatt Elementary Dad's Club Chair 2019-20 210 788 9431 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdulko at redhat.com Tue Mar 3 17:23:50 2020 From: mdulko at redhat.com (mdulko at redhat.com) Date: Tue, 03 Mar 2020 18:23:50 +0100 Subject: [kuryr] Job running open resolver In-Reply-To: <87zhcxpgqb.fsf@meyer.lemoncheese.net> References: <87zhcxpgqb.fsf@meyer.lemoncheese.net> Message-ID: <3833a1f53e5db846d6af212784050c60c81e4e42.camel@redhat.com> On Tue, 2020-03-03 at 08:04 -0800, James E. Blair wrote: > Hi, > > The openstack-infra team received a report from one of our > infrastructure donors that a gate job run by Kuryr is running a DNS > resolver open to the Internet. This is dangerous as, if discovered, it > can be used as part of DNS reflection attacks. The community and our > infrastructure donors share an interest in avoiding misuse of our > resources. > > Would you please look into whether this job is perhaps opening its > iptables ports too liberally, and whether that can be avoided? > > The job is kuryr-kubernetes-tempest-containerized-ovn, and the build > which triggered the alerting system is this one: > > https://zuul.opendev.org/t/openstack/build/166301f57b21402d8d8443bb1e17f970 Hi, The patch that disables the DNS is in review [1]. We'll come up with a way to run it locally, at the moment it should be safe for us to just disable it. [1] https://review.opendev.org/#/c/711069/ Thanks, Michał > Thanks, > > Jim > From Albert.Braden at synopsys.com Tue Mar 3 17:28:36 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 3 Mar 2020 17:28:36 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. I think I agree with the suggestion that a --os-compute-api-version=auto option might be a good solution to this conflict. Does anyone want to explain why this isn’t a good idea? From: Abhishek Kekane Sent: Monday, March 2, 2020 10:18 PM To: Artem Goncharov Cc: Sean Mooney ; Albert Braden ; openstack-discuss Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) Hi Artem, Thanks for sharing the update. The decision was collectively taken during last cycle by glance team, as we don't have enough people/resources to work on this front. I will be more than happy to change this if anyone comes forward and bridge the gaps. Thanks & Best Regards, Abhishek Kekane On Tue, Mar 3, 2020 at 11:40 AM Artem Goncharov > wrote: On Tue, 3 Mar 2020, 06:08 Abhishek Kekane, > wrote: Hi All, Thank you for making this different thread, OSC is not up to date with the current glance features and neither it has shown any interest in doing so. From glance prospective we also didn't have any bandwidth to work on adding these support to OSC. That's honestly not true this days There is some major feature gap between current OSC and Glance and that's the reason why glance does not recommend to use OSC. That's still not reason to say please don't use it anymore. 1. Support for new image import workflow Partially implemented by me and I continue working on that 2. Support for hidden images Implemented 3. Support for multihash 4. Support for multiple stores I am relying on OSC and especially for image service trying to bring it in a more useful state, thus fixing huge parts in SDK. If anyone is interested to take up this work it will be great. Thanks & Best Regards, Abhishek Kekane On Tue, Mar 3, 2020 at 12:24 AM Sean Mooney > wrote: On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: > As an openstack operator I was pretty ecstatic to hear that the assortment of clients would be replaced by a single > client. I would be disappointed to find that a component would not integrate and would continue to use a separate > client. This would be a step backward IMO. > > The discussion about microversions goes over my head, but I would hope to see the developers get together and solve > the issue and continue working toward integration. just to summerisie it in a non technical way. the project specific cli had a convention where the client would ask the api what the newest micoverion it supported and defualt to that if the clinet suported it. that meant that the same command executed against two different clouds with different versions of openstakc deploy could have different behavior and different responces. so from an interoperablity point of view that is not great but from a usablity point of view the fact enduser dont have to care about microverions and the client would try to do the right thing made some things much simpler. the unifeid client (osc) chose to priorities interoperablity by defaulting to the oldest micorverions, so for nova that would be 2.0/2.1 meaning that if you execute the same command on two different cloud with different version of nova it will behave the same but if you want to use a feature intoduced in a later micorverion you have to explcitly request that via --os-compute-api-version or set that as a env var or in you cloud.yaml so really the difference is that osc requires the end user to be explictl about what micoversion to use and therefor be explict about the behavior of the api they expect (this is what we expect application that use the the api should do) where as the project client tried to just work and use the latest microverion which mostly workd excpet where we remove a feature in a later micorverions. for example we removed the force option on some move operation in nova because allowing forcing caused many harder to fix issues. i dont thnk the nova clinet would cap at the latest micorvierion that allowed forcing. so the poject client genreally did not guarantee that a command would work without specifcing a new micorverison it just that we remove things a hell of a lot less often then we add them. so as an end user that is the main difference between using osc vs glance clinet other then the fact i belive there is a bunch of stuff you can do with glance client that is missing in osc. parity is a spereate disucssion but it is vaild concern. -----Original Message----- > From: Radosław Piliszek > > Sent: Monday, March 2, 2020 9:07 AM > To: openstack-discuss > > Subject: Re: [glance] Different checksum between CLI and curl > > Folks, > > sorry to interrupt but I think we have diverged a bit too much from the subject. > Only last Gaetan message is on topic here. > Please switch to new subject to discuss OSC future. > > -yoctozepto > > pon., 2 mar 2020 o 18:03 Tim Bell > napisał(a): > > > > > > > > On 2 Mar 2020, at 16:49, Dmitry Tantsur > wrote: > > > > Hi, > > > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano > wrote: > > > > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane > wrote: > > > > > Hi Gaëtan, > > > > > > > > > > Glance team doesn't recommend to use OSC anymore. > > > > > I will recommend you to check the same behaviour using > > > > > python-glanceclient. > > > > > > > > That's not cool - everyone has switched to OSC. It's also the first > > > > time I've heard of it. > > > > > > > > From the end user perspective, we’ve had positive feedback on the convergence to OSC from our cloud consumers. > > > > There has been great progress with Manila to get shares included ( > > https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= > > ) and it would be a pity if we’re asking our end users to understand all of the different project names and > > inconsistent options/arguments/syntax. > > > > We had hoped for a project goal to get everyone aligned on OSC but there was not consensus on this, I’d still > > encourage it to simplify the experience for OpenStack cloud consumers. > > > > Tim > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Tue Mar 3 17:49:53 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 3 Mar 2020 11:49:53 -0600 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> On 3/3/20 11:28 AM, Albert Braden wrote: > > Am I understanding correctly that the Openstack community decided to > focus on the unified client, and to deprecate the individual clients, > and that the Glance team did not agree with this decision, and that > the Glance team is now having a pissing match with the rest of the > community, and is unilaterally deciding to continue developing the > Glance client and refusing to work on the unified client, or is > something different going on? I would ask everyone involved to > remember that we operators are down here, and the yellow rain falling > on our heads does not smell very good. > I definitely would not characterize it that way. With trying not to put too much personal bias into it, here's what I would say the situation is: - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away - Glance is a very small team with very, very limited resources - The OSC team is a very small team with very, very limited resources - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends -------------- next part -------------- An HTML attachment was scrubbed... URL: From ekuvaja at redhat.com Tue Mar 3 18:04:31 2020 From: ekuvaja at redhat.com (Erno Kuvaja) Date: Tue, 3 Mar 2020 18:04:31 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: On Tue, Mar 3, 2020 at 6:14 AM Artem Goncharov wrote: > > > On Tue, 3 Mar 2020, 06:08 Abhishek Kekane, wrote: > >> Hi All, >> >> Thank you for making this different thread, >> >> OSC is not up to date with the current glance features and neither it has >> shown any interest in doing so. >> From glance prospective we also didn't have any bandwidth to work on >> adding these support to OSC. >> > > > That's honestly not true this days > It's very much true that we do not have cycles for it. If you have found the time now after we've been complaining about the issues without any concrete actions for cycles, great for those who wants to use it. > >> There is some major feature gap between current OSC and Glance and that's >> the reason why glance does not recommend to use OSC. >> > > That's still not reason to say please don't use it anymore. > But it very much is. Tells quite a bit about the communication within the community that this is the first time we hear you actively working on those bits and making progress. Yet the osc is still lacking good year+ behind the feature parity and if the _demand_ is to use osc, "I'm just one person and have only so much time for this" is not good enough. Don't get me wrong, kudos to you to actually taking it on, but too little too late I guess. If 95-100% of user issues with client gets resolved by "Have you tried to use the native glanceclient instead?" and the response is "Yes, it works, thanks." it very much tells that we should not be supporting and promoting the tooling that is not under our control and just does not work. (BTW we do encourage all those users to take their osc issues to the osc team to get fixed, yet we get these raised to us every so often.) This really got to the point where we had that very same discussion in multiple OpenStack summits in a row after the call was made that everything should move to osc and every time we got the same response "We know there are problems and we will look into it." After so many cycles and the gap growing not shrinking just got us to the point of moving forwards (or reverting back to something that actually works for our users). BTW we did announce this and it was discussed in PTG. > > 1. Support for new image import workflow >> > Partially implemented by me and I continue working on that > > 2. Support for hidden images >> > Implemented > > 3. Support for multihash >> > 4. Support for multiple stores >> > > I am relying on OSC and especially for image service trying to bring it in > a more useful state, thus fixing huge parts in SDK. > That's great and obviously you have the freedom to choose the client you prefer to use. Just like we have a moral responsibility to our users to provide them reference client that is up to date, works and the issues raised gets attention. This is all beyond the personal preference, which I get very mixed feedback of depending to whom I talk to. If I send the mail to the mailing list I get those same handful of people yelling right away how unified client is the only way to go and even thinking anything else is heresy. When I talk with people in the field, customers and users in the hallway tracks the message is much more mixed. The osc target audience prefers or just uses GUI instead, then there is a good portion of people who really don't care as they use some automation suite anyways, there is the old school guys who prefers a tool for a job as in the dedicated clients(I have to admit for disclaimer I belong to this group myself) and then there is a group of people who really don't care as long as the client they use every now and then just works. So honestly outside of those few voices in this mailing list I very rarely hear the demand of unified client and much more get the request to provide something that works, which was the major driver for our decision. Harsh, absolutely; justified, I'd like to think so. And this is just my personal experience with Glance, we're in a blessed situation of not jumping into the microversions bandwagon which seems to be totally hidden can of worms from us in this topic. For all the rest of you interested about the topic, next time you start demanding us to go to osc again, please put your money where your mouth is first and help those guys to deliver. Best, Erno "jokke" Kuvaja > >> If anyone is interested to take up this work it will be great. >> >> Thanks & Best Regards, >> >> Abhishek Kekane >> >> >> On Tue, Mar 3, 2020 at 12:24 AM Sean Mooney wrote: >> >>> On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: >>> > As an openstack operator I was pretty ecstatic to hear that the >>> assortment of clients would be replaced by a single >>> > client. I would be disappointed to find that a component would not >>> integrate and would continue to use a separate >>> > client. This would be a step backward IMO. >>> > >>> > The discussion about microversions goes over my head, but I would hope >>> to see the developers get together and solve >>> > the issue and continue working toward integration. >>> just to summerisie it in a non technical way. >>> the project specific cli had a convention where the client would ask the >>> api what the newest micoverion it supported >>> and defualt to that if the clinet suported it. that meant that the same >>> command executed against two different clouds >>> with different versions of openstakc deploy could have different >>> behavior and different responces. so from an >>> interoperablity point of view that is not great but from a usablity >>> point of view the fact enduser dont have to care >>> about microverions and the client would try to do the right thing made >>> some things much simpler. >>> >>> the unifeid client (osc) chose to priorities interoperablity by >>> defaulting to the oldest micorverions, so for nova that >>> would be 2.0/2.1 meaning that if you execute the same command on two >>> different cloud with different version of nova it >>> will behave the same but if you want to use a feature intoduced in a >>> later micorverion you have to explcitly request >>> that via --os-compute-api-version or set that as a env var or in you >>> cloud.yaml >>> >>> so really the difference is that osc requires the end user to be >>> explictl about what micoversion to use and therefor be >>> explict about the behavior of the api they expect (this is what we >>> expect application that use the the api should do) >>> where as the project client tried to just work and use the latest >>> microverion which mostly workd excpet where we remove >>> a feature in a later micorverions. for example we removed the force >>> option on some move operation in nova because >>> allowing forcing caused many harder to fix issues. i dont thnk the nova >>> clinet would cap at the latest micorvierion that >>> allowed forcing. so the poject client genreally did not guarantee that a >>> command would work without specifcing a new >>> micorverison it just that we remove things a hell of a lot less often >>> then we add them. >>> >>> so as an end user that is the main difference between using osc vs >>> glance clinet other then the fact i belive there is a >>> bunch of stuff you can do with glance client that is missing in osc. >>> parity is a spereate disucssion but it is vaild >>> concern. >>> >>> -----Original Message----- >>> > From: Radosław Piliszek >>> > Sent: Monday, March 2, 2020 9:07 AM >>> > To: openstack-discuss >>> > Subject: Re: [glance] Different checksum between CLI and curl >>> > >>> > Folks, >>> > >>> > sorry to interrupt but I think we have diverged a bit too much from >>> the subject. >>> > Only last Gaetan message is on topic here. >>> > Please switch to new subject to discuss OSC future. >>> > >>> > -yoctozepto >>> > >>> > pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): >>> > > >>> > > >>> > > >>> > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: >>> > > >>> > > Hi, >>> > > >>> > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano >>> wrote: >>> > > > >>> > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >>> > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane >>> wrote: >>> > > > > > Hi Gaëtan, >>> > > > > > >>> > > > > > Glance team doesn't recommend to use OSC anymore. >>> > > > > > I will recommend you to check the same behaviour using >>> > > > > > python-glanceclient. >>> > > > > >>> > > > > That's not cool - everyone has switched to OSC. It's also the >>> first >>> > > > > time I've heard of it. >>> > > > > >>> > > >>> > > From the end user perspective, we’ve had positive feedback on the >>> convergence to OSC from our cloud consumers. >>> > > >>> > > There has been great progress with Manila to get shares included ( >>> > > >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= >>> > > ) and it would be a pity if we’re asking our end users to >>> understand all of the different project names and >>> > > inconsistent options/arguments/syntax. >>> > > >>> > > We had hoped for a project goal to get everyone aligned on OSC but >>> there was not consensus on this, I’d still >>> > > encourage it to simplify the experience for OpenStack cloud >>> consumers. >>> > > >>> > > Tim >>> > > >>> > > >>> > >>> > >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Albert.Braden at synopsys.com Tue Mar 3 18:20:59 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 3 Mar 2020 18:20:59 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> Message-ID: Sean, thank you for clarifying that. Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. From: Sean McGinnis Sent: Tuesday, March 3, 2020 9:50 AM To: openstack-discuss at lists.openstack.org Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) On 3/3/20 11:28 AM, Albert Braden wrote: Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. I definitely would not characterize it that way. With trying not to put too much personal bias into it, here's what I would say the situation is: - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away - Glance is a very small team with very, very limited resources - The OSC team is a very small team with very, very limited resources - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnsomor at gmail.com Tue Mar 3 18:34:16 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 3 Mar 2020 10:34:16 -0800 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <2ae9b18a-0366-af33-38bb-eb1e2c789928@ham.ie> Message-ID: Putting my former PTL hat on: I agree with Slawek. I think the PTL role is still important. That said, the list of responsibilities can get daunting. I would lean toward re-affirming the "Project Team Leads" statement you linked and highlight, in the new contributor guides, that the other tasks can be delegated. Maybe we should also re-word that statement to clarify or soften the "manage day-to-day operations" part. I think over the history of OpenStack we have had "hands on" PTLs and more "delegate" PTLs, both supporting healthy projects. The lack of a clear "boundary" for the PTL role has probably lead to Fear-Uncertainty-and-Doubt. My hope is that the new contributor guide goal will help clarify the role. This may also highlight tasks that we can remove (deprecate ask.openstack.org anyone?). Michael On Tue, Mar 3, 2020 at 9:22 AM Herve Beraud wrote: > > > > Le mar. 3 mars 2020 à 18:13, Graham Hayes a écrit : >> >> On 02/03/2020 21:45, Mohammed Naser wrote: >> > Hi everyone: >> > >> > We're now in a spot where we have an increasing amount of projects >> > that don't end up with a volunteer as PTL, even if the project has >> > contributors .. no one wants to hold that responsibility alone for >> > many reasons. With time, the PTL role has become far more overloaded >> > with many extra responsibilities than what we define in our charter: >> > >> > https://governance.openstack.org/tc/reference/charter.html#project-team-leads >> > >> > I think it's time to re-evaluate the project leadership model that we >> > have. I am thinking that perhaps it would make a lot of sense to move >> > from a single PTL model to multiple maintainers. This would leave it >> > up to the maintainers to decide how they want to sort the different >> > requirements/liaisons/contact persons between them. >> > >> > The above is just a very basic idea, I don't intend to diving much >> > more in depth for now as I'd like to hear about what the rest of the >> > community thinks. >> > >> > Thanks, >> > Mohammed >> > >> >> Yeah, this is a tough spot. >> >> When we have talked about this in the past, we have theorized the role >> could be stripped back to "Project Liaison to the TC". As noted in other >> replies, the worry is that there is a lot of work that goes to the PTL >> by default currently. >> >> We should look at this work, and if is it not bringing value, just >> remove it. >> >> If it is bringing value, how do we ensure that someone does it? >> >> My consistent worry with the removal of the PTL single point >> of contact, is that without it, this work will get missed. > > > I agree the best way to miss something is to spread responsibility between members, everybody thinks that others are watchful. > > > -- > Hervé Beraud > Senior Software Engineer > Red Hat - Openstack Oslo > irc: hberaud > -----BEGIN PGP SIGNATURE----- > > wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ > Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ > RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP > F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G > 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g > glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw > m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ > hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 > qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y > F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 > B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O > v6rDpkeNksZ9fFSyoY2o > =ECSj > -----END PGP SIGNATURE----- > From openstack at nemebean.com Tue Mar 3 18:48:40 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 3 Mar 2020 12:48:40 -0600 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: On 3/3/20 9:44 AM, Jean-Philippe Evrard wrote: >> Off hand, I feel like my initial mental response was "Noooo!". Upon >> thinking of this and talking to Mohammed some, I think it is a >> necessary evolutionary step. As a burned out PTL who cares, I wonder >> "who will step up after me" and carry what I perceive as the >> organizational and co-ordination overhead, standing on stage, and >> running meetings. Nothing prevents any contributor from running a >> community meeting, standing on a stage and giving a talk or project >> update! We are a community, and single points of contact just lead >> community members to burnout. >> >> Possibly what we are lacking is a "Time for a meeting!" bot. >> > > I am not sure to understand what you are proposing. > > Wasn't the liaison's system meant for avoiding burnout by delegating > tasks, while staying clear on duties? It avoids the back and forth of > communication to some maintainer, solving the question "who is handling > that?". It still allows delegation. IMO, there was never a limitation > of the amount of liaisons for a single "kind" of liaison. You could > have 2 ppl working on the releases, 2 on the bugs, etc. Yeah, I always saw the liaison system as a way to spread the PTL workload, not as something to increase it. It was also a way to distribute work across the team without falling prey to the "someone else will do it" problem because you still have specific people assigned to specific tasks. I see the Neutron bug deputy and Keystone L1 positions similarly, and I think they're good things. As to whether all of the liaisons are still needed, I will admit there are only a handful of projects that really still have Oslo liaisons. I'm not sure eliminating it entirely benefits anyone though, it just means now I have to ping the PTL when I need to coordinate something with a project. Or if we eliminate the PTL then I have to throw issues into the IRC void and hope someone responds. When we have cross-project changes that involve a lot of projects that means inevitably someone won't respond and we have to start delivering ultimatums. Maybe that's okay, but it slows things down (we have to wait for a response, even if we don't get one) and always feels kind of unfriendly. I guess what I'm saying is that eliminating the liaison position doesn't mean we stop needing a point of contact from a project. I suspect the same thing applies to the release team. > > Don't get me wrong: on the "drop of the PTL" story, I was more in the > "we should drop this" clan. When I discussed it last time with Mohammed > (and others, but it was loooooong ago), I didn't focus on the liaisons. > But before side-tracking this thread, I would like to understand what > are the pain points in the current model (explicitly! examples!), and > how moving away from PTLs and liaisons will help the team of > maintainers. At first sight, it looks like team duties will be vague. > There are various levels of success on self-organizing teams. I've been skeptical of this before and I won't reiterate all of those arguments again unless it proves necessary, but I'm also curious what this would solve that PTL delegation can't already. Is it just a perception thing? Maybe we re-brand the PTL position "Project Delegator" or something to make it clear to potential candidates that they don't have to be responsible for everything that goes on in a project? Unless we feel there are things PTLs are doing that don't need to be done, in which case we should clearly stop doing those things, the workload remains the same no matter what we call it, "maintainers" or "PTL and liaisons". > > > Regards, > JP > > From radoslaw.piliszek at gmail.com Tue Mar 3 18:52:49 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 3 Mar 2020 19:52:49 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <2ae9b18a-0366-af33-38bb-eb1e2c789928@ham.ie> Message-ID: I agree with folks before me. As my predecessors have stated, throwing all folks into "maintainers" group/team would kill the responsibility relation. I believe we could get away with splitting the PTL role into multiple subroles and actually let each project decide on what these would be to satisfy that project's needs (but give some guidelines too maybe?). Also, it would make sense to allow assigning these on primary and secondary terms. Both would get "privileges", everyone is obviously responsible for itself, but primary is responsible if they are currently available and the role is POC-like. This way we could both lessen the load on PTL and officially allow vices. That said, these would need to get assigned and /me not sure how to best approach this atm. PS: Deprecate ask.o.o all the way. :-) -yoctozepto wt., 3 mar 2020 o 19:42 Michael Johnson napisał(a): > > Putting my former PTL hat on: > > I agree with Slawek. I think the PTL role is still important. > > That said, the list of responsibilities can get daunting. > > I would lean toward re-affirming the "Project Team Leads" statement > you linked and highlight, in the new contributor guides, that the > other tasks can be delegated. Maybe we should also re-word that > statement to clarify or soften the "manage day-to-day operations" > part. > > I think over the history of OpenStack we have had "hands on" PTLs and > more "delegate" PTLs, both supporting healthy projects. > > The lack of a clear "boundary" for the PTL role has probably lead to > Fear-Uncertainty-and-Doubt. My hope is that the new contributor guide > goal will help clarify the role. > This may also highlight tasks that we can remove (deprecate > ask.openstack.org anyone?). > > Michael From tim.bell at cern.ch Tue Mar 3 18:55:38 2020 From: tim.bell at cern.ch (Tim Bell) Date: Tue, 3 Mar 2020 19:55:38 +0100 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> Message-ID: <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> > On 3 Mar 2020, at 19:20, Albert Braden wrote: > > Sean, thank you for clarifying that. > > Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient ) BTW, I also would vote for =auto as the default. Tim > We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. > > From: Sean McGinnis > Sent: Tuesday, March 3, 2020 9:50 AM > To: openstack-discuss at lists.openstack.org > Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > > On 3/3/20 11:28 AM, Albert Braden wrote: > Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. > I definitely would not characterize it that way. > > With trying not to put too much personal bias into it, here's what I would say the situation is: > > - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away > - Glance is a very small team with very, very limited resources > - The OSC team is a very small team with very, very limited resources > - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI > - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) > - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.bell at cern.ch Tue Mar 3 19:00:35 2020 From: tim.bell at cern.ch (Tim Bell) Date: Tue, 3 Mar 2020 20:00:35 +0100 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> Message-ID: > On 3 Mar 2020, at 19:55, Tim Bell wrote: > > > >> On 3 Mar 2020, at 19:20, Albert Braden > wrote: >> >> Sean, thank you for clarifying that. >> >> Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? >> >> > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. BTW, I found the etherpad from Berlin (https://etherpad.openstack.org/p/BER-t-series-goals ) and the associated mailing list discussion at http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html > > My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. > > While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. > > Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient ) > > BTW, I also would vote for =auto as the default. > > Tim > >> We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. >> >> From: Sean McGinnis > >> Sent: Tuesday, March 3, 2020 9:50 AM >> To: openstack-discuss at lists.openstack.org >> Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) >> >> On 3/3/20 11:28 AM, Albert Braden wrote: >> Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. >> I definitely would not characterize it that way. >> >> With trying not to put too much personal bias into it, here's what I would say the situation is: >> >> - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away >> - Glance is a very small team with very, very limited resources >> - The OSC team is a very small team with very, very limited resources >> - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI >> - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) >> - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Tue Mar 3 19:00:49 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 3 Mar 2020 20:00:49 +0100 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> Message-ID: Folks, I think we should clarify the problem we are discussing. The discussion started when the response to "there could be a bug in OSC" was "we are not recommending OSC". I don't feel like OSC lagging behind per-project-specific client in terms of features is a bad thing considering the workload. On the other hand, suddenly making OSC "not recommended" for a project already supported by / supporting OSC is a <>. (TM) I feel like there should be an official on how we are going to handle the clients situation and just document it. -yoctozepto wt., 3 mar 2020 o 19:11 Erno Kuvaja napisał(a): > > On Tue, Mar 3, 2020 at 6:14 AM Artem Goncharov wrote: >> >> >> >> On Tue, 3 Mar 2020, 06:08 Abhishek Kekane, wrote: >>> >>> Hi All, >>> >>> Thank you for making this different thread, >>> >>> OSC is not up to date with the current glance features and neither it has shown any interest in doing so. >>> From glance prospective we also didn't have any bandwidth to work on adding these support to OSC. >> >> >> >> That's honestly not true this days > > > It's very much true that we do not have cycles for it. If you have found the time now after we've been complaining about the issues without any concrete actions for cycles, great for those who wants to use it. >>> >>> >>> There is some major feature gap between current OSC and Glance and that's the reason why glance does not recommend to use OSC. >> >> >> That's still not reason to say please don't use it anymore. > > > But it very much is. > > Tells quite a bit about the communication within the community that this is the first time we hear you actively working on those bits and making progress. Yet the osc is still lacking good year+ behind the feature parity and if the _demand_ is to use osc, "I'm just one person and have only so much time for this" is not good enough. Don't get me wrong, kudos to you to actually taking it on, but too little too late I guess. > > If 95-100% of user issues with client gets resolved by "Have you tried to use the native glanceclient instead?" and the response is "Yes, it works, thanks." it very much tells that we should not be supporting and promoting the tooling that is not under our control and just does not work. (BTW we do encourage all those users to take their osc issues to the osc team to get fixed, yet we get these raised to us every so often.) This really got to the point where we had that very same discussion in multiple OpenStack summits in a row after the call was made that everything should move to osc and every time we got the same response "We know there are problems and we will look into it." After so many cycles and the gap growing not shrinking just got us to the point of moving forwards (or reverting back to something that actually works for our users). BTW we did announce this and it was discussed in PTG. >> >> >>> 1. Support for new image import workflow >> >> Partially implemented by me and I continue working on that >> >>> 2. Support for hidden images >> >> Implemented >> >>> 3. Support for multihash >>> >>> 4. Support for multiple stores >> >> >> I am relying on OSC and especially for image service trying to bring it in a more useful state, thus fixing huge parts in SDK. > > > That's great and obviously you have the freedom to choose the client you prefer to use. Just like we have a moral responsibility to our users to provide them reference client that is up to date, works and the issues raised gets attention. > > This is all beyond the personal preference, which I get very mixed feedback of depending to whom I talk to. If I send the mail to the mailing list I get those same handful of people yelling right away how unified client is the only way to go and even thinking anything else is heresy. When I talk with people in the field, customers and users in the hallway tracks the message is much more mixed. The osc target audience prefers or just uses GUI instead, then there is a good portion of people who really don't care as they use some automation suite anyways, there is the old school guys who prefers a tool for a job as in the dedicated clients(I have to admit for disclaimer I belong to this group myself) and then there is a group of people who really don't care as long as the client they use every now and then just works. So honestly outside of those few voices in this mailing list I very rarely hear the demand of unified client and much more get the request to provide something that works, which was the major driver for our decision. > > Harsh, absolutely; justified, I'd like to think so. And this is just my personal experience with Glance, we're in a blessed situation of not jumping into the microversions bandwagon which seems to be totally hidden can of worms from us in this topic. > > For all the rest of you interested about the topic, next time you start demanding us to go to osc again, please put your money where your mouth is first and help those guys to deliver. > > Best, > Erno "jokke" Kuvaja > >> >>> >>> If anyone is interested to take up this work it will be great. >>> >>> Thanks & Best Regards, >>> >>> Abhishek Kekane >>> >>> >>> On Tue, Mar 3, 2020 at 12:24 AM Sean Mooney wrote: >>>> >>>> On Mon, 2020-03-02 at 18:05 +0000, Albert Braden wrote: >>>> > As an openstack operator I was pretty ecstatic to hear that the assortment of clients would be replaced by a single >>>> > client. I would be disappointed to find that a component would not integrate and would continue to use a separate >>>> > client. This would be a step backward IMO. >>>> > >>>> > The discussion about microversions goes over my head, but I would hope to see the developers get together and solve >>>> > the issue and continue working toward integration. >>>> just to summerisie it in a non technical way. >>>> the project specific cli had a convention where the client would ask the api what the newest micoverion it supported >>>> and defualt to that if the clinet suported it. that meant that the same command executed against two different clouds >>>> with different versions of openstakc deploy could have different behavior and different responces. so from an >>>> interoperablity point of view that is not great but from a usablity point of view the fact enduser dont have to care >>>> about microverions and the client would try to do the right thing made some things much simpler. >>>> >>>> the unifeid client (osc) chose to priorities interoperablity by defaulting to the oldest micorverions, so for nova that >>>> would be 2.0/2.1 meaning that if you execute the same command on two different cloud with different version of nova it >>>> will behave the same but if you want to use a feature intoduced in a later micorverion you have to explcitly request >>>> that via --os-compute-api-version or set that as a env var or in you cloud.yaml >>>> >>>> so really the difference is that osc requires the end user to be explictl about what micoversion to use and therefor be >>>> explict about the behavior of the api they expect (this is what we expect application that use the the api should do) >>>> where as the project client tried to just work and use the latest microverion which mostly workd excpet where we remove >>>> a feature in a later micorverions. for example we removed the force option on some move operation in nova because >>>> allowing forcing caused many harder to fix issues. i dont thnk the nova clinet would cap at the latest micorvierion that >>>> allowed forcing. so the poject client genreally did not guarantee that a command would work without specifcing a new >>>> micorverison it just that we remove things a hell of a lot less often then we add them. >>>> >>>> so as an end user that is the main difference between using osc vs glance clinet other then the fact i belive there is a >>>> bunch of stuff you can do with glance client that is missing in osc. parity is a spereate disucssion but it is vaild >>>> concern. >>>> >>>> -----Original Message----- >>>> > From: Radosław Piliszek >>>> > Sent: Monday, March 2, 2020 9:07 AM >>>> > To: openstack-discuss >>>> > Subject: Re: [glance] Different checksum between CLI and curl >>>> > >>>> > Folks, >>>> > >>>> > sorry to interrupt but I think we have diverged a bit too much from the subject. >>>> > Only last Gaetan message is on topic here. >>>> > Please switch to new subject to discuss OSC future. >>>> > >>>> > -yoctozepto >>>> > >>>> > pon., 2 mar 2020 o 18:03 Tim Bell napisał(a): >>>> > > >>>> > > >>>> > > >>>> > > On 2 Mar 2020, at 16:49, Dmitry Tantsur wrote: >>>> > > >>>> > > Hi, >>>> > > >>>> > > On Mon, Mar 2, 2020 at 4:29 PM Luigi Toscano wrote: >>>> > > > >>>> > > > On Monday, 2 March 2020 10:54:03 CET Mark Goddard wrote: >>>> > > > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane wrote: >>>> > > > > > Hi Gaëtan, >>>> > > > > > >>>> > > > > > Glance team doesn't recommend to use OSC anymore. >>>> > > > > > I will recommend you to check the same behaviour using >>>> > > > > > python-glanceclient. >>>> > > > > >>>> > > > > That's not cool - everyone has switched to OSC. It's also the first >>>> > > > > time I've heard of it. >>>> > > > > >>>> > > >>>> > > From the end user perspective, we’ve had positive feedback on the convergence to OSC from our cloud consumers. >>>> > > >>>> > > There has been great progress with Manila to get shares included ( >>>> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_642222_26_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=gfnHFJM7fXXAlOxyUenF0xGqH3gNiec3LxN-Gd5Ey-o&s=SYi8yPy9Dz0CgrkT5P6rTzs3141Gj4K9zO4Ht3GTYAk&e= >>>> > > ) and it would be a pity if we’re asking our end users to understand all of the different project names and >>>> > > inconsistent options/arguments/syntax. >>>> > > >>>> > > We had hoped for a project goal to get everyone aligned on OSC but there was not consensus on this, I’d still >>>> > > encourage it to simplify the experience for OpenStack cloud consumers. >>>> > > >>>> > > Tim >>>> > > >>>> > > >>>> > >>>> > >>>> >>>> From johnsomor at gmail.com Tue Mar 3 19:06:47 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 3 Mar 2020 11:06:47 -0800 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> Message-ID: Erno and I have had this discussion in the hallway at the PTGs before, so my response should be no surprise. The Octavia client is exclusively an OpenStack Client (OSC) plugin. This is partly because python-neutronclient (octavia was a neutron sub-project at the time) was already deprecated, but also because we saw the advantages and much improved user experience with OSC. We also exclusively use OSC for our devstack plugin scripts, etc. This includes interacting with glance[1]. Personally I have also moved to exclusively using OSC for my development work. I load/delete/show/tag images in glance on a daily basis. From my perspective, the basic features of glance work well, if not better due to the standardized output filtering/formatting support. So, I am an advocate for the OpenStack Client work and have contributed to it[2]. I also understand that glance has some development resource constraints. So, I have a few questions for the glance team: 1. Do we have RFEs for the feature gaps between the python-glanceclient and OSC? 2. Do we have a worklist that prioritizes these RFEs in order of use? 3. Do we have open stories for any OSC issues that may impact the above RFEs? If so, can you reply to this list with those links? I think there are folks here offering to help or enlist help to resolve these issues. Michael [1] https://github.com/openstack/octavia/blob/master/devstack/plugin.sh#L48 [2] https://review.opendev.org/#/c/662859/ On Tue, Mar 3, 2020 at 10:24 AM Albert Braden wrote: > > Sean, thank you for clarifying that. > > > > Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? > > > > We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. > > > > From: Sean McGinnis > Sent: Tuesday, March 3, 2020 9:50 AM > To: openstack-discuss at lists.openstack.org > Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > > > > On 3/3/20 11:28 AM, Albert Braden wrote: > > Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. > > I definitely would not characterize it that way. > > With trying not to put too much personal bias into it, here's what I would say the situation is: > > - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away > - Glance is a very small team with very, very limited resources > - The OSC team is a very small team with very, very limited resources > - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI > - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) > - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends > From Albert.Braden at synopsys.com Tue Mar 3 19:16:35 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 3 Mar 2020 19:16:35 +0000 Subject: Trouble with large RAM VMs - (formerly RE: Virtio memory balloon driver) Message-ID: I successfully blacklisted the "memory balloon" driver in a VM image and found that another driver is choking on the large memory. The guys in #centos think it might be the USB or PS/2 drivers. Since these are kernel drivers they cannot be blacklisted; the only way to not load them is to not call them. On the hypervisor in /etc/libvirt/qemu/.xml I see this:
I tried removing those lines, but it looks like the XML file is re-created whenever I stop and start the VM. In nova.conf I see "#pointer_model = usbtablet" If I set "pointer_model = " on the HV and then restart nova, I see this in the log: 2020-03-03 11:05:17.233 228915 ERROR nova ConfigFileValueError: Value for option pointer_model is not valid: Valid values are [None, ps2mouse, usbtablet], but found '' If I set "pointer_model = None" then I see this: 2020-03-03 11:06:24.761 229290 ERROR nova ConfigFileValueError: Value for option pointer_model is not valid: Valid values are [None, ps2mouse, usbtablet], but found 'None' What am I missing? # # Generic property to specify the pointer type. # # Input devices allow interaction with a graphical framebuffer. For # example to provide a graphic tablet for absolute cursor movement. # # If set, the 'hw_pointer_model' image property takes precedence over # this configuration option. # # Possible values: # # * None: Uses default behavior provided by drivers (mouse on PS2 for # libvirt x86) # * ps2mouse: Uses relative movement. Mouse connected by PS2 # * usbtablet: Uses absolute movement. Tablet connect by USB # # Related options: # # * usbtablet must be configured with VNC enabled or SPICE enabled and SPICE # agent disabled. When used with libvirt the instance mode should be # configured as HVM. # (string value) # Possible values: # - # ps2mouse - # usbtablet - pointer_model = None -----Original Message----- From: Albert Braden Sent: Wednesday, February 5, 2020 9:34 AM To: openstack-discuss at lists.openstack.org Subject: RE: Virtio memory balloon driver When I start and stop the giant VM I don't see any evidence of OOM errors. I suspect that the #centos guys may be correct when they say that the "Virtio memory balloon" device is not capable of addressing that much memory, and that I must disable it if I want to create VMs with 1.4T RAM. Setting "mem_stats_period_seconds = 0" doesn't seem to disable it. How are others working around this? Is anyone else creating Centos 6 VMs with 1.4T or more RAM? From fungi at yuggoth.org Tue Mar 3 19:44:27 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 3 Mar 2020 19:44:27 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> Message-ID: <20200303194427.4cmiy6rgaxfrlp6b@yuggoth.org> On 2020-03-03 18:20:59 +0000 (+0000), Albert Braden wrote: [...] > Was my understanding that the community decided to focus on the > unified client incorrect? Is the unified/individual client debate > still a matter of controversy? I'll try to avoid revisiting points other folks have made in this thread, but feel obliged to highlight that "the community" does not possess a hive mind and the idea that it "decides" things is somewhat of a misunderstanding. I would characterize the focus on the unified client as a consensus choice, but that doesn't mean that everyone who has a stake in that choice is in agreement with the direction (and with a community the size of OpenStack's, it's unlikely everyone will ever agree unanimously on a topic). > Is it possible that the unified client will be deprecated in favor > of individual clients after more discussion? [...] Anything is possible. I don't personally consider that a likely outcome, but I don't think anyone has a crystal ball which can tell us that for certain. > Do I need to start looking at individual clients again, and > telling our users to use them in some cases? [...] I would take it as a sign to not ask the Glance contributors if you run into a problem using the unified OpenStack client or SDK, and try reproducing a problem with direct API interactions before reporting a bug against the Glance service (to avoid being told that they won't even look at it if your reproducer relies on OSC). -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From mordred at inaugust.com Tue Mar 3 19:55:26 2020 From: mordred at inaugust.com (Monty Taylor) Date: Tue, 3 Mar 2020 13:55:26 -0600 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> Message-ID: > On Mar 3, 2020, at 12:20 PM, Albert Braden wrote: > > Sean, thank you for clarifying that. > > Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? Nope. Several of them even already don’t exist or are deprecated. Additiontally, several surrounding tools have explicit policies to NEVER touch python-*client libraries. Specifically Ansible - but I believe Salt has also migrated to SDK - and then any app developer who wants to be able to sanely target multiple clouds uses SDK instead of python-*client. So I can’t do anything about people preferring individual projects - but the unified stuff is DEFINITELY not getting deprecated or going away - quite simply because it cannot. And I hope that we can continue to convince more people of the power inherent in doing work to support their service in SDK/OSC instead of off in their own corner - but as I said, that I can’t do anything about. > I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? > > We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. > From: Sean McGinnis > Sent: Tuesday, March 3, 2020 9:50 AM > To: openstack-discuss at lists.openstack.org > Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > > On 3/3/20 11:28 AM, Albert Braden wrote: > Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. > I definitely would not characterize it that way. > > With trying not to put too much personal bias into it, here's what I would say the situation is: > > - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away > - Glance is a very small team with very, very limited resources > - The OSC team is a very small team with very, very limited resources > - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI > - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) > - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends From mordred at inaugust.com Tue Mar 3 20:12:30 2020 From: mordred at inaugust.com (Monty Taylor) Date: Tue, 3 Mar 2020 14:12:30 -0600 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> Message-ID: > On Mar 3, 2020, at 12:55 PM, Tim Bell wrote: > > > >> On 3 Mar 2020, at 19:20, Albert Braden wrote: >> >> Sean, thank you for clarifying that. >> >> Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? >> >> > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. > > My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. > > While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. > > Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) We’ve been working in SDK also to empower more people directly by being a bit more liberal with core. I think it’s time to start applying this approach to OSC as well. It’s never going to work to require the OSC team to implement everything, but neither is it super awesome to completely decentralize as the plugin/entrypoints issues have shown. I think SDK has been happy with blessing service humans rather quickly. > > BTW, I also would vote for =auto as the default This is what the case will be as we move towards replacing more and more of OSC’s guts with SDK. But let me describe it slightly differently: The way this works in SDK is that there is ONE user interface, which wants to track the latest as best as it can. But we can’t just do “auto” - because microversions can introduce breaking changes, so we need to add support to SDK for the most recent microversion we’re aware of. Then SDK negotiates to find the best microversion that is understands, and it always uses that. SDK has the POV that an end-user should almost never need to care about a micro version - if a user cares they are either in nova-core, or we’ve done something wrong. Case in point is this: https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/compute/v2/server.py#L457-L474 The nova team rightfully changed the semantics of live migrate because of safety. Mriedem put together the logic to express what the appropriate behavior would be, given a set of inputs, across the range of versions so that a user can do things and they’ll work. The end result is a live_migrate call that works across versions as safely as it can. I mention all of this because getting this work done was one of the key things we wanted to get right before we started transitioning OSC in earnest. It’s there - it works, and it’s being used across nova and ironic. So - I hear what people want from OSC - they want a thing that behaves like auto does. We agree - and the mechanism that makes us able to do that _safely_ is things like the above. > Tim > >> We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. >> >> From: Sean McGinnis >> Sent: Tuesday, March 3, 2020 9:50 AM >> To: openstack-discuss at lists.openstack.org >> Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) >> >> On 3/3/20 11:28 AM, Albert Braden wrote: >> Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. >> I definitely would not characterize it that way. >> >> With trying not to put too much personal bias into it, here's what I would say the situation is: >> >> - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away >> - Glance is a very small team with very, very limited resources >> - The OSC team is a very small team with very, very limited resources >> - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI >> - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) >> - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends >> From gagehugo at gmail.com Tue Mar 3 20:29:49 2020 From: gagehugo at gmail.com (Gage Hugo) Date: Tue, 3 Mar 2020 14:29:49 -0600 Subject: [openstack-helm] Virtual Midcycle - March 2020 Message-ID: Hello everyone, The openstack-helm team is looking to host a virtual midcycle this month to discuss a variety of topics in a more focused group via video conferencing. We have started an etherpad to track anyone who wishes to attend as well as any topics that want to be discussed. https://etherpad.openstack.org/p/osh-virtual-ptg-2020-03 Also there is a doodle poll to determine a time-slot in the next few weeks for the best time, we will try to choose the best time-slot based on the number of available people, please check it out here: https://doodle.com/poll/g6uvdb4rbad9s8gb We will update the etherpad to track any changes in meeting schedule, as well as when we find a web conference tool that works best. - Gage -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Tue Mar 3 20:34:27 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 3 Mar 2020 21:34:27 +0100 Subject: [nova] [neutron] multiple fixed_ip In-Reply-To: References: <20200303130104.GA29109@sync> Message-ID: Hi, If Your network has got IPv4 and IPv6 subnet, it should be done by default that created port will have one IPv4 and one IPv6 allocated. I just did it with nova client: nova boot --flavor m1.micro --image cirros-0.4.0 --nic net-name=private test-vm And my vm has IPs like: +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+ | 92385f1f-7899-40b7-94ec-bbceb6749722 | test-vm | ACTIVE | - | Running | private=fdc8:d3a9:de7b:0:f816:3eff:fe0d:16f5, 10.0.0.31 | +--------------------------------------+---------+--------+------------+-------------+————————————————————————————+ Also from novaclient help message it seems that You should be able to specify such IPv4 and IPv6 addresses: nova help boot | grep nic [--nic ] But that I didn’t try actually. > On 3 Mar 2020, at 14:23, Radosław Piliszek wrote: > > Hi Arnaud, > > Non-core here. > Last time I checked you had to decide on one and then update with > neutron (or first create the port with neutron and then give it to > nova :-) ). > Moreover, not sure if IPv6 goes through Nova directly or not (docs > suggest still nah). > > -yoctozepto > > wt., 3 mar 2020 o 14:09 Arnaud Morin napisał(a): >> >> >> Hello all, >> >> I was doing some tests to create a server using nova API. >> My objective is to create a server with one port but multiples IPs (one >> IPv4 and one IPv6). >> >> If I understand well the neutron API, I can create a port using the >> fixed_ips array parameter [1] >> >> Unfortunately, on nova side, it seems to only accept a string with only >> one ip (fixed_ip) [2] >> >> Is it mandatory for me to create the port with neutron? >> Or is there any trick that I missed on nova API side? >> >> Thanks! >> >> >> [1] https://docs.openstack.org/api-ref/network/v2/?expanded=create-port-detail#ports >> [2] https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server >> >> >> >> -- >> Arnaud Morin >> >> > — Slawek Kaplonski Senior software engineer Red Hat From pierre at stackhpc.com Tue Mar 3 21:36:59 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Tue, 3 Mar 2020 21:36:59 +0000 Subject: [glance] Different checksum between CLI and curl In-Reply-To: References: <5AC5FCDE-4F8E-478B-9BA0-34C527DDC2E2@inaugust.com> <10cb06508fa2146207462a9778253c22@incloudus.com> <40790667-B696-4CBC-9CD2-41A684D97D64@inaugust.com> Message-ID: Hi Gaëtan, Going back to your original question: have you tried to open downloaded with curl with a text editor? I am expecting that it is actually slightly bigger than the correct file downloaded with OSC, because it would include HTTP headers at the top. You should drop the `-i` option from your curl command line and try again: -i, --include (HTTP) Include the HTTP-header in the output. The HTTP-header includes things like server-name, date of the document, HTTP-version and more... Best wishes, Pierre Riteau (priteau) On Mon, 2 Mar 2020 at 15:21, wrote: > > Abhishek, > > Thansk for your answer, I tried both CLIs (Train release) and the issue > still the same. > > Paste of the "curl" command: http://paste.openstack.org/show/790197/ > > Result of the "md5sum" on the file created by the "curl": > $ md5sum /tmp/kernel.glance > c3726de8e03158305453f328d85e9957 /tmp/kernel.glance > > As Mark and Radoslaw, I'm quite surprised about OSC been deprecated. > Do you have any release note about this? > > Thanks for your help. > > Gaëtan > > curl -g -i -X GET > http://10.0.0.11:9292/v2/images/de39fc9c-b943-47e3-82c4-bd6d577a9577/file > -H "Content-Type: application/octet-stream" -H "User-Agent: > python-glanceclient" -H "X-Auth-Token: $token" --output > /tmp/kernel.glance -v > Note: Unnecessary use of -X or --request, GET is already inferred. > * Expire in 0 ms for 6 (transfer 0x557679b1de80) > * Trying 10.0.0.11... > * TCP_NODELAY set > * Expire in 200 ms for 4 (transfer 0x557679b1de80) > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0* Connected to 10.0.0.11 (10.0.0.11) port 9292 (#0) > > GET /v2/images/de39fc9c-b943-47e3-82c4-bd6d577a9577/file HTTP/1.1 > > Host: 10.0.0.11:9292 > > Accept: */* > > Content-Type: application/octet-stream > > User-Agent: python-glanceclient > > X-Auth-Token: > > gAAAAABeXRzKVS3uQIIv9t-wV7njIV-T9HIvcwFqcHNivrpyBlesDtgAj1kpWk59a20EJLUo8IeHpTdKgVFwhnAVvbSWHY-HQvxu5dwSFsK4A-7CzAOwdp3svSqxB-FdwWhsY_PElftMX4geA-y_YFZJamefZapiAv6g1gSm-BSv5GYQ0hj3yGY > > > 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- > 0< HTTP/1.1 200 OK > < Content-Type: application/octet-stream > < Content-Md5: 26c6d5c3d8ba9fd4bc4d1ee5959a827c > < Content-Length: 5631728 > < X-Openstack-Request-Id: req-e7ba2455-780f-48a8-b6a2-1c6741d0e368 > < Date: Mon, 02 Mar 2020 15:03:53 GMT > < > { [32768 bytes data] > 100 5499k 100 5499k 0 0 4269k 0 0:00:01 0:00:01 --:--:-- > 4269k > * Connection #0 to host 10.0.0.11 left intact > > > On 2020-03-02 04:54, Mark Goddard wrote: > > On Mon, 2 Mar 2020 at 06:28, Abhishek Kekane > > wrote: > >> > >> Hi Gaëtan, > >> > >> Glance team doesn't recommend to use OSC anymore. > >> I will recommend you to check the same behaviour using > >> python-glanceclient. > > > > That's not cool - everyone has switched to OSC. It's also the first > > time I've heard of it. > > > >> > >> Thanks & Best Regards, > >> > >> Abhishek Kekane > >> > >> > >> On Sat, Feb 29, 2020 at 3:54 AM Monty Taylor > >> wrote: > >>> > >>> > >>> > >>> > On Feb 28, 2020, at 4:15 PM, gaetan.trellu at incloudus.com wrote: > >>> > > >>> > Hey Monty, > >>> > > >>> > If I download the image via the CLI, the checksum of the file matches the checksum from the image details. > >>> > If I download the image via "curl", the "Content-Md5" header matches the image details but the file checksum doesn't. > >>> > > >>> > The files have the same size, this is really weird. > >>> > >>> WOW. > >>> > >>> I still don’t know the issue - but my unfounded hunch is that the > >>> curl command is likely not doing something it should be. If OSC is > >>> producing a file that matches the image details, that seems like the > >>> right choice for now. > >>> > >>> Seriously fascinating though. > >>> > >>> > Gaëtan > >>> > > >>> > On 2020-02-28 17:00, Monty Taylor wrote: > >>> >>> On Feb 28, 2020, at 2:29 PM, gaetan.trellu at incloudus.com wrote: > >>> >>> Hi guys, > >>> >>> Does anyone know why the md5 checksum is different between the "openstack image save" CLI and "curl" commands? > >>> >>> During the image creation a checksum is computed to check the image integrity, using the "openstack" CLI match the checksum generated but when "curl" is used by following the API documentation[1] the checksum change at every "download". > >>> >>> Any idea? > >>> >> That seems strange. I don’t know off the top of my head. I do know > >>> >> Artem has patches up to switch OSC to using SDK for image operations. > >>> >> https://review.opendev.org/#/c/699416/ > >>> >> That said, I’d still expect current OSC checksums to be solid. Perhaps > >>> >> there is some filtering/processing being done cloud-side in your > >>> >> glance? If you download the image to a file and run a checksum on it - > >>> >> does it match the checksum given by OSC on upload? Or the checksum > >>> >> given by glance API on download? > >>> >>> Thanks, > >>> >>> Gaëtan > >>> >>> [1] https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-binary-image-data-detail#download-binary-image-data > >>> > > >>> > >>> > From openstack at nemebean.com Tue Mar 3 23:05:53 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 3 Mar 2020 17:05:53 -0600 Subject: [oslo][infra] OpenDev git repo for oslo.policy missing commit Message-ID: <1129d4b2-0a8d-d034-5ded-7e49e6e49a77@nemebean.com> Found a weird thing today. The OpenDev oslo.policy repo[0] is missing [1]. Even stranger, I see it on the Github mirror[2]. Any idea what happened here? -Ben 0: https://opendev.org/openstack/oslo.policy/commits/branch/master 1: https://review.opendev.org/#/c/708212/ 2: https://github.com/openstack/oslo.policy/commits/master From cboylan at sapwetik.org Tue Mar 3 23:42:53 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 03 Mar 2020 15:42:53 -0800 Subject: [oslo][infra] OpenDev git repo for oslo.policy missing commit In-Reply-To: <1129d4b2-0a8d-d034-5ded-7e49e6e49a77@nemebean.com> References: <1129d4b2-0a8d-d034-5ded-7e49e6e49a77@nemebean.com> Message-ID: On Tue, Mar 3, 2020, at 3:05 PM, Ben Nemec wrote: > Found a weird thing today. The OpenDev oslo.policy repo[0] is missing > [1]. Even stranger, I see it on the Github mirror[2]. Any idea what > happened here? Some other readers may notice that the commit actually does show up for them. The reason for this is the commit is only missing from one of eight backend gitea servers. You can observe this by visiting https://gitea0X.opendev.org:3000/openstack/oslo.policy/commits/branch/master and replacing the X with 1 through 8. Number 5 is the lucky server. My hunch is that this commit merging and subsequently being replicated coincided with a restart of gitea (or related service) on gitea05. And the replication event was missed. We've tried to ensure we replicate to catch up after explicit upgrades, which implies to me that maybe the db container updated. Note that https://review.opendev.org/#/c/705804/ merged on the same day but after the missing commit. In any case I've triggered a full rereplication to gitea05 to make sure we are caught up and will work through the others as well to ensure none are missed. You should be able to confirm that the commit is present in about 20 minutes. Longer term the plan here is to run a single Gitea cluster which will allow us to do rolling restarts of services without impacting replication. Unfortunately, this requires updates to Gitea to support that. > > -Ben > > 0: https://opendev.org/openstack/oslo.policy/commits/branch/master > 1: https://review.opendev.org/#/c/708212/ > 2: https://github.com/openstack/oslo.policy/commits/master > > From kennelson11 at gmail.com Tue Mar 3 23:43:46 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 3 Mar 2020 15:43:46 -0800 Subject: [all][PTL][tc] U Community Goal: Project PTL & Contrib Docs Update #4 Message-ID: Hello! The decision has been made and the template has been merged[1] with the actual governance update well on its way to being accepted as well[2][3]. With that in mind, its time to get down to business! If You Haven't Started ================= You're in luck! Nothing has to change, just run the cookie cutter[8] and make whatever changes you need to the template and get it added to the repos for your project. Also, please keep the task tracker in StoryBoard up to date[4] as you work. If You Had 'Completed' it before Update #2 ================================ Go and check out the governance change[2] and the template[1] to make sure there isn't anything you did differently-- depending on when you 'completed' the goal, you might have some changes to make to be in line with the current (and presumably final) completion criteria. And, as always, please make sure to keep the tasks in StoryBoard up to date[4]. Previous updates if you missed them[5][6][7]. If you have any questions, please let me know. If you need help with reviewing, please feel free to add me as a reviewer. We can still get this done in time! Thanks, -Kendall (diablo_rojo) [1] Cookie Cutter Change https://review.opendev.org/#/c/708672/ [2] Governance Goal Change https://review.opendev.org/#/c/709617/ [3] Governance Goal Change Typo Fix https://review.opendev.org/#/c/710332 [4] StoryBoard Tracking: https://storyboard.openstack.org/#!/story/2007236 [5]Update #1: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012364.html [6] Update #2: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012570.html [7] Update 3: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012784.html [8] Cookiecutter Template: https://opendev.org/openstack/cookiecutter/src/branch/master/%7b%7bcookiecutter.repo_name%7d%7d/doc/source/contributor/contributing.rst -------------- next part -------------- An HTML attachment was scrubbed... URL: From gaetan.trellu at incloudus.com Wed Mar 4 00:06:37 2020 From: gaetan.trellu at incloudus.com (gaetan.trellu at incloudus.com) Date: Tue, 03 Mar 2020 19:06:37 -0500 Subject: [glance] Different checksum between CLI and curl In-Reply-To: Message-ID: <6077cc75-915c-4ed0-a973-ebefda5fd8cd@email.android.com> An HTML attachment was scrubbed... URL: From Albert.Braden at synopsys.com Wed Mar 4 00:17:28 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Wed, 4 Mar 2020 00:17:28 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> Message-ID: Thanks everyone for the helpful answers. I think I understand the situation now, and I know what to expect. I'll continue attempting to get signed up as a developer as time permits. Hopefully I can help fix some of the OSC issues. -----Original Message----- From: Monty Taylor Sent: Tuesday, March 3, 2020 12:13 PM To: Tim Bell Cc: openstack-discuss at lists.openstack.org; Sean McGinnis ; Albert Braden Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > On Mar 3, 2020, at 12:55 PM, Tim Bell wrote: > > > >> On 3 Mar 2020, at 19:20, Albert Braden wrote: >> >> Sean, thank you for clarifying that. >> >> Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? >> >> > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. > > My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. > > While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. > > Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://urldefense.proofpoint.com/v2/url?u=https-3A__www.stackalytics.com_-3Fcompany-3Dcern-26metric-3Dcommits-26module-3Dpython-2Dopenstackclient&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=PXaDUXdLASc5aN6yB6-2EAZhPajJl-7Ue1eZOWhBS-s&s=r14Sy3wGjaak4CcfQegBje22E5rxQKgxMq_x9dXcDH0&e= ) We’ve been working in SDK also to empower more people directly by being a bit more liberal with core. I think it’s time to start applying this approach to OSC as well. It’s never going to work to require the OSC team to implement everything, but neither is it super awesome to completely decentralize as the plugin/entrypoints issues have shown. I think SDK has been happy with blessing service humans rather quickly. > > BTW, I also would vote for =auto as the default This is what the case will be as we move towards replacing more and more of OSC’s guts with SDK. But let me describe it slightly differently: The way this works in SDK is that there is ONE user interface, which wants to track the latest as best as it can. But we can’t just do “auto” - because microversions can introduce breaking changes, so we need to add support to SDK for the most recent microversion we’re aware of. Then SDK negotiates to find the best microversion that is understands, and it always uses that. SDK has the POV that an end-user should almost never need to care about a micro version - if a user cares they are either in nova-core, or we’ve done something wrong. Case in point is this: https://urldefense.proofpoint.com/v2/url?u=https-3A__opendev.org_openstack_openstacksdk_src_branch_master_openstack_compute_v2_server.py-23L457-2DL474&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=PXaDUXdLASc5aN6yB6-2EAZhPajJl-7Ue1eZOWhBS-s&s=DIR_Qd19fwvb18RV_tqnnwOwFjffzojLOLIeE1oKLtU&e= The nova team rightfully changed the semantics of live migrate because of safety. Mriedem put together the logic to express what the appropriate behavior would be, given a set of inputs, across the range of versions so that a user can do things and they’ll work. The end result is a live_migrate call that works across versions as safely as it can. I mention all of this because getting this work done was one of the key things we wanted to get right before we started transitioning OSC in earnest. It’s there - it works, and it’s being used across nova and ironic. So - I hear what people want from OSC - they want a thing that behaves like auto does. We agree - and the mechanism that makes us able to do that _safely_ is things like the above. > Tim > >> We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. >> >> From: Sean McGinnis >> Sent: Tuesday, March 3, 2020 9:50 AM >> To: openstack-discuss at lists.openstack.org >> Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) >> >> On 3/3/20 11:28 AM, Albert Braden wrote: >> Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. >> I definitely would not characterize it that way. >> >> With trying not to put too much personal bias into it, here's what I would say the situation is: >> >> - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away >> - Glance is a very small team with very, very limited resources >> - The OSC team is a very small team with very, very limited resources >> - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI >> - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) >> - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends >> From sean.mcginnis at gmx.com Wed Mar 4 01:11:24 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 4 Mar 2020 02:11:24 +0100 Subject: [all] Announcing OpenStack Wallaby! Message-ID: That's right - we have the W name already. No months on end of referring to "the W release" as we start making long term plans. Wallaby https://en.wikipedia.org/wiki/Wallaby Wallabies are native to Australia, which at the start of this naming period was experiencing unprecedented wild fires. This name has passed the legal vetting phase and we are good to go. Full results of the naming poll can be found here: https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_60b98f102c5d08d5 As a reminder, the way we choose a release name had several changes this time around. Full details of the current release process can be found here: https://governance.openstack.org/tc/reference/release-naming.html There were a lot of great names proposed. I think removing the geographical restriction has been a good move. Thank you to everyone who proposed ideas for this release cycle. Happy stacking! Sean From gmann at ghanshyammann.com Wed Mar 4 01:15:49 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Tue, 03 Mar 2020 19:15:49 -0600 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> Message-ID: <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell wrote ---- > > > On 3 Mar 2020, at 19:55, Tim Bell wrote: > > > On 3 Mar 2020, at 19:20, Albert Braden wrote: > Sean, thank you for clarifying that. > > Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? > > > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. > BTW, I found the etherpad from Berlin (https://etherpad.openstack.org/p/BER-t-series-goals) and the associated mailing list discussion at http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html Yeah, we are in process of selecting the Victoria cycle community-wide goal and this can be good candidate. I agree with the idea/requirement of a multi-cycle goal. Another option is to build a pop-up team for the Victoria cycle to start burning down the keys issues/work. For both ways (either goal or pop-up team), we need some set of people to drive it. If anyone would like to volunteer for this, we can start discussing the details. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html -gmann > > My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. > While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. > Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) > BTW, I also would vote for =auto as the default. > Tim > We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. > > From: Sean McGinnis > Sent: Tuesday, March 3, 2020 9:50 AM > To: openstack-discuss at lists.openstack.org > Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > > On 3/3/20 11:28 AM, Albert Braden wrote: > Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. > I definitely would not characterize it that way. > With trying not to put too much personal bias into it, here's what I would say the situation is: > - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away > - Glance is a very small team with very, very limited resources > - The OSC team is a very small team with very, very limited resources > - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI > - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) > - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends > > > > > From zhipengh512 at gmail.com Wed Mar 4 04:06:39 2020 From: zhipengh512 at gmail.com (Zhipeng Huang) Date: Wed, 4 Mar 2020 12:06:39 +0800 Subject: [all] Announcing OpenStack Wallaby! In-Reply-To: References: Message-ID: Cuuute ! On Wed, Mar 4, 2020 at 9:15 AM Sean McGinnis wrote: > That's right - we have the W name already. No months on end of referring > to "the W release" as we start making long term plans. > > Wallaby > https://en.wikipedia.org/wiki/Wallaby > Wallabies are native to Australia, which at the start of this naming > period was experiencing unprecedented wild fires. > > This name has passed the legal vetting phase and we are good to go. Full > results of the naming poll can be found here: > > https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_60b98f102c5d08d5 > > As a reminder, the way we choose a release name had several changes this > time around. Full details of the current release process can be found here: > > https://governance.openstack.org/tc/reference/release-naming.html > > There were a lot of great names proposed. I think removing the > geographical restriction has been a good move. Thank you to everyone who > proposed ideas for this release cycle. > > Happy stacking! > > Sean > > -- Zhipeng (Howard) Huang Principle Engineer OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, DMTF, W3C -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Wed Mar 4 06:59:46 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Wed, 4 Mar 2020 06:59:46 +0000 Subject: [nova] [neutron] multiple fixed_ip In-Reply-To: References: <20200303130104.GA29109@sync> Message-ID: <20200304065946.GB29109@sync> Hello Slawek and all, You are right, I forgot to mention that I want to set a specific IP, so yes I am using the parameters you describe (v4-fixed-ip). However when trying to use both the v4-fixed-ip and v6-fixed-ip, it does not work. It seems that those params are mutually exclusives [1] on client side. Does anyone know if the server would accept more than one fixed-ips? My attemps were not sucessful regarding this matter. Another question is the accessIPv4 (and v6) params, does anyone know what they are used for? Because they seems completely ignored by neutron when used on nova client side [2]. [1] https://github.com/openstack/python-novaclient/blob/b9a7e03074cbaacc3f270b2b8228a5b85350a2de/novaclient/v2/servers.py#L798 [2] https://github.com/openstack/python-novaclient/blob/b9a7e03074cbaacc3f270b2b8228a5b85350a2de/novaclient/v2/servers.py#L815 -- Arnaud Morin On 03.03.20 - 21:34, Slawek Kaplonski wrote: > Hi, > > If Your network has got IPv4 and IPv6 subnet, it should be done by default that created port will have one IPv4 and one IPv6 allocated. I just did it with nova client: > > nova boot --flavor m1.micro --image cirros-0.4.0 --nic net-name=private test-vm > > And my vm has IPs like: > > +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+ > | ID | Name | Status | Task State | Power State | Networks | > +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+ > | 92385f1f-7899-40b7-94ec-bbceb6749722 | test-vm | ACTIVE | - | Running | private=fdc8:d3a9:de7b:0:f816:3eff:fe0d:16f5, 10.0.0.31 | > +--------------------------------------+---------+--------+------------+-------------+————————————————————————————+ > > Also from novaclient help message it seems that You should be able to specify such IPv4 and IPv6 addresses: > > nova help boot | grep nic > [--nic ] > > But that I didn’t try actually. > > > On 3 Mar 2020, at 14:23, Radosław Piliszek wrote: > > > > Hi Arnaud, > > > > Non-core here. > > Last time I checked you had to decide on one and then update with > > neutron (or first create the port with neutron and then give it to > > nova :-) ). > > Moreover, not sure if IPv6 goes through Nova directly or not (docs > > suggest still nah). > > > > -yoctozepto > > > > wt., 3 mar 2020 o 14:09 Arnaud Morin napisał(a): > >> > >> > >> Hello all, > >> > >> I was doing some tests to create a server using nova API. > >> My objective is to create a server with one port but multiples IPs (one > >> IPv4 and one IPv6). > >> > >> If I understand well the neutron API, I can create a port using the > >> fixed_ips array parameter [1] > >> > >> Unfortunately, on nova side, it seems to only accept a string with only > >> one ip (fixed_ip) [2] > >> > >> Is it mandatory for me to create the port with neutron? > >> Or is there any trick that I missed on nova API side? > >> > >> Thanks! > >> > >> > >> [1] https://docs.openstack.org/api-ref/network/v2/?expanded=create-port-detail#ports > >> [2] https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server > >> > >> > >> > >> -- > >> Arnaud Morin > >> > >> > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > From lijie at unitedstack.com Wed Mar 4 07:44:17 2020 From: lijie at unitedstack.com (=?utf-8?B?UmFtYm8=?=) Date: Wed, 4 Mar 2020 15:44:17 +0800 Subject: [nova] ask some questions about flavor Message-ID: Hi,all:         I have two questions about the flavor. One is the property "OS-FLV-DISABLED:disabled" defined when we created the flavor, but there is no method to change the property. And I see a patch about this [1], but it is abandoned. So I want to know our community how to consider this function.          Another one is when we call the "list-migrations" api, why we receive the new_instance_type_id and old_instance_type_id in response of list_migrations should be internal value for the migration-type is "resize"[2]? Maybe the value should be exposed on REST API, so that we can know which is the old flavor.          Can you tell me more about this? Thank you very much. Ref: [1]:https://review.opendev.org/#/c/61291/ [2]:https://review.opendev.org/#/c/588481/ Best Regards Rambo -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhangbailin at inspur.com Wed Mar 4 09:09:27 2020 From: zhangbailin at inspur.com (=?utf-8?B?QnJpbiBaaGFuZyjlvKDnmb7mnpcp?=) Date: Wed, 4 Mar 2020 09:09:27 +0000 Subject: =?utf-8?B?UmU6IFtsaXN0cy5vcGVuc3RhY2sub3Jn5Luj5Y+RXVtub3ZhXSBhc2sgc29t?= =?utf-8?Q?e_questions_about_flavor?= Message-ID: <28518e7a5cbb40f2b208f3a4482b3b94@inspur.com> Items: [lists.openstack.org代发][nova] ask some questions about flavor Hi,all: > I have two questions about the flavor. One is the property "OS-FLV-DISABLED:disabled" defined when we created the flavor, but there is no method to change the property. And I see a patch about this [1], but it is abandoned. So I want to know our community how > to consider this function. "OS-FLV-DISABLED:disabled", this is typically only visible to administrative users. I'm not sure if it is worth supporting the modification/update. I think it should be planned in advance for managers. If the specific extra spec bound to flavor is only used as a dedicated flavor. > Another one is when we call the "list-migrations" api, why we receive the new_instance_type_id and old_instance_type_id in response of list_migrations should be internal value for the migration-type is "resize"[2]? Maybe the value should be exposed on REST > API, so that we can know which is the old flavor. In microversion 2.23 we support to show “migration-type” in the List Migrations API [1][2], I think you call List Migrations REST API is not add the “OpenStack-API-Version” or “X-OpenStack-Nova-API-Version”, right？ And you can get the list migrations filter by “migration-type” (enum in: evacuation, live-migration, migration (cold), resize), as the call such as: http://192.168.2.11/compute/v2.1/os-migrations?migration_type=resize that show what migration type what you want. [1] API changes: https://docs.openstack.org/api-ref/compute/?expanded=list-migrations-detail,show-migration-details-detail,id320-detail#list-migrations [2] microverion 2.3: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id21 > Can you tell me more about this? Thank you very much. Ref: [1]:https://review.opendev.org/#/c/61291/ [2]:https://review.opendev.org/#/c/588481/ Best Regards Rambo -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.urdin at binero.se Wed Mar 4 09:19:59 2020 From: tobias.urdin at binero.se (Tobias Urdin) Date: Wed, 4 Mar 2020 10:19:59 +0100 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> Message-ID: Can shime in here that Puppet OpenStack has almost completely migrated to OSC since a pretty long time ago, including Glance. I think we only have some usage to the neutron CLI that needs to be replaced. (Even though I would like to talk directly to the APIs, but hey, Ruby isn't my strong suit). Best regards On 3/3/20 8:59 PM, Monty Taylor wrote: > >> On Mar 3, 2020, at 12:20 PM, Albert Braden wrote: >> >> Sean, thank you for clarifying that. >> >> Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? > Nope. Several of them even already don’t exist or are deprecated. > > Additiontally, several surrounding tools have explicit policies to NEVER touch python-*client libraries. Specifically Ansible - but I believe Salt has also migrated to SDK - and then any app developer who wants to be able to sanely target multiple clouds uses SDK instead of python-*client. > > So I can’t do anything about people preferring individual projects - but the unified stuff is DEFINITELY not getting deprecated or going away - quite simply because it cannot. And I hope that we can continue to convince more people of the power inherent in doing work to support their service in SDK/OSC instead of off in their own corner - but as I said, that I can’t do anything about. > >> I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? >> >> We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. >> From: Sean McGinnis >> Sent: Tuesday, March 3, 2020 9:50 AM >> To: openstack-discuss at lists.openstack.org >> Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) >> >> On 3/3/20 11:28 AM, Albert Braden wrote: >> Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. >> I definitely would not characterize it that way. >> >> With trying not to put too much personal bias into it, here's what I would say the situation is: >> >> - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away >> - Glance is a very small team with very, very limited resources >> - The OSC team is a very small team with very, very limited resources >> - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI >> - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) >> - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends > > From lijie at unitedstack.com Wed Mar 4 09:31:34 2020 From: lijie at unitedstack.com (=?utf-8?B?UmFtYm8=?=) Date: Wed, 4 Mar 2020 17:31:34 +0800 Subject: =?utf-8?B?UmU6UmU6IFtsaXN0cy5vcGVuc3RhY2sub3Jn5Luj?= =?utf-8?B?5Y+RXVtub3ZhXSBhc2sgc29tZSBxdWVzdGlvbnMg?= =?utf-8?B?YWJvdXQgZmxhdm9y?= In-Reply-To: <28518e7a5cbb40f2b208f3a4482b3b94@inspur.com> References: <28518e7a5cbb40f2b208f3a4482b3b94@inspur.com> Message-ID: Oh,no. I know we support to get the list migrations filter by "migration-type". Actually, I just want to know why the "new_instance_type_id" and "old_instance_type_id" in response of list_migrations the response should be an *internal* value.     ------------------ Original ------------------ From: "Brin Zhang(张百林)"; Date: 2020年3月4日(星期三) 下午5:09 To: "lijie at unitedstack.com"; "openstack-discuss at lists.openstack.org"; Subject: Re: [lists.openstack.org代发][nova] ask some questions about flavor   Items: [lists.openstack.org代发][nova] ask some questions about flavor   Hi,all: > I have two questions about the flavor. One is the property "OS-FLV-DISABLED:disabled" defined when we created the flavor, but there is no method to change the property. And I see a patch about this [1], but it is abandoned. So I want to know our community how > to consider this function.   "OS-FLV-DISABLED:disabled", this is typically only visible to administrative users. I'm not sure if it is worth supporting the modification/update. I think it should be planned in advance for managers. If the specific extra spec bound to flavor is only used as a dedicated flavor.   > Another one is when we call the "list-migrations" api, why we receive the new_instance_type_id and old_instance_type_id in response of list_migrations should be internal value for the migration-type is "resize"[2]? Maybe the value should be exposed on REST > API, so that we can know which is the old flavor.   In microversion 2.23 we support to show “migration-type” in the List Migrations API [1][2], I think you call List Migrations REST  API is not add the “OpenStack-API-Version” or “X-OpenStack-Nova-API-Version”, right？   And you can get the list migrations filter by “migration-type” (enum in: evacuation, live-migration, migration (cold), resize), as the call such as: http://192.168.2.11/compute/v2.1/os-migrations?migration_type=resize that show what migration type what you want.   [1] API changes: https://docs.openstack.org/api-ref/compute/?expanded=list-migrations-detail,show-migration-details-detail,id320-detail#list-migrations [2] microverion 2.3: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id21   > Can you tell me more about this? Thank you very much.         Ref: [1]:https://review.opendev.org/#/c/61291/ [2]:https://review.opendev.org/#/c/588481/     Best Regards Rambo -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Wed Mar 4 09:57:52 2020 From: mark at stackhpc.com (Mark Goddard) Date: Wed, 4 Mar 2020 09:57:52 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> Message-ID: On Wed, 4 Mar 2020 at 01:16, Ghanshyam Mann wrote: > > ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell wrote ---- > > > > > > On 3 Mar 2020, at 19:55, Tim Bell wrote: > > > > > > On 3 Mar 2020, at 19:20, Albert Braden wrote: > > Sean, thank you for clarifying that. > > > > Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? > > > > > > > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. > > BTW, I found the etherpad from Berlin (https://etherpad.openstack.org/p/BER-t-series-goals) and the associated mailing list discussion at http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html > > Yeah, we are in process of selecting the Victoria cycle community-wide goal and this can be good candidate. I agree with the idea/requirement of a multi-cycle goal. > Another option is to build a pop-up team for the Victoria cycle to start burning down the keys issues/work. For both ways (either goal or pop-up team), we need > some set of people to drive it. If anyone would like to volunteer for this, we can start discussing the details. > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html > > -gmann This seems like quite an important issue for OpenStack usability. Clearly there are resourcing issues within the glance team (and possibly also some personal preferences) that have prevented OSC gaining feature parity with the glance client. Having all of the core projects able to recommend using OSC seems to me like it should be quite a high priority - more so than having support for every project out there. Would cross-project goal effort be better spent swarming on filling these gaps first? Do we have any mechanisms to help drive that? I know we have the help most-wanted list. > > > > > My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. > > While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. > > Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) > > BTW, I also would vote for =auto as the default. > > Tim > > We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. > > > > From: Sean McGinnis > > Sent: Tuesday, March 3, 2020 9:50 AM > > To: openstack-discuss at lists.openstack.org > > Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > > > > On 3/3/20 11:28 AM, Albert Braden wrote: > > Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. > > I definitely would not characterize it that way. > > With trying not to put too much personal bias into it, here's what I would say the situation is: > > - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away > > - Glance is a very small team with very, very limited resources > > - The OSC team is a very small team with very, very limited resources > > - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI > > - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) > > - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends > > > > > > > > > > > From skaplons at redhat.com Wed Mar 4 10:15:40 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 4 Mar 2020 11:15:40 +0100 Subject: [neutron] Review priorities for RFEs Message-ID: <216E1F95-06AB-413B-BC9E-0509FEDF76F8@redhat.com> Hi neutrinos, I just went through patches related to our RFEs which we want to merge before Ussuri-3 and I set them review priority flag. You can find them at [1]. If I missed any patch related to any of those RFEs, please let me know by email or on IRC. And please try to review those patches if possible ;) Thx in advance. [1] https://tinyurl.com/vezk6n6 — Slawek Kaplonski Senior software engineer Red Hat From arnaud.morin at gmail.com Wed Mar 4 11:02:58 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Wed, 4 Mar 2020 11:02:58 +0000 Subject: [nova] [neutron] multiple fixed_ip In-Reply-To: References: <20200303130104.GA29109@sync> Message-ID: <20200304110258.GC29109@sync> Hello, Thanks for you answer, that's exactly what we are doing right now, I just wanted to know if there is a more straight solution :p Thanks! -- Arnaud Morin On 03.03.20 - 14:23, Radosław Piliszek wrote: > Hi Arnaud, > > Non-core here. > Last time I checked you had to decide on one and then update with > neutron (or first create the port with neutron and then give it to > nova :-) ). > Moreover, not sure if IPv6 goes through Nova directly or not (docs > suggest still nah). > > -yoctozepto > > wt., 3 mar 2020 o 14:09 Arnaud Morin napisał(a): > > > > > > Hello all, > > > > I was doing some tests to create a server using nova API. > > My objective is to create a server with one port but multiples IPs (one > > IPv4 and one IPv6). > > > > If I understand well the neutron API, I can create a port using the > > fixed_ips array parameter [1] > > > > Unfortunately, on nova side, it seems to only accept a string with only > > one ip (fixed_ip) [2] > > > > Is it mandatory for me to create the port with neutron? > > Or is there any trick that I missed on nova API side? > > > > Thanks! > > > > > > [1] https://docs.openstack.org/api-ref/network/v2/?expanded=create-port-detail#ports > > [2] https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server > > > > > > > > -- > > Arnaud Morin > > > > From radoslaw.piliszek at gmail.com Wed Mar 4 11:17:12 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Wed, 4 Mar 2020 12:17:12 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool Message-ID: Please be informed that oslo.cache 2.1.0 breaks oslo_cache.memcache_pool Kolla-Ansible gate is already RED and a quick codesearch revealed other deployment methods might be in trouble soon as well. This does not affect devstack/tempest as they use dogpile.cache.memcached instead. The error is TypeError: __init__() got an unexpected keyword argument 'dead_retry' For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 -yoctozepto From hberaud at redhat.com Wed Mar 4 12:20:46 2020 From: hberaud at redhat.com (Herve Beraud) Date: Wed, 4 Mar 2020 13:20:46 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: I think our issue is due to the fact that python-memcached accept a param named `dead_retry` [1] which is not defined in pymemcache. We just need to define it in our oslo.cache mapping. During testing we faced the same kind of issue with connection timeout. [1] https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 [2] https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 Le mer. 4 mars 2020 à 12:21, Radosław Piliszek a écrit : > Please be informed that oslo.cache 2.1.0 breaks oslo_cache.memcache_pool > > Kolla-Ansible gate is already RED and a quick codesearch revealed > other deployment methods might be in trouble soon as well. > > This does not affect devstack/tempest as they use > dogpile.cache.memcached instead. > > The error is TypeError: __init__() got an unexpected keyword argument > 'dead_retry' > > For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 > > -yoctozepto > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hberaud at redhat.com Wed Mar 4 12:28:58 2020 From: hberaud at redhat.com (Herve Beraud) Date: Wed, 4 Mar 2020 13:28:58 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: What do you think about adding a mapping between `retry_timeout` [1] and `dead_retry` [2]? [1] https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L56 [2] https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 Le mer. 4 mars 2020 à 13:20, Herve Beraud a écrit : > I think our issue is due to the fact that python-memcached accept a param > named `dead_retry` [1] which is not defined in pymemcache. > > We just need to define it in our oslo.cache mapping. During testing we > faced the same kind of issue with connection timeout. > > [1] > https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 > [2] > https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 > > Le mer. 4 mars 2020 à 12:21, Radosław Piliszek < > radoslaw.piliszek at gmail.com> a écrit : > >> Please be informed that oslo.cache 2.1.0 breaks oslo_cache.memcache_pool >> >> Kolla-Ansible gate is already RED and a quick codesearch revealed >> other deployment methods might be in trouble soon as well. >> >> This does not affect devstack/tempest as they use >> dogpile.cache.memcached instead. >> >> The error is TypeError: __init__() got an unexpected keyword argument >> 'dead_retry' >> >> For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 >> >> -yoctozepto >> >> > > -- > Hervé Beraud > Senior Software Engineer > Red Hat - Openstack Oslo > irc: hberaud > -----BEGIN PGP SIGNATURE----- > > wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ > Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ > RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP > F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G > 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g > glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw > m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ > hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 > qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y > F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 > B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O > v6rDpkeNksZ9fFSyoY2o > =ECSj > -----END PGP SIGNATURE----- > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hberaud at redhat.com Wed Mar 4 12:35:37 2020 From: hberaud at redhat.com (Herve Beraud) Date: Wed, 4 Mar 2020 13:35:37 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: `dead_timeout` [1] looks more appropriate in this case. [1] https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L58 Le mer. 4 mars 2020 à 13:28, Herve Beraud a écrit : > What do you think about adding a mapping between `retry_timeout` [1] and > `dead_retry` [2]? > > [1] > https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L56 > [2] > https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 > > Le mer. 4 mars 2020 à 13:20, Herve Beraud a écrit : > >> I think our issue is due to the fact that python-memcached accept a param >> named `dead_retry` [1] which is not defined in pymemcache. >> >> We just need to define it in our oslo.cache mapping. During testing we >> faced the same kind of issue with connection timeout. >> >> [1] >> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >> [2] >> https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 >> >> Le mer. 4 mars 2020 à 12:21, Radosław Piliszek < >> radoslaw.piliszek at gmail.com> a écrit : >> >>> Please be informed that oslo.cache 2.1.0 breaks oslo_cache.memcache_pool >>> >>> Kolla-Ansible gate is already RED and a quick codesearch revealed >>> other deployment methods might be in trouble soon as well. >>> >>> This does not affect devstack/tempest as they use >>> dogpile.cache.memcached instead. >>> >>> The error is TypeError: __init__() got an unexpected keyword argument >>> 'dead_retry' >>> >>> For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 >>> >>> -yoctozepto >>> >>> >> >> -- >> Hervé Beraud >> Senior Software Engineer >> Red Hat - Openstack Oslo >> irc: hberaud >> -----BEGIN PGP SIGNATURE----- >> >> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >> v6rDpkeNksZ9fFSyoY2o >> =ECSj >> -----END PGP SIGNATURE----- >> >> > > -- > Hervé Beraud > Senior Software Engineer > Red Hat - Openstack Oslo > irc: hberaud > -----BEGIN PGP SIGNATURE----- > > wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ > Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ > RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP > F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G > 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g > glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw > m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ > hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 > qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y > F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 > B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O > v6rDpkeNksZ9fFSyoY2o > =ECSj > -----END PGP SIGNATURE----- > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From moguimar at redhat.com Wed Mar 4 12:41:44 2020 From: moguimar at redhat.com (Moises Guimaraes de Medeiros) Date: Wed, 4 Mar 2020 13:41:44 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: `dead_timeout`++ On Wed, Mar 4, 2020 at 1:36 PM Herve Beraud wrote: > `dead_timeout` [1] looks more appropriate in this case. > > [1] > https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L58 > > Le mer. 4 mars 2020 à 13:28, Herve Beraud a écrit : > >> What do you think about adding a mapping between `retry_timeout` [1] and >> `dead_retry` [2]? >> >> [1] >> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L56 >> [2] >> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >> >> Le mer. 4 mars 2020 à 13:20, Herve Beraud a écrit : >> >>> I think our issue is due to the fact that python-memcached accept a >>> param named `dead_retry` [1] which is not defined in pymemcache. >>> >>> We just need to define it in our oslo.cache mapping. During testing we >>> faced the same kind of issue with connection timeout. >>> >>> [1] >>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>> [2] >>> https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 >>> >>> Le mer. 4 mars 2020 à 12:21, Radosław Piliszek < >>> radoslaw.piliszek at gmail.com> a écrit : >>> >>>> Please be informed that oslo.cache 2.1.0 breaks oslo_cache.memcache_pool >>>> >>>> Kolla-Ansible gate is already RED and a quick codesearch revealed >>>> other deployment methods might be in trouble soon as well. >>>> >>>> This does not affect devstack/tempest as they use >>>> dogpile.cache.memcached instead. >>>> >>>> The error is TypeError: __init__() got an unexpected keyword argument >>>> 'dead_retry' >>>> >>>> For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 >>>> >>>> -yoctozepto >>>> >>>> >>> >>> -- >>> Hervé Beraud >>> Senior Software Engineer >>> Red Hat - Openstack Oslo >>> irc: hberaud >>> -----BEGIN PGP SIGNATURE----- >>> >>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>> v6rDpkeNksZ9fFSyoY2o >>> =ECSj >>> -----END PGP SIGNATURE----- >>> >>> >> >> -- >> Hervé Beraud >> Senior Software Engineer >> Red Hat - Openstack Oslo >> irc: hberaud >> -----BEGIN PGP SIGNATURE----- >> >> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >> v6rDpkeNksZ9fFSyoY2o >> =ECSj >> -----END PGP SIGNATURE----- >> >> > > -- > Hervé Beraud > Senior Software Engineer > Red Hat - Openstack Oslo > irc: hberaud > -----BEGIN PGP SIGNATURE----- > > wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ > Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ > RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP > F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G > 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g > glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw > m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ > hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 > qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y > F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 > B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O > v6rDpkeNksZ9fFSyoY2o > =ECSj > -----END PGP SIGNATURE----- > > -- Moisés Guimarães Software Engineer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From hberaud at redhat.com Wed Mar 4 13:16:51 2020 From: hberaud at redhat.com (Herve Beraud) Date: Wed, 4 Mar 2020 14:16:51 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: Fix proposed https://review.opendev.org/#/c/711220/ Le mer. 4 mars 2020 à 13:42, Moises Guimaraes de Medeiros < moguimar at redhat.com> a écrit : > `dead_timeout`++ > > On Wed, Mar 4, 2020 at 1:36 PM Herve Beraud wrote: > >> `dead_timeout` [1] looks more appropriate in this case. >> >> [1] >> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L58 >> >> Le mer. 4 mars 2020 à 13:28, Herve Beraud a écrit : >> >>> What do you think about adding a mapping between `retry_timeout` [1] and >>> `dead_retry` [2]? >>> >>> [1] >>> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L56 >>> [2] >>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>> >>> Le mer. 4 mars 2020 à 13:20, Herve Beraud a écrit : >>> >>>> I think our issue is due to the fact that python-memcached accept a >>>> param named `dead_retry` [1] which is not defined in pymemcache. >>>> >>>> We just need to define it in our oslo.cache mapping. During testing we >>>> faced the same kind of issue with connection timeout. >>>> >>>> [1] >>>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>>> [2] >>>> https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 >>>> >>>> Le mer. 4 mars 2020 à 12:21, Radosław Piliszek < >>>> radoslaw.piliszek at gmail.com> a écrit : >>>> >>>>> Please be informed that oslo.cache 2.1.0 breaks >>>>> oslo_cache.memcache_pool >>>>> >>>>> Kolla-Ansible gate is already RED and a quick codesearch revealed >>>>> other deployment methods might be in trouble soon as well. >>>>> >>>>> This does not affect devstack/tempest as they use >>>>> dogpile.cache.memcached instead. >>>>> >>>>> The error is TypeError: __init__() got an unexpected keyword argument >>>>> 'dead_retry' >>>>> >>>>> For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 >>>>> >>>>> -yoctozepto >>>>> >>>>> >>>> >>>> -- >>>> Hervé Beraud >>>> Senior Software Engineer >>>> Red Hat - Openstack Oslo >>>> irc: hberaud >>>> -----BEGIN PGP SIGNATURE----- >>>> >>>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>>> v6rDpkeNksZ9fFSyoY2o >>>> =ECSj >>>> -----END PGP SIGNATURE----- >>>> >>>> >>> >>> -- >>> Hervé Beraud >>> Senior Software Engineer >>> Red Hat - Openstack Oslo >>> irc: hberaud >>> -----BEGIN PGP SIGNATURE----- >>> >>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>> v6rDpkeNksZ9fFSyoY2o >>> =ECSj >>> -----END PGP SIGNATURE----- >>> >>> >> >> -- >> Hervé Beraud >> Senior Software Engineer >> Red Hat - Openstack Oslo >> irc: hberaud >> -----BEGIN PGP SIGNATURE----- >> >> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >> v6rDpkeNksZ9fFSyoY2o >> =ECSj >> -----END PGP SIGNATURE----- >> >> > > -- > > Moisés Guimarães > > Software Engineer > > Red Hat > > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdopiera at redhat.com Wed Mar 4 13:37:40 2020 From: rdopiera at redhat.com (Radomir Dopieralski) Date: Wed, 4 Mar 2020 14:37:40 +0100 Subject: [release][tc][horizon] xstatic repositories marked as deprecated In-Reply-To: References: Message-ID: I think it makes a lot of sense to make them independent, especially since the releases often contain security fixes, so that they should be included in any older supported releases of Horizon as well. On Tue, Mar 3, 2020 at 3:02 PM Sean McGinnis wrote: > On 3/3/20 4:11 AM, Akihiro Motoki wrote: > > Thanks Thierry for the detail explanation. > > The horizon team will update the corresponding repos for new minor > > releases and follow the usual release process. > > One question: we have passed the milestone-2. Is it better to wait > > till Victoria dev cycle is open? > > > > Thanks, > > Akihiro > > We are past the deadline for inclusion in ussuri. But that said, these > are things that are currently being used by the team, so I think it's a > little misleading in its current state. I think we should get these new > releases done in this cycle if possible. > > Part of this is also the assumption that these will be cycle based. I > wonder if this are more appropriate as independent deliverables? That > means they are not tied to a specific release cycle and can be released > whenever there is something to be released. At least something to think > about. > > > https://releases.openstack.org/reference/release_models.html#cycle-with-intermediary > > > On Fri, Feb 28, 2020 at 1:47 AM Thierry Carrez > wrote: > >> Thierry Carrez wrote: > >>> The way we've been handling this in the past was to ignore past > releases > >>> (since they are not signed by the release team), and push a new one > >>> through the releases repository. It should replace the unofficial one > in > >>> PyPI and make sure all is in order. > >> Clarification with a practical example: > >> > >> xstatic-hogan 2.0.0.2 is on PyPI, but has no tag in the > >> openstack/xstatic-hogan repo, and no deliverable file in > openstack/releases. > >> > >> Solution is to resync everything by proposing a 2.0.0.3 release that > >> will have tag, be in openstack/releases and have a matching upload on > PyPI. > >> > >> This is done by: > >> > >> - bumping BUILD at > >> > https://opendev.org/openstack/xstatic-hogan/src/branch/master/xstatic/pkg/hogan/__init__.py# > >> > >> - adding a deliverables/_independent/xstatic-hogan.yaml file in > >> openstack/releases defining a tag for 2.0.0.3 > >> > >> - removing the "deprecated" line from > >> > https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml#L542 > >> > >> Repeat for every affected package :) > >> > >> -- > >> Thierry Carrez (ttx) > >> > > -- Radomir Dopieralski -------------- next part -------------- An HTML attachment was scrubbed... URL: From geguileo at redhat.com Wed Mar 4 15:58:50 2020 From: geguileo at redhat.com (Gorka Eguileor) Date: Wed, 4 Mar 2020 16:58:50 +0100 Subject: [CINDER] Snapshots export In-Reply-To: References: Message-ID: <20200304155850.b4ydu4vfxthih7we@localhost> On 03/03, Alfredo De Luca wrote: > Hi all. > We have our env with Openstack (Train) and cinder with CEPH (nautilus) > backend. > We are creating automatic volumes snapshots and now we'd like to export > them as a backup/restore plan. After exporting the snapshots we will use > Acronis as backup tool. > > I couldn't find the right steps/commands to exports the snapshots. > Any info? > Cheers > > -- > *Alfredo* Hi Alfredo, What kind of backup/restore plan do you have planned? Because snapshots are not meant to be used in a Disaster Recovery backup/restore plan, so the only thing available are the manage/unmanage commands. These commands are meant to add an existing volume/snapshots into Cinder together, not to unmanage/manage them independently. For example, you wouldn't be able to manage a snapshot if the volume is not already managed. Also unmanaging the snapshot would block the deletion of the RBD volume itself. Cheers, Gorka. From mordred at inaugust.com Wed Mar 4 16:19:29 2020 From: mordred at inaugust.com (Monty Taylor) Date: Wed, 4 Mar 2020 10:19:29 -0600 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams Message-ID: Hey everybody, I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re Already ensuring that SDK does what it needs to do to facilitate that goal. We also already share PTG space, and have requested a shared set of time at the upcoming Denver PTG. So really the separation is historical not practical, and these days having additional layers of governance is not super useful. I propose that we do a simple merge of the teams. This means the current SDK cores will become cores on OSC, and as most of the OSC cores are already SDK cores, it means the SDK team gains amotoki - which is always a positive. Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. Monty From artem.goncharov at gmail.com Wed Mar 4 16:28:14 2020 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Wed, 4 Mar 2020 17:28:14 +0100 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: References: Message-ID: <9709C274-FCF2-4FAB-8D3B-86EB5FDBFAA9@gmail.com> I would definitely vote for that, since it will help to address one of our biggest problems - changes in OSC take very long to get in due to lack of resources (not to blame anybody). Artem > On 4. Mar 2020, at 17:19, Monty Taylor wrote: > > Hey everybody, > > I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re > Already ensuring that SDK does what it needs to do to facilitate that goal. We also already share PTG space, and have requested a shared set of time at the upcoming Denver PTG. So really the separation is historical not practical, and these days having additional layers of governance is not super useful. > > I propose that we do a simple merge of the teams. This means the current SDK cores will become cores on OSC, and as most of the OSC cores are already SDK cores, it means the SDK team gains amotoki - which is always a positive. > > Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. > > Monty From amotoki at gmail.com Wed Mar 4 16:36:16 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Thu, 5 Mar 2020 01:36:16 +0900 Subject: [release][tc][horizon] xstatic repositories marked as deprecated In-Reply-To: References: Message-ID: I totally forgot xstatic deliverables adopts the "independent" release mode when I sent the mail. I know the policy is only applied to cycle-based deliverables, but I totally forgot it...... xstatic deliverables I am talking about fit into "independent deliverables". On Wed, Mar 4, 2020 at 10:42 PM Radomir Dopieralski wrote: > > I think it makes a lot of sense to make them independent, especially since the releases often contain security fixes, so that they should be included in any older supported releases of Horizon as well. > > On Tue, Mar 3, 2020 at 3:02 PM Sean McGinnis wrote: >> >> On 3/3/20 4:11 AM, Akihiro Motoki wrote: >> > Thanks Thierry for the detail explanation. >> > The horizon team will update the corresponding repos for new minor >> > releases and follow the usual release process. >> > One question: we have passed the milestone-2. Is it better to wait >> > till Victoria dev cycle is open? >> > >> > Thanks, >> > Akihiro >> >> We are past the deadline for inclusion in ussuri. But that said, these >> are things that are currently being used by the team, so I think it's a >> little misleading in its current state. I think we should get these new >> releases done in this cycle if possible. >> >> Part of this is also the assumption that these will be cycle based. I >> wonder if this are more appropriate as independent deliverables? That >> means they are not tied to a specific release cycle and can be released >> whenever there is something to be released. At least something to think >> about. >> >> https://releases.openstack.org/reference/release_models.html#cycle-with-intermediary >> >> > On Fri, Feb 28, 2020 at 1:47 AM Thierry Carrez wrote: >> >> Thierry Carrez wrote: >> >>> The way we've been handling this in the past was to ignore past releases >> >>> (since they are not signed by the release team), and push a new one >> >>> through the releases repository. It should replace the unofficial one in >> >>> PyPI and make sure all is in order. >> >> Clarification with a practical example: >> >> >> >> xstatic-hogan 2.0.0.2 is on PyPI, but has no tag in the >> >> openstack/xstatic-hogan repo, and no deliverable file in openstack/releases. >> >> >> >> Solution is to resync everything by proposing a 2.0.0.3 release that >> >> will have tag, be in openstack/releases and have a matching upload on PyPI. >> >> >> >> This is done by: >> >> >> >> - bumping BUILD at >> >> https://opendev.org/openstack/xstatic-hogan/src/branch/master/xstatic/pkg/hogan/__init__.py# >> >> >> >> - adding a deliverables/_independent/xstatic-hogan.yaml file in >> >> openstack/releases defining a tag for 2.0.0.3 >> >> >> >> - removing the "deprecated" line from >> >> https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml#L542 >> >> >> >> Repeat for every affected package :) >> >> >> >> -- >> >> Thierry Carrez (ttx) >> >> >> > > > -- > Radomir Dopieralski From amy at demarco.com Wed Mar 4 16:38:43 2020 From: amy at demarco.com (Amy Marrich) Date: Wed, 4 Mar 2020 10:38:43 -0600 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: References: Message-ID: Monty, That sounds like a good plan thanks for proposing it. Amy (spotz) On Wed, Mar 4, 2020 at 10:22 AM Monty Taylor wrote: > Hey everybody, > > I’d like to propose merging the SDK and OSC teams. We already share an IRC > channel, and already share a purpose in life. In OSC we have a current goal > of swapping out client implementation for SDK, and we’re > Already ensuring that SDK does what it needs to do to facilitate that > goal. We also already share PTG space, and have requested a shared set of > time at the upcoming Denver PTG. So really the separation is historical not > practical, and these days having additional layers of governance is not > super useful. > > I propose that we do a simple merge of the teams. This means the current > SDK cores will become cores on OSC, and as most of the OSC cores are > already SDK cores, it means the SDK team gains amotoki - which is always a > positive. > > Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we > remain hopeful that this will change, we’re slowly coming to terms with the > possibility that it might not. With that in mind, I’ll serve as the PTL for > the new combined team until the next election. > > Monty > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr at ham.ie Wed Mar 4 16:49:36 2020 From: gr at ham.ie (Graham Hayes) Date: Wed, 4 Mar 2020 16:49:36 +0000 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: References: Message-ID: <9be364bd-49dd-ad97-3a21-ee0cc87a9298@ham.ie> On 04/03/2020 16:19, Monty Taylor wrote: > Hey everybody, > > I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re > Already ensuring that SDK does what it needs to do to facilitate that goal. We also already share PTG space, and have requested a shared set of time at the upcoming Denver PTG. So really the separation is historical not practical, and these days having additional layers of governance is not super useful. This makes sense. > > I propose that we do a simple merge of the teams. This means the current SDK cores will become cores on OSC, and as most of the OSC cores are already SDK cores, it means the SDK team gains amotoki - which is always a positive. Yeah - projects were supposed to be mainly about common groups of people working on stuff, so if the overlap is so close already, it seems like a no brainer. > > Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. If this is good with the two teams, this is good with me :) Hopefully this can help with projects teams issues with OSC/SDK response times. > > Monty > From mordred at inaugust.com Wed Mar 4 16:56:13 2020 From: mordred at inaugust.com (Monty Taylor) Date: Wed, 4 Mar 2020 10:56:13 -0600 Subject: [sdk] Additions and subtractions from core team Message-ID: Heya, With the previous email about merging OSC and SDK teams, I’d also like to propose the following changes to the SDK core team (keeping in mind that likely means the core team of both OSC and SDK real soon now) Adds: Akihiro Motoki - The only OSC core not in sdk-core. amotoki should really be a core in all projects anyway Sean McGinnis - Sean has been reviewing things as a stable branch maint in both SDK and OSC, and as such has shown a good tendency to help things along when needed and to not approve things when he doesn’t know what’s up. Subtractions: All of these people are awesome, but they’re all long gone: Brian Curtin Clint Byrum Everett Toews Jamie Lennox Jesse Noller Ricardo Carillo Cruz Richard Theis Rosario Di Somma Sam Yaple Terry Howe Monty From artem.goncharov at gmail.com Wed Mar 4 17:02:01 2020 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Wed, 4 Mar 2020 18:02:01 +0100 Subject: [sdk] Additions and subtractions from core team In-Reply-To: References: Message-ID: +1 from me ---- typed from mobile, auto-correct typos assumed ---- On Wed, 4 Mar 2020, 18:00 Monty Taylor, wrote: > Heya, > > With the previous email about merging OSC and SDK teams, I’d also like to > propose the following changes to the SDK core team (keeping in mind that > likely means the core team of both OSC and SDK real soon now) > > Adds: > > Akihiro Motoki - The only OSC core not in sdk-core. amotoki should really > be a core in all projects anyway > Sean McGinnis - Sean has been reviewing things as a stable branch maint in > both SDK and OSC, and as such has shown a good tendency to help things > along when needed and to not approve things when he doesn’t know what’s up. > > Subtractions: > > All of these people are awesome, but they’re all long gone: > > Brian Curtin > Clint Byrum > Everett Toews > Jamie Lennox > Jesse Noller > Ricardo Carillo Cruz > Richard Theis > Rosario Di Somma > Sam Yaple > Terry Howe > > Monty > -------------- next part -------------- An HTML attachment was scrubbed... URL: From openstack at fried.cc Wed Mar 4 17:07:48 2020 From: openstack at fried.cc (Eric Fried) Date: Wed, 4 Mar 2020 11:07:48 -0600 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: References: Message-ID: +1 On 3/4/20 10:19 AM, Monty Taylor wrote: > Hey everybody, > > I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re > Already ensuring that SDK does what it needs to do to facilitate that goal. We also already share PTG space, and have requested a shared set of time at the upcoming Denver PTG. So really the separation is historical not practical, and these days having additional layers of governance is not super useful. > > I propose that we do a simple merge of the teams. This means the current SDK cores will become cores on OSC, and as most of the OSC cores are already SDK cores, it means the SDK team gains amotoki - which is always a positive. > > Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. > > Monty > From openstack at fried.cc Wed Mar 4 17:07:59 2020 From: openstack at fried.cc (Eric Fried) Date: Wed, 4 Mar 2020 11:07:59 -0600 Subject: [sdk] Additions and subtractions from core team In-Reply-To: References: Message-ID: <304f4dc4-a949-0e43-9abe-7e9dd03ca217@fried.cc> +1 On 3/4/20 10:56 AM, Monty Taylor wrote: > Heya, > > With the previous email about merging OSC and SDK teams, I’d also like to propose the following changes to the SDK core team (keeping in mind that likely means the core team of both OSC and SDK real soon now) > > Adds: > > Akihiro Motoki - The only OSC core not in sdk-core. amotoki should really be a core in all projects anyway > Sean McGinnis - Sean has been reviewing things as a stable branch maint in both SDK and OSC, and as such has shown a good tendency to help things along when needed and to not approve things when he doesn’t know what’s up. > > Subtractions: > > All of these people are awesome, but they’re all long gone: > > Brian Curtin > Clint Byrum > Everett Toews > Jamie Lennox > Jesse Noller > Ricardo Carillo Cruz > Richard Theis > Rosario Di Somma > Sam Yaple > Terry Howe > > Monty > From dtantsur at redhat.com Wed Mar 4 17:11:53 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Wed, 4 Mar 2020 18:11:53 +0100 Subject: [sdk] Additions and subtractions from core team In-Reply-To: References: Message-ID: Hi, On Wed, Mar 4, 2020 at 5:58 PM Monty Taylor wrote: > Heya, > > With the previous email about merging OSC and SDK teams, I’d also like to > propose the following changes to the SDK core team (keeping in mind that > likely means the core team of both OSC and SDK real soon now) > > Adds: > > Akihiro Motoki - The only OSC core not in sdk-core. amotoki should really > be a core in all projects anyway > Sean McGinnis - Sean has been reviewing things as a stable branch maint in > both SDK and OSC, and as such has shown a good tendency to help things > along when needed and to not approve things when he doesn’t know what’s up. > A confident +2. > > Subtractions: > > All of these people are awesome, but they’re all long gone: > > Brian Curtin > Clint Byrum > Everett Toews > Jamie Lennox > Jesse Noller > Ricardo Carillo Cruz > Richard Theis > Rosario Di Somma > Sam Yaple > Terry Howe > Mmmm, can I refuse? No? Okay, okay... Dmitry > > Monty > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Wed Mar 4 17:22:26 2020 From: satish.txt at gmail.com (Satish Patel) Date: Wed, 4 Mar 2020 12:22:26 -0500 Subject: CPU Topology confusion Message-ID: Folks, We are running openstack with KVM and i have noticed kvm presenting wrong CPU Tolopoly to VM and because of that we are seeing bad performance to our application. This is openstack compute: # lstopo-no-graphics --no-io Machine (64GB total) NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 PU L#0 (P#0) PU L#1 (P#20) L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 PU L#2 (P#1) PU L#3 (P#21) This is VM running on above compute # lstopo-no-graphics --no-io Machine (59GB total) NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) L2 L#0 (4096KB) + Core L#0 L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) L2 L#1 (4096KB) + Core L#1 L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) if you noticed P#0 and P#1 has own (32KB) cache per thread that is wrong presentation if you compare with physical CPU. This is a screenshot of AWS vs Openstack CPU Topology and looking at openstack its presentation is little odd, is that normal? https://imgur.com/a/2sPwJVC I am running CentOS7.6 with kvm 2.12 version. From gmann at ghanshyammann.com Wed Mar 4 17:22:37 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 04 Mar 2020 11:22:37 -0600 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> Message-ID: <170a6921b22.118a63397430600.1168793341050830689@ghanshyammann.com> ---- On Wed, 04 Mar 2020 03:57:52 -0600 Mark Goddard wrote ---- > On Wed, 4 Mar 2020 at 01:16, Ghanshyam Mann wrote: > > > > ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell wrote ---- > > > > > > > > > On 3 Mar 2020, at 19:55, Tim Bell wrote: > > > > > > > > > On 3 Mar 2020, at 19:20, Albert Braden wrote: > > > Sean, thank you for clarifying that. > > > > > > Was my understanding that the community decided to focus on the unified client incorrect? Is the unified/individual client debate still a matter of controversy? Is it possible that the unified client will be deprecated in favor of individual clients after more discussion? I haven’t looked at any of the individual clients since 2018 (except for osc-placement which is kind of a special case), because I thought they were all going away and could be safely ignored until they did, and I haven’t included any information about the individual clients in the documentation that I write for our users, and if they ask I have been telling them to not use the individual clients. Do I need to start looking at individual clients again, and telling our users to use them in some cases? > > > > > > > > > > > > I remember a forum discussion where a community goal was proposed to focus on OSC rather than individual project CLIs (I think Matt and I were proposers). There were concerns on the effort to do this and that it would potentially be multi-cycle. > > > BTW, I found the etherpad from Berlin (https://etherpad.openstack.org/p/BER-t-series-goals) and the associated mailing list discussion at http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html > > > > Yeah, we are in process of selecting the Victoria cycle community-wide goal and this can be good candidate. I agree with the idea/requirement of a multi-cycle goal. > > Another option is to build a pop-up team for the Victoria cycle to start burning down the keys issues/work. For both ways (either goal or pop-up team), we need > > some set of people to drive it. If anyone would like to volunteer for this, we can start discussing the details. > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html > > > > -gmann > > This seems like quite an important issue for OpenStack usability. > Clearly there are resourcing issues within the glance team (and > possibly also some personal preferences) that have prevented OSC > gaining feature parity with the glance client. Having all of the core > projects able to recommend using OSC seems to me like it should be > quite a high priority - more so than having support for every project > out there. Would cross-project goal effort be better spent swarming on > filling these gaps first? Do we have any mechanisms to help drive > that? I know we have the help most-wanted list. That is a good idea to first target big projects. We can finish this for nova, glance, cinder, keystone, glance, swift at first. Apart from Upstream Opportunity (help-most-wanted list) [2], one better way is pop-up team [1]. For that, we need a set of people from these projects or any developer to start working. Also, we can add this as the upstream opportunity for 2020 and see if we get any help. [1] https://governance.openstack.org/tc/reference/popup-teams.html [2] https://governance.openstack.org/tc/reference/upstream-investment-opportunities/2020/index.html -gmann > > > > > > > > > My experience in discussion with the CERN user community and other OpenStack operators is that OSC is felt to be the right solution for the end user facing parts of the cloud (admin commands could be another discussion if necessary). Experienced admin operators can remember that glance looks after images and nova looks after instances. Our average user can get very confused, especially given that OSC supports additional options for authentication (such as Kerberos and Certificates along with clouds.yaml) so users need to re-authenticate with a different openrc to work on their project. > > > While I understand there are limited resources all round, I would prefer that we focus on adding new project functions to OSC which will eventually lead to feature parity. > > > Attracting ‘drive-by’ contributions from operations staff for OSC work (it's more likely to be achieved if it makes the operations work less e.g. save on special end user documentation by contributing code). This is demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' functionality along with lots of random OSC updates as listed hat https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) > > > BTW, I also would vote for =auto as the default. > > > Tim > > > We are on Rocky now but I expect that we will upgrade as necessary to stay on supported versions. > > > > > > From: Sean McGinnis > > > Sent: Tuesday, March 3, 2020 9:50 AM > > > To: openstack-discuss at lists.openstack.org > > > Subject: Re: OSC future (formerly [glance] Different checksum between CLI and curl) > > > > > > On 3/3/20 11:28 AM, Albert Braden wrote: > > > Am I understanding correctly that the Openstack community decided to focus on the unified client, and to deprecate the individual clients, and that the Glance team did not agree with this decision, and that the Glance team is now having a pissing match with the rest of the community, and is unilaterally deciding to continue developing the Glance client and refusing to work on the unified client, or is something different going on? I would ask everyone involved to remember that we operators are down here, and the yellow rain falling on our heads does not smell very good. > > > I definitely would not characterize it that way. > > > With trying not to put too much personal bias into it, here's what I would say the situation is: > > > - Some part of the community has said OSC should be the only CLI and that individual CLIs should go away > > > - Glance is a very small team with very, very limited resources > > > - The OSC team is a very small team with very, very limited resources > > > - CLI capabilities need to be exposed for Glance changes and the easiest way to get them out for the is by updating the Glance CLI > > > - No one from the OSC team has been able to proactively help to make sure these changes make it into the OSC client (see bullet 3) > > > - There exists a sizable functionality gap between per-project CLIs and what OSC provides, and although a few people have done a lot of great work to close that gap, there is still a lot to be done and does not appear the gap will close at any point in the near future based on the current trends > > > > > > > > > > > > > > > > > > > From mordred at inaugust.com Wed Mar 4 17:38:54 2020 From: mordred at inaugust.com (Monty Taylor) Date: Wed, 4 Mar 2020 11:38:54 -0600 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: <9be364bd-49dd-ad97-3a21-ee0cc87a9298@ham.ie> References: <9be364bd-49dd-ad97-3a21-ee0cc87a9298@ham.ie> Message-ID: <150614DB-C9BD-413C-9790-C419635A2AFC@inaugust.com> > On Mar 4, 2020, at 10:49 AM, Graham Hayes wrote: > > On 04/03/2020 16:19, Monty Taylor wrote: >> Hey everybody, >> I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re >> Already ensuring that SDK does what it needs to do to facilitate that goal. We also already share PTG space, and have requested a shared set of time at the upcoming Denver PTG. So really the separation is historical not practical, and these days having additional layers of governance is not super useful. > > This makes sense. > >> I propose that we do a simple merge of the teams. This means the current SDK cores will become cores on OSC, and as most of the OSC cores are already SDK cores, it means the SDK team gains amotoki - which is always a positive. > > Yeah - projects were supposed to be mainly about common groups of people > working on stuff, so if the overlap is so close already, it seems like > a no brainer. > >> Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. > > If this is good with the two teams, this is good with me :) > Hopefully this can help with projects teams issues with OSC/SDK response > times. >> Monty I think it can. I’ve had some chats with some folks on the team and I think we all think this will help streamline and enable us to respond more quickly. From m2elsakha at gmail.com Wed Mar 4 18:06:43 2020 From: m2elsakha at gmail.com (Mohamed Elsakhawy) Date: Wed, 4 Mar 2020 13:06:43 -0500 Subject: Upcoming UC meeting: March 5th 2020 Message-ID: Good day everyone, As you may have heard, there is an ongoing discussion for having a single body to encompass both the UC and TC responsibilities, potentially by merging the two bodies. This is still at early stages, and thus to facilitate this discussion we will hold tomorrow's UC meeting using video conferencing to allow all audience to easily share their thoughts and input. The meeting will happen at its normal time "13:30 EST", If you are interested in joining, please use the link below https://hangouts.google.com/u/1/call/q_bcZpCRXgB2_Cak4ycyAEEI Thanks Mohamed --melsakhawy -------------- next part -------------- An HTML attachment was scrubbed... URL: From allison at openstack.org Wed Mar 4 18:22:30 2020 From: allison at openstack.org (Allison Price) Date: Wed, 4 Mar 2020 12:22:30 -0600 Subject: Upcoming UC meeting: March 5th 2020 In-Reply-To: References: Message-ID: <341F5F74-BAB4-463E-A69B-4E3F356AD1F9@openstack.org> Hi Mohamed, Did you mean CET? I believe the recurring meeting time is at 8:30 Eastern Standard Time (EST)? Thanks, Allison > On Mar 4, 2020, at 12:06 PM, Mohamed Elsakhawy wrote: > > Good day everyone, > > As you may have heard, there is an ongoing discussion for having a single body to encompass both the UC and TC responsibilities, potentially by merging the two bodies. This is still at early stages, and thus to facilitate this discussion we will hold tomorrow's UC meeting using video conferencing to allow all audience to easily share their thoughts and input. > > The meeting will happen at its normal time "13:30 EST", If you are interested in joining, please use the link below > > https://hangouts.google.com/u/1/call/q_bcZpCRXgB2_Cak4ycyAEEI > > Thanks > > Mohamed > --melsakhawy > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hberaud at redhat.com Wed Mar 4 18:23:37 2020 From: hberaud at redhat.com (Herve Beraud) Date: Wed, 4 Mar 2020 19:23:37 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: I proposed the following two patches to address the issue and improve this module beyond the current issue: - https://review.opendev.org/711220 (the fix) - https://review.opendev.org/711247 (the improvements) After these patches will be merged and the issue fixed we will blacklist the version 2.1.0 of oslo.cache and propose a new release with the previous fixes embedded. Do not hesitate to review them and leave comments. Thanks for your reading. Le mer. 4 mars 2020 à 14:16, Herve Beraud a écrit : > Fix proposed https://review.opendev.org/#/c/711220/ > > Le mer. 4 mars 2020 à 13:42, Moises Guimaraes de Medeiros < > moguimar at redhat.com> a écrit : > >> `dead_timeout`++ >> >> On Wed, Mar 4, 2020 at 1:36 PM Herve Beraud wrote: >> >>> `dead_timeout` [1] looks more appropriate in this case. >>> >>> [1] >>> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L58 >>> >>> Le mer. 4 mars 2020 à 13:28, Herve Beraud a écrit : >>> >>>> What do you think about adding a mapping between `retry_timeout` [1] >>>> and `dead_retry` [2]? >>>> >>>> [1] >>>> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L56 >>>> [2] >>>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>>> >>>> Le mer. 4 mars 2020 à 13:20, Herve Beraud a >>>> écrit : >>>> >>>>> I think our issue is due to the fact that python-memcached accept a >>>>> param named `dead_retry` [1] which is not defined in pymemcache. >>>>> >>>>> We just need to define it in our oslo.cache mapping. During testing we >>>>> faced the same kind of issue with connection timeout. >>>>> >>>>> [1] >>>>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>>>> [2] >>>>> https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 >>>>> >>>>> Le mer. 4 mars 2020 à 12:21, Radosław Piliszek < >>>>> radoslaw.piliszek at gmail.com> a écrit : >>>>> >>>>>> Please be informed that oslo.cache 2.1.0 breaks >>>>>> oslo_cache.memcache_pool >>>>>> >>>>>> Kolla-Ansible gate is already RED and a quick codesearch revealed >>>>>> other deployment methods might be in trouble soon as well. >>>>>> >>>>>> This does not affect devstack/tempest as they use >>>>>> dogpile.cache.memcached instead. >>>>>> >>>>>> The error is TypeError: __init__() got an unexpected keyword argument >>>>>> 'dead_retry' >>>>>> >>>>>> For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 >>>>>> >>>>>> -yoctozepto >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Hervé Beraud >>>>> Senior Software Engineer >>>>> Red Hat - Openstack Oslo >>>>> irc: hberaud >>>>> -----BEGIN PGP SIGNATURE----- >>>>> >>>>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>>>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>>>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>>>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>>>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>>>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>>>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>>>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>>>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>>>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>>>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>>>> v6rDpkeNksZ9fFSyoY2o >>>>> =ECSj >>>>> -----END PGP SIGNATURE----- >>>>> >>>>> >>>> >>>> -- >>>> Hervé Beraud >>>> Senior Software Engineer >>>> Red Hat - Openstack Oslo >>>> irc: hberaud >>>> -----BEGIN PGP SIGNATURE----- >>>> >>>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>>> v6rDpkeNksZ9fFSyoY2o >>>> =ECSj >>>> -----END PGP SIGNATURE----- >>>> >>>> >>> >>> -- >>> Hervé Beraud >>> Senior Software Engineer >>> Red Hat - Openstack Oslo >>> irc: hberaud >>> -----BEGIN PGP SIGNATURE----- >>> >>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>> v6rDpkeNksZ9fFSyoY2o >>> =ECSj >>> -----END PGP SIGNATURE----- >>> >>> >> >> -- >> >> Moisés Guimarães >> >> Software Engineer >> >> Red Hat >> >> >> > > > -- > Hervé Beraud > Senior Software Engineer > Red Hat - Openstack Oslo > irc: hberaud > -----BEGIN PGP SIGNATURE----- > > wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ > Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ > RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP > F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G > 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g > glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw > m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ > hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 > qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y > F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 > B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O > v6rDpkeNksZ9fFSyoY2o > =ECSj > -----END PGP SIGNATURE----- > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimmy at openstack.org Wed Mar 4 18:28:43 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Wed, 04 Mar 2020 12:28:43 -0600 Subject: Upcoming UC meeting: March 5th 2020 In-Reply-To: References: Message-ID: <5E5FF35B.2040806@openstack.org> Will the hangout be recorded and available for the rest of the community to view if they aren't able to make the meeting? Thank you, Jimmy > Mohamed Elsakhawy > March 4, 2020 at 12:06 PM > Good day everyone, > > As you may have heard, there is an ongoing discussion for having a > single body to encompass both the UC and TC responsibilities, > potentially by merging the two bodies. This is still at early stages, > and thus to facilitate this discussion we will hold tomorrow's UC > meeting using video conferencing to allow all audience to easily share > their thoughts and input. > > The meeting will happen at its normal time "13:30 EST", If you are > interested in joining, please use the link below > > https://hangouts.google.com/u/1/call/q_bcZpCRXgB2_Cak4ycyAEEI > > Thanks > > Mohamed > --melsakhawy > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m2elsakha at gmail.com Wed Mar 4 18:25:52 2020 From: m2elsakha at gmail.com (Mohamed Elsakhawy) Date: Wed, 4 Mar 2020 13:25:52 -0500 Subject: Upcoming UC meeting: March 5th 2020 In-Reply-To: <341F5F74-BAB4-463E-A69B-4E3F356AD1F9@openstack.org> References: <341F5F74-BAB4-463E-A69B-4E3F356AD1F9@openstack.org> Message-ID: Apologies, 13:30 is actually *UTC* . The meeting will happen *13:30 UTC - 8:30 EST* On Wed, Mar 4, 2020 at 1:22 PM Allison Price wrote: > Hi Mohamed, > > Did you mean CET? I believe the recurring meeting time is at 8:30 Eastern > Standard Time (EST)? > > Thanks, > Allison > > > On Mar 4, 2020, at 12:06 PM, Mohamed Elsakhawy > wrote: > > Good day everyone, > > As you may have heard, there is an ongoing discussion for having a single > body to encompass both the UC and TC responsibilities, potentially by > merging the two bodies. This is still at early stages, and thus to > facilitate this discussion we will hold tomorrow's UC meeting using video > conferencing to allow all audience to easily share their thoughts and input. > > The meeting will happen at its normal time "13:30 EST", If you are > interested in joining, please use the link below > > https://hangouts.google.com/u/1/call/q_bcZpCRXgB2_Cak4ycyAEEI > > Thanks > > Mohamed > --melsakhawy > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m2elsakha at gmail.com Wed Mar 4 18:38:14 2020 From: m2elsakha at gmail.com (Mohamed Elsakhawy) Date: Wed, 4 Mar 2020 13:38:14 -0500 Subject: Upcoming UC meeting: March 5th 2020 In-Reply-To: <5E5FF35B.2040806@openstack.org> References: <5E5FF35B.2040806@openstack.org> Message-ID: Yes, it will be recorded and available for later viewing by the community Thanks Mohamed On Wed, Mar 4, 2020 at 1:28 PM Jimmy McArthur wrote: > Will the hangout be recorded and available for the rest of the community > to view if they aren't able to make the meeting? > > Thank you, > Jimmy > > Mohamed Elsakhawy > March 4, 2020 at 12:06 PM > Good day everyone, > > As you may have heard, there is an ongoing discussion for having a single > body to encompass both the UC and TC responsibilities, potentially by > merging the two bodies. This is still at early stages, and thus to > facilitate this discussion we will hold tomorrow's UC meeting using video > conferencing to allow all audience to easily share their thoughts and input. > > The meeting will happen at its normal time "13:30 EST", If you are > interested in joining, please use the link below > > https://hangouts.google.com/u/1/call/q_bcZpCRXgB2_Cak4ycyAEEI > > Thanks > > Mohamed > --melsakhawy > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zbitter at redhat.com Wed Mar 4 18:57:38 2020 From: zbitter at redhat.com (Zane Bitter) Date: Wed, 4 Mar 2020 13:57:38 -0500 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: On 2/03/20 4:45 pm, Mohammed Naser wrote: > Hi everyone: > > We're now in a spot where we have an increasing amount of projects > that don't end up with a volunteer as PTL, even if the project has > contributors .. no one wants to hold that responsibility alone for > many reasons. With time, the PTL role has become far more overloaded > with many extra responsibilities than what we define in our charter: > > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. Just for fun I had a read through the thread from when I last proposed getting rid of PTLs, 5.5 years ago: http://lists.openstack.org/pipermail/openstack-dev/2014-August/043826.html I wrote that when I was a PTL. Now that I have been on all sides of it (Core team member, PTL, ex-PTL, TC member), let's see how well this has aged :D > First off, the PTL is not responsible for everything in a project. > *Everyone* is responsible for everything in a project. > > The PTL is *accountable* for everything in a project. PTLs are the > mechanism the TC uses to ensure that programs remain accountable to the > wider community. I still think this is true. But it's also true that if everyone is responsible then nobody is really responsible. Somebody has to be responsible for knowing all of the things that somebody needs to be responsible for and making sure that somebody is responsible for each. That can be done without a PTL as such, but the PTL system does provide a way of externally bootstrapping it in every project. > We have a heavyweight election process for PTLs once every > cycle because that used to be the process for electing the TC. Now that > it no longer serves this dual purpose, PTL elections have outlived their > usefulness. I had completely forgotten about this. From a TC perspective, we don't have a lot of visibility on internal ructions that may be going on in any particular project. The election does at least assure us that there is an outlet valve for any issues, and the fact that it is completely normalised across all of OpenStack makes it more likely that someone will actually challenge the PTL if there is a problem. > there's no need to impose that process on every project. If > they want to rotate the tech lead every week instead of every 6 months, > why not let them? We'll soon see from experimentation which models work. One cannot help wondering if we might get more Nova cores willing to sign up for a 1-week commitment to be the "PTL" than we're getting for a 6-months-and-maybe-indefinitely commitment. >> We also >> still need someone to have the final say in case of deadlocked issues. > > -1 we really don't. I still think I am mostly right about this (and I know Thierry still thinks he is right and I am wrong ;) IMHO it's never the job of the PTL to have a casting vote. It *is* the job of the PTL - and all leaders in the project - to ensure that consensus is eventually reached somehow; that discussion is not just allowed to continue forever without a resolution when people disagree. While all leaders should be doing this, I can see some benefit in having one person who sees it as specifically their responsibility, and as noted above the PTL election process ensures that this happens in every project. In summary, I still think that in a healthy project the requirement to have a PTL is probably mildly unhelpful. One thing that didn't come up in that thread but that I have mentioned elsewhere, was that when I became a PTL I very quickly learned to be very careful about what I expressed an opinion on and how, lest I accidentally close down a conversation that I was intending to open up. Because *overnight* people developed this sudden tendency to be like "HE HATH SPOKEN" whenever I weighed in. (This is very unnerving BTW, and one reason I feel like I can be more helpful by *not* running for PTL.) So having a PTL means giving up a core team member in some senses. Ultimately, from the TC perspective it's a tool for reducing the variance in outcomes compared to letting every team decide their own leadership structure. As with all interventions that act by reducing variance (rather than increasing the average), this will tend to be a burden on higher-performing teams while raising the floor for lower-performing ones. So that's the trade-off we have to make. cheers, Zane. From fungi at yuggoth.org Wed Mar 4 19:06:43 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 4 Mar 2020 19:06:43 +0000 Subject: Upcoming UC meeting: March 5th 2020 In-Reply-To: References: <5E5FF35B.2040806@openstack.org> Message-ID: <20200304190643.uqztyrgwkotkqyrg@yuggoth.org> On 2020-03-04 13:38:14 -0500 (-0500), Mohamed Elsakhawy wrote: > Yes, it will be recorded and available for later viewing by the > community [...] Since you're hosting the discussion on a meeting platform which is not reachable from China, at least make sure to publish the recording somewhere which our Chinese community members have some hope of accessing. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From james.denton at rackspace.com Wed Mar 4 19:26:54 2020 From: james.denton at rackspace.com (James Denton) Date: Wed, 4 Mar 2020 19:26:54 +0000 Subject: [neutron] security group list regression In-Reply-To: <4740f4822e7b571b40aa5dc549e3c59a2ee659c4.camel@redhat.com> References: <7DD0691D-19A3-4CDB-B377-F67829A86AD7@rackspace.com> <4740f4822e7b571b40aa5dc549e3c59a2ee659c4.camel@redhat.com> Message-ID: <59ACC4AB-95A8-49F4-91BD-8F94EF76494C@rackspace.com> Hi Rodolfo, The client we're using for Train does indeed have the patch. The Stein environment, running python-openstackclient 3.18.1, did not. I was able to patch it and speed up the DELETE operation. Real world, the user could probably just update the client and get the fix. Thanks again! On 3/3/20, 4:49 AM, "Rodolfo Alonso" wrote: CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello James: Yes, this is a known issue in OSclient: most of the "objects" (networks, subnets, routers, etc) to be retrieved, can usually can be retrieved by ID and by name. OSclient tries first to use the ID because is unique and a DB key. Then, instead of asking the server for a unique register (filtered by the name), the client retrieves the whole list and filters the results. But this problem was resolved in Train: https://review.opendev.org/#/c/637238/. Can you check, in openstacksdk, that you have this patch? At least in T. According to [1] and [2], "name" should be used as filter in the OSsdk "find" call. Regards. [1]https://review.opendev.org/#/c/637238/20/openstack/resource.py [2]https://github.com/openstack/openstacksdk/blob/master/openstack/network/v2/security_group.py#L29 On Mon, 2020-03-02 at 22:25 +0000, James Denton wrote: > Rodolfo, > > Thanks for continuing to push this on the ML and in the bug report. > > Happy to report that the client and SDK patches you provided have drastically reduced the SG list > time from ~90-120s to ~12-14s within Stein and Train lab environments. > > One last thing... when you perform an 'openstack security group delete ', the initial lookup > by name fails. In Train, the client falls back to using the 'name' parameter (/security- > groups?name=). This lookup is quick and the security group is found and deleted. However, on > Rocky/Stein (e.g. client 3.18.1), instead of searching by parameter, the client appears to perform > a GET /security-groups without limiting the fields and takes a long time. > > 'openstack security group list' with patch: > REQ: curl -g -i -X GET " > http://10.0.236.150:9696/v2.0/security-groups?fields=set%28%5B%27description%27%2C+%27project_id%27%2C+%27id%27%2C+%27tags%27%2C+%27name%27%5D%29 > " -H "Accept: application/json" -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python- > requests/2.21.0 CPython/2.7.17" -H "X-Auth-Token: > {SHA256}3e747da939e8c4befe72d5ca7105971508bd56cdf36208ba6b960d1aee6d19b6" > > 'openstack security group delete ': > > Train (notice the name param): > REQ: curl -g -i -X GET http://10.20.0.11:9696/v2.0/security-groups/train-test-1755 -H "User-Agent: > openstacksdk/0.36.0 keystoneauth1/3.17.1 python-requests/2.22.0 CPython/3.6.7" -H "X-Auth-Token: > {SHA256}bf291d5f12903876fc69151db37d295da961ba684a575e77fb6f4829b55df1bf" > http://10.20.0.11:9696 "GET /v2.0/security-groups/train-test-1755 HTTP/1.1" 404 125 > REQ: curl -g -i -X GET "http://10.20.0.11:9696/v2.0/security-groups?name=train-test-1755" -H > "Accept: application/json" -H "User-Agent: openstacksdk/0.36.0 keystoneauth1/3.17.1 python- > requests/2.22.0 CPython/3.6.7" -H "X-Auth-Token: > {SHA256}bf291d5f12903876fc69151db37d295da961ba684a575e77fb6f4829b55df1bf" > http://10.20.0.11:9696 "GET /v2.0/security-groups?name=train-test-1755 HTTP/1.1" 200 1365 > > Stein & below (notice lack of fields): > REQ: curl -g -i -X GET http://10.0.236.150:9696/v2.0/security-groups/stein-test-5189 -H "User- > Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H "X-Auth- > Token: {SHA256}e9f87afe851ff5380d8402ee81199c466be9c84fe67ed0302e8b178f33aa1fc2" > http://10.0.236.150:9696 "GET /v2.0/security-groups/stein-test-5189 HTTP/1.1" 404 125 > REQ: curl -g -i -X GET http://10.0.236.150:9696/v2.0/security-groups -H "Accept: application/json" > -H "User-Agent: openstacksdk/0.27.0 keystoneauth1/3.13.1 python-requests/2.21.0 CPython/2.7.17" -H > "X-Auth-Token: {SHA256}e9f87afe851ff5380d8402ee81199c466be9c84fe67ed0302e8b178f33aa1fc2" > > > Haven't quite figured out where fields can be used to speed up the delete process on the older > client, or if the newer client would be backwards-compatible (and how far back). > > Thanks, > James > > On 3/2/20, 9:31 AM, "James Denton" wrote: > > CAUTION: This message originated externally, please use caution when clicking on links or > opening attachments! > > > Thanks, Rodolfo. I'll take a look at each of these after coffee and clarify my position (if > needed). > > James > > On 3/2/20, 6:27 AM, "Rodolfo Alonso" wrote: > > CAUTION: This message originated externally, please use caution when clicking on links or > opening attachments! > > > Hello James: > > Just to make a quick summary of the status of the commented bugs/regressions: > > 1) https://bugs.launchpad.net/neutron/+bug/1810563: adding rules to security groups is > slow > That was addressed in https://review.opendev.org/#/c/633145/ and > https://review.opendev.org/#/c/637407/, removing the O^2 check and using lazy loading. > > > 2) https://bugzilla.redhat.com/show_bug.cgi?id=1788749: Neutron List networks API > regression > The last reply was marked as private. I've undone this and you can read now c#2. Testing > with a > similar scenario, I don't see any performance degradation between Queens and Train. > > > 3) https://bugzilla.redhat.com/show_bug.cgi?id=1721273: Neutron API List Ports Performance > regression > That problem was solved in https://review.opendev.org/#/c/667981/ and > https://review.opendev.org/#/c/667998/, by refactoring how the port QoS extension was > reading and > applying the QoS info in the port dict. > > > 4) https://bugs.launchpad.net/neutron/+bug/1865223: regression for security group list > between > Newton and Rocky+ > > This is similar to https://bugs.launchpad.net/neutron/+bug/1863201. In this case, the > regression was > detected from R to S. The performance dropped from 3 secs to 110 secs (36x). That issue > was > addressed by https://review.opendev.org/#/c/708695/. > > But while 1865223 is talking about *SG list*, 1863201 is related to *SG rule list*. I > would like to > make this differentiation, because both retrieval commands are not related. > > In this bug (1863201), the performance degradation multiplies by x3 (N->Q) the initial > time. This > could be caused by the OVO integration (O->P: https://review.opendev.org/#/c/284738/). > Instead of > using the DB object now we make this call using the OVO object containing the DB register > (something > like a DB view). That's something I still need to check. > > Just to make a concretion: the patch 708695 improves the *SG rule* retrieval, not the SG > list > command. Another punctualization is that this patch will help in the case of having a > balance > between SG rules and SG. This patch will help to retrieve from the DB only those SG rules > belonging > to the project. If, as you state in > https://bugs.launchpad.net/neutron/+bug/1865223/comments/4, most > of those SG rules belong to the same project, there is little improvement there. > > As commented, I'm still looking at improving the SG OVO performance. > > Regards > > > On Mon, 2020-03-02 at 03:03 +0000, Erik Olof Gunnar Andersson wrote: > > When we went from Mitaka to Rocky in August last year and we saw an exponential increase > in api > > times for listing security group rules. > > > > I think I last commented on this bug https://bugs.launchpad.net/neutron/+bug/1810563, > but I have > > brought it up on a few other occasions as well. > > Bug #1810563 “adding rules to security groups is slow” : Bugs : neutron Sometime > between liberty > > and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review > with > > fixes is incoming. You can repro with a vanilla devstack install on master, and this > script: > > #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') > export > > OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost > make_rules() { > > iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file > <<EOF > > {... bugs.launchpad.net > > > > > > From: Slawek Kaplonski > > Sent: Saturday, February 29, 2020 12:44 AM > > To: James Denton > > Cc: openstack-discuss > > Subject: Re: [neutron] security group list regression > > > > Hi, > > > > I just replied in Your bug report. Can You try to apply patch > > > https://urldefense.com/v3/__https://review.opendev.org/*/c/708695/__;Iw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Ul-OlUqQ$ > > to see if that will help with this problem? > > > > > On 29 Feb 2020, at 02:41, James Denton wrote: > > > > > > Hello all, > > > > > > We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty > severe > > regression in the time it takes the API to return the list of security groups. This > environment > > has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack > security > > group list’ command to complete. I don’t have actual data from the same environment > running > > Newton, but was able to replicate this behavior with the following lab environments > running a mix > > of virtual and baremetal machines: > > > > > > Newton (VM) > > > Rocky (BM) > > > Stein (VM) > > > Train (BM) > > > > > > Number of sec grps vs time in seconds: > > > > > > # Newton Rocky Stein Train > > > 200 4.1 3.7 5.4 5.2 > > > 500 5.3 7 11 9.4 > > > 1000 7.2 12.4 19.2 16 > > > 2000 9.2 24.2 35.3 30.7 > > > 3000 12.1 36.5 52 44 > > > 4000 16.1 47.2 73 58.9 > > > 5000 18.4 55 90 69 > > > > > > As you can see (hopefully), the response time increased significantly between Newton > and Rocky, > > and has grown slightly ever since. We don't know, yet, if this behavior can be seen with > other > > 'list' commands or is limited to secgroups. We're currently verifying on some > intermediate > > releases to see where things went wonky. > > > > > > There are some similar recent reports out in the wild with little feedback: > > > > > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1788749__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8Vx5jGlrA$ > > > > > > > > https://urldefense.com/v3/__https://bugzilla.redhat.com/show_bug.cgi?id=1721273__;!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8U9NbN_LA$ > > > > > > > > I opened a bug here, too: > > > > > > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/1865223__;Kw!!Ci6f514n9QsL8ck!2GsBjp6V_V3EzrzAbWgNfsURfCm2tZmlUaw2J6OxFwJZUCV71lSP1b9jg8UtMQ2-Dw$ > > > > > > > > Bottom line: Has anyone else experienced similar regressions in recent releases? If > so, were you > > able to address them with any sort of tuning? > > > > > > Thanks in advance, > > > James > > > > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > > > > > > From m2elsakha at gmail.com Wed Mar 4 19:26:54 2020 From: m2elsakha at gmail.com (Mohamed Elsakhawy) Date: Wed, 4 Mar 2020 14:26:54 -0500 Subject: Upcoming UC meeting: March 5th 2020 In-Reply-To: <20200304190643.uqztyrgwkotkqyrg@yuggoth.org> References: <5E5FF35B.2040806@openstack.org> <20200304190643.uqztyrgwkotkqyrg@yuggoth.org> Message-ID: Of course, that's a good point. On Wed, Mar 4, 2020 at 2:07 PM Jeremy Stanley wrote: > On 2020-03-04 13:38:14 -0500 (-0500), Mohamed Elsakhawy wrote: > > Yes, it will be recorded and available for later viewing by the > > community > [...] > > Since you're hosting the discussion on a meeting platform which is > not reachable from China, at least make sure to publish the > recording somewhere which our Chinese community members have some > hope of accessing. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Wed Mar 4 19:53:00 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 4 Mar 2020 14:53:00 -0500 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins Message-ID: Hello QA team and devstack-plugin-ceph-core people, The Cinder team has some proposals we'd like to float. 1. The Cinder team is interested in becoming more active in the maintenance of openstack/devstack-plugin-ceph [0]. Currently, the devstack-plugin-ceph-core is https://review.opendev.org/#/admin/groups/1196,members The cinder-core is already represented by Eric and Sean; we'd like to replace them by including the cinder-core group. 2. The Cinder team is interested in becoming more active in the maintenance of x/devstack-plugin-nfs [1]. Currently, the devstack-plugin-nfs-core is https://review.opendev.org/#/admin/groups/1330,members It's already 75% cinder-core members; we'd like to replace the individual members with the cinder-core group. We also propose that devstack-core be added as an included group. 3. The Cinder team is interested in implementing a new devstack plugin: openstack/devstack-plugin-open-cas This will enable thorough testing of a new feature [2] being introduced as experimental in Ussuri and expected to be finalized in Victoria. Our plan would be to make both cinder-core and devstack-core included groups for the gerrit group governing the new plugin. 4. This is a minor point, but can the devstack-plugin-nfs repo be moved back into the 'openstack' namespace? Let us know which of these proposals you find acceptable. [0] https://opendev.org/openstack/devstack-plugin-ceph [1] https://opendev.org/x/devstack-plugin-nfs [2] https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache From jungleboyj at gmail.com Wed Mar 4 19:56:26 2020 From: jungleboyj at gmail.com (Jay Bryant) Date: Wed, 4 Mar 2020 13:56:26 -0600 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> On 3/4/2020 12:57 PM, Zane Bitter wrote: > On 2/03/20 4:45 pm, Mohammed Naser wrote: >> Hi everyone: >> >> We're now in a spot where we have an increasing amount of projects >> that don't end up with a volunteer as PTL, even if the project has >> contributors .. no one wants to hold that responsibility alone for >> many reasons.  With time, the PTL role has become far more overloaded >> with many extra responsibilities than what we define in our charter: >> >> https://governance.openstack.org/tc/reference/charter.html#project-team-leads >> >> >> I think it's time to re-evaluate the project leadership model that we >> have.  I am thinking that perhaps it would make a lot of sense to move >> from a single PTL model to multiple maintainers.  This would leave it >> up to the maintainers to decide how they want to sort the different >> requirements/liaisons/contact persons between them. > > Just for fun I had a read through the thread from when I last proposed > getting rid of PTLs, 5.5 years ago: > > http://lists.openstack.org/pipermail/openstack-dev/2014-August/043826.html > > > I wrote that when I was a PTL. Now that I have been on all sides of it > (Core team member, PTL, ex-PTL, TC member), let's see how well this > has aged :D > >> First off, the PTL is not responsible for everything in a project. >> *Everyone* is responsible for everything in a project. >> >> The PTL is *accountable* for everything in a project. PTLs are the >> mechanism the TC uses to ensure that programs remain accountable to >> the wider community. > > I still think this is true. But it's also true that if everyone is > responsible then nobody is really responsible. Somebody has to be > responsible for knowing all of the things that somebody needs to be > responsible for and making sure that somebody is responsible for each. > > That can be done without a PTL as such, but the PTL system does > provide a way of externally bootstrapping it in every project. > >> We have a heavyweight election process for PTLs once every cycle >> because that used to be the process for electing the TC. Now that it >> no longer serves this dual purpose, PTL elections have outlived their >> usefulness. > > I had completely forgotten about this. > > From a TC perspective, we don't have a lot of visibility on internal > ructions that may be going on in any particular project. The election > does at least assure us that there is an outlet valve for any issues, > and the fact that it is completely normalised across all of OpenStack > makes it more likely that someone will actually challenge the PTL if > there is a problem. > >> there's no need to impose that process on every project. If they want >> to rotate the tech lead every week instead of every 6 months, why not >> let them? We'll soon see from experimentation which models work. > > One cannot help wondering if we might get more Nova cores willing to > sign up for a 1-week commitment to be the "PTL" than we're getting for > a 6-months-and-maybe-indefinitely commitment. > >>> We also >>> still need someone to have the final say in case of deadlocked issues. >> >> -1 we really don't. > > I still think I am mostly right about this (and I know Thierry still > thinks he is right and I am wrong ;) > > IMHO it's never the job of the PTL to have a casting vote. It *is* the > job of the PTL - and all leaders in the project - to ensure that > consensus is eventually reached somehow; that discussion is not just > allowed to continue forever without a resolution when people disagree. > > While all leaders should be doing this, I can see some benefit in > having one person who sees it as specifically their responsibility, > and as noted above the PTL election process ensures that this happens > in every project. > > > In summary, I still think that in a healthy project the requirement to > have a PTL is probably mildly unhelpful. One thing that didn't come up > in that thread but that I have mentioned elsewhere, was that when I > became a PTL I very quickly learned to be very careful about what I > expressed an opinion on and how, lest I accidentally close down a > conversation that I was intending to open up. Because *overnight* > people developed this sudden tendency to be like "HE HATH SPOKEN" > whenever I weighed in. (This is very unnerving BTW, and one reason I > feel like I can be more helpful by *not* running for PTL.) So having a > PTL means giving up a core team member in some senses. > > Ultimately, from the TC perspective it's a tool for reducing the > variance in outcomes compared to letting every team decide their own > leadership structure. As with all interventions that act by reducing > variance (rather than increasing the average), this will tend to be a > burden on higher-performing teams while raising the floor for > lower-performing ones. So that's the trade-off we have to make. > > cheers, > Zane. > I have been wanting to weigh in on this thread and was waiting for the right moment. Zane's input sums up how I feel as well.  I think that having consistent leadership structure across projects is important and helps keep us aware of the health of projects. Perhaps we can help return interest in the PTL role by providing examples of teams that share the work and have the PTL to help make final decisions.  I know that the Cinder team has been doing this for quite some time successfully. Jay From soumplis at admin.grnet.gr Wed Mar 4 19:59:52 2020 From: soumplis at admin.grnet.gr (Alexandros Soumplis) Date: Wed, 4 Mar 2020 21:59:52 +0200 Subject: [Watcher] Queue watcher_notifications Message-ID: Hi all, We are using Watcher on Train and we have noticed a problem with the watcher_notifications queue which fills up with messages from Cinder and watcher-decision-engine seems not to consume them. On the contrary it perfectly consumes nova notifications from the versioned_notifications queue. Any suggestions ? a. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3620 bytes Desc: S/MIME Cryptographic Signature URL: From openstack at nemebean.com Wed Mar 4 20:08:54 2020 From: openstack at nemebean.com (Ben Nemec) Date: Wed, 4 Mar 2020 14:08:54 -0600 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: On 3/4/20 12:57 PM, Zane Bitter wrote: > One cannot help wondering if we might get more Nova cores willing to > sign up for a 1-week commitment to be the "PTL" than we're getting for a > 6-months-and-maybe-indefinitely commitment. That's a really interesting idea. I'm not sure I'd want to go as short as one week for PTL, but shortening the term might make it easier for people to commit. It might be a small issue come project update and cycle highlights time since no one person may have the big picture of what happened in the project, but ideally those are collaborative things that the entire team has input on anyway. From skaplons at redhat.com Wed Mar 4 21:08:39 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 4 Mar 2020 22:08:39 +0100 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: <150614DB-C9BD-413C-9790-C419635A2AFC@inaugust.com> References: <9be364bd-49dd-ad97-3a21-ee0cc87a9298@ham.ie> <150614DB-C9BD-413C-9790-C419635A2AFC@inaugust.com> Message-ID: <1824F3CF-16D2-425C-8EE8-9A282A09DE4F@redhat.com> +1 from me for that idea. > On 4 Mar 2020, at 18:38, Monty Taylor wrote: > > > >> On Mar 4, 2020, at 10:49 AM, Graham Hayes wrote: >> >> On 04/03/2020 16:19, Monty Taylor wrote: >>> Hey everybody, >>> I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re >>> Already ensuring that SDK does what it needs to do to facilitate that goal. We also already share PTG space, and have requested a shared set of time at the upcoming Denver PTG. So really the separation is historical not practical, and these days having additional layers of governance is not super useful. >> >> This makes sense. >> >>> I propose that we do a simple merge of the teams. This means the current SDK cores will become cores on OSC, and as most of the OSC cores are already SDK cores, it means the SDK team gains amotoki - which is always a positive. >> >> Yeah - projects were supposed to be mainly about common groups of people >> working on stuff, so if the overlap is so close already, it seems like >> a no brainer. >> >>> Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. >> >> If this is good with the two teams, this is good with me :) >> Hopefully this can help with projects teams issues with OSC/SDK response >> times. >>> Monty > > I think it can. I’ve had some chats with some folks on the team and I think we all think this will help streamline and enable us to respond more quickly. > — Slawek Kaplonski Senior software engineer Red Hat From skaplons at redhat.com Wed Mar 4 21:09:59 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 4 Mar 2020 22:09:59 +0100 Subject: [sdk] Additions and subtractions from core team In-Reply-To: References: Message-ID: <7DF33892-5254-472F-BC0E-07211A05253E@redhat.com> Hi, > On 4 Mar 2020, at 17:56, Monty Taylor wrote: > > Heya, > > With the previous email about merging OSC and SDK teams, I’d also like to propose the following changes to the SDK core team (keeping in mind that likely means the core team of both OSC and SDK real soon now) > > Adds: > > Akihiro Motoki - The only OSC core not in sdk-core. amotoki should really be a core in all projects anyway > Sean McGinnis - Sean has been reviewing things as a stable branch maint in both SDK and OSC, and as such has shown a good tendency to help things along when needed and to not approve things when he doesn’t know what’s up. Big +1 from me here. > > Subtractions: > > All of these people are awesome, but they’re all long gone: > > Brian Curtin > Clint Byrum > Everett Toews > Jamie Lennox > Jesse Noller > Ricardo Carillo Cruz > Richard Theis > Rosario Di Somma > Sam Yaple > Terry Howe That’s sad but if that is necessary than ok. > > Monty > — Slawek Kaplonski Senior software engineer Red Hat From sean.mcginnis at gmx.com Wed Mar 4 21:10:55 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 4 Mar 2020 15:10:55 -0600 Subject: [CINDER] Snapshots export In-Reply-To: <20200304155850.b4ydu4vfxthih7we@localhost> References: <20200304155850.b4ydu4vfxthih7we@localhost> Message-ID: On 3/4/20 9:58 AM, Gorka Eguileor wrote: > On 03/03, Alfredo De Luca wrote: >> Hi all. >> We have our env with Openstack (Train) and cinder with CEPH (nautilus) >> backend. >> We are creating automatic volumes snapshots and now we'd like to export >> them as a backup/restore plan. After exporting the snapshots we will use >> Acronis as backup tool. >> >> I couldn't find the right steps/commands to exports the snapshots. >> Any info? >> Cheers >> >> -- >> *Alfredo* > Hi Alfredo, > > What kind of backup/restore plan do you have planned? > > Because snapshots are not meant to be used in a Disaster Recovery > backup/restore plan, so the only thing available are the manage/unmanage > commands. > > These commands are meant to add an existing volume/snapshots into Cinder > together, not to unmanage/manage them independently. > > For example, you wouldn't be able to manage a snapshot if the volume is > not already managed. Also unmanaging the snapshot would block the > deletion of the RBD volume itself. > > Cheers, > Gorka. If the intent is to use the snapshots as a source to backup the volume data, leaving the actual volume attached and IO running but still getting a "static" view of the code, then you would need to create a volume from the chosen snapshot, mount that volume somewhere that is accessible to your backup software, perform the copy of the data, then delete the volume when complete. I haven't used Acronis myself, but the issue for some backup software could be that the volume it is backing up from is going to be different every time. Though you could make sure it is mounted at the same place so the backup software at least *thinks* it's backing up the same thing. Then restoring the data will likely require some manual intervention, but that's pretty much always the case when something goes wrong. I would just recommend you test the full disaster recovery scenario to make sure you have that figured out and working right before you actually need it. Sean From colleen at gazlene.net Wed Mar 4 21:26:45 2020 From: colleen at gazlene.net (Colleen Murphy) Date: Wed, 04 Mar 2020 13:26:45 -0800 Subject: =?UTF-8?Q?[policy][keystone][nova][cyborg][barbican][neutron][manila][ci?= =?UTF-8?Q?nder]_Policy_Popup_team_progress_report?= Message-ID: <3f3a161a-ba33-4737-b0d1-556810aa9315@www.fastmail.com> This is an update on the progress made within the Policy Popup team[1] so far this cycle. [1] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popup_Team Why This Is Important ===================== Separating system, domain, and project-scope APIs and providing meaningful default roles is critical to facilitating secure cloud deployments and to fulfilling OpenStack's vision as a fully self-service infrastructure provider[2]. Until all projects have completed this policy migration, the "reader" role that exists in keystone is dangerously misleading, and the `[oslo_policy]/enforce_scope` option has limited usefulness as long as projects lack uniformity in how an administrator can use scoped APIs. [2] https://governance.openstack.org/tc/reference/technical-vision.html#self-service Project Progress ================ Nova ---- - Ussuri spec has merged[3] - 28 changes implementing the spec have been merged[4] - 39 additional changes have been proposed and are awaiting review[5] [3] https://review.opendev.org/686058 [4] https://review.opendev.org/#/q/topic:bp/policy-defaults-refresh+status:merged [5] https://review.opendev.org/#/q/topic:bp/policy-defaults-refresh+status:open Cyborg ------ - Ussuri spec has merged[6] and a tracking story has been created[7] - 2 changes to implement the spec have been proposed and are awaiting review[8] [6] https://review.opendev.org/699099 [7] https://storyboard.openstack.org/#!/story/2007024 [8] https://review.opendev.org/#/q/project:openstack/cyborg+topic:policy-popup+status:open Barbican -------- - A table has been created outlining the required policy changes[9] - No patches merged or proposed yet [9] https://wiki.openstack.org/wiki/Barbican/Policy Neutron ------- - No planning document - No patches merged or proposed yet Manila ------ - No planning document - No patches merged or proposed yet Cinder ------ - No planning document - No patches merged or proposed yet How You Can Help ================ If you are a contributor for these teams, please update the popup team wiki page[10] as your project starts to plan and implement policy changes. If you are a cloud operator, please help review the proposed policy rule changes to sanity-check the new scope and role defaults and to help influence these decisions. [10] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popup_Team Reminders ========= - Reach out at any time to the keystone team if you have questions on this popup team's goals. - Colleen still seeking to be replaced as co-chair, please let me know if you're interested. From openstack at nemebean.com Wed Mar 4 22:33:37 2020 From: openstack at nemebean.com (Ben Nemec) Date: Wed, 4 Mar 2020 16:33:37 -0600 Subject: [oslo][infra] OpenDev git repo for oslo.policy missing commit In-Reply-To: References: <1129d4b2-0a8d-d034-5ded-7e49e6e49a77@nemebean.com> Message-ID: <69ec9111-0617-3259-98f7-64e5125eba54@nemebean.com> On 3/3/20 5:42 PM, Clark Boylan wrote: > On Tue, Mar 3, 2020, at 3:05 PM, Ben Nemec wrote: >> Found a weird thing today. The OpenDev oslo.policy repo[0] is missing >> [1]. Even stranger, I see it on the Github mirror[2]. Any idea what >> happened here? > > Some other readers may notice that the commit actually does show up for them. The reason for this is the commit is only missing from one of eight backend gitea servers. You can observe this by visiting https://gitea0X.opendev.org:3000/openstack/oslo.policy/commits/branch/master and replacing the X with 1 through 8. Number 5 is the lucky server. > > My hunch is that this commit merging and subsequently being replicated coincided with a restart of gitea (or related service) on gitea05. And the replication event was missed. We've tried to ensure we replicate to catch up after explicit upgrades, which implies to me that maybe the db container updated. Note that https://review.opendev.org/#/c/705804/ merged on the same day but after the missing commit. > > In any case I've triggered a full rereplication to gitea05 to make sure we are caught up and will work through the others as well to ensure none are missed. You should be able to confirm that the commit is present in about 20 minutes. Great, thanks! I see it now. > > Longer term the plan here is to run a single Gitea cluster which will allow us to do rolling restarts of services without impacting replication. Unfortunately, this requires updates to Gitea to support that. > >> >> -Ben >> >> 0: https://opendev.org/openstack/oslo.policy/commits/branch/master >> 1: https://review.opendev.org/#/c/708212/ >> 2: https://github.com/openstack/oslo.policy/commits/master >> >> > From zbitter at redhat.com Wed Mar 4 22:33:57 2020 From: zbitter at redhat.com (Zane Bitter) Date: Wed, 4 Mar 2020 17:33:57 -0500 Subject: [stable][heat] Nominating Rabi Mishra for heat-stable-maint Message-ID: Rabi has been a core reviewer on Heat for more than 4 years, and is a former PTL. In that time he's done hundreds of backports: https://review.opendev.org/#/q/owner:ramishra%2540redhat.com+branch:%22%255Estable/.*%22+project:openstack/heat (Only two of which we ended up deciding not to merge, the last of which was in 2016.) As well as a good number of reviews: https://review.opendev.org/#/q/reviewedby:ramishra%2540redhat.com+branch:%22%255Estable/.*%22+project:openstack/heat+NOT+owner:%22Rabi+Mishra+%253Cramishra%2540redhat.com%253E%22 Rabi also maintains a downstream distribution of Heat, so he is fully aware of the pain that backporting inappropriate changes can cause. He is well aware of the stable branch guidelines and is, if anything, more conservative than I am in applying them. I'm 100% confident he will not be approving any feature backports. I'll be spending some extended time away from a keyboard in the near future, so it's important that we increase the bus factor (from 1) of heat-stable-maint team members. thanks, Zane. From gmann at ghanshyammann.com Wed Mar 4 22:40:39 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 04 Mar 2020 16:40:39 -0600 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: References: Message-ID: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita wrote ---- > Hello QA team and devstack-plugin-ceph-core people, > > The Cinder team has some proposals we'd like to float. > > 1. The Cinder team is interested in becoming more active in the > maintenance of openstack/devstack-plugin-ceph [0]. Currently, the > devstack-plugin-ceph-core is > https://review.opendev.org/#/admin/groups/1196,members > The cinder-core is already represented by Eric and Sean; we'd like to > replace them by including the cinder-core group. +1. This is good diea and make sense, I will do the change. > > 2. The Cinder team is interested in becoming more active in the > maintenance of x/devstack-plugin-nfs [1]. Currently, the > devstack-plugin-nfs-core is > https://review.opendev.org/#/admin/groups/1330,members > It's already 75% cinder-core members; we'd like to replace the > individual members with the cinder-core group. We also propose that > devstack-core be added as an included group. > > 3. The Cinder team is interested in implementing a new devstack plugin: > openstack/devstack-plugin-open-cas > This will enable thorough testing of a new feature [2] being introduced > as experimental in Ussuri and expected to be finalized in Victoria. Our > plan would be to make both cinder-core and devstack-core included groups > for the gerrit group governing the new plugin. +1. You want this under Cinder governance or under QA ? > > 4. This is a minor point, but can the devstack-plugin-nfs repo be moved > back into the 'openstack' namespace? If this is usable plugin for nfs testing (I am not aware if we have any other) then it make sense to bring it to openstack governance. Same question here, do you want to put this under Cinder governance or QA. Those plugins under QA governance also ok for me with your proposal of calloborative maintainance by devstack-core and cinder-core. -gmann > > Let us know which of these proposals you find acceptable. > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > [1] https://opendev.org/x/devstack-plugin-nfs > [2] https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > From zbitter at redhat.com Wed Mar 4 22:43:27 2020 From: zbitter at redhat.com (Zane Bitter) Date: Wed, 4 Mar 2020 17:43:27 -0500 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: On 4/03/20 3:08 pm, Ben Nemec wrote: > > > On 3/4/20 12:57 PM, Zane Bitter wrote: >> One cannot help wondering if we might get more Nova cores willing to >> sign up for a 1-week commitment to be the "PTL" than we're getting for >> a 6-months-and-maybe-indefinitely commitment. > > That's a really interesting idea. I'm not sure I'd want to go as short > as one week for PTL, but shortening the term might make it easier for > people to commit. The key would be to make it short enough that you can be 100% confident the next person will take over and not leave you holding the bag forever. (Hi Rico!) I've no idea where the magic number would fall, and it's probably different for every team. I'm reasonably confident it's somewhere between 1 week and 6 months though. From dms at danplanet.com Thu Mar 5 00:34:11 2020 From: dms at danplanet.com (Dan Smith) Date: Wed, 04 Mar 2020 16:34:11 -0800 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: (Zane Bitter's message of "Wed, 4 Mar 2020 17:43:27 -0500") References: Message-ID: > The key would be to make it short enough that you can be 100% > confident the next person will take over and not leave you holding the > bag forever. (Hi Rico!) > > I've no idea where the magic number would fall, and it's probably > different for every team. I'm reasonably confident it's somewhere > between 1 week and 6 months though. I think one potential benefit from this comes in the form of killing one of the other thngs that Zane mentioned (which I totally agree with): The "(s)he hath spoken" part. I can imagine getting to the point where not everyone on the team remembers exactly "who is PTL this month" and certainly the long tail of minor contributors would lose visibility into this. I think that would dis-empower (in a good way) the PTL-this-month person both from the perspective of being the constant (even if unnecessary) decider, and the sounding board for everyone trying to push their pet efforts through the works. Although I'm calling "not it" for the month containing the PTG right here and now ;P --Dan From gmann at ghanshyammann.com Thu Mar 5 00:53:55 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 04 Mar 2020 18:53:55 -0600 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <170a82f47d1.bb51993a438690.220344225005930378@ghanshyammann.com> ---- On Wed, 04 Mar 2020 16:43:27 -0600 Zane Bitter wrote ---- > On 4/03/20 3:08 pm, Ben Nemec wrote: > > > > > > On 3/4/20 12:57 PM, Zane Bitter wrote: > >> One cannot help wondering if we might get more Nova cores willing to > >> sign up for a 1-week commitment to be the "PTL" than we're getting for > >> a 6-months-and-maybe-indefinitely commitment. > > > > That's a really interesting idea. I'm not sure I'd want to go as short > > as one week for PTL, but shortening the term might make it easier for > > people to commit. > > The key would be to make it short enough that you can be 100% confident > the next person will take over and not leave you holding the bag > forever. (Hi Rico!) > > I've no idea where the magic number would fall, and it's probably > different for every team. I'm reasonably confident it's somewhere > between 1 week and 6 months though. This seems a good way to distribute the PTL overload but I am thinking what if more than one would like to server as PTL for the cycle or whatever period we decide. I am not sure we will have this case in the current situation where almost all projects are without-election but still we should have some mechanism ready. Another idea I think about co-PTLship. I remember in previous or this cycle few projects want to have the co-PTL concept. Means officially have more than PTL. To solve the single point of contact issue we can have single PTL contact and other co-PTL distribute the responsibility for that cycle. -gmann > > > From rico.lin.guanyu at gmail.com Thu Mar 5 02:08:08 2020 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Thu, 5 Mar 2020 10:08:08 +0800 Subject: [stable][heat] Nominating Rabi Mishra for heat-stable-maint In-Reply-To: References: Message-ID: Big +1 on this On Thu, Mar 5, 2020 at 6:39 AM Zane Bitter wrote: > Rabi has been a core reviewer on Heat for more than 4 years, and is a > former PTL. In that time he's done hundreds of backports: > > > https://review.opendev.org/#/q/owner:ramishra%2540redhat.com+branch:%22%255Estable/.*%22+project:openstack/heat > > (Only two of which we ended up deciding not to merge, the last of which > was in 2016.) > > As well as a good number of reviews: > > > https://review.opendev.org/#/q/reviewedby:ramishra%2540redhat.com+branch:%22%255Estable/.*%22+project:openstack/heat+NOT+owner:%22Rabi+Mishra+%253Cramishra%2540redhat.com%253E%22 > > Rabi also maintains a downstream distribution of Heat, so he is fully > aware of the pain that backporting inappropriate changes can cause. He > is well aware of the stable branch guidelines and is, if anything, more > conservative than I am in applying them. I'm 100% confident he will not > be approving any feature backports. > > I'll be spending some extended time away from a keyboard in the near > future, so it's important that we increase the bus factor (from 1) of > heat-stable-maint team members. > > thanks, > Zane. > > > -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From rui.zang at yandex.com Thu Mar 5 02:24:55 2020 From: rui.zang at yandex.com (rui zang) Date: Thu, 05 Mar 2020 10:24:55 +0800 Subject: CPU Topology confusion In-Reply-To: References: Message-ID: <23998181583375095@myt3-4825096bdc88.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From feilong at catalyst.net.nz Thu Mar 5 02:29:17 2020 From: feilong at catalyst.net.nz (Feilong Wang) Date: Thu, 5 Mar 2020 15:29:17 +1300 Subject: [stable][heat] Nominating Rabi Mishra for heat-stable-maint In-Reply-To: References: Message-ID: <47d2af35-120a-ea46-61f6-bab5d0a3548b@catalyst.net.nz> big +1 On 5/03/20 11:33 AM, Zane Bitter wrote: > Rabi has been a core reviewer on Heat for more than 4 years, and is a > former PTL. In that time he's done hundreds of backports: > > https://review.opendev.org/#/q/owner:ramishra%2540redhat.com+branch:%22%255Estable/.*%22+project:openstack/heat > > > (Only two of which we ended up deciding not to merge, the last of > which was in 2016.) > > As well as a good number of reviews: > > https://review.opendev.org/#/q/reviewedby:ramishra%2540redhat.com+branch:%22%255Estable/.*%22+project:openstack/heat+NOT+owner:%22Rabi+Mishra+%253Cramishra%2540redhat.com%253E%22 > > > Rabi also maintains a downstream distribution of Heat, so he is fully > aware of the pain that backporting inappropriate changes can cause. He > is well aware of the stable branch guidelines and is, if anything, > more conservative than I am in applying them. I'm 100% confident he > will not be approving any feature backports. > > I'll be spending some extended time away from a keyboard in the near > future, so it's important that we increase the bus factor (from 1) of > heat-stable-maint team members. > > thanks, > Zane. > > -- Cheers & Best regards, Feilong Wang (王飞龙) Head of R&D Catalyst Cloud - Cloud Native New Zealand -------------------------------------------------------------------------- Tel: +64-48032246 Email: flwang at catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington -------------------------------------------------------------------------- From rico.lin.guanyu at gmail.com Thu Mar 5 02:29:42 2020 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Thu, 5 Mar 2020 10:29:42 +0800 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <170a82f47d1.bb51993a438690.220344225005930378@ghanshyammann.com> References: <170a82f47d1.bb51993a438690.220344225005930378@ghanshyammann.com> Message-ID: On Thu, Mar 5, 2020 at 8:59 AM Ghanshyam Mann wrote: > > Another idea I think about co-PTLship. I remember in previous or this > cycle few projects want to have the co-PTL concept. Means officially have > more than PTL. To solve the single point of contact issue we can have > single PTL contact and other co-PTL distribute the responsibility for that cycle. > IMO that's a good idea for sure, and that's how the Telemetry team is doing right now (two co-PTLs) But from my part, PTL is really not that huge needed for leadership but a lot of maintaining works. This makes me tend to agree that `Maintainers` is the right word (along with other nice choices). In the long term, merge `core team` into the `Maintainers`? :) And to specifically define, I don't think this change means the project is under maintenance mode. Features should still welcome to innovate each project we have. And we should specifically mention that to avoid any confusion people might have here. -- May The Force of OpenStack Be With You, Rico Lin irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From kpuusild at gmail.com Thu Mar 5 06:45:36 2020 From: kpuusild at gmail.com (Kevin Puusild) Date: Thu, 5 Mar 2020 08:45:36 +0200 Subject: NovaVMware - vCenter / DevStack - ERROR Message-ID: Hello i have written here before but i didn't get any help so i try again. Current Setup: Ubuntu 18.04 Server with 1 vCenter + 1 ESXi i am trying to setup DevStack with vCenter as a hypervisor (not ESXI) following this wiki article: https://wiki.openstack.org/wiki/NovaVMware/DeveloperGuide to prepare to setup my devstack and following this wiki articel https://docs.openstack.org/devstack/latest/ too (compaining these two articles) *localrc* file is containing below information: ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-cpu,n-net,n-cond,n-sch,n-novnc,n-cauth,rabbit,mysql,horizon VIRT_DRIVER=vsphere VMWAREAPI_IP=#MY_VCENTER_IP VMWAREAPI_USER=#MY_VCENTER_ADMINISTRATOR_USER VMWAREAPI_PASSWORD=#MY_VCENTER_ADMINISTRATOR_PASSWORD VMWAREAPI_CLUSTER=#MY_VCENTER_CLUSTER_NAME DATABASE_PASSWORD=nova RABBIT_PASSWORD=nova SERVICE_TOKEN=nova SERVICE_PASSWORD=nova ADMIN_PASSWORD=nova HOST_IP=#MY_DEVSTACK_MACHINE_IP after running *stack.sh* script i get following error: ++functions:wait_for_compute:448 hostname +functions:wait_for_compute:448 compute_hostname=devstack +functions:wait_for_compute:450 timeout 60 bash -x ++functions:wait_for_compute:450 hostname +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +:: [[ vsphere = \f\a\k\e ]] ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host devstack --service nova-compute -c ID -f value +functions:wait_for_compute:450 rval=124 +functions:wait_for_compute:462 time_stop wait_for_service +functions-common:time_stop:2330 local name +functions-common:time_stop:2331 local end_time +functions-common:time_stop:2332 local elapsed_time +functions-common:time_stop:2333 local total +functions-common:time_stop:2334 local start_time +functions-common:time_stop:2336 name=wait_for_service +functions-common:time_stop:2337 start_time=1582790249013 +functions-common:time_stop:2339 [[ -z 1582790249013 ]] ++functions-common:time_stop:2342 date +%s%3N +functions-common:time_stop:2342 end_time=1582790309198 +functions-common:time_stop:2343 elapsed_time=60185 +functions-common:time_stop:2344 total=7679 +functions-common:time_stop:2346 _TIME_START[$name]= +functions-common:time_stop:2347 _TIME_TOTAL[$name]=67864 +functions:wait_for_compute:464 [[ 124 != 0 ]] +functions:wait_for_compute:465 echo 'Didn'\''t find service registered by hostname after 60 seconds' Didn't find service registered by hostname after 60 seconds +functions:wait_for_compute:466 openstack --os-cloud devstack-admin --os-region RegionOne compute service list +functions:wait_for_compute:468 return 124 +lib/nova:is_nova_ready:1 exit_trap +./stack.sh:exit_trap:533 local r=124 ++./stack.sh:exit_trap:534 jobs -p +./stack.sh:exit_trap:534 jobs= +./stack.sh:exit_trap:537 [[ -n '' ]] +./stack.sh:exit_trap:543 '[' -f /tmp/tmp.8iR1neh89b ']' +./stack.sh:exit_trap:544 rm /tmp/tmp.8iR1neh89b +./stack.sh:exit_trap:548 kill_spinner +./stack.sh:kill_spinner:443 '[' '!' -z '' ']' +./stack.sh:exit_trap:550 [[ 124 -ne 0 ]] +./stack.sh:exit_trap:551 echo 'Error on exit' Error on exit +./stack.sh:exit_trap:553 type -p generate-subunit +./stack.sh:exit_trap:554 generate-subunit 1582789174 1136 fail +./stack.sh:exit_trap:556 [[ -z /opt/stack/logs ]] +./stack.sh:exit_trap:559 /usr/bin/python3.6 /devstack/devstack/tools/worlddump.py -d /opt/stack/logs World dumping... see /opt/stack/logs/worlddump-2020-02-27-075831.txt for details +./stack.sh:exit_trap:568 exit 124 -- Kevin Puusild -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephenfin at redhat.com Thu Mar 5 10:55:45 2020 From: stephenfin at redhat.com (Stephen Finucane) Date: Thu, 05 Mar 2020 10:55:45 +0000 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: References: Message-ID: <2c8ccd0c2d2d7a8ae6073de8b9fc80656fa49ce0.camel@redhat.com> On Wed, 2020-03-04 at 10:19 -0600, Monty Taylor wrote: > Hey everybody, > > I'd like to propose merging the SDK and OSC teams. We already share > an IRC channel, and already share a purpose in life. In OSC we have a > current goal of swapping out client implementation for SDK, and we're > Already ensuring that SDK does what it needs to do to facilitate that > goal. We also already share PTG space, and have requested a shared > set of time at the upcoming Denver PTG. So really the separation is > historical not practical, and these days having additional layers of > governance is not super useful. > > I propose that we do a simple merge of the teams. This means the > current SDK cores will become cores on OSC, and as most of the OSC > cores are already SDK cores, it means the SDK team gains amotoki - > which is always a positive. Big +1 > Dean hasn't had time to spend on OSC quite a bit, sadly, and while we > remain hopeful that this will change, we’re slowly coming to terms > with the possibility that it might not. With that in mind, I'll serve > as the PTL for the new combined team until the next election. > > Monty > From gr at ham.ie Thu Mar 5 11:06:33 2020 From: gr at ham.ie (Graham Hayes) Date: Thu, 5 Mar 2020 11:06:33 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <1da63dab-c35e-6377-d5d8-e075a5c37408@ham.ie> On 04/03/2020 22:43, Zane Bitter wrote: > On 4/03/20 3:08 pm, Ben Nemec wrote: >> >> >> On 3/4/20 12:57 PM, Zane Bitter wrote: >>> One cannot help wondering if we might get more Nova cores willing to >>> sign up for a 1-week commitment to be the "PTL" than we're getting >>> for a 6-months-and-maybe-indefinitely commitment. >> >> That's a really interesting idea. I'm not sure I'd want to go as short >> as one week for PTL, but shortening the term might make it easier for >> people to commit. > > The key would be to make it short enough that you can be 100% confident > the next person will take over and not leave you holding the bag > forever. (Hi Rico!) And also that the person you hand it off too won't have to hand it back. (Hi Tim!) > I've no idea where the magic number would fall, and it's probably > different for every team. I'm reasonably confident it's somewhere > between 1 week and 6 months though. Yeah - I am not sure the TC should mandate a number - some teams might be OK with the 6 months, while others will need to do 1 or 2 weeks From pete.vandergiessen at canonical.com Thu Mar 5 11:19:53 2020 From: pete.vandergiessen at canonical.com (Pete Vander Giessen) Date: Thu, 5 Mar 2020 12:19:53 +0100 Subject: [MicroStack] Beta updates and strict (devmode) confinement on edge Message-ID: Hi all, It has been a little while since I’ve posted any MicroStack news. MicroStack is a base set of OpenStack services, bundled into a snap. It’s intended for OpenStack workload development (as opposed to development of OpenStack itself), and also contains some experimental features geared toward edge clouds. We’ve just released a new MicroStack beta, with preview support for clustering (e.g., running a control plane node along with several compute nodes). You can install it with: sudo snap install --classic --beta microstack There are some known issues with refreshes from the current beta version of the snap to the new version. You may need to reinstall MicroStack if you’re running the current beta. To address some issues running MicroStack on non LTS Ubuntu distros, as well as lay the groundwork for a smoother refresh/upgrade process going forward, we’re working on making MicroStack into a strictly confined snap! If you’re feeling brave, you can try the (almost) strictly confined snap in developer mode with the following invocation: sudo snap install --devmode --edge microstack Work to put together a completely confined version of the snap, with no need for the devmode flag, is ongoing. You can find more information about MicroStack at https://microstack.run, and talk to the devs either on this mailing list, or on IRC: freenode #openstack-snaps . -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Thu Mar 5 11:52:12 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 5 Mar 2020 12:52:12 +0100 Subject: [neutron][drivers team] Meeting 06.03.2020 cancelled Message-ID: <30F8DBAA-B735-4E72-A290-F23B1FC9AC58@redhat.com> Hi Neutrinos, Due to lack of agenda lets cancel tomorrow’s drivers meeting. Lets meet as usually next week on Friday 13.03.2020. But, as You have now one hour of “free” time tomorrow (:D) I would like to ask You to take a look and help triaging one new RFE: https://bugs.launchpad.net/neutron/+bug/1865889 I’m not the best expert in routed networks, so I would really like if Miguel, Brian and others could take a look at this :) Thx in advance. — Slawek Kaplonski Senior software engineer Red Hat From satish.txt at gmail.com Thu Mar 5 12:18:15 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 5 Mar 2020 07:18:15 -0500 Subject: CPU Topology confusion In-Reply-To: <23998181583375095@myt3-4825096bdc88.qloud-c.yandex.net> References: <23998181583375095@myt3-4825096bdc88.qloud-c.yandex.net> Message-ID: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> cpu-passthrough Sent from my iPhone > On Mar 4, 2020, at 9:24 PM, rui zang wrote: > >  > Hi, > > What is the value for the "cpu_mode" configuration option? > https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.html > > Thanks, > Zang, Rui > > > 05.03.2020, 01:24, "Satish Patel" : > Folks, > > We are running openstack with KVM and i have noticed kvm presenting > wrong CPU Tolopoly to VM and because of that we are seeing bad > performance to our application. > > This is openstack compute: > > # lstopo-no-graphics --no-io > Machine (64GB total) > NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) > L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 > PU L#0 (P#0) > PU L#1 (P#20) > L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 > PU L#2 (P#1) > PU L#3 (P#21) > > This is VM running on above compute > > # lstopo-no-graphics --no-io > Machine (59GB total) > NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) > L2 L#0 (4096KB) + Core L#0 > L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) > L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) > L2 L#1 (4096KB) + Core L#1 > L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) > L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) > > if you noticed P#0 and P#1 has own (32KB) cache per thread that is > wrong presentation if you compare with physical CPU. > > This is a screenshot of AWS vs Openstack CPU Topology and looking at > openstack its presentation is little odd, is that normal? > > https://imgur.com/a/2sPwJVC > > I am running CentOS7.6 with kvm 2.12 version. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tolga at etom.cloud Thu Mar 5 12:40:44 2020 From: tolga at etom.cloud (tolga at etom.cloud) Date: Thu, 05 Mar 2020 15:40:44 +0300 Subject: [trove][charm] RabbitMQ Connection Error Message-ID: <28295851583412044@myt6-636ea6dfd460.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 5 14:43:18 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 5 Mar 2020 09:43:18 -0500 Subject: CPU Topology confusion In-Reply-To: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> References: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> Message-ID: cpu_mode = cpu-passthrough cpu_model = none Do you think cpu_model make difference ? Sent from my iPhone > On Mar 5, 2020, at 7:18 AM, Satish Patel wrote: > >  > > cpu-passthrough > > Sent from my iPhone > >>> On Mar 4, 2020, at 9:24 PM, rui zang wrote: >>> >>  >> Hi, >> >> What is the value for the "cpu_mode" configuration option? >> https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.html >> >> Thanks, >> Zang, Rui >> >> >> 05.03.2020, 01:24, "Satish Patel" : >> Folks, >> >> We are running openstack with KVM and i have noticed kvm presenting >> wrong CPU Tolopoly to VM and because of that we are seeing bad >> performance to our application. >> >> This is openstack compute: >> >> # lstopo-no-graphics --no-io >> Machine (64GB total) >> NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) >> L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 >> PU L#0 (P#0) >> PU L#1 (P#20) >> L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 >> PU L#2 (P#1) >> PU L#3 (P#21) >> >> This is VM running on above compute >> >> # lstopo-no-graphics --no-io >> Machine (59GB total) >> NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) >> L2 L#0 (4096KB) + Core L#0 >> L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) >> L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) >> L2 L#1 (4096KB) + Core L#1 >> L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) >> L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) >> >> if you noticed P#0 and P#1 has own (32KB) cache per thread that is >> wrong presentation if you compare with physical CPU. >> >> This is a screenshot of AWS vs Openstack CPU Topology and looking at >> openstack its presentation is little odd, is that normal? >> >> https://imgur.com/a/2sPwJVC >> >> I am running CentOS7.6 with kvm 2.12 version. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ekuvaja at redhat.com Thu Mar 5 15:03:29 2020 From: ekuvaja at redhat.com (Erno Kuvaja) Date: Thu, 5 Mar 2020 15:03:29 +0000 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> Message-ID: On Wed, Mar 4, 2020 at 1:19 AM Ghanshyam Mann wrote: > ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell wrote > ---- > > > > > > On 3 Mar 2020, at 19:55, Tim Bell wrote: > > > > > > On 3 Mar 2020, at 19:20, Albert Braden > wrote: > > Sean, thank you for clarifying that. > > > > Was my understanding that the community decided to focus on the unified > client incorrect? Is the unified/individual client debate still a matter of > controversy? Is it possible that the unified client will be deprecated in > favor of individual clients after more discussion? I haven’t looked at any > of the individual clients since 2018 (except for osc-placement which is > kind of a special case), because I thought they were all going away and > could be safely ignored until they did, and I haven’t included any > information about the individual clients in the documentation that I write > for our users, and if they ask I have been telling them to not use the > individual clients. Do I need to start looking at individual clients again, > and telling our users to use them in some cases? > > > > > > > > I remember a forum discussion where a community goal was proposed to > focus on OSC rather than individual project CLIs (I think Matt and I were > proposers). There were concerns on the effort to do this and that it would > potentially be multi-cycle. > > BTW, I found the etherpad from Berlin ( > https://etherpad.openstack.org/p/BER-t-series-goals) and the associated > mailing list discussion at > http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html > > Yeah, we are in process of selecting the Victoria cycle community-wide > goal and this can be good candidate. I agree with the idea/requirement of a > multi-cycle goal. > Another option is to build a pop-up team for the Victoria cycle to start > burning down the keys issues/work. For both ways (either goal or pop-up > team), we need > some set of people to drive it. If anyone would like to volunteer for > this, we can start discussing the details. > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html > > -gmann > > Yeah, lets propose this as community goal again as it worked so well last time. ಠ_ಠ I think your most help wanted list/pop-up team is much more realistic approach. Lets see if there is enough interest to actually make it happen. What comes to our previous experience with Glance and moving to endorse osc, I think I'm not alone stating that we can discuss this again after osc has kept feature parity (and I mean to current release, not feature parity 2 years ago kind of thing) and actively addressed raised issues at least for a couple of cycles. Obviously if you/your users wants to use it meanwhile, that your call. If we cannot get that level of commitment, how do we expect to support this long term? I'm not willing to put our users through that misery again as it happened last time as long as I'm core in this project. - jokke > > > > My experience in discussion with the CERN user community and other > OpenStack operators is that OSC is felt to be the right solution for the > end user facing parts of the cloud (admin commands could be another > discussion if necessary). Experienced admin operators can remember that > glance looks after images and nova looks after instances. Our average user > can get very confused, especially given that OSC supports additional > options for authentication (such as Kerberos and Certificates along with > clouds.yaml) so users need to re-authenticate with a different openrc to > work on their project. > > While I understand there are limited resources all round, I would > prefer that we focus on adding new project functions to OSC which will > eventually lead to feature parity. > > Attracting ‘drive-by’ contributions from operations staff for OSC work > (it's more likely to be achieved if it makes the operations work less e.g. > save on special end user documentation by contributing code). This is > demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' > functionality along with lots of random OSC updates as listed hat > https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) > > > BTW, I also would vote for =auto as the default. > > Tim > > We are on Rocky now but I expect that we will upgrade as necessary to > stay on supported versions. > > > > From: Sean McGinnis > > Sent: Tuesday, March 3, 2020 9:50 AM > > To: openstack-discuss at lists.openstack.org > > Subject: Re: OSC future (formerly [glance] Different checksum between > CLI and curl) > > > > On 3/3/20 11:28 AM, Albert Braden wrote: > > Am I understanding correctly that the Openstack community decided to > focus on the unified client, and to deprecate the individual clients, and > that the Glance team did not agree with this decision, and that the Glance > team is now having a pissing match with the rest of the community, and is > unilaterally deciding to continue developing the Glance client and refusing > to work on the unified client, or is something different going on? I would > ask everyone involved to remember that we operators are down here, and the > yellow rain falling on our heads does not smell very good. > > I definitely would not characterize it that way. > > With trying not to put too much personal bias into it, here's what I > would say the situation is: > > - Some part of the community has said OSC should be the only CLI and > that individual CLIs should go away > > - Glance is a very small team with very, very limited resources > > - The OSC team is a very small team with very, very limited resources > > - CLI capabilities need to be exposed for Glance changes and the > easiest way to get them out for the is by updating the Glance CLI > > - No one from the OSC team has been able to proactively help to make > sure these changes make it into the OSC client (see bullet 3) > > - There exists a sizable functionality gap between per-project CLIs and > what OSC provides, and although a few people have done a lot of great work > to close that gap, there is still a lot to be done and does not appear the > gap will close at any point in the near future based on the current trends > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Thu Mar 5 15:27:21 2020 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 5 Mar 2020 15:27:21 +0000 Subject: [ansible-sig][kolla][openstack-ansible][osc][sdk][tripleo] OpenStack modules broken in Ansible 2.8.9 Message-ID: Hi, The 2.8.9 release of Ansible has a regression [1] which breaks the OpenStack modules. I've proposed a simple fix, hopefully it will be included in a 2.8.10 release soon but in the meantime you may need to blacklist 2.8.9. [1] https://github.com/ansible/ansible/issues/68042 [2] https://github.com/ansible/ansible/pull/68043 Cheers, Mark From missile0407 at gmail.com Thu Mar 5 16:26:03 2020 From: missile0407 at gmail.com (Eddie Yen) Date: Fri, 6 Mar 2020 00:26:03 +0800 Subject: CPU Topology confusion In-Reply-To: References: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> Message-ID: Hi Satish, Since you already set "cpu_mode = host-passthrough", there's no need to set cpu_model. BTW, we're not known about the CPU topology a lot. But IME we always set "hw_cpu_sockets = 2" in specified image or flavor metadata if running Windows instance. In default, KVM always allocate all vcpus into sockets in CPU topology, and this will affect the Windows VM performance since Windows only support maximum 2 CPU sockets. Perhaps you can try limit socket numbers by setting hw_cpu_sockets in image metadata (or hw:cpu_sockets in flavor metadata.) Satish Patel 於 2020年3月5日 週四 下午10:46寫道： > > cpu_mode = cpu-passthrough > cpu_model = none > > Do you think cpu_model make difference ? > > > Sent from my iPhone > > On Mar 5, 2020, at 7:18 AM, Satish Patel wrote: > >  > > cpu-passthrough > > Sent from my iPhone > > On Mar 4, 2020, at 9:24 PM, rui zang wrote: > >  > Hi, > > What is the value for the "cpu_mode" configuration option? > > https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.html > > Thanks, > Zang, Rui > > > 05.03.2020, 01:24, "Satish Patel" : > > Folks, > > We are running openstack with KVM and i have noticed kvm presenting > wrong CPU Tolopoly to VM and because of that we are seeing bad > performance to our application. > > This is openstack compute: > > # lstopo-no-graphics --no-io > Machine (64GB total) > NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) > L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 > PU L#0 (P#0) > PU L#1 (P#20) > L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 > PU L#2 (P#1) > PU L#3 (P#21) > > This is VM running on above compute > > # lstopo-no-graphics --no-io > Machine (59GB total) > NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) > L2 L#0 (4096KB) + Core L#0 > L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) > L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) > L2 L#1 (4096KB) + Core L#1 > L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) > L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) > > if you noticed P#0 and P#1 has own (32KB) cache per thread that is > wrong presentation if you compare with physical CPU. > > This is a screenshot of AWS vs Openstack CPU Topology and looking at > openstack its presentation is little odd, is that normal? > > https://imgur.com/a/2sPwJVC > > I am running CentOS7.6 with kvm 2.12 version. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mordred at inaugust.com Thu Mar 5 16:40:38 2020 From: mordred at inaugust.com (Monty Taylor) Date: Thu, 5 Mar 2020 10:40:38 -0600 Subject: [sdk] Additions and subtractions from core team In-Reply-To: <7DF33892-5254-472F-BC0E-07211A05253E@redhat.com> References: <7DF33892-5254-472F-BC0E-07211A05253E@redhat.com> Message-ID: <5A17C548-2F70-458D-B54E-91E160DF1AB0@inaugust.com> > On Mar 4, 2020, at 3:09 PM, Slawek Kaplonski wrote: > > Hi, > >> On 4 Mar 2020, at 17:56, Monty Taylor wrote: >> >> Heya, >> >> With the previous email about merging OSC and SDK teams, I’d also like to propose the following changes to the SDK core team (keeping in mind that likely means the core team of both OSC and SDK real soon now) >> >> Adds: >> >> Akihiro Motoki - The only OSC core not in sdk-core. amotoki should really be a core in all projects anyway >> Sean McGinnis - Sean has been reviewing things as a stable branch maint in both SDK and OSC, and as such has shown a good tendency to help things along when needed and to not approve things when he doesn’t know what’s up. > > Big +1 from me here. >> >> Subtractions: >> >> All of these people are awesome, but they’re all long gone: >> >> Brian Curtin >> Clint Byrum >> Everett Toews >> Jamie Lennox >> Jesse Noller >> Ricardo Carillo Cruz >> Richard Theis >> Rosario Di Somma >> Sam Yaple >> Terry Howe > > That’s sad but if that is necessary than ok. I agree, quite sad. :( >> >> Monty >> > > — > Slawek Kaplonski > Senior software engineer > Red Hat This has been done. From a.settle at outlook.com Thu Mar 5 16:45:55 2020 From: a.settle at outlook.com (Alexandra Settle) Date: Thu, 5 Mar 2020 16:45:55 +0000 Subject: [all][tc] Stepping down from TC Message-ID: Hi all, This should come as no shock as I have been relatively quite for some time now, but I will not standing for the Technical Committee for a second term. I have thoroughly enjoyed my tenure, learning so much about open source governance than I ever thought I needed 😉 My work takes me elsewhere, as it did several years ago, and I simply do not have the time to manage both. I encourage anyone who is interested in governance, or is passionate about OpenStack and wants to learn more, to stand for the TC elections. As was proven by my own nomination and subsequent successful election, you do not have to be "purely technical" to stand and be a part of something great. Diversity of skill is so important to our survival. Thanks to all those that have supported me to get to the point, I appreciate you all and will miss working intimately with the community. Please do not hesitate to reach out and ask any questions if you are interested in the positions available, happy to help encourage and answer any questions you may have. All the best, Alex ________________________________ Alexandra Settle Senior Technical Writer London, United Kingdom (GMT) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mordred at inaugust.com Thu Mar 5 16:50:36 2020 From: mordred at inaugust.com (Monty Taylor) Date: Thu, 5 Mar 2020 10:50:36 -0600 Subject: [ansible-sig][kolla][openstack-ansible][osc][sdk][tripleo] OpenStack modules broken in Ansible 2.8.9 In-Reply-To: References: Message-ID: <08B1D0C9-E613-45D3-8A07-E5B7C8270326@inaugust.com> > On Mar 5, 2020, at 9:27 AM, Mark Goddard wrote: > > Hi, > > The 2.8.9 release of Ansible has a regression [1] which breaks the > OpenStack modules. I've proposed a simple fix, hopefully it will be > included in a 2.8.10 release soon but in the meantime you may need to > blacklist 2.8.9. > > [1] https://github.com/ansible/ansible/issues/68042 > [2] https://github.com/ansible/ansible/pull/68043 We have jobs in OpenDev Zuul that are supposed to help catch this sort of thing … and they weren’t configured to run on stable-2.8. :( That has been rectified, as well as stable-2.9. For post-2.9 the modules live in OpenDev Gerrit in the new ansible-collections-openstack collection - so we should be in a better position to keep stuff covered. From satish.txt at gmail.com Thu Mar 5 17:11:37 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 5 Mar 2020 12:11:37 -0500 Subject: CPU Topology confusion In-Reply-To: References: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> Message-ID: Eddie, I have tried everything to match or fix CPU Topology layout but its never come down to correct as i mentioned in screenshot, I have check on Alicloud and they are also running KVM and their virtual machine lstopo output is really match with physical machine, like L1i / L1d cache layout etc. if you look at following output its strange i am using "-cpu host" option but still there are lots of missing flags on my virtual machine cpuinfo, is that normal? This is my VM output (virtual machine) # grep flags /proc/cpuinfo | uniq flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm arat fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt This is compute machine (physical host) # grep flags /proc/cpuinfo | uniq flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d On Thu, Mar 5, 2020 at 11:26 AM Eddie Yen wrote: > > Hi Satish, > > Since you already set "cpu_mode = host-passthrough", there's no need > to set cpu_model. > > BTW, we're not known about the CPU topology a lot. But IME we always > set "hw_cpu_sockets = 2" in specified image or flavor metadata if running > Windows instance. In default, KVM always allocate all vcpus into sockets > in CPU topology, and this will affect the Windows VM performance since > Windows only support maximum 2 CPU sockets. > > Perhaps you can try limit socket numbers by setting hw_cpu_sockets in > image metadata (or hw:cpu_sockets in flavor metadata.) > > Satish Patel 於 2020年3月5日 週四 下午10:46寫道： >> >> >> cpu_mode = cpu-passthrough >> cpu_model = none >> >> Do you think cpu_model make difference ? >> >> >> Sent from my iPhone >> >> On Mar 5, 2020, at 7:18 AM, Satish Patel wrote: >> >>  >> >> cpu-passthrough >> >> Sent from my iPhone >> >> On Mar 4, 2020, at 9:24 PM, rui zang wrote: >> >>  >> Hi, >> >> What is the value for the "cpu_mode" configuration option? >> https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.html >> >> Thanks, >> Zang, Rui >> >> >> 05.03.2020, 01:24, "Satish Patel" : >> >> Folks, >> >> We are running openstack with KVM and i have noticed kvm presenting >> wrong CPU Tolopoly to VM and because of that we are seeing bad >> performance to our application. >> >> This is openstack compute: >> >> # lstopo-no-graphics --no-io >> Machine (64GB total) >> NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) >> L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 >> PU L#0 (P#0) >> PU L#1 (P#20) >> L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 >> PU L#2 (P#1) >> PU L#3 (P#21) >> >> This is VM running on above compute >> >> # lstopo-no-graphics --no-io >> Machine (59GB total) >> NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) >> L2 L#0 (4096KB) + Core L#0 >> L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) >> L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) >> L2 L#1 (4096KB) + Core L#1 >> L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) >> L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) >> >> if you noticed P#0 and P#1 has own (32KB) cache per thread that is >> wrong presentation if you compare with physical CPU. >> >> This is a screenshot of AWS vs Openstack CPU Topology and looking at >> openstack its presentation is little odd, is that normal? >> >> https://imgur.com/a/2sPwJVC >> >> I am running CentOS7.6 with kvm 2.12 version. >> From whayutin at redhat.com Thu Mar 5 17:14:31 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Thu, 5 Mar 2020 10:14:31 -0700 Subject: [ansible-sig][kolla][openstack-ansible][osc][sdk][tripleo] OpenStack modules broken in Ansible 2.8.9 In-Reply-To: <08B1D0C9-E613-45D3-8A07-E5B7C8270326@inaugust.com> References: <08B1D0C9-E613-45D3-8A07-E5B7C8270326@inaugust.com> Message-ID: On Thu, Mar 5, 2020 at 9:51 AM Monty Taylor wrote: > > > > On Mar 5, 2020, at 9:27 AM, Mark Goddard wrote: > > > > Hi, > > > > The 2.8.9 release of Ansible has a regression [1] which breaks the > > OpenStack modules. I've proposed a simple fix, hopefully it will be > > included in a 2.8.10 release soon but in the meantime you may need to > > blacklist 2.8.9. > > > > [1] https://github.com/ansible/ansible/issues/68042 > > [2] https://github.com/ansible/ansible/pull/68043 > > We have jobs in OpenDev Zuul that are supposed to help catch this sort of > thing … and they weren’t configured to run on stable-2.8. :( That has been > rectified, as well as stable-2.9. > > For post-2.9 the modules live in OpenDev Gerrit in the new > ansible-collections-openstack collection - so we should be in a better > position to keep stuff covered. > > > Thanks for the heads up and for fixing the tests you guys!! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mordred at inaugust.com Thu Mar 5 17:54:49 2020 From: mordred at inaugust.com (Monty Taylor) Date: Thu, 5 Mar 2020 11:54:49 -0600 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK Message-ID: Heya, I’d like to try something. I’d like to try adding some project-specific people to the core team so that they can more directly help maintain the support for their service in SDK. In some of these cases the person I’m suggestion has next to no review experience in SDK. I think let’s be fine with that for now - we’re still a 2x +2 in general thing - but I know currently when reviewing neutron or ironic changes I always want to see a +2 from slaweq or dtantsur … so in the spirit of trying new things and trying to move the project forward in a healthy and welcoming way - how about we give this a try? The idea here is that we’re trusting people to use their good judgement and to only use their new +2 powers for good in their project. Over time, if they feel like they’ve gotten a handle on things more widely, there’s nothing stopping them from reviewing other patches - but I think that most of us aren’t looking for additional review work anyway. Specifically this would be: Shogo Saito - congress Adam Harwell - octavia Graham Hayes - designate Bharat Kumar - magnum Erik Olof Gunnar Andersson - senlin Tim Burke - swift I think we should also add a file in the repo that lists “subject matter experts” for each service we’ve got support for, where we have them. My list of current cores who I’d ping for specific service suitability are: Sean McGinnis - cinder Slawek Kaplonski - neutron Dmitry Tantsur - ironic Eric Fried - nova (at least until tomorrow my friend) How does that sound to folks? Monty From kennelson11 at gmail.com Thu Mar 5 18:01:53 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 5 Mar 2020 10:01:53 -0800 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: References: Message-ID: Seems like a good way to kind of unblock/get things moving a little faster for some more projects. Also, we did something similar to this in releases where a few new cores were added, but we were instructed to not +W things. Basically we could give the first +2 to move things along, but weren't making the merge decision and I would say its gone rather well. -Kendall (diablo_rojo) On Thu, Mar 5, 2020 at 9:56 AM Monty Taylor wrote: > Heya, > > I’d like to try something. > > I’d like to try adding some project-specific people to the core team so > that they can more directly help maintain the support for their service in > SDK. In some of these cases the person I’m suggestion has next to no review > experience in SDK. I think let’s be fine with that for now - we’re still a > 2x +2 in general thing - but I know currently when reviewing neutron or > ironic changes I always want to see a +2 from slaweq or dtantsur … so in > the spirit of trying new things and trying to move the project forward in a > healthy and welcoming way - how about we give this a try? > > The idea here is that we’re trusting people to use their good judgement > and to only use their new +2 powers for good in their project. Over time, > if they feel like they’ve gotten a handle on things more widely, there’s > nothing stopping them from reviewing other patches - but I think that most > of us aren’t looking for additional review work anyway. > > Specifically this would be: > > Shogo Saito - congress > Adam Harwell - octavia > Graham Hayes - designate > Bharat Kumar - magnum > Erik Olof Gunnar Andersson - senlin > Tim Burke - swift > > I think we should also add a file in the repo that lists “subject matter > experts” for each service we’ve got support for, where we have them. My > list of current cores who I’d ping for specific service suitability are: > > Sean McGinnis - cinder > Slawek Kaplonski - neutron > Dmitry Tantsur - ironic > Eric Fried - nova (at least until tomorrow my friend) > > How does that sound to folks? > > Monty > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Thu Mar 5 18:29:57 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 5 Mar 2020 10:29:57 -0800 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <1da63dab-c35e-6377-d5d8-e075a5c37408@ham.ie> References: <1da63dab-c35e-6377-d5d8-e075a5c37408@ham.ie> Message-ID: On Thu, Mar 5, 2020 at 3:07 AM Graham Hayes wrote: > On 04/03/2020 22:43, Zane Bitter wrote: > > On 4/03/20 3:08 pm, Ben Nemec wrote: > >> > >> > >> On 3/4/20 12:57 PM, Zane Bitter wrote: > >>> One cannot help wondering if we might get more Nova cores willing to > >>> sign up for a 1-week commitment to be the "PTL" than we're getting > >>> for a 6-months-and-maybe-indefinitely commitment. > >> > >> That's a really interesting idea. I'm not sure I'd want to go as short > >> as one week for PTL, but shortening the term might make it easier for > >> people to commit. > > > > The key would be to make it short enough that you can be 100% confident > > the next person will take over and not leave you holding the bag > > forever. (Hi Rico!) > > And also that the person you hand it off too won't have to hand it back. > (Hi Tim!) > > > I've no idea where the magic number would fall, and it's probably > > different for every team. I'm reasonably confident it's somewhere > > between 1 week and 6 months though. > > Yeah - I am not sure the TC should mandate a number - some teams > might be OK with the 6 months, while others will need to do 1 or 2 weeks > > I would like to think elections would NOT get held every 1-2 weeks or whatever the chosen PTL term is for a project? Its just a like...signup sheet sort of thing? What if more than one person wants to sign up for the same week( I can't think of why this would happen, just thinking about all the details)? -Kendall (diablo_rojo) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr at ham.ie Thu Mar 5 18:53:01 2020 From: gr at ham.ie (Graham Hayes) Date: Thu, 5 Mar 2020 18:53:01 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <1da63dab-c35e-6377-d5d8-e075a5c37408@ham.ie> Message-ID: On 05/03/2020 18:29, Kendall Nelson wrote: > > > On Thu, Mar 5, 2020 at 3:07 AM Graham Hayes > wrote: > > On 04/03/2020 22:43, Zane Bitter wrote: > > On 4/03/20 3:08 pm, Ben Nemec wrote: > >> > >> > >> On 3/4/20 12:57 PM, Zane Bitter wrote: > >>> One cannot help wondering if we might get more Nova cores > willing to > >>> sign up for a 1-week commitment to be the "PTL" than we're getting > >>> for a 6-months-and-maybe-indefinitely commitment. > >> > >> That's a really interesting idea. I'm not sure I'd want to go as > short > >> as one week for PTL, but shortening the term might make it > easier for > >> people to commit. > > > > The key would be to make it short enough that you can be 100% > confident > > the next person will take over and not leave you holding the bag > > forever. (Hi Rico!) > > And also that the person you hand it off too won't have to hand it back. > (Hi Tim!) > > > I've no idea where the magic number would fall, and it's probably > > different for every team. I'm reasonably confident it's somewhere > > between 1 week and 6 months though. > > Yeah - I am not sure the TC should mandate a number - some teams > might be OK with the 6 months, while others will need to do 1 or 2 weeks > > > I would like to think elections would NOT get held every 1-2 weeks or > whatever the chosen PTL term is for a project? Its just a like...signup > sheet sort of thing? What if more than one person wants to sign up for > the same week( I can't think of why this would happen, just thinking > about all the details)? I will admit to not thinking the whole way through to elections ... Yes - ideally, it would not be an election every 2 weeks - that would be an insane overhead. How to resolve conflicts in this is more problematic alright .... > -Kendall (diablo_rojo) From duc.openstack at gmail.com Thu Mar 5 19:14:53 2020 From: duc.openstack at gmail.com (Duc Truong) Date: Thu, 5 Mar 2020 11:14:53 -0800 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: References: Message-ID: +1 from me for Erik on Senlin. On Thu, Mar 5, 2020 at 10:02 AM Kendall Nelson wrote: > > Seems like a good way to kind of unblock/get things moving a little faster for some more projects. Also, we did something similar to this in releases where a few new cores were added, but we were instructed to not +W things. Basically we could give the first +2 to move things along, but weren't making the merge decision and I would say its gone rather well. > > -Kendall (diablo_rojo) > > On Thu, Mar 5, 2020 at 9:56 AM Monty Taylor wrote: >> >> Heya, >> >> I’d like to try something. >> >> I’d like to try adding some project-specific people to the core team so that they can more directly help maintain the support for their service in SDK. In some of these cases the person I’m suggestion has next to no review experience in SDK. I think let’s be fine with that for now - we’re still a 2x +2 in general thing - but I know currently when reviewing neutron or ironic changes I always want to see a +2 from slaweq or dtantsur … so in the spirit of trying new things and trying to move the project forward in a healthy and welcoming way - how about we give this a try? >> >> The idea here is that we’re trusting people to use their good judgement and to only use their new +2 powers for good in their project. Over time, if they feel like they’ve gotten a handle on things more widely, there’s nothing stopping them from reviewing other patches - but I think that most of us aren’t looking for additional review work anyway. >> >> Specifically this would be: >> >> Shogo Saito - congress >> Adam Harwell - octavia >> Graham Hayes - designate >> Bharat Kumar - magnum >> Erik Olof Gunnar Andersson - senlin >> Tim Burke - swift >> >> I think we should also add a file in the repo that lists “subject matter experts” for each service we’ve got support for, where we have them. My list of current cores who I’d ping for specific service suitability are: >> >> Sean McGinnis - cinder >> Slawek Kaplonski - neutron >> Dmitry Tantsur - ironic >> Eric Fried - nova (at least until tomorrow my friend) >> >> How does that sound to folks? >> >> Monty From mike243512 at gmail.com Thu Mar 5 13:24:13 2020 From: mike243512 at gmail.com (Mike Manning) Date: Thu, 5 Mar 2020 08:24:13 -0500 Subject: [training-labs] Current training lab for Rocky not working on Windows Message-ID: Hello, I've attempted to install the training-labs for the Rocky release onto my Windows 10 laptop on VirtualBox 6.0, but it is not working properly. The windows batch files are not functioning properly, and at least one reason is that the shell scripts that the batch files are copying are using "ifup" and "ifdown" which are not supported in Ubuntu 18.04. This one issue seems to imply that the scripts and batch files may need some review. Also, after install the ifupdown package on the system, the batch files and shell scripts still get hung up often on the existence of a file named "done" that never gets created. Can someone help me with this installation of the training-labs software in preparation for the COA exam? Mike Manning mike243512 at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Thu Mar 5 19:49:11 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Thu, 5 Mar 2020 14:49:11 -0500 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> Message-ID: <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita wrote ---- > > Hello QA team and devstack-plugin-ceph-core people, > > > > The Cinder team has some proposals we'd like to float. > > > > 1. The Cinder team is interested in becoming more active in the > > maintenance of openstack/devstack-plugin-ceph [0]. Currently, the > > devstack-plugin-ceph-core is > > https://review.opendev.org/#/admin/groups/1196,members > > The cinder-core is already represented by Eric and Sean; we'd like to > > replace them by including the cinder-core group. > > +1. This is good diea and make sense, I will do the change. Great, thanks! > > > > 2. The Cinder team is interested in becoming more active in the > > maintenance of x/devstack-plugin-nfs [1]. Currently, the > > devstack-plugin-nfs-core is > > https://review.opendev.org/#/admin/groups/1330,members > > It's already 75% cinder-core members; we'd like to replace the > > individual members with the cinder-core group. We also propose that > > devstack-core be added as an included group. > > > > 3. The Cinder team is interested in implementing a new devstack plugin: > > openstack/devstack-plugin-open-cas > > This will enable thorough testing of a new feature [2] being introduced > > as experimental in Ussuri and expected to be finalized in Victoria. Our > > plan would be to make both cinder-core and devstack-core included groups > > for the gerrit group governing the new plugin. > > +1. You want this under Cinder governance or under QA ? I think it makes sense for these to be under QA governance -- QA would own the repo with both QA and Cinder having permission to make changes. > > > > 4. This is a minor point, but can the devstack-plugin-nfs repo be moved > > back into the 'openstack' namespace? > > If this is usable plugin for nfs testing (I am not aware if we have any other) then > it make sense to bring it to openstack governance. > Same question here, do you want to put this under Cinder governance or QA. Same here, I think QA should "own" the repo, but Cinder will have permission to make changes there. > > Those plugins under QA governance also ok for me with your proposal of calloborative maintainance by > devstack-core and cinder-core. > > -gmann Thanks for the quick response! > > > > Let us know which of these proposals you find acceptable. > > > > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > > [1] https://opendev.org/x/devstack-plugin-nfs > > [2] https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > > > > From johnsomor at gmail.com Thu Mar 5 19:54:20 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Thu, 5 Mar 2020 11:54:20 -0800 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: <2c8ccd0c2d2d7a8ae6073de8b9fc80656fa49ce0.camel@redhat.com> References: <2c8ccd0c2d2d7a8ae6073de8b9fc80656fa49ce0.camel@redhat.com> Message-ID: We have been drifting this way for a while, so "yes please". Michael On Thu, Mar 5, 2020 at 2:59 AM Stephen Finucane wrote: > > On Wed, 2020-03-04 at 10:19 -0600, Monty Taylor wrote: > > Hey everybody, > > > > I'd like to propose merging the SDK and OSC teams. We already share > > an IRC channel, and already share a purpose in life. In OSC we have a > > current goal of swapping out client implementation for SDK, and we're > > Already ensuring that SDK does what it needs to do to facilitate that > > goal. We also already share PTG space, and have requested a shared > > set of time at the upcoming Denver PTG. So really the separation is > > historical not practical, and these days having additional layers of > > governance is not super useful. > > > > I propose that we do a simple merge of the teams. This means the > > current SDK cores will become cores on OSC, and as most of the OSC > > cores are already SDK cores, it means the SDK team gains amotoki - > > which is always a positive. > > Big +1 > > > Dean hasn't had time to spend on OSC quite a bit, sadly, and while we > > remain hopeful that this will change, we’re slowly coming to terms > > with the possibility that it might not. With that in mind, I'll serve > > as the PTL for the new combined team until the next election. > > > > Monty > > > > From nate.johnston at redhat.com Thu Mar 5 19:54:32 2020 From: nate.johnston at redhat.com (Nate Johnston) Date: Thu, 5 Mar 2020 14:54:32 -0500 Subject: [all][tc] Stepping down from TC In-Reply-To: References: Message-ID: <20200305195432.qdsmkyr7jebx3y5c@firewall> Thank you for everything you have done over the years Alex. When I went to my first summit (Tokyo), you and Lana were two of the first people I met and your warm welcome helped put me on a course to become a part of the community. Thank you for being a person always looking out for the user and the newbie and making all of OpenStack better for it. Nate On Thu, Mar 05, 2020 at 04:45:55PM +0000, Alexandra Settle wrote: > Hi all, > > This should come as no shock as I have been relatively quite for some time > now, but I will not standing for the Technical Committee for a second term. > > I have thoroughly enjoyed my tenure, learning so much about open source > governance than I ever thought I needed 😉 > > My work takes me elsewhere, as it did several years ago, and I simply do not have > the time to manage both. > > I encourage anyone who is interested in governance, or is passionate about OpenStack > and wants to learn more, to stand for the TC elections. As was proven by my own > nomination and subsequent successful election, you do not have to be "purely technical" > to stand and be a part of something great. Diversity of skill is so important to our > survival. > > Thanks to all those that have supported me to get to the point, I appreciate you all and > will miss working intimately with the community. > > Please do not hesitate to reach out and ask any questions if you are interested in the > positions available, happy to help encourage and answer any questions you may have. > > All the best, > > Alex > > ________________________________ > Alexandra Settle > Senior Technical Writer > London, United Kingdom (GMT) > From flux.adam at gmail.com Thu Mar 5 20:11:05 2020 From: flux.adam at gmail.com (Adam Harwell) Date: Fri, 6 Mar 2020 05:11:05 +0900 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> Message-ID: Well, part of maintaining feature parity is that the features should be added to the OSC by the project team at the time they're made -- you're already doing the work to add them to your own client, so instead, do the same amount of work but add them in OSC instead! It doesn't seem incredibly onerous to me. If the OSC plugin for your project IS the official client, then there's no duplication of effort. I think saying "someone else had better implement our features in a timely fashion" is a bit irresponsible. Though, this is coming from working on a project where we aren't used to being included as a "core piece" and having any work done for us, ever... Also, things are also definitely moving in a better direction now with the probable addition of project team liasons as cores in SDK/OSC, which should alleviate a lot of the issues around "response time" on reviews, when you do put in the effort to add features yourself. --Adam On Fri, Mar 6, 2020, 00:15 Erno Kuvaja wrote: > On Wed, Mar 4, 2020 at 1:19 AM Ghanshyam Mann > wrote: > >> ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell >> wrote ---- >> > >> > >> > On 3 Mar 2020, at 19:55, Tim Bell wrote: >> > >> > >> > On 3 Mar 2020, at 19:20, Albert Braden >> wrote: >> > Sean, thank you for clarifying that. >> > >> > Was my understanding that the community decided to focus on the >> unified client incorrect? Is the unified/individual client debate still a >> matter of controversy? Is it possible that the unified client will be >> deprecated in favor of individual clients after more discussion? I haven’t >> looked at any of the individual clients since 2018 (except for >> osc-placement which is kind of a special case), because I thought they were >> all going away and could be safely ignored until they did, and I haven’t >> included any information about the individual clients in the documentation >> that I write for our users, and if they ask I have been telling them to not >> use the individual clients. Do I need to start looking at individual >> clients again, and telling our users to use them in some cases? >> > >> > >> > >> > I remember a forum discussion where a community goal was proposed to >> focus on OSC rather than individual project CLIs (I think Matt and I were >> proposers). There were concerns on the effort to do this and that it would >> potentially be multi-cycle. >> > BTW, I found the etherpad from Berlin ( >> https://etherpad.openstack.org/p/BER-t-series-goals) and the associated >> mailing list discussion at >> http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html >> >> Yeah, we are in process of selecting the Victoria cycle community-wide >> goal and this can be good candidate. I agree with the idea/requirement of a >> multi-cycle goal. >> Another option is to build a pop-up team for the Victoria cycle to start >> burning down the keys issues/work. For both ways (either goal or pop-up >> team), we need >> some set of people to drive it. If anyone would like to volunteer for >> this, we can start discussing the details. >> >> [1] >> http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html >> >> -gmann >> >> Yeah, lets propose this as community goal again as it worked so well last > time. ಠ_ಠ > > I think your most help wanted list/pop-up team is much more realistic > approach. Lets see if there is enough interest to actually make it happen. > What comes to our previous experience with Glance and moving to endorse > osc, I think I'm not alone stating that we can discuss this again after osc > has kept feature parity (and I mean to current release, not feature parity > 2 years ago kind of thing) and actively addressed raised issues at least > for a couple of cycles. Obviously if you/your users wants to use it > meanwhile, that your call. If we cannot get that level of commitment, how > do we expect to support this long term? > > I'm not willing to put our users through that misery again as it happened > last time as long as I'm core in this project. > > - jokke > > >> > >> > My experience in discussion with the CERN user community and other >> OpenStack operators is that OSC is felt to be the right solution for the >> end user facing parts of the cloud (admin commands could be another >> discussion if necessary). Experienced admin operators can remember that >> glance looks after images and nova looks after instances. Our average user >> can get very confused, especially given that OSC supports additional >> options for authentication (such as Kerberos and Certificates along with >> clouds.yaml) so users need to re-authenticate with a different openrc to >> work on their project. >> > While I understand there are limited resources all round, I would >> prefer that we focus on adding new project functions to OSC which will >> eventually lead to feature parity. >> > Attracting ‘drive-by’ contributions from operations staff for OSC work >> (it's more likely to be achieved if it makes the operations work less e.g. >> save on special end user documentation by contributing code). This is >> demonstrated from the CERN team contribution to the OSC ‘coe' and ‘share' >> functionality along with lots of random OSC updates as listed hat >> https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) >> >> > BTW, I also would vote for =auto as the default. >> > Tim >> > We are on Rocky now but I expect that we will upgrade as necessary to >> stay on supported versions. >> > >> > From: Sean McGinnis >> > Sent: Tuesday, March 3, 2020 9:50 AM >> > To: openstack-discuss at lists.openstack.org >> > Subject: Re: OSC future (formerly [glance] Different checksum between >> CLI and curl) >> > >> > On 3/3/20 11:28 AM, Albert Braden wrote: >> > Am I understanding correctly that the Openstack community decided to >> focus on the unified client, and to deprecate the individual clients, and >> that the Glance team did not agree with this decision, and that the Glance >> team is now having a pissing match with the rest of the community, and is >> unilaterally deciding to continue developing the Glance client and refusing >> to work on the unified client, or is something different going on? I would >> ask everyone involved to remember that we operators are down here, and the >> yellow rain falling on our heads does not smell very good. >> > I definitely would not characterize it that way. >> > With trying not to put too much personal bias into it, here's what I >> would say the situation is: >> > - Some part of the community has said OSC should be the only CLI and >> that individual CLIs should go away >> > - Glance is a very small team with very, very limited resources >> > - The OSC team is a very small team with very, very limited resources >> > - CLI capabilities need to be exposed for Glance changes and the >> easiest way to get them out for the is by updating the Glance CLI >> > - No one from the OSC team has been able to proactively help to make >> sure these changes make it into the OSC client (see bullet 3) >> > - There exists a sizable functionality gap between per-project CLIs >> and what OSC provides, and although a few people have done a lot of great >> work to close that gap, there is still a lot to be done and does not appear >> the gap will close at any point in the near future based on the current >> trends >> > >> > >> > >> > >> > >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From openstack at nemebean.com Thu Mar 5 20:24:54 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 5 Mar 2020 14:24:54 -0600 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <1da63dab-c35e-6377-d5d8-e075a5c37408@ham.ie> Message-ID: On 3/5/20 12:29 PM, Kendall Nelson wrote: > > > On Thu, Mar 5, 2020 at 3:07 AM Graham Hayes > wrote: > > On 04/03/2020 22:43, Zane Bitter wrote: > > On 4/03/20 3:08 pm, Ben Nemec wrote: > >> > >> > >> On 3/4/20 12:57 PM, Zane Bitter wrote: > >>> One cannot help wondering if we might get more Nova cores > willing to > >>> sign up for a 1-week commitment to be the "PTL" than we're getting > >>> for a 6-months-and-maybe-indefinitely commitment. > >> > >> That's a really interesting idea. I'm not sure I'd want to go as > short > >> as one week for PTL, but shortening the term might make it > easier for > >> people to commit. > > > > The key would be to make it short enough that you can be 100% > confident > > the next person will take over and not leave you holding the bag > > forever. (Hi Rico!) > > And also that the person you hand it off too won't have to hand it back. > (Hi Tim!) > > > I've no idea where the magic number would fall, and it's probably > > different for every team. I'm reasonably confident it's somewhere > > between 1 week and 6 months though. > > Yeah - I am not sure the TC should mandate a number - some teams > might be OK with the 6 months, while others will need to do 1 or 2 weeks > > > I would like to think elections would NOT get held every 1-2 weeks or > whatever the chosen PTL term is for a project? Its just a like...signup > sheet sort of thing? What if more than one person wants to sign up for > the same week( I can't think of why this would happen, just thinking > about all the details)? Yeah, this logistical problem is one of the reasons I didn't want it to be a one week rotation. I guess maybe you could solve that by holding elections each cycle, but selecting a pool of PTLs who could then trade off as desired, but at that point I feel like you're back to not having a PTL and instead having a maintainer team. It also seems like a potentially complicated election system. Plus it kind of introduces a problem with the PTL being the point of contact for the project. If it's changing weekly then you lose most of the benefits of having a single person other people know they can go to. I'm also wondering if this actually solves the "Hi Rico!" case anyway. :-) If a PTL leaves the position because their focus has changed, they aren't likely to take it back even if the new person only technically signs up for a week. When your PTL candidate pool is exactly 1 then it doesn't matter if they're getting elected weekly or six monthly. I guess the idea is for this to increase the pool size, and whether that will work probably depends on the circumstances for each project. Anyway, I still think it's an interesting idea, even if I have come up with a few new concerns after further consideration. Maybe it's something a few teams could experiment with initially and see if it works and what timeframes are appropriate? > > -Kendall (diablo_rojo) From anlin.kong at gmail.com Thu Mar 5 20:54:53 2020 From: anlin.kong at gmail.com (Lingxian Kong) Date: Fri, 6 Mar 2020 09:54:53 +1300 Subject: [trove][charm] RabbitMQ Connection Error In-Reply-To: <28295851583412044@myt6-636ea6dfd460.qloud-c.yandex.net> References: <28295851583412044@myt6-636ea6dfd460.qloud-c.yandex.net> Message-ID: Hi Tolga, I am not familiar with how Trove Charm works, but I've never seen such issue in my devstack for Trove testing. I would recommend you install Trove by devstack and check differences of the config options between installations. Sorry maybe I didn't provide much help but please don't hesitate to let me know if you have any other questions. - Best regards, Lingxian Kong Catalyst Cloud On Fri, Mar 6, 2020 at 1:45 AM wrote: > Hi everyone, > > I updated retired Trove Charm. The deployment runs on bionic-train release. > > I able to run trove-api successfully however trove-conductor and > trove-taskmanager services are returning following error: > > > ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno > 111] ECONNREFUSED (retrying in 30.0 seconds): ConnectionRefusedError: > [Errno 111] ECONNREFUSED > > > I validated trove user has registered to RabbitMQ successfully and it can > be reachable from trove node. > > I have also checked active connections via netstat -nputw command but > there is no sign to connection attempt to RabbitMQ Server. > > Here is latest trove.conf for reference. > > http://paste.openstack.org/show/790336/ > > I also noticed that both services are using trove.conf instead of other > conf files like trove-taskmanager.conf. > > Thanks, > > PS: I would like to start to maintain charm-trove project. Please also > inform me about the process. > > -- > Tolga KAPROL > ETOM Teknoloji ARGE > Etom.io > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Thu Mar 5 22:26:16 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 05 Mar 2020 16:26:16 -0600 Subject: [all][tc] Stepping down from TC In-Reply-To: References: Message-ID: <170acce76ec.c8b9b5e4488373.3611812206878074759@ghanshyammann.com> Thanks, Alex for all the contributions to TC. It was great working with you. -gmann ---- On Thu, 05 Mar 2020 10:45:55 -0600 Alexandra Settle wrote ---- > div.zm_-5094580936618874778_parse_-1866959152293090290 P { margin-top: 0; margin-bottom: 0 }Hi all, > This should come as no shock as I have been relatively quite for some timenow, but I will not standing for the Technical Committee for a second term. > I have thoroughly enjoyed my tenure, learning so much about open sourcegovernance than I ever thought I needed 😉 > My work takes me elsewhere, as it did several years ago, and I simply do not havethe time to manage both. > I encourage anyone who is interested in governance, or is passionate about OpenStackand wants to learn more, to stand for the TC elections. As was proven by my ownnomination and subsequent successful election, you do not have to be "purely technical"to stand and be a part of something great. Diversity of skill is so important to oursurvival. > Thanks to all those that have supported me to get to the point, I appreciate you all andwill miss working intimately with the community. > Please do not hesitate to reach out and ask any questions if you are interested in thepositions available, happy to help encourage and answer any questions you may have. > All the best, > Alex > > Alexandra Settle > Senior Technical Writer > London, United Kingdom (GMT) > From melwittt at gmail.com Thu Mar 5 22:57:29 2020 From: melwittt at gmail.com (melanie witt) Date: Thu, 5 Mar 2020 14:57:29 -0800 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> Message-ID: On 3/5/20 12:11, Adam Harwell wrote: > Well, part of maintaining feature parity is that the features should be > added to the OSC by the project team at the time they're made -- you're > already doing the work to add them to your own client, so instead, do > the same amount of work but add them in OSC instead! It doesn't seem > incredibly onerous to me. If the OSC plugin for your project IS the > official client, then there's no duplication of effort. I think saying > "someone else had better implement our features in a timely fashion" is > a bit irresponsible. Though, this is coming from working on a project > where we aren't used to being included as a "core piece" and having any > work done for us, ever... I think this is the key point regarding the lack of feature parity in OSC for some projects. If you are a new-enough project to have begun your CLI as an OSC plugin (examples: ironic client, placement client, and more) then adding a feature to the client is one shot. You add support in the plugin and you're done. If you are an older project (examples: nova client, glance client) then you have a two-step process for adding a feature to OSC. For older projects, OSC imports the legacy clients and calls their python bindings to make the API calls. So for nova, if you want to add a feature to the client, you have to add it to the legacy nova client. This is required. Then, to add it to OSC you have to add it to OSC and have OSC call the newly added legacy binding for the feature. This is [technically] optional. This is why parity is missing. It pains me a bit to write it ^ because you may be thinking, "what's so difficult about going the extra step to add a feature to OSC after adding it to nova client?" I don't know. Maybe people are too stressed and busy. If it's not "required", it gets deferred. Maybe people don't feel familiar enough with OSC to add the feature there as well. There could be a lot of different reasons. So, not trying to make excuses here but just sharing my opinion on why adding features to OSC is not so simple for some projects. Cheers, -melanie > Also, things are also definitely moving in a better direction now with > the probable addition of project team liasons as cores in SDK/OSC, which > should alleviate a lot of the issues around "response time" on reviews, > when you do put in the effort to add features yourself. > >     --Adam > > On Fri, Mar 6, 2020, 00:15 Erno Kuvaja > wrote: > > On Wed, Mar 4, 2020 at 1:19 AM Ghanshyam Mann > > wrote: > > ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell > > wrote ---- >  > >  > >  > On 3 Mar 2020, at 19:55, Tim Bell > wrote: >  > >  > >  > On 3 Mar 2020, at 19:20, Albert Braden > > > wrote: >  > Sean, thank you for clarifying that. >  > >  > Was my understanding that the community decided to focus on > the unified client incorrect? Is the unified/individual client > debate still a matter of controversy? Is it possible that the > unified client will be deprecated in favor of individual clients > after more discussion? I haven’t looked at any of the individual > clients since 2018 (except for osc-placement which is kind of a > special case), because I thought they were all going away and > could be safely ignored until they did, and I haven’t included > any information about the individual clients in the > documentation that I write for our users, and if they ask I have > been telling them to not use the individual clients. Do I need > to start looking at individual clients again, and telling our > users to use them in some cases? >  > >  > >  > >  > I remember a forum discussion where a community goal was > proposed to focus on OSC rather than individual project CLIs (I > think Matt and I were proposers).  There were concerns on the > effort to do this and that it would potentially be multi-cycle. >  > BTW, I found the etherpad from Berlin > (https://etherpad.openstack.org/p/BER-t-series-goals) and the > associated mailing list discussion at > http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html > > Yeah, we are in process of selecting the Victoria cycle > community-wide goal and this can be good candidate. I agree with > the idea/requirement of a multi-cycle goal. > Another option is to build a pop-up team for the Victoria cycle > to start burning down the keys issues/work. For both ways > (either goal or pop-up team), we need > some set of people to drive it. If anyone would like to > volunteer for this, we can start discussing the details. > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html > > -gmann > > Yeah, lets propose this as community goal again as it worked so well > last time. |ಠ_ಠ| > > I think your most help wanted list/pop-up team is much more > realistic approach. Lets see if there is enough interest to actually > make it happen. What comes to our previous experience with Glance > and moving to endorse osc, I think I'm not alone stating that we can > discuss this again after osc has kept feature parity (and I mean to > current release, not feature parity 2 years ago kind of thing) and > actively addressed raised issues at least for a couple of cycles. > Obviously if you/your users wants to use it meanwhile, that your > call. If we cannot get that level of commitment, how do we expect to > support this long term? > > I'm not willing to put our users through that misery again as it > happened last time as long as I'm core in this project. > > - jokke > >  > >  > My experience in discussion with the CERN user community and > other OpenStack operators is that OSC is felt to be the right > solution for the end user facing parts of the cloud (admin > commands could be another discussion if necessary). Experienced > admin operators can remember that glance looks after images and > nova looks after instances. Our average user can get very > confused, especially given that OSC supports additional options > for authentication (such as Kerberos and Certificates along with > clouds.yaml) so users need to re-authenticate with a different > openrc to work on their project. >  > While I understand there are limited resources all round, I > would prefer that we focus on adding new project functions to > OSC which will eventually lead to feature parity. >  > Attracting ‘drive-by’ contributions from operations staff > for OSC work (it's more likely to be achieved if it makes the > operations work less e.g. save on special end user documentation > by contributing code).  This is demonstrated from the CERN team > contribution to the OSC  ‘coe' and ‘share' functionality along > with lots of random OSC updates as listed hat > https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient) > >  > BTW, I also would vote for =auto as the default. >  > Tim >  > We are on Rocky now but I expect that we will upgrade as > necessary to stay on supported versions. >  > >  > From: Sean McGinnis > >  > Sent: Tuesday, March 3, 2020 9:50 AM >  > To: openstack-discuss at lists.openstack.org > >  > Subject: Re: OSC future (formerly [glance] Different > checksum between CLI and curl) >  > >  > On 3/3/20 11:28 AM, Albert Braden wrote: >  > Am I understanding correctly that the Openstack community > decided to focus on the unified client, and to deprecate the > individual clients, and that the Glance team did not agree with > this decision, and that the Glance team is now having a pissing > match with the rest of the community, and is unilaterally > deciding to continue developing the Glance client and refusing > to work on the unified client, or is something different going > on? I would ask everyone involved to remember that we operators > are down here, and the yellow rain falling on our heads does not > smell very good. >  > I definitely would not characterize it that way. >  > With trying not to put too much personal bias into it, > here's what I would say the situation is: >  > - Some part of the community has said OSC should be the only > CLI and that individual CLIs should go away >  > - Glance is a very small team with very, very limited resources >  > - The OSC team is a very small team with very, very limited > resources >  > - CLI capabilities need to be exposed for Glance changes and > the easiest way to get them out for the is by updating the > Glance CLI >  > - No one from the OSC team has been able to proactively help > to make sure these changes make it into the OSC client (see > bullet 3) >  > - There exists a sizable functionality gap between > per-project CLIs and what OSC provides, and although a few > people have done a lot of great work to close that gap, there is > still a lot to be done and does not appear the gap will close at > any point in the near future based on the current trends >  > >  > >  > >  > >  > > From dtroyer at gmail.com Thu Mar 5 23:15:12 2020 From: dtroyer at gmail.com (Dean Troyer) Date: Thu, 5 Mar 2020 17:15:12 -0600 Subject: [osc][sdk] Merging OpenStack SDK and OpenStack Client teams In-Reply-To: References: Message-ID: On Wed, Mar 4, 2020 at 10:24 AM Monty Taylor wrote: > I’d like to propose merging the SDK and OSC teams. We already share an IRC channel, and already share a purpose in life. In OSC we have a current goal of swapping out client implementation for SDK, and we’re ++ > Dean hasn’t had time to spend on OSC quite a bit, sadly, and while we remain hopeful that this will change, we’re slowly coming to terms with the possibility that it might not. With that in mind, I’ll serve as the PTL for the new combined team until the next election. ++ This is the right move whether I am able to continue to work on OpenStack or not. Thank you for picking up results from a cab-ride-to-SFO conversation with Dolph. It is in good hands with the combined team. dt -- Dean Troyer dtroyer at gmail.com From johnsomor at gmail.com Fri Mar 6 00:53:05 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Thu, 5 Mar 2020 16:53:05 -0800 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: References: Message-ID: Naming names is a highly effective demotivator for those whose contributions are not recognized. https://www.stackalytics.com/?module=openstacksdk&metric=loc Maybe the project teams should select their own liaison? Michael On Thu, Mar 5, 2020 at 9:58 AM Monty Taylor wrote: > > Heya, > > I’d like to try something. > > I’d like to try adding some project-specific people to the core team so that they can more directly help maintain the support for their service in SDK. In some of these cases the person I’m suggestion has next to no review experience in SDK. I think let’s be fine with that for now - we’re still a 2x +2 in general thing - but I know currently when reviewing neutron or ironic changes I always want to see a +2 from slaweq or dtantsur … so in the spirit of trying new things and trying to move the project forward in a healthy and welcoming way - how about we give this a try? > > The idea here is that we’re trusting people to use their good judgement and to only use their new +2 powers for good in their project. Over time, if they feel like they’ve gotten a handle on things more widely, there’s nothing stopping them from reviewing other patches - but I think that most of us aren’t looking for additional review work anyway. > > Specifically this would be: > > Shogo Saito - congress > Adam Harwell - octavia > Graham Hayes - designate > Bharat Kumar - magnum > Erik Olof Gunnar Andersson - senlin > Tim Burke - swift > > I think we should also add a file in the repo that lists “subject matter experts” for each service we’ve got support for, where we have them. My list of current cores who I’d ping for specific service suitability are: > > Sean McGinnis - cinder > Slawek Kaplonski - neutron > Dmitry Tantsur - ironic > Eric Fried - nova (at least until tomorrow my friend) > > How does that sound to folks? > > Monty From missile0407 at gmail.com Fri Mar 6 01:15:43 2020 From: missile0407 at gmail.com (Eddie Yen) Date: Fri, 6 Mar 2020 09:15:43 +0800 Subject: CPU Topology confusion In-Reply-To: References: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> Message-ID: Hi Satish, Using host-passthrough on KVM is not only passthrough the physical host CPU model, but will also "try" passthrough the same CPU flags. That means the vcpu will contain the flags same as host CPU, but it still depends on how KVM can support. In other words, KVM will only set the flags what the host CPU have and what KVM itself can support. Since QEMU has released to 4.2.0, perhaps you can try the latest version and doing the pure KVM running to see if it bring up the performance and consider upgrade KVM on compute nodes. Satish Patel 於 2020年3月6日 週五 上午1:11寫道： > Eddie, > > I have tried everything to match or fix CPU Topology layout but its > never come down to correct as i mentioned in screenshot, I have check > on Alicloud and they are also running KVM and their virtual machine > lstopo output is really match with physical machine, like L1i / L1d > cache layout etc. > > if you look at following output its strange i am using "-cpu host" > option but still there are lots of missing flags on my virtual machine > cpuinfo, is that normal? > > This is my VM output (virtual machine) > > # grep flags /proc/cpuinfo | uniq > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov > pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm > constant_tsc arch_perfmon rep_good nopl xtopology eagerfpu pni > pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt > tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm > arat fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt > > This is compute machine (physical host) > > # grep flags /proc/cpuinfo | uniq > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx > pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl > xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor > ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 > sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c > rdrand lahf_lm abm epb invpcid_single intel_ppin ssbd ibrs ibpb stibp > tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 > smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida > arat pln pts md_clear spec_ctrl intel_stibp flush_l1d > > On Thu, Mar 5, 2020 at 11:26 AM Eddie Yen wrote: > > > > Hi Satish, > > > > Since you already set "cpu_mode = host-passthrough", there's no need > > to set cpu_model. > > > > BTW, we're not known about the CPU topology a lot. But IME we always > > set "hw_cpu_sockets = 2" in specified image or flavor metadata if running > > Windows instance. In default, KVM always allocate all vcpus into sockets > > in CPU topology, and this will affect the Windows VM performance since > > Windows only support maximum 2 CPU sockets. > > > > Perhaps you can try limit socket numbers by setting hw_cpu_sockets in > > image metadata (or hw:cpu_sockets in flavor metadata.) > > > > Satish Patel 於 2020年3月5日 週四 下午10:46寫道： > >> > >> > >> cpu_mode = cpu-passthrough > >> cpu_model = none > >> > >> Do you think cpu_model make difference ? > >> > >> > >> Sent from my iPhone > >> > >> On Mar 5, 2020, at 7:18 AM, Satish Patel wrote: > >> > >>  > >> > >> cpu-passthrough > >> > >> Sent from my iPhone > >> > >> On Mar 4, 2020, at 9:24 PM, rui zang wrote: > >> > >>  > >> Hi, > >> > >> What is the value for the "cpu_mode" configuration option? > >> > https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.html > >> > >> Thanks, > >> Zang, Rui > >> > >> > >> 05.03.2020, 01:24, "Satish Patel" : > >> > >> Folks, > >> > >> We are running openstack with KVM and i have noticed kvm presenting > >> wrong CPU Tolopoly to VM and because of that we are seeing bad > >> performance to our application. > >> > >> This is openstack compute: > >> > >> # lstopo-no-graphics --no-io > >> Machine (64GB total) > >> NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) > >> L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 > >> PU L#0 (P#0) > >> PU L#1 (P#20) > >> L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 > >> PU L#2 (P#1) > >> PU L#3 (P#21) > >> > >> This is VM running on above compute > >> > >> # lstopo-no-graphics --no-io > >> Machine (59GB total) > >> NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) > >> L2 L#0 (4096KB) + Core L#0 > >> L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) > >> L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) > >> L2 L#1 (4096KB) + Core L#1 > >> L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) > >> L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) > >> > >> if you noticed P#0 and P#1 has own (32KB) cache per thread that is > >> wrong presentation if you compare with physical CPU. > >> > >> This is a screenshot of AWS vs Openstack CPU Topology and looking at > >> openstack its presentation is little odd, is that normal? > >> > >> https://imgur.com/a/2sPwJVC > >> > >> I am running CentOS7.6 with kvm 2.12 version. > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Fri Mar 6 03:01:00 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 5 Mar 2020 22:01:00 -0500 Subject: CPU Topology confusion In-Reply-To: References: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> Message-ID: I am running CentOS 7.6 and it doesn't have RPM available for 4.x release i have to compile it or try to install fedora one. Let me tell you why i am doing all these exercise. We are planning to run Erlang (mongooseIM) application on openstack instance, before i move to production i started doing some load-testing on openstack vm and this is what i did. - I have HP Gen9 server which has 40 core CPU (with HT) so i add this machine in openstack and reserve 8 cpu for host using vcpu_pin_set. - I have created 32 vcpu core virtual machine on this compute node with --property hw:numa_nodes=2 option and also added dedicated and hugepage properties for best performance - In lstopo command i can see two numa with 2 socket / 8 core / 2 thread per numa topology on my guest VM - I have installed erlang (mongooseIM) application and start load-testing and found very very poor result (i would say worst) - For experiment i told erlang to bind all process to numa0 (0-16 vcpu) and found benchmark result was good much better. - Problem is if i run erlang on single numa then i am wasting my CPU core for second numa (i want to utilize all CPU to get best performance) - For experiment i have disable hyperthreading on openstack compute node and build new VM with hw:numa_node=2 option and found best result in benchmark. Now question is why erlang doesn't like dual numa openstack vm with HT enable, it seems erlang looking at CPU Topology and something is missing or broken in TOPOLOGY of kvm when trying to utilize both numa and result poor performance. last few days i am trying to solve this mystery and even i contact erlang developer but didn't get any help. On Thu, Mar 5, 2020 at 8:15 PM Eddie Yen wrote: > > Hi Satish, > > Using host-passthrough on KVM is not only passthrough the physical > host CPU model, but will also "try" passthrough the same CPU flags. > That means the vcpu will contain the flags same as host CPU, but it > still depends on how KVM can support. In other words, KVM will only > set the flags what the host CPU have and what KVM itself can support. > > Since QEMU has released to 4.2.0, perhaps you can try the latest > version and doing the pure KVM running to see if it bring up the > performance and consider upgrade KVM on compute nodes. > > Satish Patel 於 2020年3月6日 週五 上午1:11寫道： >> >> Eddie, >> >> I have tried everything to match or fix CPU Topology layout but its >> never come down to correct as i mentioned in screenshot, I have check >> on Alicloud and they are also running KVM and their virtual machine >> lstopo output is really match with physical machine, like L1i / L1d >> cache layout etc. >> >> if you look at following output its strange i am using "-cpu host" >> option but still there are lots of missing flags on my virtual machine >> cpuinfo, is that normal? >> >> This is my VM output (virtual machine) >> >> # grep flags /proc/cpuinfo | uniq >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov >> pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm >> constant_tsc arch_perfmon rep_good nopl xtopology eagerfpu pni >> pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt >> tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm >> arat fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt >> >> This is compute machine (physical host) >> >> # grep flags /proc/cpuinfo | uniq >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov >> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx >> pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl >> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor >> ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 >> sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c >> rdrand lahf_lm abm epb invpcid_single intel_ppin ssbd ibrs ibpb stibp >> tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 >> smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida >> arat pln pts md_clear spec_ctrl intel_stibp flush_l1d >> >> On Thu, Mar 5, 2020 at 11:26 AM Eddie Yen wrote: >> > >> > Hi Satish, >> > >> > Since you already set "cpu_mode = host-passthrough", there's no need >> > to set cpu_model. >> > >> > BTW, we're not known about the CPU topology a lot. But IME we always >> > set "hw_cpu_sockets = 2" in specified image or flavor metadata if running >> > Windows instance. In default, KVM always allocate all vcpus into sockets >> > in CPU topology, and this will affect the Windows VM performance since >> > Windows only support maximum 2 CPU sockets. >> > >> > Perhaps you can try limit socket numbers by setting hw_cpu_sockets in >> > image metadata (or hw:cpu_sockets in flavor metadata.) >> > >> > Satish Patel 於 2020年3月5日 週四 下午10:46寫道： >> >> >> >> >> >> cpu_mode = cpu-passthrough >> >> cpu_model = none >> >> >> >> Do you think cpu_model make difference ? >> >> >> >> >> >> Sent from my iPhone >> >> >> >> On Mar 5, 2020, at 7:18 AM, Satish Patel wrote: >> >> >> >>  >> >> >> >> cpu-passthrough >> >> >> >> Sent from my iPhone >> >> >> >> On Mar 4, 2020, at 9:24 PM, rui zang wrote: >> >> >> >>  >> >> Hi, >> >> >> >> What is the value for the "cpu_mode" configuration option? >> >> https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.html >> >> >> >> Thanks, >> >> Zang, Rui >> >> >> >> >> >> 05.03.2020, 01:24, "Satish Patel" : >> >> >> >> Folks, >> >> >> >> We are running openstack with KVM and i have noticed kvm presenting >> >> wrong CPU Tolopoly to VM and because of that we are seeing bad >> >> performance to our application. >> >> >> >> This is openstack compute: >> >> >> >> # lstopo-no-graphics --no-io >> >> Machine (64GB total) >> >> NUMANode L#0 (P#0 32GB) + Package L#0 + L3 L#0 (25MB) >> >> L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 >> >> PU L#0 (P#0) >> >> PU L#1 (P#20) >> >> L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 >> >> PU L#2 (P#1) >> >> PU L#3 (P#21) >> >> >> >> This is VM running on above compute >> >> >> >> # lstopo-no-graphics --no-io >> >> Machine (59GB total) >> >> NUMANode L#0 (P#0 29GB) + Package L#0 + L3 L#0 (16MB) >> >> L2 L#0 (4096KB) + Core L#0 >> >> L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0) >> >> L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1) >> >> L2 L#1 (4096KB) + Core L#1 >> >> L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2) >> >> L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3) >> >> >> >> if you noticed P#0 and P#1 has own (32KB) cache per thread that is >> >> wrong presentation if you compare with physical CPU. >> >> >> >> This is a screenshot of AWS vs Openstack CPU Topology and looking at >> >> openstack its presentation is little odd, is that normal? >> >> >> >> https://imgur.com/a/2sPwJVC >> >> >> >> I am running CentOS7.6 with kvm 2.12 version. >> >> From openstack at fried.cc Fri Mar 6 03:40:54 2020 From: openstack at fried.cc (Eric Fried) Date: Thu, 5 Mar 2020 21:40:54 -0600 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK Message-ID: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> Any chance of reusing the “PTL ack” bot that has recently appeared in the releases repo? But as a “SME ack” that would recognize anyone from $project’s core team? (How does the releases bot know which project the patch is for? Might have to be a bit fuzzy on that logic for SDK/OSC.) Then the team could adopt a policy of single core approval if the patch has this SME +1, and no real danger of “abuse”. Eric Fried From agarwalvishakha18 at gmail.com Wed Mar 4 11:42:15 2020 From: agarwalvishakha18 at gmail.com (Vishakha Agarwal) Date: Wed, 4 Mar 2020 17:12:15 +0530 Subject: [keystone] Keystone Team Update - Week of 2 February 2020 Message-ID: # Keystone Team Update - Week of 2 March 2020 ## News ### User Support and Bug Duty Every week the duty is being rotated between the members. The person-in-charge for bug duty for current and upcoming week can be seen on the etherpad [1] [1] https://etherpad.openstack.org/p/keystone-l1-duty ## Open Specs Ussuri specs: https://bit.ly/2XDdpkU Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 4 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 24 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ### Priority Reviews * Ussuri Roadmap Stories - Groups in keystone SAML assertion https://tree.taiga.io/project/keystone-ussuri-roadmap/us/33 https://review.opendev.org/#/c/588211/ Add openstack_groups to assertion - Add support for modifying resource options to CLI tool https://tree.taiga.io/project/keystone-ussuri-roadmap/us/53 https://review.opendev.org/#/c/697444/ Adding options to user cli * Special Requests https://review.opendev.org/#/c/710734/ Correcting api-ref for users ## Bugs This week we opened 1 new bug and closed 1. Bugs opened (1) Bug #1865121 (keystone:Undecided): 'openstack token issue' command doesn't issue token for MFA enabled user - Opened by Abhishek Sharma M https://bugs.launchpad.net/keystone/+bug/1865121 Bugs closed (1) Bug #1865121 (keystone:Undecided) https://bugs.launchpad.net/keystone/+bug/1865121 ## Milestone Outlook https://releases.openstack.org/ussuri/schedule.html Feature proposal freeze is NEXT WEEK (March 9- March 16). Spec implementations that are not submitted or still in a WIP state by the end of the week will need to be postponed until next cycle unless we agree on an exception. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From artem.goncharov at gmail.com Fri Mar 6 06:19:19 2020 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Fri, 6 Mar 2020 07:19:19 +0100 Subject: OSC future (formerly [glance] Different checksum between CLI and curl) In-Reply-To: References: <2beb58bd79afea58ec342fe3c565f3b4e4bc3005.camel@redhat.com> <714d6f56-5e6b-2784-483e-e767f76442cd@gmx.com> <36FB0C7D-C3E1-4C3A-B923-1F68764D44A8@cern.ch> <170a31cf7da.c168b6b0389449.3073076279707922843@ghanshyammann.com> Message-ID: On Fri, 6 Mar 2020, 00:03 melanie witt, wrote: > On 3/5/20 12:11, Adam Harwell wrote: > > Well, part of maintaining feature parity is that the features should be > > added to the OSC by the project team at the time they're made -- you're > > already doing the work to add them to your own client, so instead, do > > the same amount of work but add them in OSC instead! It doesn't seem > > incredibly onerous to me. If the OSC plugin for your project IS the > > official client, then there's no duplication of effort. I think saying > > "someone else had better implement our features in a timely fashion" is > > a bit irresponsible. Though, this is coming from working on a project > > where we aren't used to being included as a "core piece" and having any > > work done for us, ever... > > I think this is the key point regarding the lack of feature parity in > OSC for some projects. > > If you are a new-enough project to have begun your CLI as an OSC plugin > (examples: ironic client, placement client, and more) then adding a > feature to the client is one shot. You add support in the plugin and > you're done. > > If you are an older project (examples: nova client, glance client) then > you have a two-step process for adding a feature to OSC. For older > projects, OSC imports the legacy clients and calls their python bindings > to make the API calls. So for nova, if you want to add a feature to the > client, you have to add it to the legacy nova client. This is required. > Then, to add it to OSC you have to add it to OSC and have OSC call the > newly added legacy binding for the feature. This is [technically] > optional. This is why parity is missing. > Hopefully in some days for glance this will not be necessary anymore, since there is a patch waiting to be merged, which switches OSC to SDK for Glance and removes glanceclient from dependencies. For neutron it is not required since long time. But still, what you say was real way to implement things, but it doesn't mean it is a correct or the easiest way. Instead if team once invests time to bring OSC Plugin, which bases on SDK in parity ... - the old client might be deprecated and you don't have this double efforts. The SDK/OSC team can not and should not try to catch all projects with their features each release - it is impossible (there are just 3 active cores in SDK now but hundreds of projects we might want to support). This team enables projects in a "unified technology stack" so that they become responsible for implementing all new features there. > It pains me a bit to write it ^ because you may be thinking, "what's so > difficult about going the extra step to add a feature to OSC after > adding it to nova client?" I don't know. Maybe people are too stressed > and busy. If it's not "required", it gets deferred. Maybe people don't > feel familiar enough with OSC to add the feature there as well. There > could be a lot of different reasons. > > So, not trying to make excuses here but just sharing my opinion on why > adding features to OSC is not so simple for some projects. > It's not if you prioritize things not correctly and chooses complex direction. I'm laughing so loud, that this "storm" started just out of one question "OSC works correct, but curl not, so what am I doing wrong". It just one more time convinces me in some form of radicalisation of some projects against bringing things in order. Regards, Artem > Cheers, > -melanie > > > Also, things are also definitely moving in a better direction now with > > the probable addition of project team liasons as cores in SDK/OSC, which > > should alleviate a lot of the issues around "response time" on reviews, > > when you do put in the effort to add features yourself. > > > > --Adam > > > > On Fri, Mar 6, 2020, 00:15 Erno Kuvaja > > wrote: > > > > On Wed, Mar 4, 2020 at 1:19 AM Ghanshyam Mann > > > wrote: > > > > ---- On Tue, 03 Mar 2020 13:00:35 -0600 Tim Bell > > > wrote ---- > > > > > > > > > On 3 Mar 2020, at 19:55, Tim Bell > > wrote: > > > > > > > > > On 3 Mar 2020, at 19:20, Albert Braden > > > > > wrote: > > > Sean, thank you for clarifying that. > > > > > > Was my understanding that the community decided to focus on > > the unified client incorrect? Is the unified/individual client > > debate still a matter of controversy? Is it possible that the > > unified client will be deprecated in favor of individual clients > > after more discussion? I haven’t looked at any of the individual > > clients since 2018 (except for osc-placement which is kind of a > > special case), because I thought they were all going away and > > could be safely ignored until they did, and I haven’t included > > any information about the individual clients in the > > documentation that I write for our users, and if they ask I have > > been telling them to not use the individual clients. Do I need > > to start looking at individual clients again, and telling our > > users to use them in some cases? > > > > > > > > > > > > I remember a forum discussion where a community goal was > > proposed to focus on OSC rather than individual project CLIs (I > > think Matt and I were proposers). There were concerns on the > > effort to do this and that it would potentially be multi-cycle. > > > BTW, I found the etherpad from Berlin > > (https://etherpad.openstack.org/p/BER-t-series-goals) and the > > associated mailing list discussion at > > > http://lists.openstack.org/pipermail/openstack-dev/2018-September/135107.html > > > > Yeah, we are in process of selecting the Victoria cycle > > community-wide goal and this can be good candidate. I agree with > > the idea/requirement of a multi-cycle goal. > > Another option is to build a pop-up team for the Victoria cycle > > to start burning down the keys issues/work. For both ways > > (either goal or pop-up team), we need > > some set of people to drive it. If anyone would like to > > volunteer for this, we can start discussing the details. > > > > [1] > > > http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012866.html > > > > -gmann > > > > Yeah, lets propose this as community goal again as it worked so well > > last time. |ಠ_ಠ| > > > > I think your most help wanted list/pop-up team is much more > > realistic approach. Lets see if there is enough interest to actually > > make it happen. What comes to our previous experience with Glance > > and moving to endorse osc, I think I'm not alone stating that we can > > discuss this again after osc has kept feature parity (and I mean to > > current release, not feature parity 2 years ago kind of thing) and > > actively addressed raised issues at least for a couple of cycles. > > Obviously if you/your users wants to use it meanwhile, that your > > call. If we cannot get that level of commitment, how do we expect to > > support this long term? > > > > I'm not willing to put our users through that misery again as it > > happened last time as long as I'm core in this project. > > > > - jokke > > > > > > > > My experience in discussion with the CERN user community and > > other OpenStack operators is that OSC is felt to be the right > > solution for the end user facing parts of the cloud (admin > > commands could be another discussion if necessary). Experienced > > admin operators can remember that glance looks after images and > > nova looks after instances. Our average user can get very > > confused, especially given that OSC supports additional options > > for authentication (such as Kerberos and Certificates along with > > clouds.yaml) so users need to re-authenticate with a different > > openrc to work on their project. > > > While I understand there are limited resources all round, I > > would prefer that we focus on adding new project functions to > > OSC which will eventually lead to feature parity. > > > Attracting ‘drive-by’ contributions from operations staff > > for OSC work (it's more likely to be achieved if it makes the > > operations work less e.g. save on special end user documentation > > by contributing code). This is demonstrated from the CERN team > > contribution to the OSC ‘coe' and ‘share' functionality along > > with lots of random OSC updates as listed hat > > > https://www.stackalytics.com/?company=cern&metric=commits&module=python-openstackclient > ) > > > > > BTW, I also would vote for =auto as the default. > > > Tim > > > We are on Rocky now but I expect that we will upgrade as > > necessary to stay on supported versions. > > > > > > From: Sean McGinnis > > > > > Sent: Tuesday, March 3, 2020 9:50 AM > > > To: openstack-discuss at lists.openstack.org > > > > > Subject: Re: OSC future (formerly [glance] Different > > checksum between CLI and curl) > > > > > > On 3/3/20 11:28 AM, Albert Braden wrote: > > > Am I understanding correctly that the Openstack community > > decided to focus on the unified client, and to deprecate the > > individual clients, and that the Glance team did not agree with > > this decision, and that the Glance team is now having a pissing > > match with the rest of the community, and is unilaterally > > deciding to continue developing the Glance client and refusing > > to work on the unified client, or is something different going > > on? I would ask everyone involved to remember that we operators > > are down here, and the yellow rain falling on our heads does not > > smell very good. > > > I definitely would not characterize it that way. > > > With trying not to put too much personal bias into it, > > here's what I would say the situation is: > > > - Some part of the community has said OSC should be the only > > CLI and that individual CLIs should go away > > > - Glance is a very small team with very, very limited > resources > > > - The OSC team is a very small team with very, very limited > > resources > > > - CLI capabilities need to be exposed for Glance changes and > > the easiest way to get them out for the is by updating the > > Glance CLI > > > - No one from the OSC team has been able to proactively help > > to make sure these changes make it into the OSC client (see > > bullet 3) > > > - There exists a sizable functionality gap between > > per-project CLIs and what OSC provides, and although a few > > people have done a lot of great work to close that gap, there is > > still a lot to be done and does not appear the gap will close at > > any point in the near future based on the current trends > > > > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Fri Mar 6 07:31:07 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 6 Mar 2020 08:31:07 +0100 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> References: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> Message-ID: <019B33AF-7ADC-4C08-A131-C9CF0B919D07@redhat.com> Hi, I’m fine with adding those new people to the core team. As Monty said, I’m not doing too many reviews in sdk project but I’m trying to always check neutron related changes and I think that having such expert for other projects would be good too. > On 6 Mar 2020, at 04:40, Eric Fried wrote: > > Any chance of reusing the “PTL ack” bot that has recently appeared in the releases repo? But as a “SME ack” that would recognize anyone from $project’s core team? (How does the releases bot know which project the patch is for? Might have to be a bit fuzzy on that logic for SDK/OSC.) That also seems like potential solution but as changes in this repo are a bit differently then in e.g. releases repo how bot will exactly know which PTL should approve the patch? It may be much harder to do here than in releases repo, no? > > Then the team could adopt a policy of single core approval if the patch has this SME +1, and no real danger of “abuse”. > > Eric Fried > — Slawek Kaplonski Senior software engineer Red Hat From rui.zang at yandex.com Fri Mar 6 07:52:40 2020 From: rui.zang at yandex.com (rui zang) Date: Fri, 06 Mar 2020 15:52:40 +0800 Subject: CPU Topology confusion In-Reply-To: References: <3F1F97A3-4DEE-4CA9-9147-892D6E7355E7@gmail.com> Message-ID: <1166211583479894@vla4-4046ec513d04.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From qiujunting at inspur.com Fri Mar 6 07:57:24 2020 From: qiujunting at inspur.com (=?gb2312?B?SnVudGluZ3FpdSBRaXVqdW50aW5nICjH8b785sMp?=) Date: Fri, 6 Mar 2020 07:57:24 +0000 Subject: [nova][qa] When creating a instance using a flavor with "hw:cpu_policy=dedicated" "hw:cpu_realtime=yes" and "hw:cpu_realtime_mask=^0-1" failed. Message-ID: When I create a instance using a flavor with "hw:cpu_policy=dedicated" "hw:cpu_realtime=yes" and "hw:cpu_realtime_mask=^0-1". The error is "libvirtError:Cannot set scheduler parameters for pid 3666:operation not permitted". https://bugs.launchpad.net/starlingx/+bug/1866311 The instance xml as following: 4 The error log as following: ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1099, in createWithFlags ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] libvirtError: Cannot set scheduler parameters for pid 6504: Operation not permitted ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Fri Mar 6 08:11:19 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?Q?Bal=E1zs_Gibizer?=) Date: Fri, 6 Mar 2020 08:11:19 +0000 Subject: [nova][ptl] Temporary Nova PTL until election Message-ID: <1583482276.12170.14@est.tech> Hi, Since Eric announced that he has to leave us [1] I have been working internally with my employee to be able to take over the Nova PTL position. Now I've got the necessary approvals. The official PTL election is close [2] and I'm ready to fill the PTL gap until the proper PTL election in April. Is this a workable solution for the community? Cheers, gibi [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012663.html [2] https://governance.openstack.org/election/ From zhangbailin at inspur.com Fri Mar 6 08:49:06 2020 From: zhangbailin at inspur.com (=?gb2312?B?QnJpbiBaaGFuZyjVxbDZwdYp?=) Date: Fri, 6 Mar 2020 08:49:06 +0000 Subject: =?gb2312?B?tPC4tDogW25vdmFdW3B0bF0gVGVtcG9yYXJ5IE5vdmEgUFRMIHVudGlsIGVs?= =?gb2312?Q?ection?= In-Reply-To: <1583482276.12170.14@est.tech> References: <1583482276.12170.14@est.tech> Message-ID: > 主题: [lists.openstack.org代发][nova][ptl] Temporary Nova PTL until election > > Hi, > Since Eric announced that he has to leave us [1] I have been working internally with my employee to be able to take > over the Nova PTL position. Now I've got the necessary approvals. The official PTL election is close [2] and I'm ready to > fill the PTL gap until the proper PTL election in April. > > Is this a workable solution for the community? +1, Yes, at 2020-03-05 (Thursday) nova meeting talked this topic [#1]. After long-term cooperation with gibi in community work, I agree that he will take over the PTL vacancy brought by Eric, and I think we need to do so. [#1]http://eavesdrop.openstack.org/irclogs/%23openstack-meeting/%23openstack-meeting.2020-03-05.log.html#t2020-03-05T14:19:38-2-3 > Cheers, > gibi > [1]http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012663.html > [2] https://governance.openstack.org/election/ From alfredo.deluca at gmail.com Fri Mar 6 08:52:02 2020 From: alfredo.deluca at gmail.com (Alfredo De Luca) Date: Fri, 6 Mar 2020 09:52:02 +0100 Subject: [CINDER] Distributed storage alternatives In-Reply-To: References: Message-ID: Hi Donny. thanks for your inputs. Appreciate it On Sun, Feb 23, 2020 at 7:16 AM Donny Davis wrote: > I use local NVME storage for FortNebula. Its very fast, and for "cloudy" > things I prefer availability of data to be above the infrastructure layer. > I used to use Ceph for all things, but in my experience... if performance > is a requirement, local storage is pretty hard to beat. > I am in the process of moving the object store to ceph, and all seems to > be well in terms of performance using ceph for that use case. > > > > On Tue, Feb 18, 2020 at 6:41 AM Alfredo De Luca > wrote: > >> Thanks Burak and Ignazio. >> Appreciate it >> >> >> >> On Thu, Feb 13, 2020 at 10:19 PM Burak Hoban >> wrote: >> >>> Hi guys, >>> >>> We use Dell EMC VxFlex OS, which in its current version allows for free >>> use and commercial (in version 3.5 a licence is needed, but its perpetual). >>> It's similar to Ceph but more geared towards scale and performance etc (it >>> use to be called ScaleIO). >>> >>> Other than that, I know of a couple sites using SAN storage, but a lot >>> of people just seem to use Ceph. >>> >>> Cheers, >>> >>> Burak >>> >>> ------------------------------ >>> >>> Message: 2 >>> Date: Thu, 13 Feb 2020 18:20:29 +0100 >>> From: Ignazio Cassano >>> To: Alfredo De Luca >>> Cc: openstack-discuss >>> Subject: Re: [CINDER] Distributed storage alternatives >>> Message-ID: >>> < >>> CAB7j8cXLQWh5fx-E9AveUEa6OncDwCL6BOGc-Pm2TX4FKwnUKg at mail.gmail.com> >>> Content-Type: text/plain; charset="utf-8" >>> >>> Hello Alfredo, I think best opensource solution is ceph. >>> As far as commercial solutions are concerned we are working with network >>> appliance (netapp) and emc unity. >>> Regards >>> Ignazio >>> >>> Il Gio 13 Feb 2020, 13:48 Alfredo De Luca ha >>> scritto: >>> >>> > Hi all. >>> > we 'd like to explore storage back end alternatives to CEPH for >>> > Openstack >>> > >>> > I am aware of GlusterFS but what would you recommend for distributed >>> > storage like Ceph and specifically for block device provisioning? >>> > Of course must be: >>> > >>> > 1. *Reliable* >>> > 2. *Fast* >>> > 3. *Capable of good performance over WAN given a good network back >>> > end* >>> > >>> > Both open source and commercial technologies and ideas are welcome. >>> > >>> > Cheers >>> > >>> > -- >>> > *Alfredo* >>> > >>> > >>> >>> _____________________________________________________________________ >>> >>> The information transmitted in this message and its attachments (if any) >>> is intended >>> only for the person or entity to which it is addressed. >>> The message may contain confidential and/or privileged material. Any >>> review, >>> retransmission, dissemination or other use of, or taking of any action >>> in reliance >>> upon this information, by persons or entities other than the intended >>> recipient is >>> prohibited. >>> >>> If you have received this in error, please contact the sender and delete >>> this e-mail >>> and associated material from any computer. >>> >>> The intended recipient of this e-mail may only use, reproduce, disclose >>> or distribute >>> the information contained in this e-mail and any attached files, with >>> the permission >>> of the sender. >>> >>> This message has been scanned for viruses. >>> _____________________________________________________________________ >>> >> >> >> -- >> *Alfredo* >> >> > > -- > ~/DonnyD > C: 805 814 6800 > "No mission too difficult. No sacrifice too great. Duty First" > -- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyarwood at redhat.com Fri Mar 6 09:11:11 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Fri, 6 Mar 2020 09:11:11 +0000 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: <1583482276.12170.14@est.tech> References: <1583482276.12170.14@est.tech> Message-ID: <20200306091111.kxkd6mtk4mwqo5vs@lyarwood.usersys.redhat.com> On 06-03-20 08:11:19, Balázs Gibizer wrote: > Hi, > > Since Eric announced that he has to leave us [1] I have been working > internally with my employee to be able to take over the Nova PTL > position. Now I've got the necessary approvals. The official PTL > election is close [2] and I'm ready to fill the PTL gap until the > proper PTL election in April. > > Is this a workable solution for the community? Yes definitely for me, thanks for stepping up gibi! -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From hberaud at redhat.com Fri Mar 6 09:35:29 2020 From: hberaud at redhat.com (Herve Beraud) Date: Fri, 6 Mar 2020 10:35:29 +0100 Subject: oslo.cache 2.1.0 breaks oslo_cache.memcache_pool In-Reply-To: References: Message-ID: Oslo.cache version 2.1.0 is now blacklisted [1] to avoid similar issues for now. A new `memcache_pool` backend job have been introduced (not yet merged) through a patch [2] to oslo.cache to help us to catch similar errors during CI. Now we have 2 ways to definitely fix the situation on oslo.cache: - fix the broken code [3] and release a new patched version (2.1.1), still WIP; - revert the initial changes [4] and release a new version free from this bug (2.2.0). After some discussions with other oslo cores we want to give priority to the fix [3] first. Do not hesitate to correct me if something is wrong here. Cheers, PS: improvements and fix described in my previous email have been merged together [3]. [1] https://review.opendev.org/#/c/711427/ [2] https://review.opendev.org/#/c/711422/ [3] https://review.opendev.org/#/c/711220/ [4] https://review.opendev.org/#/c/711439/ Le mer. 4 mars 2020 à 19:23, Herve Beraud a écrit : > I proposed the following two patches to address the issue and improve this > module beyond the current issue: > - https://review.opendev.org/711220 (the fix) > - https://review.opendev.org/711247 (the improvements) > > After these patches will be merged and the issue fixed we will blacklist > the version 2.1.0 of oslo.cache and propose a new release with the previous > fixes embedded. > > Do not hesitate to review them and leave comments. > > Thanks for your reading. > > Le mer. 4 mars 2020 à 14:16, Herve Beraud a écrit : > >> Fix proposed https://review.opendev.org/#/c/711220/ >> >> Le mer. 4 mars 2020 à 13:42, Moises Guimaraes de Medeiros < >> moguimar at redhat.com> a écrit : >> >>> `dead_timeout`++ >>> >>> On Wed, Mar 4, 2020 at 1:36 PM Herve Beraud wrote: >>> >>>> `dead_timeout` [1] looks more appropriate in this case. >>>> >>>> [1] >>>> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L58 >>>> >>>> Le mer. 4 mars 2020 à 13:28, Herve Beraud a >>>> écrit : >>>> >>>>> What do you think about adding a mapping between `retry_timeout` [1] >>>>> and `dead_retry` [2]? >>>>> >>>>> [1] >>>>> https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/hash.py#L56 >>>>> [2] >>>>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>>>> >>>>> Le mer. 4 mars 2020 à 13:20, Herve Beraud a >>>>> écrit : >>>>> >>>>>> I think our issue is due to the fact that python-memcached accept a >>>>>> param named `dead_retry` [1] which is not defined in pymemcache. >>>>>> >>>>>> We just need to define it in our oslo.cache mapping. During testing >>>>>> we faced the same kind of issue with connection timeout. >>>>>> >>>>>> [1] >>>>>> https://github.com/linsomniac/python-memcached/blob/bad41222379102e3f18f6f2f7be3ee608de6fbff/memcache.py#L183 >>>>>> [2] >>>>>> https://github.com/openstack/oslo.cache/blob/8a8248d764bbb1db6c0089a58745803c03e38fdb/oslo_cache/_memcache_pool.py#L193,L201 >>>>>> >>>>>> Le mer. 4 mars 2020 à 12:21, Radosław Piliszek < >>>>>> radoslaw.piliszek at gmail.com> a écrit : >>>>>> >>>>>>> Please be informed that oslo.cache 2.1.0 breaks >>>>>>> oslo_cache.memcache_pool >>>>>>> >>>>>>> Kolla-Ansible gate is already RED and a quick codesearch revealed >>>>>>> other deployment methods might be in trouble soon as well. >>>>>>> >>>>>>> This does not affect devstack/tempest as they use >>>>>>> dogpile.cache.memcached instead. >>>>>>> >>>>>>> The error is TypeError: __init__() got an unexpected keyword argument >>>>>>> 'dead_retry' >>>>>>> >>>>>>> For details see: https://bugs.launchpad.net/oslo.cache/+bug/1866008 >>>>>>> >>>>>>> -yoctozepto >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Hervé Beraud >>>>>> Senior Software Engineer >>>>>> Red Hat - Openstack Oslo >>>>>> irc: hberaud >>>>>> -----BEGIN PGP SIGNATURE----- >>>>>> >>>>>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>>>>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>>>>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>>>>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>>>>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>>>>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>>>>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>>>>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>>>>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>>>>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>>>>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>>>>> v6rDpkeNksZ9fFSyoY2o >>>>>> =ECSj >>>>>> -----END PGP SIGNATURE----- >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Hervé Beraud >>>>> Senior Software Engineer >>>>> Red Hat - Openstack Oslo >>>>> irc: hberaud >>>>> -----BEGIN PGP SIGNATURE----- >>>>> >>>>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>>>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>>>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>>>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>>>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>>>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>>>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>>>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>>>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>>>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>>>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>>>> v6rDpkeNksZ9fFSyoY2o >>>>> =ECSj >>>>> -----END PGP SIGNATURE----- >>>>> >>>>> >>>> >>>> -- >>>> Hervé Beraud >>>> Senior Software Engineer >>>> Red Hat - Openstack Oslo >>>> irc: hberaud >>>> -----BEGIN PGP SIGNATURE----- >>>> >>>> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >>>> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >>>> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >>>> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >>>> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >>>> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >>>> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >>>> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >>>> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >>>> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >>>> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >>>> v6rDpkeNksZ9fFSyoY2o >>>> =ECSj >>>> -----END PGP SIGNATURE----- >>>> >>>> >>> >>> -- >>> >>> Moisés Guimarães >>> >>> Software Engineer >>> >>> Red Hat >>> >>> >>> >> >> >> -- >> Hervé Beraud >> Senior Software Engineer >> Red Hat - Openstack Oslo >> irc: hberaud >> -----BEGIN PGP SIGNATURE----- >> >> wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ >> Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ >> RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP >> F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G >> 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g >> glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw >> m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ >> hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 >> qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y >> F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 >> B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O >> v6rDpkeNksZ9fFSyoY2o >> =ECSj >> -----END PGP SIGNATURE----- >> >> > > -- > Hervé Beraud > Senior Software Engineer > Red Hat - Openstack Oslo > irc: hberaud > -----BEGIN PGP SIGNATURE----- > > wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ > Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ > RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP > F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G > 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g > glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw > m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ > hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 > qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y > F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 > B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O > v6rDpkeNksZ9fFSyoY2o > =ECSj > -----END PGP SIGNATURE----- > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From kchamart at redhat.com Fri Mar 6 11:26:53 2020 From: kchamart at redhat.com (Kashyap Chamarthy) Date: Fri, 6 Mar 2020 12:26:53 +0100 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: <1583482276.12170.14@est.tech> References: <1583482276.12170.14@est.tech> Message-ID: <20200306112653.GA30104@paraplu> On Fri, Mar 06, 2020 at 08:11:19AM +0000, Balázs Gibizer wrote: > Hi, > > Since Eric announced that he has to leave us [1] I have been working > internally with my employee to be able to take over the Nova PTL > position. Now I've got the necessary approvals. The official PTL > election is close [2] and I'm ready to fill the PTL gap until the > proper PTL election in April. > > Is this a workable solution for the community? Absolutely! Thanks for raising your hand to do the unthankful work. -- /kashyap From openstack at fried.cc Fri Mar 6 13:01:15 2020 From: openstack at fried.cc (Eric Fried) Date: Fri, 6 Mar 2020 07:01:15 -0600 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: <1583482276.12170.14@est.tech> References: <1583482276.12170.14@est.tech> Message-ID: <4692F106-6B00-41FC-9BA9-1DF62A24EDAB@fried.cc> Big +1 from me. Many thanks, gibi. Not that you‘ll need it, but please don’t hesitate to reach out to me if you have questions. efried_gone > On Mar 6, 2020, at 02:16, Balázs Gibizer wrote: > > Hi, > > Since Eric announced that he has to leave us [1] I have been working > internally with my employee to be able to take over the Nova PTL > position. Now I've got the necessary approvals. The official PTL > election is close [2] and I'm ready to fill the PTL gap until the > proper PTL election in April. > > Is this a workable solution for the community? > > Cheers, > gibi > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012663.html > [2] https://governance.openstack.org/election/ > > > From gmann at ghanshyammann.com Fri Mar 6 13:51:25 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 06 Mar 2020 07:51:25 -0600 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: <4692F106-6B00-41FC-9BA9-1DF62A24EDAB@fried.cc> References: <1583482276.12170.14@est.tech> <4692F106-6B00-41FC-9BA9-1DF62A24EDAB@fried.cc> Message-ID: <170b01d7595.10341333e516143.4131462912712933865@ghanshyammann.com> ---- On Fri, 06 Mar 2020 07:01:15 -0600 Eric Fried wrote ---- > Big +1 from me. Many thanks, gibi. Not that you‘ll need it, but please don’t hesitate to reach out to me if you have questions. Indeed. Thanks gibi for helping out here. -gmann > > efried_gone > > > On Mar 6, 2020, at 02:16, Balázs Gibizer wrote: > > > > Hi, > > > > Since Eric announced that he has to leave us [1] I have been working > > internally with my employee to be able to take over the Nova PTL > > position. Now I've got the necessary approvals. The official PTL > > election is close [2] and I'm ready to fill the PTL gap until the > > proper PTL election in April. > > > > Is this a workable solution for the community? > > > > Cheers, > > gibi > > > > [1] > > http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012663.html > > [2] https://governance.openstack.org/election/ > > > > > > > > > From mordred at inaugust.com Fri Mar 6 15:01:22 2020 From: mordred at inaugust.com (Monty Taylor) Date: Fri, 6 Mar 2020 09:01:22 -0600 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: References: Message-ID: > On Mar 5, 2020, at 6:53 PM, Michael Johnson wrote: > > Naming names is a highly effective demotivator for those whose > contributions are not recognized. > > https://www.stackalytics.com/?module=openstacksdk&metric=loc > > Maybe the project teams should select their own liaison? > Great point - and totally! I was mostly keying off of recent interactions but you’re absolutely right about that. Also - apologies for my brain not immediately calling you up there… it’s been a week. I think adding you is a great idea. There’s some other thoughts in the responses here that are interesting that we should try as well, but I think those will take a bit longer to sort out. > On Thu, Mar 5, 2020 at 9:58 AM Monty Taylor wrote: >> >> Heya, >> >> I’d like to try something. >> >> I’d like to try adding some project-specific people to the core team so that they can more directly help maintain the support for their service in SDK. In some of these cases the person I’m suggestion has next to no review experience in SDK. I think let’s be fine with that for now - we’re still a 2x +2 in general thing - but I know currently when reviewing neutron or ironic changes I always want to see a +2 from slaweq or dtantsur … so in the spirit of trying new things and trying to move the project forward in a healthy and welcoming way - how about we give this a try? >> >> The idea here is that we’re trusting people to use their good judgement and to only use their new +2 powers for good in their project. Over time, if they feel like they’ve gotten a handle on things more widely, there’s nothing stopping them from reviewing other patches - but I think that most of us aren’t looking for additional review work anyway. >> >> Specifically this would be: >> >> Shogo Saito - congress >> Adam Harwell - octavia >> Graham Hayes - designate >> Bharat Kumar - magnum >> Erik Olof Gunnar Andersson - senlin >> Tim Burke - swift >> >> I think we should also add a file in the repo that lists “subject matter experts” for each service we’ve got support for, where we have them. My list of current cores who I’d ping for specific service suitability are: >> >> Sean McGinnis - cinder >> Slawek Kaplonski - neutron >> Dmitry Tantsur - ironic >> Eric Fried - nova (at least until tomorrow my friend) >> >> How does that sound to folks? >> >> Monty > From openstack at fried.cc Fri Mar 6 15:06:03 2020 From: openstack at fried.cc (Eric Fried) Date: Fri, 6 Mar 2020 09:06:03 -0600 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: <019B33AF-7ADC-4C08-A131-C9CF0B919D07@redhat.com> References: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> <019B33AF-7ADC-4C08-A131-C9CF0B919D07@redhat.com> Message-ID: <55bfaec1-1880-e779-2a5d-288528c4c26e@fried.cc> > That also seems like potential solution but as changes in this repo are a bit differently then in e.g. releases repo how bot will exactly know which PTL should approve the patch? It may be much harder to do here than in releases repo, no? Yeah, I agree, which is why I asked how the releases repo does it. I can envision the bot knowing which packages/files pertain to a given project and doing the mapping that way. And like if multiple projects' files are touched, the bot doesn't register the SME+1 until there's a +1 from each project. But yeah, this is getting a bit heavy weight. Then again, it allays diablo_rojo's concern about SMEs' ability to +W, and johnsom's concern about impacting morale by singling out individuals. Anyway, just a thought. efried_gone . From mordred at inaugust.com Fri Mar 6 15:12:18 2020 From: mordred at inaugust.com (Monty Taylor) Date: Fri, 6 Mar 2020 09:12:18 -0600 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: <019B33AF-7ADC-4C08-A131-C9CF0B919D07@redhat.com> References: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> <019B33AF-7ADC-4C08-A131-C9CF0B919D07@redhat.com> Message-ID: > On Mar 6, 2020, at 1:31 AM, Slawek Kaplonski wrote: > > Hi, > > I’m fine with adding those new people to the core team. As Monty said, I’m not doing too many reviews in sdk project but I’m trying to always check neutron related changes and I think that having such expert for other projects would be good too. > >> On 6 Mar 2020, at 04:40, Eric Fried wrote: >> >> Any chance of reusing the “PTL ack” bot that has recently appeared in the releases repo? But as a “SME ack” that would recognize anyone from $project’s core team? (How does the releases bot know which project the patch is for? Might have to be a bit fuzzy on that logic for SDK/OSC.) > > That also seems like potential solution but as changes in this repo are a bit differently then in e.g. releases repo how bot will exactly know which PTL should approve the patch? It may be much harder to do here than in releases repo, no I agree with Slawek - I like this idea but I think it *is* a harder one to accomplish. Also, sometimes patches are straightforward enough that we don’t really need a service-specific ack from … especially when the service has good and clear API documentation. I think I’m leaning towards liking Michael’s suggestion a bit better as a next step - having the teams suggest a person, liaison-style - mostly because it’s easy. But for now, even with that - and even with my flub of completely blanking on Michael, let’s see how playing it by ear goes before we add more process. >> >> Then the team could adopt a policy of single core approval if the patch has this SME +1, and no real danger of “abuse”. >> >> Eric Fried >> > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > > From fungi at yuggoth.org Fri Mar 6 15:25:33 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 6 Mar 2020 15:25:33 +0000 Subject: [sdk][congress][octavia][designate][magnum][senlin][swift] Adding project-specific cores to SDK In-Reply-To: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> References: <7BBB4E00-5AA1-441B-9872-F6AF5415FCDB@fried.cc> Message-ID: <20200306152533.jbkcwtitvsgds44k@yuggoth.org> On 2020-03-05 21:40:54 -0600 (-0600), Eric Fried wrote: > Any chance of reusing the “PTL ack” bot that has recently appeared > in the releases repo? But as a “SME ack” that would recognize > anyone from $project’s core team? (How does the releases bot know > which project the patch is for? Might have to be a bit fuzzy on > that logic for SDK/OSC.) [...] The openstack/releases repository has its critical data organized by OpenStack governance project team deliverable, and there's a Zuul job which gets triggered on every review comment which looks to see if the Gerrit account leaving the comment maps through https://opendev.org/openstack/releases/src/branch/master/data/release_liaisons.yaml and https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml to match the reviewer to the deliverable(s) covered by the proposed change. To do something similar for the OSC/SDK repos, you'd need 1. a means of programmatically identifying the relevant project team for any proposed change, and 2. some list of unique Gerrit account identifiers (most likely E-mail addresses) of the people whose reviews are relevant for them. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From skaplons at redhat.com Fri Mar 6 15:28:59 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 6 Mar 2020 16:28:59 +0100 Subject: [neutron] Propose Lajos Katona for Neutron core team Message-ID: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Hi neutrinos, I would like to propose Lajos Katona (irc: lajoskatona) as a member of the Neutron core team. Lajos is Neutron contributor Neutron since around Queens cycle and now he is one of the most active reviewers in the Neutron group projects. He was one of the key contributors in cooperation with Nova and Placement teams to deliver guaranteed minimum bandwidth feature in OpenStack. He is very active and helpful with triaging and fixing Neutron bugs and issues in our CI. During last few cycles he proved that he has wide knowledge about Neutron code base. He is currently also a maintainer of some neutron stadium projects which shows that he has knowledge about code base not only about neutron but also Neutron stadium. The quality and number of his reviews are comparable to other members of the Neutron core team: https://www.stackalytics.com/?release=ussuri&module=neutron-group and are higher every cycle :) I think he will be great addition to our core team. I will keep this nomination open for a week or until all current cores will respond. — Slawek Kaplonski Senior software engineer Red Hat From nate.johnston at redhat.com Fri Mar 6 15:42:09 2020 From: nate.johnston at redhat.com (Nate Johnston) Date: Fri, 6 Mar 2020 10:42:09 -0500 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Message-ID: <20200306154209.763y4s3xfcymtqtt@firewall> Lajos is a great contributor in all ways. I have been watching his reviews and he is an insightful, thorough reviewer. Big +1 from me. Nate On Fri, Mar 06, 2020 at 04:28:59PM +0100, Slawek Kaplonski wrote: > Hi neutrinos, > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the Neutron core team. > Lajos is Neutron contributor Neutron since around Queens cycle and now he is one of the most active reviewers in the Neutron group projects. > He was one of the key contributors in cooperation with Nova and Placement teams to deliver guaranteed minimum bandwidth feature in OpenStack. > He is very active and helpful with triaging and fixing Neutron bugs and issues in our CI. > > During last few cycles he proved that he has wide knowledge about Neutron code base. He is currently also a maintainer of some neutron stadium projects which shows that he has knowledge about code base not only about neutron but also Neutron stadium. > > The quality and number of his reviews are comparable to other members of the Neutron core team: https://www.stackalytics.com/?release=ussuri&module=neutron-group and are higher every cycle :) > I think he will be great addition to our core team. > > I will keep this nomination open for a week or until all current cores will respond. > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From miguel at mlavalle.com Fri Mar 6 16:26:07 2020 From: miguel at mlavalle.com (Miguel Lavalle) Date: Fri, 6 Mar 2020 10:26:07 -0600 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Message-ID: Hi, Yes I agree, Lajos will be a great addition to the Neutron core team. Big +1 from me Regards On Fri, Mar 6, 2020 at 9:29 AM Slawek Kaplonski wrote: > Hi neutrinos, > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the > Neutron core team. > Lajos is Neutron contributor Neutron since around Queens cycle and now he > is one of the most active reviewers in the Neutron group projects. > He was one of the key contributors in cooperation with Nova and Placement > teams to deliver guaranteed minimum bandwidth feature in OpenStack. > He is very active and helpful with triaging and fixing Neutron bugs and > issues in our CI. > > During last few cycles he proved that he has wide knowledge about Neutron > code base. He is currently also a maintainer of some neutron stadium > projects which shows that he has knowledge about code base not only about > neutron but also Neutron stadium. > > The quality and number of his reviews are comparable to other members of > the Neutron core team: > https://www.stackalytics.com/?release=ussuri&module=neutron-group and are > higher every cycle :) > I think he will be great addition to our core team. > > I will keep this nomination open for a week or until all current cores > will respond. > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralonsoh at redhat.com Fri Mar 6 16:34:59 2020 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Fri, 06 Mar 2020 16:34:59 +0000 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <20200306154209.763y4s3xfcymtqtt@firewall> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> <20200306154209.763y4s3xfcymtqtt@firewall> Message-ID: <484abed16b6277582a765578b08bbc7fc14742e9.camel@redhat.com> Indeed Lajos is a detail oriented reviewer and has contributed for a long time to this project. Welcome aboard! +1 On Fri, 2020-03-06 at 10:42 -0500, Nate Johnston wrote: > Lajos is a great contributor in all ways. I have been watching his reviews and > he is an insightful, thorough reviewer. Big +1 from me. > > Nate > > On Fri, Mar 06, 2020 at 04:28:59PM +0100, Slawek Kaplonski wrote: > > Hi neutrinos, > > > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the Neutron core team. > > Lajos is Neutron contributor Neutron since around Queens cycle and now he is one of the most > > active reviewers in the Neutron group projects. > > He was one of the key contributors in cooperation with Nova and Placement teams to deliver > > guaranteed minimum bandwidth feature in OpenStack. > > He is very active and helpful with triaging and fixing Neutron bugs and issues in our CI. > > > > During last few cycles he proved that he has wide knowledge about Neutron code base. He is > > currently also a maintainer of some neutron stadium projects which shows that he has knowledge > > about code base not only about neutron but also Neutron stadium. > > > > The quality and number of his reviews are comparable to other members of the Neutron core team: > > https://www.stackalytics.com/?release=ussuri&module=neutron-group and are higher every cycle :) > > I think he will be great addition to our core team. > > > > I will keep this nomination open for a week or until all current cores will respond. > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > > > From mark at stackhpc.com Fri Mar 6 16:39:51 2020 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 6 Mar 2020 16:39:51 +0000 Subject: [kolla][uc] Kolla SIG Message-ID: Hi, I'd like to propose the creation of a Special Interest Group (SIG) [0] for Kolla. The main aim of the group would be to improve communication between operators and developers. The SIG would host regular virtual project onboarding, project update, and feedback sessions, ideally via video calls. This should remove the necessity of being physically present at Open Infra Summits for participation in the project. I like to think of this as the fifth open [1] (name TBD). I propose that in addition to the above sessions, the SIG should host more informal discussions, probably every 2-4 weeks with the aim of meeting other community members, discussing successes and failures, sharing knowledge, and generally getting to know each other a bit better. These could be via video call, IRC, or a mix. Finally - I propose that we build and maintain a list of Kolla users, including details of their environments and areas of interest and expertise. Of course this would be opt-in. This would help us to connect with subject matter experts and interested parties to help answer queries in IRC, or when making changes to a specific area. This is all up for discussion, and subject to sufficient interest. If you are interested, please add your name and email address to the Etherpad [2], along with any comments, thoughts or suggestions. [0] https://governance.openstack.org/sigs/ [1] https://www.openstack.org/four-opens/ [2] https://etherpad.openstack.org/p/kolla-sig Cheers, Mark From kchamart at redhat.com Fri Mar 6 16:45:37 2020 From: kchamart at redhat.com (Kashyap Chamarthy) Date: Fri, 6 Mar 2020 17:45:37 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <20200306164537.GB30104@paraplu> On Mon, Mar 02, 2020 at 04:45:47PM -0500, Mohammed Naser wrote: [...] > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. Personally, I've operated with the mindset of "multiple maintainers" / SMEs approach. Also that's the model I've seen successfully used in other upstream communities (10+ years old) that I participate in. It makes a lot of sense in the long-term, and sustainability-wise. IOW, "Vehement ACK" for the multiple maintiners model. [...] -- /kashyap From smooney at redhat.com Fri Mar 6 18:35:11 2020 From: smooney at redhat.com (Sean Mooney) Date: Fri, 06 Mar 2020 18:35:11 +0000 Subject: [nova][qa] When creating a instance using a flavor with "hw:cpu_policy=dedicated" "hw:cpu_realtime=yes" and "hw:cpu_realtime_mask=^0-1" failed. In-Reply-To: References: Message-ID: <318b8178190191764df5e4de63e145b888f9203b.camel@redhat.com> On Fri, 2020-03-06 at 07:57 +0000, Juntingqiu Qiujunting (邱军婷) wrote: > When I create a instance using a flavor with "hw:cpu_policy=dedicated" "hw:cpu_realtime=yes" and > "hw:cpu_realtime_mask=^0-1". > > > > The error is "libvirtError:Cannot set scheduler parameters for pid 3666:operation not permitted". i would guess that ytou are hiting an selinux or apparmor issue. in either case this is not a nova/openstack bug but rather a libvirt/host os configuration issue. i would check the output of dmesg and see if there are any messages that relate to this. > > > > https://bugs.launchpad.net/starlingx/+bug/1866311 > > > > The instance xml as following: > > 4 > > > > > > > > > > > The error log as following: > > ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] File "/usr/lib64/python2.7/site- > packages/libvirt.py", line 1099, in createWithFlags > ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] if ret == -1: raise libvirtError > ('virDomainCreateWithFlags() failed', dom=self) > ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] libvirtError: Cannot set scheduler > parameters for pid 6504: Operation not permitted > ERROR nova.compute.manager [instance: 74dcc0a1-9ed4-4d13-bef7-ac9623bca2d0] > > > > > From haleyb.dev at gmail.com Fri Mar 6 19:45:04 2020 From: haleyb.dev at gmail.com (Brian Haley) Date: Fri, 6 Mar 2020 14:45:04 -0500 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Message-ID: +1 from me, Lajos would be a great addition to the team. On 3/6/20 10:28 AM, Slawek Kaplonski wrote: > Hi neutrinos, > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the Neutron core team. > Lajos is Neutron contributor Neutron since around Queens cycle and now he is one of the most active reviewers in the Neutron group projects. > He was one of the key contributors in cooperation with Nova and Placement teams to deliver guaranteed minimum bandwidth feature in OpenStack. > He is very active and helpful with triaging and fixing Neutron bugs and issues in our CI. > > During last few cycles he proved that he has wide knowledge about Neutron code base. He is currently also a maintainer of some neutron stadium projects which shows that he has knowledge about code base not only about neutron but also Neutron stadium. > > The quality and number of his reviews are comparable to other members of the Neutron core team: https://www.stackalytics.com/?release=ussuri&module=neutron-group and are higher every cycle :) > I think he will be great addition to our core team. > > I will keep this nomination open for a week or until all current cores will respond. > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From zbitter at redhat.com Fri Mar 6 19:52:21 2020 From: zbitter at redhat.com (Zane Bitter) Date: Fri, 6 Mar 2020 14:52:21 -0500 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <1da63dab-c35e-6377-d5d8-e075a5c37408@ham.ie> Message-ID: <271de548-3f54-4b9e-99bb-ee819378ae77@redhat.com> On 5/03/20 1:29 pm, Kendall Nelson wrote: > I would like to think elections would NOT get held every 1-2 weeks or > whatever the chosen PTL term is for a project? Its just a like...signup > sheet sort of thing? I'd imagine in this model the most likely implementation would be a roster where core team members sign up for a slot. But I also imagine it being left up to the project to decide the mechanics. > What if more than one person wants to sign up for > the same week( I can't think of why this would happen, just thinking > about all the details)? I think just let people sort it out amongst themselves. Nobody is going to get too exercised over a problem that will literally resolve _itself_ in 1-2 weeks. From mark at openstack.org Fri Mar 6 20:56:41 2020 From: mark at openstack.org (Mark Collier) Date: Fri, 6 Mar 2020 14:56:41 -0600 (CST) Subject: FW: 2020 OSF Events & coronavirus Message-ID: <1583528201.853712216@emailsrvr.com> I wanted to make sure everyone saw this thread on the foundation mailing list, since I know not everyone is subscribed to both lists: Archive: http://lists.openstack.org/pipermail/foundation/2020-March/002852.html Please join that ML thread to share feedback on this topic, or you can reach out directly to myself or jonathan at openstack.org I saw first hand how we all pulled together during the Snowpenstack in Dublin, so I know we'll once again pull together as a community to get through this! Mark On Friday, March 6, 2020 12:55pm, "Mark Collier" said: > Stackers, >   > Before I get into the current plans for the OSF events in Vancouver and Berlin, I > wanted to say a few words in general about the virus impacting so many people > right now. >   > First, I wanted to acknowledge the very difficult situation many are facing > because of COVID-19 (Coronavirus), across open source communities and local > communities in general (tech or otherwise). I also want to say to everyone who is > on the front lines managing events, from the full time staffers to the volunteers, > to the contractors and production partners, that we have some idea of what you're > going through and we know this is a tough time. If there's anything we can do to > help, please reach out. In the best of times, event organization can be grueling > and thankless, and so I just want to say THANK YOU to everyone who does the > organizing work in the communities we all care so much about. >   > OSF 2020 EVENTS >   > When it comes to the 2020 events OSF is managing, namely the OpenDev + PTG in > Vancouver June 8-11 and the Open Infrastructure Summit in Berlin October 19-23, > please read and bookmark this status page which we will continue to update:  > https://www.openstack.org/events/covid-19-coronavirus-disease-updates >   > When it comes to our community, the health of every individual is of paramount > concern. We have always aimed to produce events "of the community, by the > community" and the upcoming event in Vancouver is no exception. The OpenDev tracks > each morning will be programmed by volunteers from the community, and the project > teams will be organizing their own conversations as well each afternoon M-W, and > all day Thursday.  >   > But the larger question is here: should the show go on?  > > The short answer is that as of now, the Vancouver and Berlin events are still > scheduled to happen in June (8-11) and October (19-23), respectively.  >   > However, we are willing to cancel or approach the events in a different way (i.e. > virtual) if the facts indicate that is the best path, and we know the facts are > changing rapidly. One of the most critical inputs we need is to hear from each of > you. We know that many of you rely on the twice-annual events to get together and > make rapid progress on the software, which is one reason we are not making any > decisions in haste. We also know that many of you may be unable or unwilling to > travel in June, and that is critical information to hear as we get closer to the > event so that we can make the most informed decision.  >   > I also wanted to answer a FAQ by letting everyone know that if either event is > cancelled, event tickets and sponsorships will be fully refunded. Please note that > if you're making travel arrangements (e.g. flights, hotels) those are outside of > our control. >   > So as we continue to monitor the news and listen to health experts to make an > informed decision on any changes to our event plans, we'd like to hear from > everyone in the community who has a stake in these events. Our most pressing topic > is of course Vanvouver, but if you have questions or concerns about the Berlin > plan feel free to share those as well. >   > If you'd like to connect directly, you can always contact Executive Director > Jonathan Bryce (jonathan at openstack.org) or myself (mark at openstack.org). >   > Key Links: > - STATUS PAGE: > https://www.openstack.org/events/covid-19-coronavirus-disease-updates > - Vancouver OpenDev + PTG https://www.openstack.org/events/opendev-ptg-2020/ > - Berlin Open Infrastructure Summit: https://www.openstack.org/summit/berlin-2020/ > > Key Dates for OpenDev + PTG in Vancouver: > - Schedule will be published in early April > - Early bird deadline is April 4 > - Final day to sponsor will be May 4 > - Final registration price increase will be in early May >   > Mark Collier > COO, OpenStack Foundation > @sparkycollier > > From Albert.Braden at synopsys.com Fri Mar 6 23:05:19 2020 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 6 Mar 2020 23:05:19 +0000 Subject: Goodbye for now Message-ID: My contract at Synopsys ends today, so I will have to continue my efforts to sign up as an Openstack developer at my next role. Thanks to everyone for all of the help and advice. Best wishes! Albert -------------- next part -------------- An HTML attachment was scrubbed... URL: From gouthampravi at gmail.com Fri Mar 6 23:12:37 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Fri, 6 Mar 2020 15:12:37 -0800 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> Message-ID: On Thu, Mar 5, 2020 at 11:53 AM Brian Rosmaita wrote: > On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > > ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita < > rosmaita.fossdev at gmail.com> wrote ---- > > > Hello QA team and devstack-plugin-ceph-core people, > > > > > > The Cinder team has some proposals we'd like to float. > > > > > > 1. The Cinder team is interested in becoming more active in the > > > maintenance of openstack/devstack-plugin-ceph [0]. Currently, the > > > devstack-plugin-ceph-core is > > > https://review.opendev.org/#/admin/groups/1196,members > > > The cinder-core is already represented by Eric and Sean; we'd like to > > > replace them by including the cinder-core group. > > > > +1. This is good diea and make sense, I will do the change. > > Great, thanks! > I agree this is a great idea to have more members of Cinder joining the devstack-plugin-ceph team. I would like to have atleast a sub team of manila core reviewers added to this project if it makes sense. The Manila CephFS drivers (cephfs-native and cephfs-nfs) are currently being tested with the help of the devstack integration in devstack-plugin-ceph. We have Tom Barron (tbarron) in the team, i'd like to propose myself (gouthamr) and Victoria Martinez de la Cruz (vkmc) Please let me know what you think of the idea. > > > > > > > 2. The Cinder team is interested in becoming more active in the > > > maintenance of x/devstack-plugin-nfs [1]. Currently, the > > > devstack-plugin-nfs-core is > > > https://review.opendev.org/#/admin/groups/1330,members > > > It's already 75% cinder-core members; we'd like to replace the > > > individual members with the cinder-core group. We also propose that > > > devstack-core be added as an included group. > > > > > > 3. The Cinder team is interested in implementing a new devstack > plugin: > > > openstack/devstack-plugin-open-cas > > > This will enable thorough testing of a new feature [2] being > introduced > > > as experimental in Ussuri and expected to be finalized in Victoria. > Our > > > plan would be to make both cinder-core and devstack-core included > groups > > > for the gerrit group governing the new plugin. > > > > +1. You want this under Cinder governance or under QA ? > > I think it makes sense for these to be under QA governance -- QA would > own the repo with both QA and Cinder having permission to make changes. > > > > > > > 4. This is a minor point, but can the devstack-plugin-nfs repo be > moved > > > back into the 'openstack' namespace? > > > > If this is usable plugin for nfs testing (I am not aware if we have any > other) then > > it make sense to bring it to openstack governance. > > Same question here, do you want to put this under Cinder governance or > QA. > > Same here, I think QA should "own" the repo, but Cinder will have > permission to make changes there. > > > > > Those plugins under QA governance also ok for me with your proposal of > calloborative maintainance by > > devstack-core and cinder-core. > > > > -gmann > > Thanks for the quick response! > > > > > > > Let us know which of these proposals you find acceptable. > > > > > > > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > > > [1] https://opendev.org/x/devstack-plugin-nfs > > > [2] > https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hongbin034 at gmail.com Sat Mar 7 03:07:42 2020 From: hongbin034 at gmail.com (Hongbin Lu) Date: Fri, 6 Mar 2020 22:07:42 -0500 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Message-ID: +1 from me. Best regards, Hongbin On Fri, Mar 6, 2020 at 10:34 AM Slawek Kaplonski wrote: > Hi neutrinos, > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the > Neutron core team. > Lajos is Neutron contributor Neutron since around Queens cycle and now he > is one of the most active reviewers in the Neutron group projects. > He was one of the key contributors in cooperation with Nova and Placement > teams to deliver guaranteed minimum bandwidth feature in OpenStack. > He is very active and helpful with triaging and fixing Neutron bugs and > issues in our CI. > > During last few cycles he proved that he has wide knowledge about Neutron > code base. He is currently also a maintainer of some neutron stadium > projects which shows that he has knowledge about code base not only about > neutron but also Neutron stadium. > > The quality and number of his reviews are comparable to other members of > the Neutron core team: > https://www.stackalytics.com/?release=ussuri&module=neutron-group and are > higher every cycle :) > I think he will be great addition to our core team. > > I will keep this nomination open for a week or until all current cores > will respond. > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsharma1818 at outlook.com Sat Mar 7 13:29:33 2020 From: rsharma1818 at outlook.com (Rahul Sharma) Date: Sat, 7 Mar 2020 13:29:33 +0000 Subject: [Horizon] Unable to access the dashboard page Message-ID: Hi, I am trying to get OpenStack (Train) up and running on CentOS 7 by following the "OpenStack Installation Guide" provided on OpenStack's website and have completed installation of below components - Keystone - Glance - Placement - Nova - Networking After installing each component, I have also verified its operation and it seems to be working successfully. However, there is a problem I am facing after installing "Horizon" for dashboard services. In order to verify its operation, one is supposed to browse to the URL "http://controller/horizon/" where "controller" could be the hostname or IP address of the Node which is running the controller Browsing to the above URL throws an error "The requested URL /horizon was not found on this server." In the apache access logs, I see below error: " 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" " If I browse to the URL "http://controller/", the default "Testing123" page of apache gets loaded. Please assist. Thanks, Rahul -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Sat Mar 7 15:15:55 2020 From: donny at fortnebula.com (Donny Davis) Date: Sat, 7 Mar 2020 10:15:55 -0500 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: Message-ID: Try /dashboard Donny Davis c: 805 814 6800 On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma wrote: > Hi, > > I am trying to get OpenStack (Train) up and running on CentOS 7 by > following the "OpenStack Installation Guide" provided on OpenStack's > website and have completed installation of below components > > > - Keystone > - Glance > - Placement > - Nova > - Networking > > > After installing each component, I have also verified its operation and it > seems to be working successfully. > > However, there is a problem I am facing after installing "Horizon" for > dashboard services. In order to verify its operation, one is supposed to > browse to the URL "http://controller/horizon/" where "controller" could > be the hostname or IP address of the Node which is running the controller > > Browsing to the above URL throws an error "The requested URL /horizon was > not found on this server." > > In the apache access logs, I see below error: > > " > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 > 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 > (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" > 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; > x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 > Safari/537.36" > " > > If I browse to the URL "http://controller/", the default "Testing123" > page of apache gets loaded. > > > Please assist. > > Thanks, > Rahul > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsharma1818 at outlook.com Sat Mar 7 17:41:31 2020 From: rsharma1818 at outlook.com (Rahul Sharma) Date: Sat, 7 Mar 2020 17:41:31 +0000 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: , Message-ID: Hi, Tried with /dashboard but it ain't working Getting error "The requested URL /auth/login/ was not found on this server." in the browser (Error 404) ________________________________ From: Donny Davis Sent: Saturday, March 7, 2020 8:45 PM To: Rahul Sharma Cc: OpenStack Discuss Subject: Re: [Horizon] Unable to access the dashboard page Try /dashboard Donny Davis c: 805 814 6800 On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma > wrote: Hi, I am trying to get OpenStack (Train) up and running on CentOS 7 by following the "OpenStack Installation Guide" provided on OpenStack's website and have completed installation of below components - Keystone - Glance - Placement - Nova - Networking After installing each component, I have also verified its operation and it seems to be working successfully. However, there is a problem I am facing after installing "Horizon" for dashboard services. In order to verify its operation, one is supposed to browse to the URL "http://controller/horizon/" where "controller" could be the hostname or IP address of the Node which is running the controller Browsing to the above URL throws an error "The requested URL /horizon was not found on this server." In the apache access logs, I see below error: " 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" " If I browse to the URL "http://controller/", the default "Testing123" page of apache gets loaded. Please assist. Thanks, Rahul -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 7 17:45:17 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 7 Mar 2020 18:45:17 +0100 Subject: [queens] [neutron]security_groups_log] Message-ID: Hello, I have queens installation based on centos7. Before implementing security groups logs, I had the following configuration in /etc/neutron/plugins/ml2/openvswitch_agent.ini: firewall_driver = iptables_hybrid Enabling security groups log I had to change it in : firewall_driver = openvswitch It seems to work end security logs are logged . After restarting kvm nodes and controllers, virtual machines do not live migrate. The firewall driver change could be the cause of my problem ? firewall_driver = openvswitch is mandatory for security groups log ? Please, any help ? I cannot reproduce the problem rebooting all my nodes. I rebooted them because I hat to transfer from a rack to another. Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 7 17:48:44 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 7 Mar 2020 18:48:44 +0100 Subject: [queens] [neutron]security_groups_log] issues Message-ID: Hello, I have queens installation based on centos7. Before implementing security groups logs, I had the following configuration in /etc/neutron/plugins/ml2/openvswitch_agent.ini: firewall_driver = iptables_hybrid Enabling security groups log I had to change it in : firewall_driver = openvswitch It seems to work end security logs are logged . After restarting kvm nodes and controllers, virtual machines do not live migrate. I rolled back my configuration without security groups logs feature and now all works fine. The firewall driver change could be the cause of my problem ? firewall_driver = openvswitch is mandatory for security groups log ? Please, any help ? I cannot reproduce the problem rebooting all my nodes. I rebooted them because I hat to transfer from a rack to another. Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Sat Mar 7 18:02:25 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Sat, 7 Mar 2020 19:02:25 +0100 Subject: [queens] [neutron]security_groups_log] In-Reply-To: References: Message-ID: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> Hi, > On 7 Mar 2020, at 18:45, Ignazio Cassano wrote: > > Hello, I have queens installation based on centos7. > > Before implementing security groups logs, I had the following configuration in > /etc/neutron/plugins/ml2/openvswitch_agent.ini: > > firewall_driver = iptables_hybrid > > > Enabling security groups log I had to change it in : > > firewall_driver = openvswitch > > > It seems to work end security logs are logged . > After restarting kvm nodes and controllers, virtual machines do not live migrate. > The firewall driver change could be the cause of my problem ? Yes, In queens there wasn’t yet migration between various firewall drivers so that can be an issue. It should works fine since Rocky release with “multiple port bindings” feature. > firewall_driver = openvswitch is mandatory for security groups log ? Yes, logging isn’t supported by iptables_hybrid driver. > > Please, any help ? > > > I cannot reproduce the problem rebooting all my nodes. > I rebooted them because I hat to transfer from a rack to another. > > Ignazio > > — Slawek Kaplonski Senior software engineer Red Hat From ignaziocassano at gmail.com Sat Mar 7 18:09:59 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 7 Mar 2020 19:09:59 +0100 Subject: [queens] [neutron]security_groups_log] In-Reply-To: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> References: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> Message-ID: Many thanks Slawek. Your help is always very appreciated. Ignazio Il giorno sab 7 mar 2020 alle ore 19:02 Slawek Kaplonski < skaplons at redhat.com> ha scritto: > Hi, > > > On 7 Mar 2020, at 18:45, Ignazio Cassano > wrote: > > > > Hello, I have queens installation based on centos7. > > > > Before implementing security groups logs, I had the following > configuration in > > /etc/neutron/plugins/ml2/openvswitch_agent.ini: > > > > firewall_driver = iptables_hybrid > > > > > > Enabling security groups log I had to change it in : > > > > firewall_driver = openvswitch > > > > > > It seems to work end security logs are logged . > > After restarting kvm nodes and controllers, virtual machines do not live > migrate. > > The firewall driver change could be the cause of my problem ? > > Yes, In queens there wasn’t yet migration between various firewall drivers > so that can be an issue. It should works fine since Rocky release with > “multiple port bindings” feature. > > > firewall_driver = openvswitch is mandatory for security groups log ? > > Yes, logging isn’t supported by iptables_hybrid driver. > > > > > Please, any help ? > > > > > > I cannot reproduce the problem rebooting all my nodes. > > I rebooted them because I hat to transfer from a rack to another. > > > > Ignazio > > > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 7 20:45:52 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 7 Mar 2020 21:45:52 +0100 Subject: [queens] [neutron]security_groups_log] In-Reply-To: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> References: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> Message-ID: Slawek, forgive me if I take advantage of your patience. Before rebooting nodes, I modified nodes and controllers with security groups logs, modifying neutron.conf, ml2 and openvswitch agents, moving from iptables_hybrid to openvswitch firewall etc etc..... I only restarted neutron components and before rebooting nodes and controllers, I saw security groups logs and I was able to migrate instances. Why after rebooting not ? And, please, what about “multiple port bindings” ? Thanks Ignazio Il giorno sab 7 mar 2020 alle ore 19:02 Slawek Kaplonski < skaplons at redhat.com> ha scritto: > Hi, > > > On 7 Mar 2020, at 18:45, Ignazio Cassano > wrote: > > > > Hello, I have queens installation based on centos7. > > > > Before implementing security groups logs, I had the following > configuration in > > /etc/neutron/plugins/ml2/openvswitch_agent.ini: > > > > firewall_driver = iptables_hybrid > > > > > > Enabling security groups log I had to change it in : > > > > firewall_driver = openvswitch > > > > > > It seems to work end security logs are logged . > > After restarting kvm nodes and controllers, virtual machines do not live > migrate. > > The firewall driver change could be the cause of my problem ? > > Yes, In queens there wasn’t yet migration between various firewall drivers > so that can be an issue. It should works fine since Rocky release with > “multiple port bindings” feature. > > > firewall_driver = openvswitch is mandatory for security groups log ? > > Yes, logging isn’t supported by iptables_hybrid driver. > > > > > Please, any help ? > > > > > > I cannot reproduce the problem rebooting all my nodes. > > I rebooted them because I hat to transfer from a rack to another. > > > > Ignazio > > > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Sun Mar 8 03:00:40 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sat, 07 Mar 2020 21:00:40 -0600 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> Message-ID: <170b81663ff.c070e14d541882.3735232197665233208@ghanshyammann.com> ---- On Thu, 05 Mar 2020 13:49:11 -0600 Brian Rosmaita wrote ---- > On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > > ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita wrote ---- > > > Hello QA team and devstack-plugin-ceph-core people, > > > > > > The Cinder team has some proposals we'd like to float. > > > > > > 1. The Cinder team is interested in becoming more active in the > > > maintenance of openstack/devstack-plugin-ceph [0]. Currently, the > > > devstack-plugin-ceph-core is > > > https://review.opendev.org/#/admin/groups/1196,members > > > The cinder-core is already represented by Eric and Sean; we'd like to > > > replace them by including the cinder-core group. > > > > +1. This is good diea and make sense, I will do the change. > > Great, thanks! Done. > > > > > > > 2. The Cinder team is interested in becoming more active in the > > > maintenance of x/devstack-plugin-nfs [1]. Currently, the > > > devstack-plugin-nfs-core is > > > https://review.opendev.org/#/admin/groups/1330,members > > > It's already 75% cinder-core members; we'd like to replace the > > > individual members with the cinder-core group. We also propose that > > > devstack-core be added as an included group. > > > > > > 3. The Cinder team is interested in implementing a new devstack plugin: > > > openstack/devstack-plugin-open-cas > > > This will enable thorough testing of a new feature [2] being introduced > > > as experimental in Ussuri and expected to be finalized in Victoria. Our > > > plan would be to make both cinder-core and devstack-core included groups > > > for the gerrit group governing the new plugin. > > > > +1. You want this under Cinder governance or under QA ? > > I think it makes sense for these to be under QA governance -- QA would > own the repo with both QA and Cinder having permission to make changes. Sure. Please let me know once it is ready or propose it under QA and I will review that. > > > > > > > 4. This is a minor point, but can the devstack-plugin-nfs repo be moved > > > back into the 'openstack' namespace? > > > > If this is usable plugin for nfs testing (I am not aware if we have any other) then > > it make sense to bring it to openstack governance. > > Same question here, do you want to put this under Cinder governance or QA. > > Same here, I think QA should "own" the repo, but Cinder will have > permission to make changes there. Sounds good. I proposed the patches: https://review.opendev.org/#/q/topic:devstack-plugin-nfs+(status:open+OR+status:merged) -gmann > > > > > Those plugins under QA governance also ok for me with your proposal of calloborative maintainance by > > devstack-core and cinder-core. > > > > -gmann > > Thanks for the quick response! > > > > > > > Let us know which of these proposals you find acceptable. > > > > > > > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > > > [1] https://opendev.org/x/devstack-plugin-nfs > > > [2] https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > > > > > > > > > > From donny at fortnebula.com Sun Mar 8 12:42:53 2020 From: donny at fortnebula.com (Donny Davis) Date: Sun, 8 Mar 2020 08:42:53 -0400 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: Message-ID: You may also want to check this setting https://docs.openstack.org/horizon/train/configuration/settings.html#allowed-hosts Donny Davis c: 805 814 6800 On Sat, Mar 7, 2020, 12:41 PM Rahul Sharma wrote: > Hi, > > Tried with /dashboard but it ain't working > > Getting error "The requested URL /auth/login/ was not found on this > server." in the browser (Error 404) > > > ------------------------------ > *From:* Donny Davis > *Sent:* Saturday, March 7, 2020 8:45 PM > *To:* Rahul Sharma > *Cc:* OpenStack Discuss > *Subject:* Re: [Horizon] Unable to access the dashboard page > > Try /dashboard > > Donny Davis > c: 805 814 6800 > > On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma wrote: > > Hi, > > I am trying to get OpenStack (Train) up and running on CentOS 7 by > following the "OpenStack Installation Guide" provided on OpenStack's > website and have completed installation of below components > > > - Keystone > - Glance > - Placement > - Nova > - Networking > > > After installing each component, I have also verified its operation and it > seems to be working successfully. > > However, there is a problem I am facing after installing "Horizon" for > dashboard services. In order to verify its operation, one is supposed to > browse to the URL "http://controller/horizon/" where "controller" could > be the hostname or IP address of the Node which is running the controller > > Browsing to the above URL throws an error "The requested URL /horizon was > not found on this server." > > In the apache access logs, I see below error: > > " > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 > 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 > (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" > 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; > x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 > Safari/537.36" > " > > If I browse to the URL "http://controller/", the default "Testing123" > page of apache gets loaded. > > > Please assist. > > Thanks, > Rahul > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Sun Mar 8 14:07:47 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Sun, 8 Mar 2020 15:07:47 +0100 Subject: [queens] [neutron]security_groups_log] In-Reply-To: References: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> Message-ID: Hi, > On 7 Mar 2020, at 21:45, Ignazio Cassano wrote: > > Slawek, forgive me if I take advantage of your patience. > > Before rebooting nodes, I modified nodes and controllers with security groups logs, modifying neutron.conf, ml2 and openvswitch agents, moving from iptables_hybrid to openvswitch firewall etc etc..... > I only restarted neutron components and before rebooting nodes and controllers, I saw security groups logs and I was able to migrate instances. > Why after rebooting not ? To be honest I don’t know why it’s like that. You probably will need to give more info there, what errors You have exactly during the migration. > And, please, what about “multiple port bindings” ? Spec for this feature is at https://specs.openstack.org/openstack/neutron-specs/specs/ocata/portbinding_information_for_nova.html - You should find more details about it there. > > Thanks > Ignazio > > > > Il giorno sab 7 mar 2020 alle ore 19:02 Slawek Kaplonski ha scritto: > Hi, > > > On 7 Mar 2020, at 18:45, Ignazio Cassano wrote: > > > > Hello, I have queens installation based on centos7. > > > > Before implementing security groups logs, I had the following configuration in > > /etc/neutron/plugins/ml2/openvswitch_agent.ini: > > > > firewall_driver = iptables_hybrid > > > > > > Enabling security groups log I had to change it in : > > > > firewall_driver = openvswitch > > > > > > It seems to work end security logs are logged . > > After restarting kvm nodes and controllers, virtual machines do not live migrate. > > The firewall driver change could be the cause of my problem ? > > Yes, In queens there wasn’t yet migration between various firewall drivers so that can be an issue. It should works fine since Rocky release with “multiple port bindings” feature. > > > firewall_driver = openvswitch is mandatory for security groups log ? > > Yes, logging isn’t supported by iptables_hybrid driver. > > > > > Please, any help ? > > > > > > I cannot reproduce the problem rebooting all my nodes. > > I rebooted them because I hat to transfer from a rack to another. > > > > Ignazio > > > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > — Slawek Kaplonski Senior software engineer Red Hat From amy at demarco.com Sun Mar 8 14:32:57 2020 From: amy at demarco.com (Amy) Date: Sun, 8 Mar 2020 09:32:57 -0500 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: Message-ID: Another thing to check is the indentation in your config file. Amy (spotz) > On Mar 8, 2020, at 7:46 AM, Donny Davis wrote: > >  > You may also want to check this setting > https://docs.openstack.org/horizon/train/configuration/settings.html#allowed-hosts > > Donny Davis > c: 805 814 6800 > >> On Sat, Mar 7, 2020, 12:41 PM Rahul Sharma wrote: >> Hi, >> >> Tried with /dashboard but it ain't working >> >> Getting error "The requested URL /auth/login/ was not found on this server." in the browser (Error 404) >> >> >> From: Donny Davis >> Sent: Saturday, March 7, 2020 8:45 PM >> To: Rahul Sharma >> Cc: OpenStack Discuss >> Subject: Re: [Horizon] Unable to access the dashboard page >> >> Try /dashboard >> >> Donny Davis >> c: 805 814 6800 >> >> On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma wrote: >> Hi, >> >> I am trying to get OpenStack (Train) up and running on CentOS 7 by following the "OpenStack Installation Guide" provided on OpenStack's website and have completed installation of below components >> >> >> - Keystone >> - Glance >> - Placement >> - Nova >> - Networking >> >> >> After installing each component, I have also verified its operation and it seems to be working successfully. >> >> However, there is a problem I am facing after installing "Horizon" for dashboard services. In order to verify its operation, one is supposed to browse to the URL "http://controller/horizon/" where "controller" could be the hostname or IP address of the Node which is running the controller >> >> Browsing to the above URL throws an error "The requested URL /horizon was not found on this server." >> >> In the apache access logs, I see below error: >> >> " >> 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" >> 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" >> " >> >> If I browse to the URL "http://controller/", the default "Testing123" page of apache gets loaded. >> >> >> Please assist. >> >> Thanks, >> Rahul -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sun Mar 8 16:17:53 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 8 Mar 2020 17:17:53 +0100 Subject: [queens] [neutron]security_groups_log] In-Reply-To: References: <2B63B1E5-F914-41C5-9A37-C2E0E96665A5@redhat.com> Message-ID: Hi, I think the problem is the migration from iptables_hybrid to openvswitch firewall : https://docs.openstack.org/neutron/rocky/contributor/internals/openvswitch_firewall.html Thanks Ignazio Il Dom 8 Mar 2020, 15:07 Slawek Kaplonski ha scritto: > Hi, > > > On 7 Mar 2020, at 21:45, Ignazio Cassano > wrote: > > > > Slawek, forgive me if I take advantage of your patience. > > > > Before rebooting nodes, I modified nodes and controllers with security > groups logs, modifying neutron.conf, ml2 and openvswitch agents, moving > from iptables_hybrid to openvswitch firewall etc etc..... > > I only restarted neutron components and before rebooting nodes and > controllers, I saw security groups logs and I was able to migrate instances. > > Why after rebooting not ? > > To be honest I don’t know why it’s like that. You probably will need to > give more info there, what errors You have exactly during the migration. > > > And, please, what about “multiple port bindings” ? > > Spec for this feature is at > https://specs.openstack.org/openstack/neutron-specs/specs/ocata/portbinding_information_for_nova.html > - You should find more details about it there. > > > > > Thanks > > Ignazio > > > > > > > > Il giorno sab 7 mar 2020 alle ore 19:02 Slawek Kaplonski < > skaplons at redhat.com> ha scritto: > > Hi, > > > > > On 7 Mar 2020, at 18:45, Ignazio Cassano > wrote: > > > > > > Hello, I have queens installation based on centos7. > > > > > > Before implementing security groups logs, I had the following > configuration in > > > /etc/neutron/plugins/ml2/openvswitch_agent.ini: > > > > > > firewall_driver = iptables_hybrid > > > > > > > > > Enabling security groups log I had to change it in : > > > > > > firewall_driver = openvswitch > > > > > > > > > It seems to work end security logs are logged . > > > After restarting kvm nodes and controllers, virtual machines do not > live migrate. > > > The firewall driver change could be the cause of my problem ? > > > > Yes, In queens there wasn’t yet migration between various firewall > drivers so that can be an issue. It should works fine since Rocky release > with “multiple port bindings” feature. > > > > > firewall_driver = openvswitch is mandatory for security groups log ? > > > > Yes, logging isn’t supported by iptables_hybrid driver. > > > > > > > > Please, any help ? > > > > > > > > > I cannot reproduce the problem rebooting all my nodes. > > > I rebooted them because I hat to transfer from a rack to another. > > > > > > Ignazio > > > > > > > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.deluca at gmail.com Sun Mar 8 21:52:10 2020 From: alfredo.deluca at gmail.com (Alfredo De Luca) Date: Sun, 8 Mar 2020 22:52:10 +0100 Subject: [CINDER] Snapshots export In-Reply-To: References: <20200304155850.b4ydu4vfxthih7we@localhost> Message-ID: Hi Sean. Sorry for the late reply. What we want to do is backing up snapshots in case of a complete compute lost of as a plan for disaster recovery. So after recreating the environment we can restore snapshots and start the VMs again. Cheers On Wed, Mar 4, 2020 at 10:14 PM Sean McGinnis wrote: > On 3/4/20 9:58 AM, Gorka Eguileor wrote: > > On 03/03, Alfredo De Luca wrote: > >> Hi all. > >> We have our env with Openstack (Train) and cinder with CEPH (nautilus) > >> backend. > >> We are creating automatic volumes snapshots and now we'd like to export > >> them as a backup/restore plan. After exporting the snapshots we will use > >> Acronis as backup tool. > >> > >> I couldn't find the right steps/commands to exports the snapshots. > >> Any info? > >> Cheers > >> > >> -- > >> *Alfredo* > > Hi Alfredo, > > > > What kind of backup/restore plan do you have planned? > > > > Because snapshots are not meant to be used in a Disaster Recovery > > backup/restore plan, so the only thing available are the manage/unmanage > > commands. > > > > These commands are meant to add an existing volume/snapshots into Cinder > > together, not to unmanage/manage them independently. > > > > For example, you wouldn't be able to manage a snapshot if the volume is > > not already managed. Also unmanaging the snapshot would block the > > deletion of the RBD volume itself. > > > > Cheers, > > Gorka. > > If the intent is to use the snapshots as a source to backup the volume > data, leaving the actual volume attached and IO running but still > getting a "static" view of the code, then you would need to create a > volume from the chosen snapshot, mount that volume somewhere that is > accessible to your backup software, perform the copy of the data, then > delete the volume when complete. > > I haven't used Acronis myself, but the issue for some backup software > could be that the volume it is backing up from is going to be different > every time. Though you could make sure it is mounted at the same place > so the backup software at least *thinks* it's backing up the same thing. > > Then restoring the data will likely require some manual intervention, > but that's pretty much always the case when something goes wrong. I > would just recommend you test the full disaster recovery scenario to > make sure you have that figured out and working right before you > actually need it. > > Sean > > > -- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Sun Mar 8 22:20:54 2020 From: donny at fortnebula.com (Donny Davis) Date: Sun, 8 Mar 2020 18:20:54 -0400 Subject: [CINDER] Snapshots export In-Reply-To: References: <20200304155850.b4ydu4vfxthih7we@localhost> Message-ID: On Sun, Mar 8, 2020, 5:55 PM Alfredo De Luca wrote: > Hi Sean. Sorry for the late reply. > What we want to do is backing up snapshots in case of a complete compute > lost of as a plan for disaster recovery. > So after recreating the environment we can restore snapshots and start the > VMs again. > > Cheers > > > On Wed, Mar 4, 2020 at 10:14 PM Sean McGinnis > wrote: > >> On 3/4/20 9:58 AM, Gorka Eguileor wrote: >> > On 03/03, Alfredo De Luca wrote: >> >> Hi all. >> >> We have our env with Openstack (Train) and cinder with CEPH (nautilus) >> >> backend. >> >> We are creating automatic volumes snapshots and now we'd like to export >> >> them as a backup/restore plan. After exporting the snapshots we will >> use >> >> Acronis as backup tool. >> >> >> >> I couldn't find the right steps/commands to exports the snapshots. >> >> Any info? >> >> Cheers >> >> >> >> -- >> >> *Alfredo* >> > Hi Alfredo, >> > >> > What kind of backup/restore plan do you have planned? >> > >> > Because snapshots are not meant to be used in a Disaster Recovery >> > backup/restore plan, so the only thing available are the manage/unmanage >> > commands. >> > >> > These commands are meant to add an existing volume/snapshots into Cinder >> > together, not to unmanage/manage them independently. >> > >> > For example, you wouldn't be able to manage a snapshot if the volume is >> > not already managed. Also unmanaging the snapshot would block the >> > deletion of the RBD volume itself. >> > >> > Cheers, >> > Gorka. >> >> If the intent is to use the snapshots as a source to backup the volume >> data, leaving the actual volume attached and IO running but still >> getting a "static" view of the code, then you would need to create a >> volume from the chosen snapshot, mount that volume somewhere that is >> accessible to your backup software, perform the copy of the data, then >> delete the volume when complete. >> >> I haven't used Acronis myself, but the issue for some backup software >> could be that the volume it is backing up from is going to be different >> every time. Though you could make sure it is mounted at the same place >> so the backup software at least *thinks* it's backing up the same thing. >> >> Then restoring the data will likely require some manual intervention, >> but that's pretty much always the case when something goes wrong. I >> would just recommend you test the full disaster recovery scenario to >> make sure you have that figured out and working right before you >> actually need it. >> >> Sean >> >> >> > > -- > *Alfredo* > > Is there a reason not to use the cinder backup feature? This function works for me backing up ceph volumes to swift. Once the backup is in swift it's very easy to pull it down to replicate it somewhere else. There are also other backup targets using the built in provider. It's worth checking out. Donny Davis > c: 805 814 6800 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amotoki at gmail.com Mon Mar 9 03:00:32 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Mon, 9 Mar 2020 12:00:32 +0900 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: Message-ID: I think you are hitting https://bugs.launchpad.net/horizon/+bug/1853651. The bug says WEBROOT needs to be configured. It was reported against the horizon installation guide on RHEL/CentOS, but I believe it is a bug on CentOS packaging as a package should work with the default config provided by the package. On Sun, Mar 8, 2020 at 2:45 AM Rahul Sharma wrote: > > Hi, > > Tried with /dashboard but it ain't working > > Getting error "The requested URL /auth/login/ was not found on this server." in the browser (Error 404) > > > ________________________________ > From: Donny Davis > Sent: Saturday, March 7, 2020 8:45 PM > To: Rahul Sharma > Cc: OpenStack Discuss > Subject: Re: [Horizon] Unable to access the dashboard page > > Try /dashboard > > Donny Davis > c: 805 814 6800 > > On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma wrote: > > Hi, > > I am trying to get OpenStack (Train) up and running on CentOS 7 by following the "OpenStack Installation Guide" provided on OpenStack's website and have completed installation of below components > > > - Keystone > - Glance > - Placement > - Nova > - Networking > > > After installing each component, I have also verified its operation and it seems to be working successfully. > > However, there is a problem I am facing after installing "Horizon" for dashboard services. In order to verify its operation, one is supposed to browse to the URL "http://controller/horizon/" where "controller" could be the hostname or IP address of the Node which is running the controller > > Browsing to the above URL throws an error "The requested URL /horizon was not found on this server." > > In the apache access logs, I see below error: > > " > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" > " > > If I browse to the URL "http://controller/", the default "Testing123" page of apache gets loaded. > > > Please assist. > > Thanks, > Rahul From jean-philippe at evrard.me Mon Mar 9 07:45:24 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Mon, 09 Mar 2020 08:45:24 +0100 Subject: [tc] March meeting Message-ID: <26f010de7fedd1125073c0d7b8221f9cb86988c5.camel@evrard.me> Hello, Here are the minutes for the march meeting [1]. There are a few action points for you, please have a look! Regards, Jean-Philippe Evrard (evrardjp) [1]: http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-03-05-14.00.html From jean-philippe at evrard.me Mon Mar 9 07:54:01 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Mon, 09 Mar 2020 08:54:01 +0100 Subject: [tc] April meeting Message-ID: <8ac6572483e17fb25a67f2858e69f4117fa8d624.camel@evrard.me> Hello everyone, It would be nice if you could update the agenda on the wiki [1] for the April meeting, happening on April, 2nd. Thank you in advance, Jean-Philippe Evrard (evrardjp). [1]: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting From jean-philippe at evrard.me Mon Mar 9 08:58:13 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Mon, 09 Mar 2020 09:58:13 +0100 Subject: [all][tc] What happened in OpenStack Governance recently Message-ID: Hello, It's been a while there was no update on "what happened in governance recently?" (February). So here is a summary for you! We had two meetings since the last community update, February [1] and March [2]. We have established our first (business-focused) upstream investment opportunities for 2020, which currently consists of: - Goal champions - Consistent and secure policy defaults - QA developers You can see the latest version in [3]. The timing for the next PTL and TC elections is now determined [4]. We've introduced the ideas framework, allowing you to propose, follow, and track large scale changes of OpenStack, with their conversations history [5]. We are starting to split OpenDev out of OpenStack-infra. Our next release names are Victoria and Wallaby. In terms of community goals [6], we've changed the U goal for documentation, so please have a look if you haven't already [7][8], and/or talk to the goal champion Kendall (diablo_rojo). The 'Switch legacy Zuul jobs to native' goal has been selected for Victoria. Please talk to its champion, Luigi Toscano (tosky)! On the projects' side, we've clarified the guidelines to drop official project teams [9]. We want to retire teams more actively if necessary, to allow us to increase our focus. Some projects have changed: - networking-ovn won't get releases anymore, as it's merged back into neutron in U [10] - neutron-lbaas was retired in Ussuri [11] Have a good day to you all! Regards, Jean-Philippe Evrard (evrardjp) [1]: http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-02-06-14.00.html [2]: http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-03-05-14.00.html [3]: https://governance.openstack.org/tc/reference/upstream-investment-opportunities/index.html [4]: https://review.opendev.org/#/c/708470/3/configuration.yaml [5]: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012847.html [6]: https://governance.openstack.org/tc/goals/selected/index.html [7]: https://review.opendev.org/#/c/708672/ [8]: https://review.opendev.org/#/c/709617/ [9]: https://review.opendev.org/#/c/707421/ [10]: https://review.opendev.org/#/c/705781/ [11]: https://review.opendev.org/#/c/705780/ From balazs.gibizer at est.tech Mon Mar 9 08:59:00 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 09 Mar 2020 09:59:00 +0100 Subject: [nova] US meeting slot Message-ID: <1583744340.12170.17@est.tech> Hi, Nova has alternate meeting slots on Thursdays to try to cover contributors from different time zones. * 14:00 UTC * 21:00 UTC As I'm taking over the PTL role form Eric I need to figure out how to run the nova meetings. I cannot really run the UTC 21:00 as it is pretty late for me. (I will run the 14:00 UTC slot). I see different options: a) Somebody from the US side of the globe volunteers to run the 21:00 UTC slot. Please speak up if you would like to run it. I can help you with agenda refresh and technicalities if needed. b) Have only one meeting time, and move that to 16:00 UTC. In this case I will be able to run it most of the weeks. c) Do not have a dedicated meeting slot but switch to office hours. Here we also need to find a time slot. I think 16:00 UTC could work there as well. Please share your view! Any other proposal is very welcome. Cheers, gibi From Cyrille.CARTIER at antemeta.fr Mon Mar 9 09:00:18 2020 From: Cyrille.CARTIER at antemeta.fr (Cyrille CARTIER) Date: Mon, 9 Mar 2020 09:00:18 +0000 Subject: [CINDER] Snapshots export In-Reply-To: References: <20200304155850.b4ydu4vfxthih7we@localhost> Message-ID: Hi Alfredo, In addition of cinder backup feature, you may try Freezer Project. With Freezer, you’ll be able to backup in swift or on a remote storage. Cheers, Cyrille De : Donny Davis [mailto:donny at fortnebula.com] Envoyé : dimanche 8 mars 2020 23:21 À : Alfredo De Luca Cc : Sean McGinnis ; openstack-discuss Objet : Re: [CINDER] Snapshots export On Sun, Mar 8, 2020, 5:55 PM Alfredo De Luca > wrote: Hi Sean. Sorry for the late reply. What we want to do is backing up snapshots in case of a complete compute lost of as a plan for disaster recovery. So after recreating the environment we can restore snapshots and start the VMs again. Cheers On Wed, Mar 4, 2020 at 10:14 PM Sean McGinnis > wrote: On 3/4/20 9:58 AM, Gorka Eguileor wrote: > On 03/03, Alfredo De Luca wrote: >> Hi all. >> We have our env with Openstack (Train) and cinder with CEPH (nautilus) >> backend. >> We are creating automatic volumes snapshots and now we'd like to export >> them as a backup/restore plan. After exporting the snapshots we will use >> Acronis as backup tool. >> >> I couldn't find the right steps/commands to exports the snapshots. >> Any info? >> Cheers >> >> -- >> *Alfredo* > Hi Alfredo, > > What kind of backup/restore plan do you have planned? > > Because snapshots are not meant to be used in a Disaster Recovery > backup/restore plan, so the only thing available are the manage/unmanage > commands. > > These commands are meant to add an existing volume/snapshots into Cinder > together, not to unmanage/manage them independently. > > For example, you wouldn't be able to manage a snapshot if the volume is > not already managed. Also unmanaging the snapshot would block the > deletion of the RBD volume itself. > > Cheers, > Gorka. If the intent is to use the snapshots as a source to backup the volume data, leaving the actual volume attached and IO running but still getting a "static" view of the code, then you would need to create a volume from the chosen snapshot, mount that volume somewhere that is accessible to your backup software, perform the copy of the data, then delete the volume when complete. I haven't used Acronis myself, but the issue for some backup software could be that the volume it is backing up from is going to be different every time. Though you could make sure it is mounted at the same place so the backup software at least *thinks* it's backing up the same thing. Then restoring the data will likely require some manual intervention, but that's pretty much always the case when something goes wrong. I would just recommend you test the full disaster recovery scenario to make sure you have that figured out and working right before you actually need it. Sean -- Alfredo Is there a reason not to use the cinder backup feature? This function works for me backing up ceph volumes to swift. Once the backup is in swift it's very easy to pull it down to replicate it somewhere else. There are also other backup targets using the built in provider. It's worth checking out. Donny Davis c: 805 814 6800 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Mon Mar 9 09:04:21 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Mon, 9 Mar 2020 10:04:21 +0100 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: <1583528201.853712216@emailsrvr.com> References: <1583528201.853712216@emailsrvr.com> Message-ID: Hi, It seems that the critical bit we don't know for sure is whether and to what extent the virus is affected by warmer weather. If it behaves similarly to flu, June should be safe, otherwise we will likely have to cancel it. Nothing can be reliably said about Berlin at this point... Personally, I've been a long-term proponent of virtual events for many reasons, health situation included. As much as I would miss hanging out with everyone... Dmitry On Fri, Mar 6, 2020 at 9:58 PM Mark Collier wrote: > I wanted to make sure everyone saw this thread on the foundation mailing > list, since I know not everyone is subscribed to both lists: > > Archive: > http://lists.openstack.org/pipermail/foundation/2020-March/002852.html > > Please join that ML thread to share feedback on this topic, or you can > reach out directly to myself or jonathan at openstack.org > > I saw first hand how we all pulled together during the Snowpenstack in > Dublin, so I know we'll once again pull together as a community to get > through this! > > Mark > > > > On Friday, March 6, 2020 12:55pm, "Mark Collier" > said: > > > Stackers, > > > > Before I get into the current plans for the OSF events in Vancouver and > Berlin, I > > wanted to say a few words in general about the virus impacting so many > people > > right now. > > > > First, I wanted to acknowledge the very difficult situation many are > facing > > because of COVID-19 (Coronavirus), across open source communities and > local > > communities in general (tech or otherwise). I also want to say to > everyone who is > > on the front lines managing events, from the full time staffers to the > volunteers, > > to the contractors and production partners, that we have some idea of > what you're > > going through and we know this is a tough time. If there's anything we > can do to > > help, please reach out. In the best of times, event organization can be > grueling > > and thankless, and so I just want to say THANK YOU to everyone who does > the > > organizing work in the communities we all care so much about. > > > > OSF 2020 EVENTS > > > > When it comes to the 2020 events OSF is managing, namely the OpenDev + > PTG in > > Vancouver June 8-11 and the Open Infrastructure Summit in Berlin October > 19-23, > > please read and bookmark this status page which we will continue to > update: > > https://www.openstack.org/events/covid-19-coronavirus-disease-updates > > > > When it comes to our community, the health of every individual is of > paramount > > concern. We have always aimed to produce events "of the community, by the > > community" and the upcoming event in Vancouver is no exception. The > OpenDev tracks > > each morning will be programmed by volunteers from the community, and > the project > > teams will be organizing their own conversations as well each afternoon > M-W, and > > all day Thursday. > > > > But the larger question is here: should the show go on? > > > > The short answer is that as of now, the Vancouver and Berlin events are > still > > scheduled to happen in June (8-11) and October (19-23), respectively. > > > > However, we are willing to cancel or approach the events in a different > way (i.e. > > virtual) if the facts indicate that is the best path, and we know the > facts are > > changing rapidly. One of the most critical inputs we need is to hear > from each of > > you. We know that many of you rely on the twice-annual events to get > together and > > make rapid progress on the software, which is one reason we are not > making any > > decisions in haste. We also know that many of you may be unable or > unwilling to > > travel in June, and that is critical information to hear as we get > closer to the > > event so that we can make the most informed decision. > > > > I also wanted to answer a FAQ by letting everyone know that if either > event is > > cancelled, event tickets and sponsorships will be fully refunded. Please > note that > > if you're making travel arrangements (e.g. flights, hotels) those are > outside of > > our control. > > > > So as we continue to monitor the news and listen to health experts to > make an > > informed decision on any changes to our event plans, we'd like to hear > from > > everyone in the community who has a stake in these events. Our most > pressing topic > > is of course Vanvouver, but if you have questions or concerns about the > Berlin > > plan feel free to share those as well. > > > > If you'd like to connect directly, you can always contact Executive > Director > > Jonathan Bryce (jonathan at openstack.org) or myself (mark at openstack.org). > > > > Key Links: > > - STATUS PAGE: > > https://www.openstack.org/events/covid-19-coronavirus-disease-updates > > - Vancouver OpenDev + PTG > https://www.openstack.org/events/opendev-ptg-2020/ > > - Berlin Open Infrastructure Summit: > https://www.openstack.org/summit/berlin-2020/ > > > > Key Dates for OpenDev + PTG in Vancouver: > > - Schedule will be published in early April > > - Early bird deadline is April 4 > > - Final day to sponsor will be May 4 > > - Final registration price increase will be in early May > > > > Mark Collier > > COO, OpenStack Foundation > > @sparkycollier > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Mon Mar 9 09:15:38 2020 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Mon, 9 Mar 2020 10:15:38 +0100 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> Message-ID: You are very right Dmitriy. One thing to add: it is currently absolutely unpredictable how situation will look like in June, but we need to get travel approvals, visas, reservations already now. And at least in central Europe all companies (especially big ones) are now in the alarmed state with incredibly strengthen travel regulations, so that at least for me even to start discussion about travel is very hard. Artem ---- typed from mobile, auto-correct typos assumed ---- On Mon, 9 Mar 2020, 10:10 Dmitry Tantsur, wrote: > Hi, > > It seems that the critical bit we don't know for sure is whether and to > what extent the virus is affected by warmer weather. If it behaves > similarly to flu, June should be safe, otherwise we will likely have to > cancel it. Nothing can be reliably said about Berlin at this point... > > Personally, I've been a long-term proponent of virtual events for many > reasons, health situation included. As much as I would miss hanging out > with everyone... > > Dmitry > > On Fri, Mar 6, 2020 at 9:58 PM Mark Collier wrote: > >> I wanted to make sure everyone saw this thread on the foundation mailing >> list, since I know not everyone is subscribed to both lists: >> >> Archive: >> http://lists.openstack.org/pipermail/foundation/2020-March/002852.html >> >> Please join that ML thread to share feedback on this topic, or you can >> reach out directly to myself or jonathan at openstack.org >> >> I saw first hand how we all pulled together during the Snowpenstack in >> Dublin, so I know we'll once again pull together as a community to get >> through this! >> >> Mark >> >> >> >> On Friday, March 6, 2020 12:55pm, "Mark Collier" >> said: >> >> > Stackers, >> > >> > Before I get into the current plans for the OSF events in Vancouver and >> Berlin, I >> > wanted to say a few words in general about the virus impacting so many >> people >> > right now. >> > >> > First, I wanted to acknowledge the very difficult situation many are >> facing >> > because of COVID-19 (Coronavirus), across open source communities and >> local >> > communities in general (tech or otherwise). I also want to say to >> everyone who is >> > on the front lines managing events, from the full time staffers to the >> volunteers, >> > to the contractors and production partners, that we have some idea of >> what you're >> > going through and we know this is a tough time. If there's anything we >> can do to >> > help, please reach out. In the best of times, event organization can be >> grueling >> > and thankless, and so I just want to say THANK YOU to everyone who does >> the >> > organizing work in the communities we all care so much about. >> > >> > OSF 2020 EVENTS >> > >> > When it comes to the 2020 events OSF is managing, namely the OpenDev + >> PTG in >> > Vancouver June 8-11 and the Open Infrastructure Summit in Berlin >> October 19-23, >> > please read and bookmark this status page which we will continue to >> update: >> > https://www.openstack.org/events/covid-19-coronavirus-disease-updates >> > >> > When it comes to our community, the health of every individual is of >> paramount >> > concern. We have always aimed to produce events "of the community, by >> the >> > community" and the upcoming event in Vancouver is no exception. The >> OpenDev tracks >> > each morning will be programmed by volunteers from the community, and >> the project >> > teams will be organizing their own conversations as well each afternoon >> M-W, and >> > all day Thursday. >> > >> > But the larger question is here: should the show go on? >> > >> > The short answer is that as of now, the Vancouver and Berlin events are >> still >> > scheduled to happen in June (8-11) and October (19-23), respectively. >> > >> > However, we are willing to cancel or approach the events in a different >> way (i.e. >> > virtual) if the facts indicate that is the best path, and we know the >> facts are >> > changing rapidly. One of the most critical inputs we need is to hear >> from each of >> > you. We know that many of you rely on the twice-annual events to get >> together and >> > make rapid progress on the software, which is one reason we are not >> making any >> > decisions in haste. We also know that many of you may be unable or >> unwilling to >> > travel in June, and that is critical information to hear as we get >> closer to the >> > event so that we can make the most informed decision. >> > >> > I also wanted to answer a FAQ by letting everyone know that if either >> event is >> > cancelled, event tickets and sponsorships will be fully refunded. >> Please note that >> > if you're making travel arrangements (e.g. flights, hotels) those are >> outside of >> > our control. >> > >> > So as we continue to monitor the news and listen to health experts to >> make an >> > informed decision on any changes to our event plans, we'd like to hear >> from >> > everyone in the community who has a stake in these events. Our most >> pressing topic >> > is of course Vanvouver, but if you have questions or concerns about the >> Berlin >> > plan feel free to share those as well. >> > >> > If you'd like to connect directly, you can always contact Executive >> Director >> > Jonathan Bryce (jonathan at openstack.org) or myself (mark at openstack.org). >> > >> > Key Links: >> > - STATUS PAGE: >> > https://www.openstack.org/events/covid-19-coronavirus-disease-updates >> > - Vancouver OpenDev + PTG >> https://www.openstack.org/events/opendev-ptg-2020/ >> > - Berlin Open Infrastructure Summit: >> https://www.openstack.org/summit/berlin-2020/ >> > >> > Key Dates for OpenDev + PTG in Vancouver: >> > - Schedule will be published in early April >> > - Early bird deadline is April 4 >> > - Final day to sponsor will be May 4 >> > - Final registration price increase will be in early May >> > >> > Mark Collier >> > COO, OpenStack Foundation >> > @sparkycollier >> > >> > >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bcafarel at redhat.com Mon Mar 9 09:22:44 2020 From: bcafarel at redhat.com (Bernard Cafarelli) Date: Mon, 9 Mar 2020 10:22:44 +0100 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Message-ID: Well that's only a "stable core +1", but big personal +1! On Fri, 6 Mar 2020 at 16:31, Slawek Kaplonski wrote: > Hi neutrinos, > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the > Neutron core team. > Lajos is Neutron contributor Neutron since around Queens cycle and now he > is one of the most active reviewers in the Neutron group projects. > He was one of the key contributors in cooperation with Nova and Placement > teams to deliver guaranteed minimum bandwidth feature in OpenStack. > He is very active and helpful with triaging and fixing Neutron bugs and > issues in our CI. > > During last few cycles he proved that he has wide knowledge about Neutron > code base. He is currently also a maintainer of some neutron stadium > projects which shows that he has knowledge about code base not only about > neutron but also Neutron stadium. > > The quality and number of his reviews are comparable to other members of > the Neutron core team: > https://www.stackalytics.com/?release=ussuri&module=neutron-group and are > higher every cycle :) > I think he will be great addition to our core team. > > I will keep this nomination open for a week or until all current cores > will respond. > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > > -- Bernard Cafarelli -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Mon Mar 9 09:22:57 2020 From: mark at stackhpc.com (Mark Goddard) Date: Mon, 9 Mar 2020 09:22:57 +0000 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> Message-ID: On Mon, 9 Mar 2020 at 09:05, Dmitry Tantsur wrote: > > Hi, > > It seems that the critical bit we don't know for sure is whether and to what extent the virus is affected by warmer weather. If it behaves similarly to flu, June should be safe, otherwise we will likely have to cancel it. Nothing can be reliably said about Berlin at this point... > > Personally, I've been a long-term proponent of virtual events for many reasons, health situation included. As much as I would miss hanging out with everyone... We've been doing virtual-only design sessions in Kolla for a while now. My recent proposal for a Kolla SIG is partly an effort to close the gaps with other Summit sessions (onboarding, updates, ops feedback), and bring the wider community into the virtual net. Follow along to see how it goes... [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013122.html > > Dmitry > > On Fri, Mar 6, 2020 at 9:58 PM Mark Collier wrote: >> >> I wanted to make sure everyone saw this thread on the foundation mailing list, since I know not everyone is subscribed to both lists: >> >> Archive: http://lists.openstack.org/pipermail/foundation/2020-March/002852.html >> >> Please join that ML thread to share feedback on this topic, or you can reach out directly to myself or jonathan at openstack.org >> >> I saw first hand how we all pulled together during the Snowpenstack in Dublin, so I know we'll once again pull together as a community to get through this! >> >> Mark >> >> >> >> On Friday, March 6, 2020 12:55pm, "Mark Collier" said: >> >> > Stackers, >> > >> > Before I get into the current plans for the OSF events in Vancouver and Berlin, I >> > wanted to say a few words in general about the virus impacting so many people >> > right now. >> > >> > First, I wanted to acknowledge the very difficult situation many are facing >> > because of COVID-19 (Coronavirus), across open source communities and local >> > communities in general (tech or otherwise). I also want to say to everyone who is >> > on the front lines managing events, from the full time staffers to the volunteers, >> > to the contractors and production partners, that we have some idea of what you're >> > going through and we know this is a tough time. If there's anything we can do to >> > help, please reach out. In the best of times, event organization can be grueling >> > and thankless, and so I just want to say THANK YOU to everyone who does the >> > organizing work in the communities we all care so much about. >> > >> > OSF 2020 EVENTS >> > >> > When it comes to the 2020 events OSF is managing, namely the OpenDev + PTG in >> > Vancouver June 8-11 and the Open Infrastructure Summit in Berlin October 19-23, >> > please read and bookmark this status page which we will continue to update: >> > https://www.openstack.org/events/covid-19-coronavirus-disease-updates >> > >> > When it comes to our community, the health of every individual is of paramount >> > concern. We have always aimed to produce events "of the community, by the >> > community" and the upcoming event in Vancouver is no exception. The OpenDev tracks >> > each morning will be programmed by volunteers from the community, and the project >> > teams will be organizing their own conversations as well each afternoon M-W, and >> > all day Thursday. >> > >> > But the larger question is here: should the show go on? >> > >> > The short answer is that as of now, the Vancouver and Berlin events are still >> > scheduled to happen in June (8-11) and October (19-23), respectively. >> > >> > However, we are willing to cancel or approach the events in a different way (i.e. >> > virtual) if the facts indicate that is the best path, and we know the facts are >> > changing rapidly. One of the most critical inputs we need is to hear from each of >> > you. We know that many of you rely on the twice-annual events to get together and >> > make rapid progress on the software, which is one reason we are not making any >> > decisions in haste. We also know that many of you may be unable or unwilling to >> > travel in June, and that is critical information to hear as we get closer to the >> > event so that we can make the most informed decision. >> > >> > I also wanted to answer a FAQ by letting everyone know that if either event is >> > cancelled, event tickets and sponsorships will be fully refunded. Please note that >> > if you're making travel arrangements (e.g. flights, hotels) those are outside of >> > our control. >> > >> > So as we continue to monitor the news and listen to health experts to make an >> > informed decision on any changes to our event plans, we'd like to hear from >> > everyone in the community who has a stake in these events. Our most pressing topic >> > is of course Vanvouver, but if you have questions or concerns about the Berlin >> > plan feel free to share those as well. >> > >> > If you'd like to connect directly, you can always contact Executive Director >> > Jonathan Bryce (jonathan at openstack.org) or myself (mark at openstack.org). >> > >> > Key Links: >> > - STATUS PAGE: >> > https://www.openstack.org/events/covid-19-coronavirus-disease-updates >> > - Vancouver OpenDev + PTG https://www.openstack.org/events/opendev-ptg-2020/ >> > - Berlin Open Infrastructure Summit: https://www.openstack.org/summit/berlin-2020/ >> > >> > Key Dates for OpenDev + PTG in Vancouver: >> > - Schedule will be published in early April >> > - Early bird deadline is April 4 >> > - Final day to sponsor will be May 4 >> > - Final registration price increase will be in early May >> > >> > Mark Collier >> > COO, OpenStack Foundation >> > @sparkycollier >> > >> > >> >> >> From balazs.gibizer at est.tech Mon Mar 9 09:23:27 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 09 Mar 2020 10:23:27 +0100 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> Message-ID: <1583745807.12170.18@est.tech> On Mon, Mar 9, 2020 at 10:15, Artem Goncharov wrote: > You are very right Dmitriy. > > One thing to add: it is currently absolutely unpredictable how > situation will look like in June, but we need to get travel > approvals, visas, reservations already now. And at least in central > Europe all companies (especially big ones) are now in the alarmed > state with incredibly strengthen travel regulations, so that at least > for me even to start discussion about travel is very hard. I can second that. In theory I have the travel approval to go to Vancouver but at the same time all non business critical travel is suspended until further notice by my employer. So right now I can only wait for the situation to unfold. gibi > > Artem > > ---- > typed from mobile, auto-correct typos assumed > ---- > > On Mon, 9 Mar 2020, 10:10 Dmitry Tantsur, wrote: >> Hi, >> >> It seems that the critical bit we don't know for sure is whether and >> to what extent the virus is affected by warmer weather. If it >> behaves similarly to flu, June should be safe, otherwise we will >> likely have to cancel it. Nothing can be reliably said about Berlin >> at this point... >> >> Personally, I've been a long-term proponent of virtual events for >> many reasons, health situation included. As much as I would miss >> hanging out with everyone... >> >> Dmitry >> >> On Fri, Mar 6, 2020 at 9:58 PM Mark Collier >> wrote: >>> I wanted to make sure everyone saw this thread on the foundation >>> mailing list, since I know not everyone is subscribed to both lists: >>> >>> Archive: >>> http://lists.openstack.org/pipermail/foundation/2020-March/002852.html >>> >>> Please join that ML thread to share feedback on this topic, or you >>> can reach out directly to myself or jonathan at openstack.org >>> >>> I saw first hand how we all pulled together during the >>> Snowpenstack in Dublin, so I know we'll once again pull together as >>> a community to get through this! >>> >>> Mark >>> >>> >>> >>> On Friday, March 6, 2020 12:55pm, "Mark Collier" >>> said: >>> >>> > Stackers, >>> > >>> > Before I get into the current plans for the OSF events in >>> Vancouver and Berlin, I >>> > wanted to say a few words in general about the virus impacting >>> so many people >>> > right now. >>> > >>> > First, I wanted to acknowledge the very difficult situation many >>> are facing >>> > because of COVID-19 (Coronavirus), across open source >>> communities and local >>> > communities in general (tech or otherwise). I also want to say >>> to everyone who is >>> > on the front lines managing events, from the full time staffers >>> to the volunteers, >>> > to the contractors and production partners, that we have some >>> idea of what you're >>> > going through and we know this is a tough time. If there's >>> anything we can do to >>> > help, please reach out. In the best of times, event organization >>> can be grueling >>> > and thankless, and so I just want to say THANK YOU to everyone >>> who does the >>> > organizing work in the communities we all care so much about. >>> > >>> > OSF 2020 EVENTS >>> > >>> > When it comes to the 2020 events OSF is managing, namely the >>> OpenDev + PTG in >>> > Vancouver June 8-11 and the Open Infrastructure Summit in Berlin >>> October 19-23, >>> > please read and bookmark this status page which we will continue >>> to update: >>> > >>> https://www.openstack.org/events/covid-19-coronavirus-disease-updates >>> > >>> > When it comes to our community, the health of every individual >>> is of paramount >>> > concern. We have always aimed to produce events "of the >>> community, by the >>> > community" and the upcoming event in Vancouver is no exception. >>> The OpenDev tracks >>> > each morning will be programmed by volunteers from the >>> community, and the project >>> > teams will be organizing their own conversations as well each >>> afternoon M-W, and >>> > all day Thursday. >>> > >>> > But the larger question is here: should the show go on? >>> > >>> > The short answer is that as of now, the Vancouver and Berlin >>> events are still >>> > scheduled to happen in June (8-11) and October (19-23), >>> respectively. >>> > >>> > However, we are willing to cancel or approach the events in a >>> different way (i.e. >>> > virtual) if the facts indicate that is the best path, and we >>> know the facts are >>> > changing rapidly. One of the most critical inputs we need is to >>> hear from each of >>> > you. We know that many of you rely on the twice-annual events to >>> get together and >>> > make rapid progress on the software, which is one reason we are >>> not making any >>> > decisions in haste. We also know that many of you may be unable >>> or unwilling to >>> > travel in June, and that is critical information to hear as we >>> get closer to the >>> > event so that we can make the most informed decision. >>> > >>> > I also wanted to answer a FAQ by letting everyone know that if >>> either event is >>> > cancelled, event tickets and sponsorships will be fully >>> refunded. Please note that >>> > if you're making travel arrangements (e.g. flights, hotels) >>> those are outside of >>> > our control. >>> > >>> > So as we continue to monitor the news and listen to health >>> experts to make an >>> > informed decision on any changes to our event plans, we'd like >>> to hear from >>> > everyone in the community who has a stake in these events. Our >>> most pressing topic >>> > is of course Vanvouver, but if you have questions or concerns >>> about the Berlin >>> > plan feel free to share those as well. >>> > >>> > If you'd like to connect directly, you can always contact >>> Executive Director >>> > Jonathan Bryce (jonathan at openstack.org) or myself >>> (mark at openstack.org). >>> > >>> > Key Links: >>> > - STATUS PAGE: >>> > >>> https://www.openstack.org/events/covid-19-coronavirus-disease-updates >>> > - Vancouver OpenDev + PTG >>> https://www.openstack.org/events/opendev-ptg-2020/ >>> > - Berlin Open Infrastructure Summit: >>> https://www.openstack.org/summit/berlin-2020/ >>> > >>> > Key Dates for OpenDev + PTG in Vancouver: >>> > - Schedule will be published in early April >>> > - Early bird deadline is April 4 >>> > - Final day to sponsor will be May 4 >>> > - Final registration price increase will be in early May >>> > >>> > Mark Collier >>> > COO, OpenStack Foundation >>> > @sparkycollier >>> > >>> > >>> >>> >>> From balazs.gibizer at est.tech Mon Mar 9 10:01:27 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 09 Mar 2020 11:01:27 +0100 Subject: [nova] bug triage Message-ID: <1583748087.12170.20@est.tech> Hi, We surpassed the somewhat magical line of having more than 100 untriaged nova bugs [1]. For me it seems that how we, as a team, are handling the incoming bugs is not sustainable. I'm guilty as well of not doing bug triage in a last two months. So I'm personally trying to change now and dedicate a weekly times lot to look at the bug list. But I also want to open a discussion about bug triage in genera. How can we handle the incoming bugs? I see that neutron does a weekly rotation of bug deputy and for them it works nicely. What do you think? Do we want to try that? Do we have enough volunteers to create a nice rotation period? Cheers, gibi [1] https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New From stephenfin at redhat.com Mon Mar 9 10:04:44 2020 From: stephenfin at redhat.com (Stephen Finucane) Date: Mon, 09 Mar 2020 10:04:44 +0000 Subject: [nova] US meeting slot In-Reply-To: <1583744340.12170.17@est.tech> References: <1583744340.12170.17@est.tech> Message-ID: <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> On Mon, 2020-03-09 at 09:59 +0100, Balázs Gibizer wrote: > Hi, > > Nova has alternate meeting slots on Thursdays to try to cover > contributors from different time zones. > * 14:00 UTC > * 21:00 UTC > > As I'm taking over the PTL role form Eric I need to figure out how to > run the nova meetings. I cannot really run the UTC 21:00 as it is > pretty late for me. (I will run the 14:00 UTC slot). I see different > options: > > a) Somebody from the US side of the globe volunteers to run the 21:00 > UTC slot. Please speak up if you would like to run it. I can help you > with agenda refresh and technicalities if needed. > > b) Have only one meeting time, and move that to 16:00 UTC. In this case > I will be able to run it most of the weeks. >From a European perspective, this is preferable for me since I could never attend the 21:00 UTC slot. I don't know how it works for the folks on the US west coast or in China though. > c) Do not have a dedicated meeting slot but switch to office hours. > Here we also need to find a time slot. I think 16:00 UTC could work > there as well. Do you mean move all meetings to office hours or just the one in the US timezone? Personally, I'd like to have a regular meeting with an agenda at least every couple of weeks. Stephen > Please share your view! Any other proposal is very welcome. > > Cheers, > gibi From balazs.gibizer at est.tech Mon Mar 9 10:16:29 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 09 Mar 2020 11:16:29 +0100 Subject: [nova] US meeting slot In-Reply-To: <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> References: <1583744340.12170.17@est.tech> <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> Message-ID: <1583748989.12170.21@est.tech> On Mon, Mar 9, 2020 at 10:04, Stephen Finucane wrote: > On Mon, 2020-03-09 at 09:59 +0100, Balázs Gibizer wrote: >> >> c) Do not have a dedicated meeting slot but switch to office hours. >> Here we also need to find a time slot. I think 16:00 UTC could work >> there as well. > > Do you mean move all meetings to office hours or just the one in the > US > timezone? Personally, I'd like to have a regular meeting with an > agenda > at least every couple of weeks. To clarify, I meant to stop having meetings and have office hours instead. But a mixed setup also works for me if there will be folks around on the nova channel at 21:00 UTC. gibi From zhangbailin at inspur.com Mon Mar 9 10:21:38 2020 From: zhangbailin at inspur.com (=?utf-8?B?QnJpbiBaaGFuZyjlvKDnmb7mnpcp?=) Date: Mon, 9 Mar 2020 10:21:38 +0000 Subject: =?utf-8?B?562U5aSNOiBbbGlzdHMub3BlbnN0YWNrLm9yZ+S7o+WPkV1SZTogW25vdmFd?= =?utf-8?Q?_US_meeting_slot?= In-Reply-To: <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> References: <25204fddabe1a05ee593ec9170dc512f@sslemail.net> <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> Message-ID: <70294733ed48434bb161033c294f562a@inspur.com> > 发件人: Stephen Finucane [mailto:stephenfin at redhat.com] > 发送时间: 2020年3月9日 18:05 > 收件人: Balázs Gibizer ; OpenStack Discuss > 主题: [lists.openstack.org代发]Re: [nova] US meeting slot > On Mon, 2020-03-09 at 09:59 +0100, Balázs Gibizer wrote: >> Hi, >> >> Nova has alternate meeting slots on Thursdays to try to cover >> contributors from different time zones. >> * 14:00 UTC >> * 21:00 UTC >> >> As I'm taking over the PTL role form Eric I need to figure out how to >> run the nova meetings. I cannot really run the UTC 21:00 as it is >> pretty late for me. (I will run the 14:00 UTC slot). I see different >> options: >> >> a) Somebody from the US side of the globe volunteers to run the 21:00 >> UTC slot. Please speak up if you would like to run it. I can help you >> with agenda refresh and technicalities if needed. >> >> b) Have only one meeting time, and move that to 16:00 UTC. In this >> case I will be able to run it most of the weeks. > From a European perspective, this is preferable for me since I could never attend the 21:00 UTC slot. I don't know how it works for the folks on the US west coast or in China though. To be honest, if 16:00UTC in China is 24:00 at night, if we attend to this meeting, then maybe the whole day will be listless and distressing. How about 10:30UTC or 11:00UTC? I know you are works at this time, but maybe dansmith will be still absent like the regular meeting at 21:00 UTC. >> c) Do not have a dedicated meeting slot but switch to office hours. >> Here we also need to find a time slot. I think 16:00 UTC could work >> there as well. > Do you mean move all meetings to office hours or just the one in the US timezone? > Personally, I'd like to have a regular meeting with an agenda at least every couple of weeks. Agree. > Stephen > Please share your view! Any other proposal is very welcome. > > Cheers, > gibi From geguileo at redhat.com Mon Mar 9 10:47:21 2020 From: geguileo at redhat.com (Gorka Eguileor) Date: Mon, 9 Mar 2020 11:47:21 +0100 Subject: [CINDER] Snapshots export In-Reply-To: References: <20200304155850.b4ydu4vfxthih7we@localhost> Message-ID: <20200309104721.fyqjvi4miiefon24@localhost> On 08/03, Alfredo De Luca wrote: > Hi Sean. Sorry for the late reply. > What we want to do is backing up snapshots in case of a complete compute > lost of as a plan for disaster recovery. > So after recreating the environment we can restore snapshots and start the > VMs again. > > Cheers Hi, Snapshots are stored in the same medium as the original volume, therefore are not valid for disaster recovery. In case of a disaster you would lose both, the volume and the snapshot. Depending on the type of scenario you want to guard against you will need different methods: - Snapshots - Backups - Replication Snapshots in general are only useful in case your volume gets corrupted, you accidentally delete data in the disk, etc. If you lose your compute host your volume would still be safe, so you don't need to do anything fancy, you can just attach the volume again. Cheers, Gorka. > > > On Wed, Mar 4, 2020 at 10:14 PM Sean McGinnis wrote: > > > On 3/4/20 9:58 AM, Gorka Eguileor wrote: > > > On 03/03, Alfredo De Luca wrote: > > >> Hi all. > > >> We have our env with Openstack (Train) and cinder with CEPH (nautilus) > > >> backend. > > >> We are creating automatic volumes snapshots and now we'd like to export > > >> them as a backup/restore plan. After exporting the snapshots we will use > > >> Acronis as backup tool. > > >> > > >> I couldn't find the right steps/commands to exports the snapshots. > > >> Any info? > > >> Cheers > > >> > > >> -- > > >> *Alfredo* > > > Hi Alfredo, > > > > > > What kind of backup/restore plan do you have planned? > > > > > > Because snapshots are not meant to be used in a Disaster Recovery > > > backup/restore plan, so the only thing available are the manage/unmanage > > > commands. > > > > > > These commands are meant to add an existing volume/snapshots into Cinder > > > together, not to unmanage/manage them independently. > > > > > > For example, you wouldn't be able to manage a snapshot if the volume is > > > not already managed. Also unmanaging the snapshot would block the > > > deletion of the RBD volume itself. > > > > > > Cheers, > > > Gorka. > > > > If the intent is to use the snapshots as a source to backup the volume > > data, leaving the actual volume attached and IO running but still > > getting a "static" view of the code, then you would need to create a > > volume from the chosen snapshot, mount that volume somewhere that is > > accessible to your backup software, perform the copy of the data, then > > delete the volume when complete. > > > > I haven't used Acronis myself, but the issue for some backup software > > could be that the volume it is backing up from is going to be different > > every time. Though you could make sure it is mounted at the same place > > so the backup software at least *thinks* it's backing up the same thing. > > > > Then restoring the data will likely require some manual intervention, > > but that's pretty much always the case when something goes wrong. I > > would just recommend you test the full disaster recovery scenario to > > make sure you have that figured out and working right before you > > actually need it. > > > > Sean > > > > > > > > -- > *Alfredo* From smooney at redhat.com Mon Mar 9 10:54:08 2020 From: smooney at redhat.com (Sean Mooney) Date: Mon, 09 Mar 2020 10:54:08 +0000 Subject: [nova] US meeting slot In-Reply-To: <1583748989.12170.21@est.tech> References: <1583744340.12170.17@est.tech> <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> <1583748989.12170.21@est.tech> Message-ID: <2d73108df2cac8732dd439b290c46d833e4c3bce.camel@redhat.com> On Mon, 2020-03-09 at 11:16 +0100, Balázs Gibizer wrote: > > On Mon, Mar 9, 2020 at 10:04, Stephen Finucane > wrote: > > On Mon, 2020-03-09 at 09:59 +0100, Balázs Gibizer wrote: > > > > > > c) Do not have a dedicated meeting slot but switch to office hours. > > > Here we also need to find a time slot. I think 16:00 UTC could work > > > there as well. > > > > Do you mean move all meetings to office hours or just the one in the > > US > > timezone? Personally, I'd like to have a regular meeting with an > > agenda > > at least every couple of weeks. > > To clarify, I meant to stop having meetings and have office hours > instead. But a mixed setup also works for me if there will be folks > around on the nova channel at 21:00 UTC. i would prefer to have the meeting or a mix rather then change to just office hours. i dont always rememebr to join the meeting unless there is a ping on the nova channel before but i generally am online for both slots. 16:00 UTC would be fine too but im not sure that would work for non eu/us folks. > > gibi > > > From rsharma1818 at outlook.com Mon Mar 9 11:27:31 2020 From: rsharma1818 at outlook.com (Rahul Sharma) Date: Mon, 9 Mar 2020 11:27:31 +0000 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: , Message-ID: Thanks Akihiro Adding the line "WEBROOT = '/dashboard'" to the local settings file worked ________________________________ From: Akihiro Motoki Sent: Monday, March 9, 2020 8:30 AM To: Rahul Sharma Cc: Donny Davis ; OpenStack Discuss Subject: Re: [Horizon] Unable to access the dashboard page I think you are hitting https://bugs.launchpad.net/horizon/+bug/1853651. The bug says WEBROOT needs to be configured. It was reported against the horizon installation guide on RHEL/CentOS, but I believe it is a bug on CentOS packaging as a package should work with the default config provided by the package. On Sun, Mar 8, 2020 at 2:45 AM Rahul Sharma wrote: > > Hi, > > Tried with /dashboard but it ain't working > > Getting error "The requested URL /auth/login/ was not found on this server." in the browser (Error 404) > > > ________________________________ > From: Donny Davis > Sent: Saturday, March 7, 2020 8:45 PM > To: Rahul Sharma > Cc: OpenStack Discuss > Subject: Re: [Horizon] Unable to access the dashboard page > > Try /dashboard > > Donny Davis > c: 805 814 6800 > > On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma wrote: > > Hi, > > I am trying to get OpenStack (Train) up and running on CentOS 7 by following the "OpenStack Installation Guide" provided on OpenStack's website and have completed installation of below components > > > - Keystone > - Glance > - Placement > - Nova > - Networking > > > After installing each component, I have also verified its operation and it seems to be working successfully. > > However, there is a problem I am facing after installing "Horizon" for dashboard services. In order to verify its operation, one is supposed to browse to the URL "http://controller/horizon/" where "controller" could be the hostname or IP address of the Node which is running the controller > > Browsing to the above URL throws an error "The requested URL /horizon was not found on this server." > > In the apache access logs, I see below error: > > " > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" > 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" > " > > If I browse to the URL "http://controller/", the default "Testing123" page of apache gets loaded. > > > Please assist. > > Thanks, > Rahul -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsharma1818 at outlook.com Mon Mar 9 11:27:50 2020 From: rsharma1818 at outlook.com (Rahul Sharma) Date: Mon, 9 Mar 2020 11:27:50 +0000 Subject: [Horizon] Unable to access the dashboard page In-Reply-To: References: , Message-ID: I checked the indentation and there's nothing wrong with it Rahul ________________________________ From: Amy Sent: Sunday, March 8, 2020 8:02 PM To: Rahul Sharma Cc: OpenStack Discuss Subject: Re: [Horizon] Unable to access the dashboard page Another thing to check is the indentation in your config file. Amy (spotz) On Mar 8, 2020, at 7:46 AM, Donny Davis wrote:  You may also want to check this setting https://docs.openstack.org/horizon/train/configuration/settings.html#allowed-hosts Donny Davis c: 805 814 6800 On Sat, Mar 7, 2020, 12:41 PM Rahul Sharma > wrote: Hi, Tried with /dashboard but it ain't working Getting error "The requested URL /auth/login/ was not found on this server." in the browser (Error 404) ________________________________ From: Donny Davis > Sent: Saturday, March 7, 2020 8:45 PM To: Rahul Sharma > Cc: OpenStack Discuss > Subject: Re: [Horizon] Unable to access the dashboard page Try /dashboard Donny Davis c: 805 814 6800 On Sat, Mar 7, 2020, 8:33 AM Rahul Sharma > wrote: Hi, I am trying to get OpenStack (Train) up and running on CentOS 7 by following the "OpenStack Installation Guide" provided on OpenStack's website and have completed installation of below components - Keystone - Glance - Placement - Nova - Networking After installing each component, I have also verified its operation and it seems to be working successfully. However, there is a problem I am facing after installing "Horizon" for dashboard services. In order to verify its operation, one is supposed to browse to the URL "http://controller/horizon/" where "controller" could be the hostname or IP address of the Node which is running the controller Browsing to the above URL throws an error "The requested URL /horizon was not found on this server." In the apache access logs, I see below error: " 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /horizon/ HTTP/1.1" 404 206 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" 103.44.50.92 - - [07/Mar/2020:13:14:52 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "http://3.21.90.63/horizon/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36" " If I browse to the URL "http://controller/", the default "Testing123" page of apache gets loaded. Please assist. Thanks, Rahul -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Mon Mar 9 11:31:50 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Mar 2020 12:31:50 +0100 Subject: Goodbye for now In-Reply-To: References: Message-ID: Albert Braden wrote: > My contract at Synopsys ends today, so I will have to continue my > efforts to sign up as an Openstack developer at my next role. Thanks to > everyone for all of the help and advice. Best wishes! Hoping to see you active again soon! -- Thierry Carrez (ttx) From thierry at openstack.org Mon Mar 9 13:50:02 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Mar 2020 14:50:02 +0100 Subject: [largescale-sig] Next meeting: Mar 11, 9utc Message-ID: <69398fb2-d2b0-d12e-e04f-7a9f7531fa7b@openstack.org> Hi everyone, The Large Scale SIG will have a meeting this week on Wednesday, Mar 11 at 9 UTC[1] in #openstack-meeting on IRC. As we evolve in DST hell (US having moved to summer time while Europe hasn't), please doublecheck when that falls for you: [1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200311T09 As always, the agenda for our meeting is available at: https://etherpad.openstack.org/p/large-scale-sig-meeting Feel free to add topics to it! Talk to you all on Wednesday, -- Thierry Carrez From dms at danplanet.com Mon Mar 9 14:10:17 2020 From: dms at danplanet.com (Dan Smith) Date: Mon, 09 Mar 2020 07:10:17 -0700 Subject: [nova] US meeting slot In-Reply-To: <1583744340.12170.17@est.tech> (=?utf-8?Q?=22Bal=C3=A1zs?= Gibizer"'s message of "Mon, 09 Mar 2020 09:59:00 +0100") References: <1583744340.12170.17@est.tech> Message-ID: > a) Somebody from the US side of the globe volunteers to run the 21:00 > UTC slot. Please speak up if you would like to run it. I can help you > with agenda refresh and technicalities if needed. > > b) Have only one meeting time, and move that to 16:00 UTC. In this > case I will be able to run it most of the weeks. > > c) Do not have a dedicated meeting slot but switch to office > hours. Here we also need to find a time slot. I think 16:00 UTC could > work there as well. I'd prefer 1600 to the 2100 actually, so that's fine with me. During DST I can make 1400, but no earlier. The 2100 meeting isn't very convenient for me, and very few people show up to it anymore anyway. I'd say it's not worth keeping that spot regardless. For a while now, I've felt that the meeting is unnecessary and overly repetitive. For that reason, I'd definitely be in favor of moving to office hours entirely (or as much as possible), although it sounds like most people aren't. --Dan From thierry at openstack.org Mon Mar 9 14:11:12 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Mar 2020 15:11:12 +0100 Subject: [Release-job-failures] Release of openstack/monasca-agent for ref refs/tags/2.8.1 failed In-Reply-To: References: Message-ID: zuul at openstack.org wrote: > Build failed. > > - release-openstack-python https://zuul.opendev.org/t/openstack/build/25e1809c970044708c503cda05ac84f9 : SUCCESS in 7m 24s > - announce-release https://zuul.opendev.org/t/openstack/build/646a0bbaa0054504a863b3900b476006 : FAILURE in 7m 55s > - propose-update-constraints https://zuul.opendev.org/t/openstack/build/5d85de0f42c740299fb5c6c7f24f3aa1 : SUCCESS in 4m 42s Analysis: Release announcement for monasca-agent failed due to the following transient email error: 451 Temporary local problem - please try later Release went out OK, only the announce was missed. -- Thierry Carrez (ttx) From gmann at ghanshyammann.com Mon Mar 9 14:20:34 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 09 Mar 2020 09:20:34 -0500 Subject: [nova] US meeting slot In-Reply-To: <2d73108df2cac8732dd439b290c46d833e4c3bce.camel@redhat.com> References: <1583744340.12170.17@est.tech> <29fc2d4b8daa903715a3e620f6eb77a9be1d34e9.camel@redhat.com> <1583748989.12170.21@est.tech> <2d73108df2cac8732dd439b290c46d833e4c3bce.camel@redhat.com> Message-ID: <170bfab37b8.116cafb576317.5873131666573593661@ghanshyammann.com> ---- On Mon, 09 Mar 2020 05:54:08 -0500 Sean Mooney wrote ---- > On Mon, 2020-03-09 at 11:16 +0100, Balázs Gibizer wrote: > > > > On Mon, Mar 9, 2020 at 10:04, Stephen Finucane > > wrote: > > > On Mon, 2020-03-09 at 09:59 +0100, Balázs Gibizer wrote: > > > > > > > > c) Do not have a dedicated meeting slot but switch to office hours. > > > > Here we also need to find a time slot. I think 16:00 UTC could work > > > > there as well. > > > > > > Do you mean move all meetings to office hours or just the one in the > > > US > > > timezone? Personally, I'd like to have a regular meeting with an > > > agenda > > > at least every couple of weeks. > > > > To clarify, I meant to stop having meetings and have office hours > > instead. But a mixed setup also works for me if there will be folks > > around on the nova channel at 21:00 UTC. > i would prefer to have the meeting or a mix rather then change to just office hours. > i dont always rememebr to join the meeting unless there is a ping on the nova channel > before but i generally am online for both slots. 16:00 UTC would be fine too but im not sure that would > work for non eu/us folks. 16:00 UTC works for me too. I agree to keep meeting as of now at least from the meeting content point of view which is more towards the status side. If we start discussing more technical things than status then it makes sense to move to office hours. -gmann > > > > gibi > > > > > > > > > From smooney at redhat.com Mon Mar 9 14:30:24 2020 From: smooney at redhat.com (Sean Mooney) Date: Mon, 09 Mar 2020 14:30:24 +0000 Subject: [nova] US meeting slot In-Reply-To: References: <1583744340.12170.17@est.tech> Message-ID: On Mon, 2020-03-09 at 07:10 -0700, Dan Smith wrote: > > a) Somebody from the US side of the globe volunteers to run the 21:00 > > UTC slot. Please speak up if you would like to run it. I can help you > > with agenda refresh and technicalities if needed. > > > > b) Have only one meeting time, and move that to 16:00 UTC. In this > > case I will be able to run it most of the weeks. > > > > c) Do not have a dedicated meeting slot but switch to office > > hours. Here we also need to find a time slot. I think 16:00 UTC could > > work there as well. > > I'd prefer 1600 to the 2100 actually, so that's fine with me. During DST > I can make 1400, but no earlier. The 2100 meeting isn't very convenient > for me, and very few people show up to it anymore anyway. I'd say it's > not worth keeping that spot regardless. > > For a while now, I've felt that the meeting is unnecessary and overly > repetitive. For that reason, I'd definitely be in favor of moving to > office hours entirely (or as much as possible), although it sounds like > most people aren't. maybe we could try alternating. so like we alternate the time now alternate between office hours and meeting. every other week. that said im not that pushed either way. i do think the status updates are useful but that is most because if the updates were sent as emails i would just ignore them and since they are done the meeting when i attend i actually learn something. > --Dan > From thierry at openstack.org Mon Mar 9 14:32:01 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Mar 2020 15:32:01 +0100 Subject: [kolla][uc] Kolla SIG In-Reply-To: References: Message-ID: Mark Goddard wrote: > Hi, > > I'd like to propose the creation of a Special Interest Group (SIG) [0] > for Kolla. > [...] I'm a bit skeptical. We have a history of creating a lot of groups and structures. This was very helpful in the early years to cope with the crazy growth of openstack and to capture all the energy sent toward the project. But today we really have too many groups, meetings, channels compared to the number of active people. We still have thousands of contributors, and yet we feel spread thin. So I'm skeptical of creating new groups and/or meetings (at least not without eliminating a number of other groups/meetings as a result). Creation of a Kolla SIG would IMHO duplicate "Kolla" groups, and create a bit of confusion and uncertainty as to what is handled by the Kolla SIG vs. what is handled by the Kolla project team. I'd rather encourage the Kolla project team to directly engage with its users by holding "Ops feedback" sessions and other activities. Basically I'm not sure what the Kolla SIG would do that the Kolla project team cannot currently do... -- Thierry From amy at demarco.com Mon Mar 9 14:44:34 2020 From: amy at demarco.com (Amy Marrich) Date: Mon, 9 Mar 2020 09:44:34 -0500 Subject: [kolla][uc] Kolla SIG In-Reply-To: References: Message-ID: Mark, I agree with Thierry on this as I think this would cause confusion as well as split focus between two things vs including OPS more with the development. A SiG should bring together people from different projects or different interests together with a common interest. This is solely about Kolla which already has a specific project unlike say the Finance SiG which had users with a common interest of installing OpenStack for financial use. Have you utilized Forum or BoF sessions at events yet? Or maybe reach out to the OPS Meetup team about including Kolla more at their events? If I can be of any help let me know, Amy (spotz) On Mon, Mar 9, 2020 at 9:33 AM Thierry Carrez wrote: > Mark Goddard wrote: > > Hi, > > > > I'd like to propose the creation of a Special Interest Group (SIG) [0] > > for Kolla. > > [...] > > I'm a bit skeptical. > > We have a history of creating a lot of groups and structures. This was > very helpful in the early years to cope with the crazy growth of > openstack and to capture all the energy sent toward the project. But > today we really have too many groups, meetings, channels compared to the > number of active people. We still have thousands of contributors, and > yet we feel spread thin. So I'm skeptical of creating new groups and/or > meetings (at least not without eliminating a number of other > groups/meetings as a result). > > Creation of a Kolla SIG would IMHO duplicate "Kolla" groups, and create > a bit of confusion and uncertainty as to what is handled by the Kolla > SIG vs. what is handled by the Kolla project team. > > I'd rather encourage the Kolla project team to directly engage with its > users by holding "Ops feedback" sessions and other activities. Basically > I'm not sure what the Kolla SIG would do that the Kolla project team > cannot currently do... > > -- > Thierry > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Mon Mar 9 15:05:12 2020 From: mark at stackhpc.com (Mark Goddard) Date: Mon, 9 Mar 2020 15:05:12 +0000 Subject: [kolla][uc] Kolla SIG In-Reply-To: References: Message-ID: On Mon, 9 Mar 2020 at 14:45, Amy Marrich wrote: > > Mark, > > I agree with Thierry on this as I think this would cause confusion as well as split focus between two things vs including OPS more with the development. A SiG should bring together people from different projects or different interests together with a common interest. This is solely about Kolla which already has a specific project unlike say the Finance SiG which had users with a common interest of installing OpenStack for financial use. > > Have you utilized Forum or BoF sessions at events yet? Or maybe reach out to the OPS Meetup team about including Kolla more at their events? > > If I can be of any help let me know, > > Amy (spotz) > > On Mon, Mar 9, 2020 at 9:33 AM Thierry Carrez wrote: >> >> Mark Goddard wrote: >> > Hi, >> > >> > I'd like to propose the creation of a Special Interest Group (SIG) [0] >> > for Kolla. >> > [...] >> >> I'm a bit skeptical. >> >> We have a history of creating a lot of groups and structures. This was >> very helpful in the early years to cope with the crazy growth of >> openstack and to capture all the energy sent toward the project. But >> today we really have too many groups, meetings, channels compared to the >> number of active people. We still have thousands of contributors, and >> yet we feel spread thin. So I'm skeptical of creating new groups and/or >> meetings (at least not without eliminating a number of other >> groups/meetings as a result). >> >> Creation of a Kolla SIG would IMHO duplicate "Kolla" groups, and create >> a bit of confusion and uncertainty as to what is handled by the Kolla >> SIG vs. what is handled by the Kolla project team. >> >> I'd rather encourage the Kolla project team to directly engage with its >> users by holding "Ops feedback" sessions and other activities. Basically >> I'm not sure what the Kolla SIG would do that the Kolla project team >> cannot currently do... Thanks for the feedback. I don't particularly mind what we call it - if a SIG is not appropriate then that's fine. What I want is to be able to include those on the periphery of the community who for a variety of reasons are not particularly active upstream. I want to provide some way for these people to feel more of a part of the community, and if possible help them to become more active. They might not have time to sit in #openstack-kolla or attend summits, but an hour or two a month is probably something they could commit to. A big part of the drive behind this is simply gathering a list of members. So often we put things on openstack-discuss asking about who uses particular features, knowing it mostly goes to the void. Many operators simply don't have time to follow it. When you suggest Ops feedback sessions, do you mean at the summit? My view is that we are at the point where that is a fairly exclusive club. I'd much rather something virtual with a low barrier to entry. >> >> -- >> Thierry >> From balazs.gibizer at est.tech Mon Mar 9 15:27:25 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 09 Mar 2020 16:27:25 +0100 Subject: [nova][stable] Nova stable branch liaison Message-ID: <1583767645.12170.27@est.tech> Hi Team, According to the wiki [1] we still have Matt as a stable branch liaison for nova. Obviously we need to find somebody else as Matt is gone. I'm not part of the nova-stable-core team so I cannot assume default ownership of that. Also I see that we have a small but active stable team so I hope somebody from that team can step forward and can take this role. Cheers, gibi [1] https://wiki.openstack.org/wiki/CrossProjectLiaisons#Stable_Branch From thierry at openstack.org Mon Mar 9 15:40:50 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Mar 2020 16:40:50 +0100 Subject: [kolla][uc] Kolla SIG In-Reply-To: References: Message-ID: <6407c2a8-b778-4163-3fd1-031d9e073209@openstack.org> Mark Goddard wrote: > [...] > Thanks for the feedback. I don't particularly mind what we call it - > if a SIG is not appropriate then that's fine. What I want is to be > able to include those on the periphery of the community who for a > variety of reasons are not particularly active upstream. I want to > provide some way for these people to feel more of a part of the > community, and if possible help them to become more active. They might > not have time to sit in #openstack-kolla or attend summits, but an > hour or two a month is probably something they could commit to. > > A big part of the drive behind this is simply gathering a list of > members. So often we put things on openstack-discuss asking about who > uses particular features, knowing it mostly goes to the void. Many > operators simply don't have time to follow it. I think you can do that from the Kolla project team. You can call it the Kolla users club and create a number of events around it (virtual or face-to-face). My point is that you do not need a specific governance structure to do this. > When you suggest Ops feedback sessions, do you mean at the summit? My > view is that we are at the point where that is a fairly exclusive > club. I'd much rather something virtual with a low barrier to entry. No, I was just suggesting finding a catchy name for those outreach activities, one that encourages users to reach out, and make it less intimidating than joining the IRC meeting for a project team. -- Thierry From jimmy at openstack.org Mon Mar 9 15:42:37 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Mon, 09 Mar 2020 10:42:37 -0500 Subject: [kolla][uc] Kolla SIG In-Reply-To: <6407c2a8-b778-4163-3fd1-031d9e073209@openstack.org> References: <6407c2a8-b778-4163-3fd1-031d9e073209@openstack.org> Message-ID: <5E6663ED.5060609@openstack.org> Kollaborators? Thierry Carrez wrote: > No, I was just suggesting finding a catchy name for those outreach > activities, one that encourages users to reach out, and make it less > intimidating than joining the IRC meeting for a project team. From bence.romsics at gmail.com Mon Mar 9 15:57:55 2020 From: bence.romsics at gmail.com (Bence Romsics) Date: Mon, 9 Mar 2020 16:57:55 +0100 Subject: [neutron] bug deputy report for week of 2020-03-02 Message-ID: Hi All, Here's the deputy report for the week of 2020-03-02. Please note we have a new rotation schedule at the usual place: https://wiki.openstack.org/wiki/Network/Meetings High * https://bugs.launchpad.net/neutron/+bug/1865453 neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before fails randomly Random error in the gate. Maciej is working on it. * https://bugs.launchpad.net/neutron/+bug/1866039 [OVN] QoS gives different bandwidth limit measures than ml2/ovs Clearing prerequisites of testing qos in tempest against ovn backend. Work in progress by Maciej: https://review.opendev.org/711048 * https://bugs.launchpad.net/neutron/+bug/1866336 Binding of floating ip agent gateway port and agent_id isn't removed Work in progress by Slawek: https://review.opendev.org/711623 * https://bugs.launchpad.net/neutron/+bug/1866560 FIP Port forwarding description API extension don't work Work in progress by Slawek: https://review.opendev.org/711888 Medium * https://bugs.launchpad.net/neutron/+bug/1865891 Race condition during removal of subnet from the router and removal of subnet Downstream bug reproduced on master too. Slawek is working on it. * https://bugs.launchpad.net/neutron/+bug/1866068 [OVN] neutron_pg_drop port group table creation race condition Work in progress by Jakub: https://review.opendev.org/711404 * https://bugs.launchpad.net/neutron/+bug/1866160 Update security group failed with the same stateful data Work in progress by Lina: https://review.opendev.org/711385 RFE * https://bugs.launchpad.net/neutron/+bug/1865889 Routed provider networks support in OVN To be scheduled and discussed on the drivers meeting. Could turn into an RFE * https://bugs.launchpad.net/neutron/+bug/1866077 [L3][IPv6] IPv6 traffic with DVR in compute host Incomplete * https://bugs.launchpad.net/neutron/+bug/1866139 GARP not sent on provider network after live migration * https://bugs.launchpad.net/neutron/+bug/1866445 br-int bridge in one compute can't learn MAC addresses of VMs in other compute nodes Invalid * https://bugs.launchpad.net/neutron/+bug/1866353 Neutron API returning HTTP 201 for SG rule create when not fully created yet octavia-ovn-provider * https://review.opendev.org/711244 [OVN Octavia Provider] Deleting of listener fails Work in progress by Maciej: https://review.opendev.org/711244 Cheers, Bence (rubasov) From gr at ham.ie Mon Mar 9 16:06:11 2020 From: gr at ham.ie (Graham Hayes) Date: Mon, 9 Mar 2020 16:06:11 +0000 Subject: [all][tc] Stepping down from TC In-Reply-To: References: Message-ID: <308304c3-7314-b01e-e1b4-6b15f926a8b3@ham.ie> On 05/03/2020 16:45, Alexandra Settle wrote: > Hi all, > > This should come as no shock as I have been relatively quite for some time > now, but I will not standing for the Technical Committee for a second term. > > I have thoroughly enjoyed my tenure, learning so much about open source > governance than I ever thought I needed 😉 > > My work takes me elsewhere, as it did several years ago, and I simply do > not have > the time to manage both. > > I encourage anyone who is interested in governance, or is passionate > about OpenStack > and wants to learn more, to stand for the TC elections. As was proven by > my own > nomination and subsequent successful election, you do not have to be > "purely technical" > to stand and be a part of something great. Diversity of skill is so > important to our > survival. > > Thanks to all those that have supported me to get to the point, I > appreciate you all and > will miss working intimately with the community. > > Please do not hesitate to reach out and ask any questions if you are > interested in the > positions available, happy to help encourage and answer any questions > you may have. > > All the best, > > Alex > > ------------------------------------------------------------------------ > Alexandra Settle > Senior Technical Writer > London, United Kingdom (GMT) > Sad to see you go! Thanks for all the work, and much needed perspective you brought to the TC and the community. From sbauza at redhat.com Mon Mar 9 16:08:14 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Mon, 9 Mar 2020 17:08:14 +0100 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: <170b01d7595.10341333e516143.4131462912712933865@ghanshyammann.com> References: <1583482276.12170.14@est.tech> <4692F106-6B00-41FC-9BA9-1DF62A24EDAB@fried.cc> <170b01d7595.10341333e516143.4131462912712933865@ghanshyammann.com> Message-ID: Thanks indeed gibi for this. On Fri, Mar 6, 2020 at 2:58 PM Ghanshyam Mann wrote: > ---- On Fri, 06 Mar 2020 07:01:15 -0600 Eric Fried > wrote ---- > > Big +1 from me. Many thanks, gibi. Not that you‘ll need it, but please > don’t hesitate to reach out to me if you have questions. > > Indeed. Thanks gibi for helping out here. > > -gmann > > > > efried_gone > > > > > On Mar 6, 2020, at 02:16, Balázs Gibizer > wrote: > > > > > > Hi, > > > > > > Since Eric announced that he has to leave us [1] I have been working > > > internally with my employee to be able to take over the Nova PTL > > > position. Now I've got the necessary approvals. The official PTL > > > election is close [2] and I'm ready to fill the PTL gap until the > > > proper PTL election in April. > > > > > > Is this a workable solution for the community? > > > > > > Cheers, > > > gibi > > > > > > [1] > > > > http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012663.html > > > [2] https://governance.openstack.org/election/ > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From melwittt at gmail.com Mon Mar 9 16:23:37 2020 From: melwittt at gmail.com (melanie witt) Date: Mon, 9 Mar 2020 09:23:37 -0700 Subject: [nova] US meeting slot In-Reply-To: References: <1583744340.12170.17@est.tech> Message-ID: On 3/9/20 07:10, Dan Smith wrote: >> a) Somebody from the US side of the globe volunteers to run the 21:00 >> UTC slot. Please speak up if you would like to run it. I can help you >> with agenda refresh and technicalities if needed. >> >> b) Have only one meeting time, and move that to 16:00 UTC. In this >> case I will be able to run it most of the weeks. >> >> c) Do not have a dedicated meeting slot but switch to office >> hours. Here we also need to find a time slot. I think 16:00 UTC could >> work there as well. > > I'd prefer 1600 to the 2100 actually, so that's fine with me. During DST > I can make 1400, but no earlier. The 2100 meeting isn't very convenient > for me, and very few people show up to it anymore anyway. I'd say it's > not worth keeping that spot regardless. +1 to the opinion that it's not worth keeping the 2100 slot regardless. I think there are too few attendees during that time to make the meeting useful. 1400 is usually too early for me but I'm OK with that -- I catch up on the 1400 meetings by reading the meeting IRC logs. And if I need input on a blueprint or spec, I can use the ML if I can't make the meeting. Cheers, -melanie From rosmaita.fossdev at gmail.com Mon Mar 9 18:19:32 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 9 Mar 2020 14:19:32 -0400 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> Message-ID: On 3/6/20 6:12 PM, Goutham Pacha Ravi wrote: > > On Thu, Mar 5, 2020 at 11:53 AM Brian Rosmaita > > wrote: > > On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > >   ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita > > > wrote ---- > >   > Hello QA team and devstack-plugin-ceph-core people, > >   > > >   > The Cinder team has some proposals we'd like to float. > >   > > >   > 1. The Cinder team is interested in becoming more active in the > >   > maintenance of openstack/devstack-plugin-ceph [0]. > Currently, the > >   > devstack-plugin-ceph-core is > >   > https://review.opendev.org/#/admin/groups/1196,members > >   > The cinder-core is already represented by Eric and Sean; we'd > like to > >   > replace them by including the cinder-core group. > > > > +1. This is good diea and make sense, I will do the change. > > Great, thanks! > > > > I agree this is a great idea to have more members of Cinder joining the > devstack-plugin-ceph team. I would like to have atleast a sub team of > manila core reviewers added to this project if it makes sense. The > Manila CephFS drivers (cephfs-native and cephfs-nfs) are currently being > tested with the help of the devstack integration in devstack-plugin-ceph. > > We have Tom Barron (tbarron) in the team, i'd like to propose myself > (gouthamr) and Victoria Martinez de la Cruz (vkmc) > > Please let me know what you think of the idea. I've got no objection from the Cinder side. I would also not object to adding the manila-core group instead of individuals. It's certainly in your team's interest to keep this thing stable and working, just as it is for the Cinder team. > > >   > > >   > 2. The Cinder team is interested in becoming more active in the > >   > maintenance of x/devstack-plugin-nfs [1].  Currently, the > >   > devstack-plugin-nfs-core is > >   > https://review.opendev.org/#/admin/groups/1330,members > >   > It's already 75% cinder-core members; we'd like to replace the > >   > individual members with the cinder-core group.  We also > propose that > >   > devstack-core be added as an included group. > >   > > >   > 3. The Cinder team is interested in implementing a new > devstack plugin: > >   >      openstack/devstack-plugin-open-cas > >   > This will enable thorough testing of a new feature [2] being > introduced > >   > as experimental in Ussuri and expected to be finalized in > Victoria.  Our > >   > plan would be to make both cinder-core and devstack-core > included groups > >   > for the gerrit group governing the new plugin. > > > > +1. You want this under Cinder governance or under QA ? > > I think it makes sense for these to be under QA governance -- QA would > own the repo with both QA and Cinder having permission to make changes. > > >   > > >   > 4. This is a minor point, but can the devstack-plugin-nfs > repo be moved > >   > back into the 'openstack' namespace? > > > > If this is usable plugin for nfs testing (I am not aware if we > have any other) then > > it make sense to bring it to openstack governance. > > Same question here, do you want to put this under Cinder > governance or QA. > > Same here, I think QA should "own" the repo, but Cinder will have > permission to make changes there. > > > > > Those plugins under QA governance also ok for me with your > proposal of calloborative maintainance by > > devstack-core and cinder-core. > > > > -gmann > > Thanks for the quick response! > > >   > > >   > Let us know which of these proposals you find acceptable. > >   > > >   > > >   > [0] https://opendev.org/openstack/devstack-plugin-ceph > >   > [1] https://opendev.org/x/devstack-plugin-nfs > >   > [2] > https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > >   > > >   > > > > > From lyarwood at redhat.com Mon Mar 9 19:01:44 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Mon, 9 Mar 2020 19:01:44 +0000 Subject: [nova][stable] Nova stable branch liaison In-Reply-To: <1583767645.12170.27@est.tech> References: <1583767645.12170.27@est.tech> Message-ID: <20200309190144.yad4y7wrcnpsfk4j@lyarwood.usersys.redhat.com> On 09-03-20 16:27:25, Balázs Gibizer wrote: > Hi Team, > > According to the wiki [1] we still have Matt as a stable branch liaison for > nova. Obviously we need to find somebody else as Matt is gone. I'm not part > of the nova-stable-core team so I cannot assume default ownership of that. > Also I see that we have a small but active stable team so I hope somebody > from that team can step forward and can take this role. Yup I would be happy to help with this. Should I update the page assuming we don't have any other volunteers? Cheers, Lee > [1] https://wiki.openstack.org/wiki/CrossProjectLiaisons#Stable_Branch -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From gmann at ghanshyammann.com Mon Mar 9 19:10:04 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 09 Mar 2020 14:10:04 -0500 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> Message-ID: <170c0b444a5.e5378d4f17622.7872527250831207185@ghanshyammann.com> ---- On Mon, 09 Mar 2020 13:19:32 -0500 Brian Rosmaita wrote ---- > On 3/6/20 6:12 PM, Goutham Pacha Ravi wrote: > > > > On Thu, Mar 5, 2020 at 11:53 AM Brian Rosmaita > > > wrote: > > > > On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > > > ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita > > > > > wrote ---- > > > > Hello QA team and devstack-plugin-ceph-core people, > > > > > > > > The Cinder team has some proposals we'd like to float. > > > > > > > > 1. The Cinder team is interested in becoming more active in the > > > > maintenance of openstack/devstack-plugin-ceph [0]. > > Currently, the > > > > devstack-plugin-ceph-core is > > > > https://review.opendev.org/#/admin/groups/1196,members > > > > The cinder-core is already represented by Eric and Sean; we'd > > like to > > > > replace them by including the cinder-core group. > > > > > > +1. This is good diea and make sense, I will do the change. > > > > Great, thanks! > > > > > > > > I agree this is a great idea to have more members of Cinder joining the > > devstack-plugin-ceph team. I would like to have atleast a sub team of > > manila core reviewers added to this project if it makes sense. The > > Manila CephFS drivers (cephfs-native and cephfs-nfs) are currently being > > tested with the help of the devstack integration in devstack-plugin-ceph. > > > > We have Tom Barron (tbarron) in the team, i'd like to propose myself > > (gouthamr) and Victoria Martinez de la Cruz (vkmc) > > > > Please let me know what you think of the idea. > > I've got no objection from the Cinder side. I would also not object to > adding the manila-core group instead of individuals. It's certainly in > your team's interest to keep this thing stable and working, just as it > is for the Cinder team. Agree, I think adding manila group will be helpful, let me know if ok for you and accordinfgly I will make changes. -gmann > > > > > > > > > > > 2. The Cinder team is interested in becoming more active in the > > > > maintenance of x/devstack-plugin-nfs [1]. Currently, the > > > > devstack-plugin-nfs-core is > > > > https://review.opendev.org/#/admin/groups/1330,members > > > > It's already 75% cinder-core members; we'd like to replace the > > > > individual members with the cinder-core group. We also > > propose that > > > > devstack-core be added as an included group. > > > > > > > > 3. The Cinder team is interested in implementing a new > > devstack plugin: > > > > openstack/devstack-plugin-open-cas > > > > This will enable thorough testing of a new feature [2] being > > introduced > > > > as experimental in Ussuri and expected to be finalized in > > Victoria. Our > > > > plan would be to make both cinder-core and devstack-core > > included groups > > > > for the gerrit group governing the new plugin. > > > > > > +1. You want this under Cinder governance or under QA ? > > > > I think it makes sense for these to be under QA governance -- QA would > > own the repo with both QA and Cinder having permission to make changes. > > > > > > > > > > 4. This is a minor point, but can the devstack-plugin-nfs > > repo be moved > > > > back into the 'openstack' namespace? > > > > > > If this is usable plugin for nfs testing (I am not aware if we > > have any other) then > > > it make sense to bring it to openstack governance. > > > Same question here, do you want to put this under Cinder > > governance or QA. > > > > Same here, I think QA should "own" the repo, but Cinder will have > > permission to make changes there. > > > > > > > > Those plugins under QA governance also ok for me with your > > proposal of calloborative maintainance by > > > devstack-core and cinder-core. > > > > > > -gmann > > > > Thanks for the quick response! > > > > > > > > > > Let us know which of these proposals you find acceptable. > > > > > > > > > > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > > > > [1] https://opendev.org/x/devstack-plugin-nfs > > > > [2] > > https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > > > > > > > > > > > > > > > > > From gouthampravi at gmail.com Mon Mar 9 19:21:09 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Mon, 9 Mar 2020 12:21:09 -0700 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: <170c0b444a5.e5378d4f17622.7872527250831207185@ghanshyammann.com> References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> <170c0b444a5.e5378d4f17622.7872527250831207185@ghanshyammann.com> Message-ID: On Mon, Mar 9, 2020 at 12:10 PM Ghanshyam Mann wrote: > ---- On Mon, 09 Mar 2020 13:19:32 -0500 Brian Rosmaita < > rosmaita.fossdev at gmail.com> wrote ---- > > On 3/6/20 6:12 PM, Goutham Pacha Ravi wrote: > > > > > > On Thu, Mar 5, 2020 at 11:53 AM Brian Rosmaita > > > > > wrote: > > > > > > On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > > > > ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita > > > > > > > wrote ---- > > > > > Hello QA team and devstack-plugin-ceph-core people, > > > > > > > > > > The Cinder team has some proposals we'd like to float. > > > > > > > > > > 1. The Cinder team is interested in becoming more active > in the > > > > > maintenance of openstack/devstack-plugin-ceph [0]. > > > Currently, the > > > > > devstack-plugin-ceph-core is > > > > > https://review.opendev.org/#/admin/groups/1196,members > > > > > The cinder-core is already represented by Eric and Sean; > we'd > > > like to > > > > > replace them by including the cinder-core group. > > > > > > > > +1. This is good diea and make sense, I will do the change. > > > > > > Great, thanks! > > > > > > > > > > > > I agree this is a great idea to have more members of Cinder joining > the > > > devstack-plugin-ceph team. I would like to have atleast a sub team of > > > manila core reviewers added to this project if it makes sense. The > > > Manila CephFS drivers (cephfs-native and cephfs-nfs) are currently > being > > > tested with the help of the devstack integration in > devstack-plugin-ceph. > > > > > > We have Tom Barron (tbarron) in the team, i'd like to propose myself > > > (gouthamr) and Victoria Martinez de la Cruz (vkmc) > > > > > > Please let me know what you think of the idea. > > > > I've got no objection from the Cinder side. I would also not object to > > adding the manila-core group instead of individuals. It's certainly in > > your team's interest to keep this thing stable and working, just as it > > is for the Cinder team. > > Agree, I think adding manila group will be helpful, let me know if ok for > you > and accordinfgly I will make changes. > Sure thing, works for me. Thanks Brian and Ghanshyam. > > -gmann > > > > > > > > > > > > > > > > 2. The Cinder team is interested in becoming more active > in the > > > > > maintenance of x/devstack-plugin-nfs [1]. Currently, the > > > > > devstack-plugin-nfs-core is > > > > > https://review.opendev.org/#/admin/groups/1330,members > > > > > It's already 75% cinder-core members; we'd like to replace > the > > > > > individual members with the cinder-core group. We also > > > propose that > > > > > devstack-core be added as an included group. > > > > > > > > > > 3. The Cinder team is interested in implementing a new > > > devstack plugin: > > > > > openstack/devstack-plugin-open-cas > > > > > This will enable thorough testing of a new feature [2] > being > > > introduced > > > > > as experimental in Ussuri and expected to be finalized in > > > Victoria. Our > > > > > plan would be to make both cinder-core and devstack-core > > > included groups > > > > > for the gerrit group governing the new plugin. > > > > > > > > +1. You want this under Cinder governance or under QA ? > > > > > > I think it makes sense for these to be under QA governance -- QA > would > > > own the repo with both QA and Cinder having permission to make > changes. > > > > > > > > > > > > > 4. This is a minor point, but can the devstack-plugin-nfs > > > repo be moved > > > > > back into the 'openstack' namespace? > > > > > > > > If this is usable plugin for nfs testing (I am not aware if we > > > have any other) then > > > > it make sense to bring it to openstack governance. > > > > Same question here, do you want to put this under Cinder > > > governance or QA. > > > > > > Same here, I think QA should "own" the repo, but Cinder will have > > > permission to make changes there. > > > > > > > > > > > Those plugins under QA governance also ok for me with your > > > proposal of calloborative maintainance by > > > > devstack-core and cinder-core. > > > > > > > > -gmann > > > > > > Thanks for the quick response! > > > > > > > > > > > > > Let us know which of these proposals you find acceptable. > > > > > > > > > > > > > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > > > > > [1] https://opendev.org/x/devstack-plugin-nfs > > > > > [2] > > > > https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > > > > > > > > > > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Mon Mar 9 19:49:53 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 9 Mar 2020 15:49:53 -0400 Subject: [cinder] reminder: ussuri virtual mid-cycle 16 march at 12:00 UTC Message-ID: (Feedback from the last virtual meet-up was that it needed more promotion, so if you know anyone who might be (or should be) interested, please tell them, in addition to marking your own calendar.) Session Two of the Cinder Ussuri virtual mid-cycle will be held: DATE: Monday, 16 March 2020 TIME: 1200-1400 UTC LOCATION: https://bluejeans.com/3228528973 The meeting will be recorded. Please add topics to the planning etherpad: https://etherpad.openstack.org/p/cinder-ussuri-mid-cycle-planning cheers, brian From openstack at nemebean.com Mon Mar 9 20:51:12 2020 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 9 Mar 2020 15:51:12 -0500 Subject: [oslo][infra] Oslo core security team on Launchpad Message-ID: <6427416f-ed83-e7c2-e40e-a5013202d5ce@nemebean.com> Hi, I just noticed that the Oslo core security team includes a number of people no longer active in Oslo and also only me for current cores. We should really clean that up so random people aren't getting notified of private security bugs and ideally add some current cores so we have more eyes on said security bugs. How do we go about doing that? I see it's owned by the OpenStack Administrators team, so do I put in a request with the changes or can they just make me an administrator for that group? Thanks. -Ben From fungi at yuggoth.org Mon Mar 9 21:18:03 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 9 Mar 2020 21:18:03 +0000 Subject: [oslo][infra] Oslo core security team on Launchpad In-Reply-To: <6427416f-ed83-e7c2-e40e-a5013202d5ce@nemebean.com> References: <6427416f-ed83-e7c2-e40e-a5013202d5ce@nemebean.com> Message-ID: <20200309211802.fejswnds7n6t55zt@yuggoth.org> On 2020-03-09 15:51:12 -0500 (-0500), Ben Nemec wrote: > I just noticed that the Oslo core security team includes a number > of people no longer active in Oslo and also only me for current > cores. We should really clean that up so random people aren't > getting notified of private security bugs and ideally add some > current cores so we have more eyes on said security bugs. It's been languishing on my to do list to remind all projects with the vulnerability:managed governance tag to review those group memberships in LP regularly and keep them groomed to fit the recommendations in requirement #2 here: https://governance.openstack.org/tc/reference/tags/vulnerability_managed.html#requirements 2. The deliverable must have a dedicated point of contact for security issues (which could be shared by multiple deliverables in a given project-team if needed), so that the VMT can engage them to triage reports of potential vulnerabilities. Deliverables with more than five core reviewers should (so as to limit the unnecessary exposure of private reports) settle on a subset of these to act as security core reviewers whose responsibility it is to be able to confirm whether a bug report is accurate/applicable or at least know other subject matter experts they can in turn subscribe to perform those activities in a timely manner. They should also be able to review and provide pre-approval of patches attached to private bugs, which is why at least a majority are expected to be core reviewers for the deliverable. These should be members of a group contact (for example a -coresec team) in the deliverable’s defect tracker so that the VMT can easily subscribe them to new bugs." We're also trying to keep the liaisons and links to corresponding security teams tracked here for faster VMT response: https://wiki.openstack.org/wiki/CrossProjectLiaisons#Vulnerability_management > How do we go about doing that? A group member marked as an "administrator" for it should add and remove members as needed. Generally this group would include the current PTL or active liaison for vulnerability reports as an administrative member to take care of the duty of maintaining group membership, including proper hand-off during transitions of leadership. > I see it's owned by the OpenStack Administrators team, so do I put > in a request with the changes or can they just make me an > administrator for that group? Since I'm in the OpenStack Administrators group on LP I've gone ahead and flagged your membership in oslo-coresec as having administrative privileges. We require these groups to be owned by OpenStack Administrators so that it can act as a fallback in situations like this where expected group admin hand-off has been forgotten. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From openstack at nemebean.com Mon Mar 9 21:42:20 2020 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 9 Mar 2020 16:42:20 -0500 Subject: [oslo][infra] Oslo core security team on Launchpad In-Reply-To: <20200309211802.fejswnds7n6t55zt@yuggoth.org> References: <6427416f-ed83-e7c2-e40e-a5013202d5ce@nemebean.com> <20200309211802.fejswnds7n6t55zt@yuggoth.org> Message-ID: On 3/9/20 4:18 PM, Jeremy Stanley wrote: > On 2020-03-09 15:51:12 -0500 (-0500), Ben Nemec wrote: >> I just noticed that the Oslo core security team includes a number >> of people no longer active in Oslo and also only me for current >> cores. We should really clean that up so random people aren't >> getting notified of private security bugs and ideally add some >> current cores so we have more eyes on said security bugs. > > It's been languishing on my to do list to remind all projects with > the vulnerability:managed governance tag to review those group > memberships in LP regularly and keep them groomed to fit the > recommendations in requirement #2 here: > > https://governance.openstack.org/tc/reference/tags/vulnerability_managed.html#requirements > > > 2. The deliverable must have a dedicated point of contact for > security issues (which could be shared by multiple deliverables > in a given project-team if needed), so that the VMT can engage > them to triage reports of potential vulnerabilities. Deliverables > with more than five core reviewers should (so as to limit the > unnecessary exposure of private reports) settle on a subset of > these to act as security core reviewers whose responsibility it > is to be able to confirm whether a bug report is > accurate/applicable or at least know other subject matter experts > they can in turn subscribe to perform those activities in a > timely manner. They should also be able to review and provide > pre-approval of patches attached to private bugs, which is why at > least a majority are expected to be core reviewers for the > deliverable. These should be members of a group contact (for > example a -coresec team) in the deliverable’s defect > tracker so that the VMT can easily subscribe them to new bugs." > > We're also trying to keep the liaisons and links to corresponding > security teams tracked here for faster VMT response: > > https://wiki.openstack.org/wiki/CrossProjectLiaisons#Vulnerability_management > >> How do we go about doing that? > > A group member marked as an "administrator" for it should add and > remove members as needed. Generally this group would include the > current PTL or active liaison for vulnerability reports as an > administrative member to take care of the duty of maintaining group > membership, including proper hand-off during transitions of > leadership. > >> I see it's owned by the OpenStack Administrators team, so do I put >> in a request with the changes or can they just make me an >> administrator for that group? > > Since I'm in the OpenStack Administrators group on LP I've gone > ahead and flagged your membership in oslo-coresec as having > administrative privileges. We require these groups to be owned by > OpenStack Administrators so that it can act as a fallback in > situations like this where expected group admin hand-off has been > forgotten. > Great, thanks! I have something to add to my shiny new Oslo PTL guide. :-) From kennelson11 at gmail.com Tue Mar 10 00:25:59 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Mon, 9 Mar 2020 17:25:59 -0700 Subject: [all] Collecting Virtual Midcycle Best Practices Message-ID: Hello Everyone! I wanted to collect best practices and pitfalls to avoid wrt projects experiences with virtual midcycles. I know of a few projects that have done them in the past and with how travel is hard for a lot of people right now, I expect more projects to have midcycles. I think it would be helpful to have all of the data we can collect in one place for those not just new to virtual midcycles but the whole community. I threw some categories into this etherpad[1] and filled in some options. Please add to it :) -Kendall (diablo_rojo) [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Tue Mar 10 01:30:11 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 09 Mar 2020 20:30:11 -0500 Subject: [qa][cinder][devstack] proposed governance changes for some devstack plugins In-Reply-To: References: <170a7b5430a.1155e6495437733.1575830632912803163@ghanshyammann.com> <69fcb574-1ae1-08cb-e8e2-8bd08bef80f4@gmail.com> <170c0b444a5.e5378d4f17622.7872527250831207185@ghanshyammann.com> Message-ID: <170c21047ef.1139d333621430.2800684447022470438@ghanshyammann.com> ---- On Mon, 09 Mar 2020 14:21:09 -0500 Goutham Pacha Ravi wrote ---- > > > On Mon, Mar 9, 2020 at 12:10 PM Ghanshyam Mann wrote: > ---- On Mon, 09 Mar 2020 13:19:32 -0500 Brian Rosmaita wrote ---- > > On 3/6/20 6:12 PM, Goutham Pacha Ravi wrote: > > > > > > On Thu, Mar 5, 2020 at 11:53 AM Brian Rosmaita > > > > wrote: > > > > > > On 3/4/20 5:40 PM, Ghanshyam Mann wrote: > > > > ---- On Wed, 04 Mar 2020 13:53:00 -0600 Brian Rosmaita > > > > > > > wrote ---- > > > > > Hello QA team and devstack-plugin-ceph-core people, > > > > > > > > > > The Cinder team has some proposals we'd like to float. > > > > > > > > > > 1. The Cinder team is interested in becoming more active in the > > > > > maintenance of openstack/devstack-plugin-ceph [0]. > > > Currently, the > > > > > devstack-plugin-ceph-core is > > > > > https://review.opendev.org/#/admin/groups/1196,members > > > > > The cinder-core is already represented by Eric and Sean; we'd > > > like to > > > > > replace them by including the cinder-core group. > > > > > > > > +1. This is good diea and make sense, I will do the change. > > > > > > Great, thanks! > > > > > > > > > > > > I agree this is a great idea to have more members of Cinder joining the > > > devstack-plugin-ceph team. I would like to have atleast a sub team of > > > manila core reviewers added to this project if it makes sense. The > > > Manila CephFS drivers (cephfs-native and cephfs-nfs) are currently being > > > tested with the help of the devstack integration in devstack-plugin-ceph. > > > > > > We have Tom Barron (tbarron) in the team, i'd like to propose myself > > > (gouthamr) and Victoria Martinez de la Cruz (vkmc) > > > > > > Please let me know what you think of the idea. > > > > I've got no objection from the Cinder side. I would also not object to > > adding the manila-core group instead of individuals. It's certainly in > > your team's interest to keep this thing stable and working, just as it > > is for the Cinder team. > > Agree, I think adding manila group will be helpful, let me know if ok for you > and accordinfgly I will make changes. > > > Sure thing, works for me. Thanks Brian and Ghanshyam. Done. Replace the individual with manila group. -gmann > > -gmann > > > > > > > > > > > > > > > > 2. The Cinder team is interested in becoming more active in the > > > > > maintenance of x/devstack-plugin-nfs [1]. Currently, the > > > > > devstack-plugin-nfs-core is > > > > > https://review.opendev.org/#/admin/groups/1330,members > > > > > It's already 75% cinder-core members; we'd like to replace the > > > > > individual members with the cinder-core group. We also > > > propose that > > > > > devstack-core be added as an included group. > > > > > > > > > > 3. The Cinder team is interested in implementing a new > > > devstack plugin: > > > > > openstack/devstack-plugin-open-cas > > > > > This will enable thorough testing of a new feature [2] being > > > introduced > > > > > as experimental in Ussuri and expected to be finalized in > > > Victoria. Our > > > > > plan would be to make both cinder-core and devstack-core > > > included groups > > > > > for the gerrit group governing the new plugin. > > > > > > > > +1. You want this under Cinder governance or under QA ? > > > > > > I think it makes sense for these to be under QA governance -- QA would > > > own the repo with both QA and Cinder having permission to make changes. > > > > > > > > > > > > > 4. This is a minor point, but can the devstack-plugin-nfs > > > repo be moved > > > > > back into the 'openstack' namespace? > > > > > > > > If this is usable plugin for nfs testing (I am not aware if we > > > have any other) then > > > > it make sense to bring it to openstack governance. > > > > Same question here, do you want to put this under Cinder > > > governance or QA. > > > > > > Same here, I think QA should "own" the repo, but Cinder will have > > > permission to make changes there. > > > > > > > > > > > Those plugins under QA governance also ok for me with your > > > proposal of calloborative maintainance by > > > > devstack-core and cinder-core. > > > > > > > > -gmann > > > > > > Thanks for the quick response! > > > > > > > > > > > > > Let us know which of these proposals you find acceptable. > > > > > > > > > > > > > > > [0] https://opendev.org/openstack/devstack-plugin-ceph > > > > > [1] https://opendev.org/x/devstack-plugin-nfs > > > > > [2] > > > https://blueprints.launchpad.net/cinder/+spec/support-volume-local-cache > > > > > > > > > > > > > > > > > > > > > > > > > > > From Dong.Ding at dell.com Tue Mar 10 01:55:29 2020 From: Dong.Ding at dell.com (Dong.Ding at dell.com) Date: Tue, 10 Mar 2020 01:55:29 +0000 Subject: [manila] share group replication spike/questions In-Reply-To: References: <55d84e2e29cb4758aaff0b8c07aaa0bd@KULX13MDC124.APAC.DELL.COM> Message-ID: <3b103cb9a3894762a8664815fff5771c@KULX13MDC124.APAC.DELL.COM> Hi, Gotham, After checked the manila DB, I noticed there is table called ‘share_instances’ which was added for share replication and snapshot. Now, for group replication, do you think we also need a new table like ‘share_group_instances’ ? Thanks, Ding Dong From: Goutham Pacha Ravi Sent: Saturday, February 29, 2020 7:43 AM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Fri, Feb 28, 2020 at 12:21 AM > wrote: Thanks Gotham, We are talking about this feature after U release. Cannot get it done in recently. Just do some prepare first. Great, thanks for confirming. We'll hash out the design on the specification, and if necessary, we can work through it during the Open Infra Project Technical Gathering in June [8][9] [8] https://www.openstack.org/ptg/ [9] https://etherpad.openstack.org/p/vancouver-ptg-manila-planning BR, Ding Dong From: Goutham Pacha Ravi > Sent: Friday, February 28, 2020 7:10 AM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Tue, Feb 25, 2020 at 12:53 AM > wrote: Hi, guys, As we talked about the topic in a virtual PTG few months ago. https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual (Support promoting several shares in group (DELL EMC: dingdong) I’m trying to write a manila-spec for it. Hi, thank you for working on this, and for submitting a specification [0]. We're targeting this for the Victoria release, correct? I like working on these major changes as soon as possible giving us enough air time for testing and hardening. It’s my first experience to implement such feature in framework. I need to double check with you something, and hope you can give me some guides like: 1. Where is the extra-spec defined for group/group type, it’s in Manila repo, right? (like manila.db.sqlalchemy.models….) Group type extra specs are added as storage capabilities first, you begin by modifying the driver interface to report this group type capability. When share drivers report their support for group replication, operators can use the corresponding string in their group type extra-specs to schedule appropriately. I suggest taking a look at an existing share group type capability called "consistent_snapshot_support". [1] and [2] are reviews that added it. 2. The command cli should be implemented for ‘python-manilaclinet’ repo, right? (I have never touched this repo before) Yes. python-manilaclient encompasses - a python SDK to version 2 of the manila API - two shell implementations: manila and openstack client (actively being developed) Group type extra-specs are passed transparently through the SDK and CLI, you may probably add some documentation or shell hint text (like [3] if needed). 3. Where is the rest-api should be implemented? The rest API is in the openstack/manila repository. [4][5] contain some documentation regarding how to change the manila API. 4. And more tips you have? like any other related project should be changed? For any new feature, we need these additional things besides working code: - A first party driver implementation where possible so we can test this feature in the upstream CI (if no first party driver can support this feature, you'll need to make the best approximation of this feature through the Dummy/Fake driver [6]) - The feature must be tested with adequate test cases in manila-tempest-plugin - Documentation must be added to the manila documentation [7] Just list what I know, and more details questions will be raised when implementing, I think. FYI Thanks, Ding Dong Happy to answer any more questions, here or on your specification [0] Thanks, Goutham [0] https://review.opendev.org/#/c/710166/ [1] https://review.opendev.org/#/c/446044/ [2] https://review.opendev.org/#/c/447474/ [3] https://opendev.org/openstack/python-manilaclient/src/commit/ac5ca461e8c8dd11fe737de7b90ab5c33366ab35/manilaclient/v2/shell.py#L4543 [4] https://docs.openstack.org/manila/latest/contributor/addmethod.openstackapi.html [5] https://docs.openstack.org/manila/latest/contributor/api_microversion_dev.html [6] https://opendev.org/openstack/manila/src/commit/68a18f49472ac7686ceab15e9788dcef05764822/manila/tests/share/drivers/dummy.py [7] https://docs.openstack.org/manila/latest/contributor/documenting_your_work.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From zbitter at redhat.com Tue Mar 10 02:21:11 2020 From: zbitter at redhat.com (Zane Bitter) Date: Mon, 9 Mar 2020 22:21:11 -0400 Subject: [all][ideas] Introducing Project Teapot: a baremetal cloud for the 2020s Message-ID: <1dce98d8-e92d-f648-282d-2491d32ae1bd@redhat.com> I'm pleased to announce the first entry on the new Ideas site (I wanted to call it Crazy Ideas, but JP overruled me and he did all the work to set it up), which also happens to be the reason the Ideas site exists: Project Teapot. (Before we go any further; we all wear a lot of hats in this community, so I'd like to make clear that I'm writing this email wearing my cap-and-bells in my capacity as self-appointed spokes-jester for Project Teapot, and no other.) What is Project Teapot? It's a new kind of cloud that has only one type of workload: Kubernetes on bare metal. Intrigued? Read on: https://governance.openstack.org/ideas/ideas/teapot/index.html What does this have to do with OpenStack? Well, it turns out that the OpenStack community has already built a lot (but not all) of the implementations of things that a cloud like this would need, in the form of projects like Ironic, Manila, Cinder, Keystone, Designate, Barbican. Plus we think it could probably be run as an OpenStack service alongside an existing OpenStack cloud when desired as well. Thanks are due to all of the folks who helped develop this idea, and the domain experts who reviewed it to hopefully eliminate the most egregious errors. Now it's over to you. If you need this, or want to help implement it, we'd like to hear from you on this thread. If you think this is a terrible idea and we should all be run out of town on a rail, we want to hear that too! (Pro tip: use the hashtag #ProjectCrackpot when you complain about it on Twitter.) If you have corrections or additional implementation ideas to add, feel free to submit a patch to the openstack/ideas repo. You can also add questions as inline comments on the original review (https://review.opendev.org/710173) if you want. It might pay to flag anything you post to Gerrit in this thread as well to make sure it's not missed. Finally, if you have ideas of your own, please submit them to the Ideas repo. Remember, they can be as crazy as you want. Let's not let the wisdom of the OpenStack community remain locked up in individual heads. cheers, Zane. From whayutin at redhat.com Tue Mar 10 02:57:37 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Mon, 9 Mar 2020 20:57:37 -0600 Subject: [tripleo] Message-ID: Greetings, Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours. Sorry again, and hopefully you'll be back up and running shortly. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From whayutin at redhat.com Tue Mar 10 04:24:40 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Mon, 9 Mar 2020 22:24:40 -0600 Subject: [tripleo] In-Reply-To: References: Message-ID: On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin wrote: > Greetings, > > Everyone you are going to find your jobs going red atm. My apologies some > containers were not pushed manually properly and I'm having to repull and > and repush to docker.io atm. We should have this fixed in about 2 hours. > > Sorry again, and hopefully you'll be back up and running shortly. > > Thanks > Quick update.. We're about 1/2 done w/ pushing the new containers for centos-8, the tag is af182654cc32d30ea7f4774eb06ed9fd Hopefully all the mirrors will get seeded quickly. Sorry for the inconvenience. -------------- next part -------------- An HTML attachment was scrubbed... URL: From soulxu at gmail.com Tue Mar 10 05:45:13 2020 From: soulxu at gmail.com (Alex Xu) Date: Tue, 10 Mar 2020 13:45:13 +0800 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: References: <1583482276.12170.14@est.tech> <4692F106-6B00-41FC-9BA9-1DF62A24EDAB@fried.cc> <170b01d7595.10341333e516143.4131462912712933865@ghanshyammann.com> Message-ID: gibi, thanks. Sylvain Bauza 于2020年3月10日周二 上午12:09写道： > Thanks indeed gibi for this. > > On Fri, Mar 6, 2020 at 2:58 PM Ghanshyam Mann > wrote: > >> ---- On Fri, 06 Mar 2020 07:01:15 -0600 Eric Fried >> wrote ---- >> > Big +1 from me. Many thanks, gibi. Not that you‘ll need it, but >> please don’t hesitate to reach out to me if you have questions. >> >> Indeed. Thanks gibi for helping out here. >> >> -gmann >> > >> > efried_gone >> > >> > > On Mar 6, 2020, at 02:16, Balázs Gibizer >> wrote: >> > > >> > > Hi, >> > > >> > > Since Eric announced that he has to leave us [1] I have been working >> > > internally with my employee to be able to take over the Nova PTL >> > > position. Now I've got the necessary approvals. The official PTL >> > > election is close [2] and I'm ready to fill the PTL gap until the >> > > proper PTL election in April. >> > > >> > > Is this a workable solution for the community? >> > > >> > > Cheers, >> > > gibi >> > > >> > > [1] >> > > >> http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012663.html >> > > [2] https://governance.openstack.org/election/ >> > > >> > > >> > > >> > >> > >> > >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From licanwei_cn at 163.com Tue Mar 10 06:30:57 2020 From: licanwei_cn at 163.com (licanwei) Date: Tue, 10 Mar 2020 14:30:57 +0800 (GMT+08:00) Subject: [Watcher]about meeting on March 11 Message-ID: <3555fb1f.2d8e.170c323a2a3.Coremail.licanwei_cn@163.com> Hi, We will have the team meeting tomorrow at 08:00 UTC on #openstack-meeting-alt. pls update the meeting agenda if you have something want to be discussed. if nothing need to be discussed, maybe i'll cancel the meeting. thanks, licanwei | | licanwei_cn | | 邮箱：licanwei_cn at 163.com | 签名由 网易邮箱大师 定制 -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Tue Mar 10 08:16:54 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 10 Mar 2020 09:16:54 +0100 Subject: [nova][stable] Nova stable branch liaison In-Reply-To: <20200309190144.yad4y7wrcnpsfk4j@lyarwood.usersys.redhat.com> References: <1583767645.12170.27@est.tech> <20200309190144.yad4y7wrcnpsfk4j@lyarwood.usersys.redhat.com> Message-ID: <1583828214.12170.28@est.tech> On Mon, Mar 9, 2020 at 19:01, Lee Yarwood wrote: > On 09-03-20 16:27:25, Balázs Gibizer wrote: >> Hi Team, >> >> According to the wiki [1] we still have Matt as a stable branch >> liaison for >> nova. Obviously we need to find somebody else as Matt is gone. I'm >> not part >> of the nova-stable-core team so I cannot assume default ownership >> of that. >> Also I see that we have a small but active stable team so I hope >> somebody >> from that team can step forward and can take this role. > > Yup I would be happy to help with this. Thank you for taking this work up. > > Should I update the page assuming we don't have any other volunteers? I trust you that you can do it and I handle this in a first come first serve basis. But anyone who want to help in stable maint can reach out to me (or I guess to you as well) publicly or privately and we can arrange further load sharing if needed. Also after the PTL election the next PTL (if any) can re-evaluate the liaison situation if needed. I updated the wiki accordingly. Cheers, gibi > > Cheers, > > Lee > >> [1] >> https://wiki.openstack.org/wiki/CrossProjectLiaisons#Stable_Branch > > -- > Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 > F672 2D76 From thierry at openstack.org Tue Mar 10 09:15:50 2020 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 10 Mar 2020 10:15:50 +0100 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: Kendall Nelson wrote: > I wanted to collect best practices and pitfalls to avoid wrt projects > experiences with virtual midcycles. I know of a few projects that have > done them in the past and with how travel is hard for a lot of people > right now, I expect more projects to have midcycles. I think it would be > helpful to have all of the data we can collect in one place for those > not just new to virtual midcycles but the whole community. > [...] Also interested in feedback from teams that had virtual PTGs in the past (keeping all possibilities on the table). I think Kolla, Telemetry and a few others did that. -- Thierry Carrez (ttx) From mark at stackhpc.com Tue Mar 10 09:45:22 2020 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 10 Mar 2020 09:45:22 +0000 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: On Tue, 10 Mar 2020 at 09:17, Thierry Carrez wrote: > > Kendall Nelson wrote: > > I wanted to collect best practices and pitfalls to avoid wrt projects > > experiences with virtual midcycles. I know of a few projects that have > > done them in the past and with how travel is hard for a lot of people > > right now, I expect more projects to have midcycles. I think it would be > > helpful to have all of the data we can collect in one place for those > > not just new to virtual midcycles but the whole community. > > [...] > > Also interested in feedback from teams that had virtual PTGs in the past > (keeping all possibilities on the table). I think Kolla, Telemetry and a > few others did that. Kolla has now had two virtual PTGs. Overall I think they went fairly well, particularly the most recent one. We tried Zoom, then moved to Google Meet. I forget the problems with Zoom. There were inevitably a few teething problems with the video, but I think we worked it out after 15-20 minutes. Etherpad for Ussuri vPTG here: https://etherpad.openstack.org/p/kolla-ussuri-ptg. Without seeing people's faces it can be hard to ensure everyone keeps focussed. It's quite rare for the whole room to be focussed at physical discussions though. Going around the room giving short intros helps to get people talking, and it may be better to do these ~1 hour in as people may miss the start. Directing questions at non-cores can help overcome that pesky imposter syndrome. Keeping video on definitely helps with engagement, up to the point where it impacts audio quality. There was also the Denver PTG where the PTL and a number of cores were remote where we struggled to make any progress. I think there were a few reasons for this. The fixed time of the PTG was not optimal for many remote attendees living in Europe or Asia. When there are a number of participants in one location, it can be easy to forget to direct speech at the microphone, allow time for remote callers to ask questions/respond etc. This makes it difficult and frustrating for them to join in, making it easier to get distracted and drop off. Not too much hard data in there, but hopefully a feel for how it went for us. > > -- > Thierry Carrez (ttx) > From balazs.gibizer at est.tech Tue Mar 10 09:49:13 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 10 Mar 2020 10:49:13 +0100 Subject: [nova] US meeting slot In-Reply-To: <1583744340.12170.17@est.tech> References: <1583744340.12170.17@est.tech> Message-ID: <1583833753.12170.29@est.tech> On Mon, Mar 9, 2020 at 08:59, Balázs Gibizer wrote: > Hi, > > Nova has alternate meeting slots on Thursdays to try to cover > contributors from different time zones. > * 14:00 UTC > * 21:00 UTC > > As I'm taking over the PTL role form Eric I need to figure out how to > run the nova meetings. I cannot really run the UTC 21:00 as it is > pretty late for me. (I will run the 14:00 UTC slot). I see different > options: > > a) Somebody from the US side of the globe volunteers to run the 21:00 > UTC slot. Please speak up if you would like to run it. I can help you > with agenda refresh and technicalities if needed. > > b) Have only one meeting time, and move that to 16:00 UTC. In this > case I will be able to run it most of the weeks. > > c) Do not have a dedicated meeting slot but switch to office hours. > Here we also need to find a time slot. I think 16:00 UTC could work > there as well. > > > Please share your view! Any other proposal is very welcome. Thank you all for the feedback. I see a consensus forming around moving the weekly meeting slot both from 21:00 UTC and from 14:00 UTC to a single 16:00 UTC slot. I'm aware that this is bad for our contributors from China. To help with this pain I offer to be available on Thurday 8:00 - 9:00 UTC on #openstack-nova to discuss any issues that needs to be brought up on the 16:00 UTC meeting (almost like an office hour). Let's see how this works out and please give me feedback any time. // gibi > > Cheers, > gibi > From thierry at openstack.org Tue Mar 10 10:12:13 2020 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 10 Mar 2020 11:12:13 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: <2e142636-0070-704c-c5f7-1e035bc9d406@openstack.org> Mohammed Naser wrote: > [...] > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. > > The above is just a very basic idea, I don't intend to diving much > more in depth for now as I'd like to hear about what the rest of the > community thinks. I agree that in the current age we need to take steps to avoid overwhelming roles and long commitments. As others said, we also need to preserve some accountability, but I don't think those goals are incompatible. The original design goal of the "PTL" system was to have a clear "bucket stops here" for technical decisions at project-team level, as well as a safety valve for contributors at large (through elections) to reset the core reviewers team if it's gone wild. The "bucket stops here" power was very rarely exercised (probably due to its mere existence). I'd agree that today this is less needed, and we could have equal-power maintainers/corereviewers. We still have the TC above project teams as a safety valve, and we could agree that petitions from enough contributors can trigger a reset of the core reviewers structure. The real benefit of the "PTL" system today is to facilitate the work of people outside the project team. When you try to put out a coordinated release (or organize a PTG), having a clear person that can "speak for the team", without having to get into specifics for each of our 60+ teams, is invaluable. That said, there is really no reason why that clear person should be always the same person, for 6 months. We've always said that those subroles (release liaison, meeting chair, event liaison...) should be decomposed and delegated to multiple people. That the PTL should only be involved if the role was not delegated. Yet in most teams the PTL has trouble delegating and still fills all those roles. We need to change the perception. So one solution might be: - Define multiple roles (release liaison, event liaison, meeting chair...) and allow them to be filled by the team as they want, for the duration they want, replaced when they want (would just need +1 from previous and new holder of the role) - Use the TC as a governance safety valve to resolve any conflict (instead of PTL elections) -- Thierry Carrez (ttx) From balazs.gibizer at est.tech Tue Mar 10 11:39:51 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 10 Mar 2020 12:39:51 +0100 Subject: [nova] US meeting slot In-Reply-To: <1583833753.12170.29@est.tech> References: <1583744340.12170.17@est.tech> <1583833753.12170.29@est.tech> Message-ID: <1583840391.12170.30@est.tech> On Tue, Mar 10, 2020 at 10:49, Balázs Gibizer wrote: > > > On Mon, Mar 9, 2020 at 08:59, Balázs Gibizer > wrote: >> Hi, >> >> Nova has alternate meeting slots on Thursdays to try to cover >> contributors from different time zones. >> * 14:00 UTC >> * 21:00 UTC >> >> As I'm taking over the PTL role form Eric I need to figure out how >> to run the nova meetings. I cannot really run the UTC 21:00 as it >> is pretty late for me. (I will run the 14:00 UTC slot). I see >> different options: >> >> a) Somebody from the US side of the globe volunteers to run the >> 21:00 UTC slot. Please speak up if you would like to run it. I can >> help you with agenda refresh and technicalities if needed. >> >> b) Have only one meeting time, and move that to 16:00 UTC. In this >> case I will be able to run it most of the weeks. >> >> c) Do not have a dedicated meeting slot but switch to office hours. >> Here we also need to find a time slot. I think 16:00 UTC could work >> there as well. >> >> >> Please share your view! Any other proposal is very welcome. > > Thank you all for the feedback. I see a consensus forming around > moving the weekly meeting slot both from 21:00 UTC and from 14:00 UTC > to a single 16:00 UTC slot. I'm aware that this is bad for our > contributors from China. To help with this pain I offer to be > available on Thurday 8:00 - 9:00 UTC on #openstack-nova to discuss > any issues that needs to be brought up on the 16:00 UTC meeting > (almost like an office hour). Let's see how this works out and please > give me feedback any time. Patch is up to re-schedule meeting https://review.opendev.org/#/c/712052 gibi > > // > gibi > >> >> Cheers, >> gibi >> > > > From sean.mcginnis at gmx.com Tue Mar 10 13:56:33 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 10 Mar 2020 08:56:33 -0500 Subject: [all] Victoria schedule Message-ID: Hello everyone, The proposed schedule for the Victoria release has now been approved and the schedule published: https://releases.openstack.org/victoria/schedule.html Some of the key dates for Victoria: * Milestone 1 - June 18 * Milestone 2 - July 30 * Milestone 3 - Sept 10 * RC1 deadline - Sept 24 * Final RC deadline - Oct 8 * Victoria coordinated release - Oct 14 Thanks! Sean From whayutin at redhat.com Tue Mar 10 14:24:27 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Tue, 10 Mar 2020 08:24:27 -0600 Subject: [tripleo] In-Reply-To: References: Message-ID: Greetings, Follow up.. new containers have been pushed to docker.io. Thank you for your patience! On Mon, Mar 9, 2020 at 10:24 PM Wesley Hayutin wrote: > > > On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin wrote: > >> Greetings, >> >> Everyone you are going to find your jobs going red atm. My >> apologies some containers were not pushed manually properly and I'm having >> to repull and and repush to docker.io atm. We should have this fixed in >> about 2 hours. >> >> Sorry again, and hopefully you'll be back up and running shortly. >> >> Thanks >> > > Quick update.. > We're about 1/2 done w/ pushing the new containers for centos-8, the tag > is af182654cc32d30ea7f4774eb06ed9fd > > > > Hopefully all the mirrors will get seeded quickly. > Sorry for the inconvenience. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mordred at inaugust.com Tue Mar 10 14:34:13 2020 From: mordred at inaugust.com (Monty Taylor) Date: Tue, 10 Mar 2020 09:34:13 -0500 Subject: [tripleo] In-Reply-To: References: Message-ID: <1B1A2143-A018-408D-9515-A367CEA952B5@inaugust.com> > On Mar 10, 2020, at 9:24 AM, Wesley Hayutin wrote: > > Greetings, > > Follow up.. new containers have been pushed to docker.io. > Thank you for your patience! Yay! When you have brainspace after firefighting (always fun) - maybe we should find a time to talk about whether our image building and publishing automation could help you out here. No rush - this is one of those “we’ve got some tools we might be able to leverage to help” - just ping me whenever. > On Mon, Mar 9, 2020 at 10:24 PM Wesley Hayutin wrote: > > > On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin wrote: > Greetings, > > Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours. > > Sorry again, and hopefully you'll be back up and running shortly. > > Thanks > > Quick update.. > We're about 1/2 done w/ pushing the new containers for centos-8, the tag is af182654cc32d30ea7f4774eb06ed9fd > > Hopefully all the mirrors will get seeded quickly. > Sorry for the inconvenience. > From whayutin at redhat.com Tue Mar 10 14:49:08 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Tue, 10 Mar 2020 08:49:08 -0600 Subject: [tripleo] In-Reply-To: <1B1A2143-A018-408D-9515-A367CEA952B5@inaugust.com> References: <1B1A2143-A018-408D-9515-A367CEA952B5@inaugust.com> Message-ID: On Tue, Mar 10, 2020 at 8:34 AM Monty Taylor wrote: > > > > On Mar 10, 2020, at 9:24 AM, Wesley Hayutin wrote: > > > > Greetings, > > > > Follow up.. new containers have been pushed to docker.io. > > Thank you for your patience! > > Yay! > > When you have brainspace after firefighting (always fun) - maybe we should > find a time to talk about whether our image building and publishing > automation could help you out here. No rush - this is one of those “we’ve > got some tools we might be able to leverage to help” - just ping me > whenever. > > Definitely interested!! Some additional context on what we're up to is helpful as well. Our current tooling is here [1] and we're under the gun w/ getting centos-8 ready for ussuri and DLRN's new component feature [2]. If there are upstream tooling and processes we can incorporate we'd be interested in picking our heads up and listening! Thanks as usual Monty! [1] https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/dlrnapi_promoter [2] https://review.rdoproject.org/r/#/c/24818/ https://trunk.rdoproject.org/centos8-master/component/ https://dlrn.readthedocs.io/en/latest/api.html > > On Mon, Mar 9, 2020 at 10:24 PM Wesley Hayutin > wrote: > > > > > > On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin > wrote: > > Greetings, > > > > Everyone you are going to find your jobs going red atm. My apologies > some containers were not pushed manually properly and I'm having to repull > and and repush to docker.io atm. We should have this fixed in about 2 > hours. > > > > Sorry again, and hopefully you'll be back up and running shortly. > > > > Thanks > > > > Quick update.. > > We're about 1/2 done w/ pushing the new containers for centos-8, the tag > is af182654cc32d30ea7f4774eb06ed9fd > > > > Hopefully all the mirrors will get seeded quickly. > > Sorry for the inconvenience. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyarwood at redhat.com Tue Mar 10 15:22:03 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Tue, 10 Mar 2020 15:22:03 +0000 Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 Message-ID: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> Hello all, I've started PoC'ing some ideas around $subject in the topic below and wanted to ask the wider team for feedback on the approach I'm taking: https://review.opendev.org/#/q/topic:nova-live-migration-zuulv3 My initial idea is to break the job up into the following smaller multinode jobs that are hopefully easier to understand and maintain. * nova-multinode-live-migration-py3 A simple LM job using the qcow2 imagebackend and LVM/iSCSI c-vol. * nova-multinode-live-migration-ceph-py3 A ceph based LM job using rbd for both imagebackend and c-vol. * nova-multinode-evacuate-py3 A separate evacuation job using qcow2 imagebackend and LVM/iSCSI c-vol. The existing script *could* then be ported to an Ansible role as part of the migration to Zuulv3. Hopefully this is pretty straight forward but I'd appreciate any feedback on this all the same. Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From cboylan at sapwetik.org Tue Mar 10 15:25:38 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 10 Mar 2020 08:25:38 -0700 Subject: =?UTF-8?Q?Re:_[nova]_Breaking_up_and_migrating_the_nova-live-migration_j?= =?UTF-8?Q?ob_to_Zuulv3?= In-Reply-To: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> References: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> Message-ID: <00363f3f-ed3d-488d-98d4-c3025b7e179f@www.fastmail.com> On Tue, Mar 10, 2020, at 8:22 AM, Lee Yarwood wrote: > Hello all, > > I've started PoC'ing some ideas around $subject in the topic below and > wanted to ask the wider team for feedback on the approach I'm taking: > > https://review.opendev.org/#/q/topic:nova-live-migration-zuulv3 > > My initial idea is to break the job up into the following smaller > multinode jobs that are hopefully easier to understand and maintain. > > * nova-multinode-live-migration-py3 > > A simple LM job using the qcow2 imagebackend and LVM/iSCSI c-vol. > > * nova-multinode-live-migration-ceph-py3 > > A ceph based LM job using rbd for both imagebackend and c-vol. > > * nova-multinode-evacuate-py3 > > A separate evacuation job using qcow2 imagebackend and LVM/iSCSI c-vol. > The existing script *could* then be ported to an Ansible role as part of > the migration to Zuulv3. > > Hopefully this is pretty straight forward but I'd appreciate any > feedback on this all the same. Just a note that you can probably drop the -py3 suffix as I imagine that is assumed at this point? > > Cheers, > > -- > Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 From gagehugo at gmail.com Tue Mar 10 15:36:09 2020 From: gagehugo at gmail.com (Gage Hugo) Date: Tue, 10 Mar 2020 10:36:09 -0500 Subject: [openstack-helm] VIrtual Midcycle - March 2020 Message-ID: Hello everyone, just a reminder that we are looking to host a virtual midcycle later this month. If you wish to attend, please fill out the poll[0] and etherpad[1] if you have any topics to discuss. We are looking to host a roughly 4 hour session virtually either next week or the following week. [0] https://doodle.com/poll/g6uvdb4rbad9s8gb [1] https://etherpad.openstack.org/p/osh-virtual-ptg-2020-03 -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcin.juszkiewicz at linaro.org Tue Mar 10 16:43:12 2020 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Tue, 10 Mar 2020 17:43:12 +0100 Subject: [horizon][kolla] pyscss failure on newest setuptools Message-ID: <79855e85-adcf-bf3b-dfa4-c017ed2ae329@linaro.org> One of Horizon's requirements is pyscss package. Which had last release over 4 years ago... Two days ago setuptools v46 got released. One of changes was drop of Features feature. Today Kolla builds started to fail: INFO:kolla.common.utils.horizon:Collecting pyScss===1.3.4 INFO:kolla.common.utils.horizon: Downloading http://mirror.ord.rax.opendev.org:8080/pypifiles/packages/1d/4a/221ae7561c8f51c4f28b2b172366ccd0820b14bb947350df82428dfce381/pyScss-1.3.4.tar.gz (120 kB) INFO:kolla.common.utils.horizon: ERROR: Command errored out with exit status 1: INFO:kolla.common.utils.horizon: command: /var/lib/kolla/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rr0db3qs/pyScss/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rr0db3qs/pyScss/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-rr0db3qs/pyScss/pip-egg-info INFO:kolla.common.utils.horizon: cwd: /tmp/pip-install-rr0db3qs/pyScss/ INFO:kolla.common.utils.horizon: Complete output (5 lines): INFO:kolla.common.utils.horizon: Traceback (most recent call last): INFO:kolla.common.utils.horizon: File "", line 1, in INFO:kolla.common.utils.horizon: File "/tmp/pip-install-rr0db3qs/pyScss/setup.py", line 9, in INFO:kolla.common.utils.horizon: from setuptools import setup, Extension, Feature INFO:kolla.common.utils.horizon: ImportError: cannot import name 'Feature' Are there any plans to fix it? pyscss project got issue: https://github.com/Kronuz/pyScss/issues/385 In Kolla I made ugly workaround: https://paste.centos.org/view/2e29d284 What are plans of Horizon team? From Arkady.Kanevsky at dell.com Tue Mar 10 17:04:19 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Tue, 10 Mar 2020 17:04:19 +0000 Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 In-Reply-To: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> References: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> Message-ID: Thank Lee. Sound approach. A few questions/comments. 1. Assume that we have unwritten assumption that all nova nodes have access to volumes on the backend. So we rely on it except for ephemeral storage. 2. What need to be done for volumes that use FC not iSCSI? 3. You have one for Ceph. Does that mean that we need an analog for other cinder back ends? 4. Do we need to anything analogous for Manila? 5. How do we address multi-attach volumes and multipathing? Expect that if we have multipthaing on origin node we laso have multipathing at destination at the end. Thanks, Arkady -----Original Message----- From: Lee Yarwood Sent: Tuesday, March 10, 2020 10:22 AM To: openstack-discuss at lists.openstack.org Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 Hello all, I've started PoC'ing some ideas around $subject in the topic below and wanted to ask the wider team for feedback on the approach I'm taking: https://review.opendev.org/#/q/topic:nova-live-migration-zuulv3 My initial idea is to break the job up into the following smaller multinode jobs that are hopefully easier to understand and maintain. * nova-multinode-live-migration-py3 A simple LM job using the qcow2 imagebackend and LVM/iSCSI c-vol. * nova-multinode-live-migration-ceph-py3 A ceph based LM job using rbd for both imagebackend and c-vol. * nova-multinode-evacuate-py3 A separate evacuation job using qcow2 imagebackend and LVM/iSCSI c-vol. The existing script *could* then be ported to an Ansible role as part of the migration to Zuulv3. Hopefully this is pretty straight forward but I'd appreciate any feedback on this all the same. Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 From grant at civo.com Tue Mar 10 19:18:58 2020 From: grant at civo.com (Grant Morley) Date: Tue, 10 Mar 2020 19:18:58 +0000 Subject: Neutron RabbitMQ issues Message-ID: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> Hi all, We are currently experiencing some fairly major issues with our OpenStack cluster. It all appears to be with Neutron and RabbitMQ.  We are seeing a lot of time out messages in responses to replies and because of this instance creation or anything to do with instances and networking is broken. We are running OpenStack Queens. We have already tuned Rabbit for Neutron by doing the following on neutron: heartbeat_timeout_threshold = 0 rpc_conn_pool_size = 300 rpc_thread_pool_size = 2048 rpc_response_timeout = 3600 rpc_poll_timeout = 60 ## Rpc all executor_thread_pool_size = 64 rpc_response_timeout = 3600 What we are seeing in the error logs for neutron for all services (l3-agent, dhcp, linux-bridge etc ) are these timeouts: https://pastebin.com/Fjh23A5a We have manually tried to get everything in sync by forcing fail-over of the networking which seems to get routers in sync. We are also seeing that there are a lot of "unacknowledged" messages in RabbitMQ for 'q-plugin' in the neutron queues. Some times restarting of the services on neutron gets these back acknowledged again, however the timeouts come back. The RabbitMQ servers themselves are not loaded at all. All memory, file descriptors and errlang processes have plenty of resources available. We are also seeing a lot of rpc issues: Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: MessagingTimeout: Timed out waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC method release_dhcp_port. Waiting for 3347 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: MessagingTimeout: Timed out waiting for a reply to message ID 7937465f15634fbfa443fe1758a12a9c Does anyone know if there is anymore tuning to be done at all? Upgrading for us at the moment to a newer version isn't really an option unfortunately. Because of our setup, we also have roughly 800 routers enabled and I know that will be putting a load on the system. However these problems have only started to happen roughly 1 week ago and have steadily got worse. If anyone has any use cases for this or any more recommendations that would be great. Many thanks, From radoslaw.piliszek at gmail.com Tue Mar 10 20:47:29 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 10 Mar 2020 21:47:29 +0100 Subject: OSC version Message-ID: Hiya Folks, while doing something else entirely (as usual!), I noticed something related to OSC version. OSC metapackage README [1] states that: "The major version of openstackclient will always correspond to the major version of python-openstackclient that will be installed." But OSC metapackage is 4.0.0 atm and installs latest python-osc which is 5.0.0. I don't follow that "correspondence" unless it was meant to mean "always not less than". :-) Still confusing. [1] https://pypi.org/project/openstackclient/ -yoctozepto From jim at jimrollenhagen.com Tue Mar 10 20:59:48 2020 From: jim at jimrollenhagen.com (Jim Rollenhagen) Date: Tue, 10 Mar 2020 16:59:48 -0400 Subject: Not running for TC next election Message-ID: Hi all, I won't be running for TC next election. As you probably noticed, I don't really have enough time these days to meaningfully contribute, so leaving it open for someone new. It's been fun and a great learning experience, so I highly encourage others in the community to run! I'll still be around to heckle in the background, don't worry. :) // jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Tue Mar 10 21:19:00 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 10 Mar 2020 14:19:00 -0700 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: Thanks for sharing Mark! I think there is a lot of good information in there. How many people were joining approximately? How did you coordinate the when you would do it? Would you mind adding some of that to the etherpad[1] I am collecting info into? -Kendall (diablo_rojo) [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices On Tue, Mar 10, 2020 at 2:47 AM Mark Goddard wrote: > On Tue, 10 Mar 2020 at 09:17, Thierry Carrez > wrote: > > > > Kendall Nelson wrote: > > > I wanted to collect best practices and pitfalls to avoid wrt projects > > > experiences with virtual midcycles. I know of a few projects that have > > > done them in the past and with how travel is hard for a lot of people > > > right now, I expect more projects to have midcycles. I think it would > be > > > helpful to have all of the data we can collect in one place for those > > > not just new to virtual midcycles but the whole community. > > > [...] > > > > Also interested in feedback from teams that had virtual PTGs in the past > > (keeping all possibilities on the table). I think Kolla, Telemetry and a > > few others did that. > > Kolla has now had two virtual PTGs. Overall I think they went fairly > well, particularly the most recent one. We tried Zoom, then moved to > Google Meet. I forget the problems with Zoom. There were inevitably a > few teething problems with the video, but I think we worked it out > after 15-20 minutes. Etherpad for Ussuri vPTG here: > https://etherpad.openstack.org/p/kolla-ussuri-ptg. > > Without seeing people's faces it can be hard to ensure everyone keeps > focussed. It's quite rare for the whole room to be focussed at > physical discussions though. > > Going around the room giving short intros helps to get people talking, > and it may be better to do these ~1 hour in as people may miss the > start. Directing questions at non-cores can help overcome that pesky > imposter syndrome. Keeping video on definitely helps with engagement, > up to the point where it impacts audio quality. > > There was also the Denver PTG where the PTL and a number of cores were > remote where we struggled to make any progress. I think there were a > few reasons for this. The fixed time of the PTG was not optimal for > many remote attendees living in Europe or Asia. When there are a > number of participants in one location, it can be easy to forget to > direct speech at the microphone, allow time for remote callers to ask > questions/respond etc. This makes it difficult and frustrating for > them to join in, making it easier to get distracted and drop off. > > Not too much hard data in there, but hopefully a feel for how it went for > us. > > > > > -- > > Thierry Carrez (ttx) > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emilien at redhat.com Tue Mar 10 23:11:25 2020 From: emilien at redhat.com (Emilien Macchi) Date: Tue, 10 Mar 2020 19:11:25 -0400 Subject: [tripleo] Missing tag in cron container image - no recheck please Message-ID: Hi folks, We seem to have an issue with container images, where one (at least) has a missing tag: https://bugs.launchpad.net/tripleo/+bug/1866927 It is causing most of our jobs to go red and fail on: tripleo_common.image.exception.ImageNotFoundException: Not found image: docker:// docker.io/tripleomaster/centos-binary-cron:3621159be13b41f8ead2e873b357f4a5 Please refrain from approving or rechecking patches until we have sorted this out. Thanks and stay tuned. -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From emilien at redhat.com Tue Mar 10 23:38:37 2020 From: emilien at redhat.com (Emilien Macchi) Date: Tue, 10 Mar 2020 19:38:37 -0400 Subject: [tripleo] In-Reply-To: <1B1A2143-A018-408D-9515-A367CEA952B5@inaugust.com> References: <1B1A2143-A018-408D-9515-A367CEA952B5@inaugust.com> Message-ID: On Tue, Mar 10, 2020 at 10:41 AM Monty Taylor wrote: > Yay! > > When you have brainspace after firefighting (always fun) - maybe we should > find a time to talk about whether our image building and publishing > automation could help you out here. No rush - this is one of those “we’ve > got some tools we might be able to leverage to help” - just ping me > whenever. > Hey Monty, The CI team is presently busy with CentOS 8 fires but I would be happy to help and work together on convergence. Maybe I can start by explaining how our process works, then you can do the same and we see where we can collaborate. The TL;DR is that we have built TripleO CLI and Ansible roles to consume Kolla tooling and build our images. 1) How a TripleO user would build an image? By using the "openstack overcloud container image build" command ("overcloud" is here by legacy, please ignore it). The magic happens here: https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleoclient/v1/container_image.py#L104-L252 It's basically wrapping out the kolla-build CLI; with proper options for us. In fact, since podman/buildah, we only use kolla-build to render Kolla Dockerfiles templates to merge them with our TripleO overrides: https://opendev.org/openstack/tripleo-common/src/branch/master/container-images/tripleo_kolla_template_overrides.j2 kolla-build would generate directories for each image and inside you would have their Dockerfiles. We don't use kolla-build to build the containers because Kolla doesn't support Buildah, and none of us has taken the time to do it yet. To build the images from Dockerfiles, we use that code: https://opendev.org/openstack/tripleo-common/src/branch/master/tripleo_common/image/builder/buildah.py It's basically running "buildah bud" with concurrency (to make it faster). This code could be converted to an Ansible module eventually; which could be consumed by more than us. Once images are built, the code runs "buildah push"; to push it to a remote (or local) registry. That's it, that's all. If we resume, we use kolla-build to generate Dockerfiles for our containers (since TripleO images use Kolla format) and then we have our own crap to use Buildah to build & push the image. I guess the second part is something we could share. 2) How TripleO CI builds containers? We have an Ansible role for that: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers It basically: - Install repositories needed to deploy TripleO - Deploy a local docker registry with ansible-role-container-registry (also used in production when Docker is deployed, so before Stein) - Install and configure Kolla - Runs "openstack overcloud container image build" (which was described earlier) to build, tag and push images I skipped a few details but this is the big picture. I'm sure there is a lot where we can share and I would be more than happy to contribute in that effort, please let me know how it works on your side and we'll find ways to collaborate. Thanks, -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ekcs.openstack at gmail.com Wed Mar 11 02:17:36 2020 From: ekcs.openstack at gmail.com (Eric Kao) Date: Tue, 10 Mar 2020 19:17:36 -0700 Subject: [congress][ptl] new PTL for V-cycle Message-ID: Hello all, Due to a change in my role, I do not intend to seek another term as Congress project PTL. I will continue through the end of the U-cycle. After that, I look forward to new voices and new perspectives to help us take the next step in the Congress journey. Cheers, Eric From mark.kirkwood at catalyst.net.nz Wed Mar 11 02:27:58 2020 From: mark.kirkwood at catalyst.net.nz (Mark Kirkwood) Date: Wed, 11 Mar 2020 15:27:58 +1300 Subject: [swift] Rolling upgrade, any version relationships? Message-ID: <5f55bc36-b51c-5c6d-ad0f-63a32fcba2d4@catalyst.net.nz> Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are: - If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)? - If not is there a preferred version to jump to first? - Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time? regards Mark From sundar.nadathur at intel.com Wed Mar 11 06:02:08 2020 From: sundar.nadathur at intel.com (Nadathur, Sundar) Date: Wed, 11 Mar 2020 06:02:08 +0000 Subject: [cyborg] Updating the list of core reviewers Message-ID: Hello all, Brin Zhang has been actively contributing to Cyborg in various areas, adding new features, improving quality, reviewing patches, and generally helping others in the community. Despite the relatively short time, he has been one of the most prolific contributors, and brings an enthusiastic and active mindset. I would like to thank him and acknowledge his significant contributions by proposing him as a core reviewer for Cyborg. Shogo Saito has been active in Cyborg since Train release. He has been driving the Cyborg client improvements, including its revamp to use OpenStackSDK. Previously he was instrumental in the transition to Python 3, testing and fixing issues in the process. As he has access to real FPGA hardware, he brings a users' perspective and also tests Cyborg with real hardware. I would like to thank and acknowledge him for his steady valuable contributions, and propose him as a core reviewer for Cyborg. Some of the currently listed core reviewers have not been participating for a lengthy period of time. It is proposed that those who have had no contributions for the past 18 months - i.e. no participation in meetings, no code contributions and no reviews - be removed from the list of core reviewers. If no objections are made known by March 20, I will make the changes proposed above. Thanks. Regards, Sundar -------------- next part -------------- An HTML attachment was scrubbed... URL: From gouthampravi at gmail.com Wed Mar 11 06:09:37 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Tue, 10 Mar 2020 23:09:37 -0700 Subject: [manila] share group replication spike/questions In-Reply-To: <3b103cb9a3894762a8664815fff5771c@KULX13MDC124.APAC.DELL.COM> References: <55d84e2e29cb4758aaff0b8c07aaa0bd@KULX13MDC124.APAC.DELL.COM> <3b103cb9a3894762a8664815fff5771c@KULX13MDC124.APAC.DELL.COM> Message-ID: On Mon, Mar 9, 2020 at 6:55 PM wrote: > Hi, Gotham, > > > > After checked the manila DB, I noticed there is table called > ‘share_instances’ which was added for share replication and snapshot. > > Now, for group replication, do you think we also need a new table like > ‘share_group_instances’ ? > Agree, I think that's a sane approach to capture source and destination replicas adequately. Could you please discuss this through your specification? > > > Thanks, > > Ding Dong > > > > *From:* Goutham Pacha Ravi > *Sent:* Saturday, February 29, 2020 7:43 AM > *To:* Ding, Dong > *Cc:* OpenStack Discuss > *Subject:* Re: [manila] share group replication spike/questions > > > > [EXTERNAL EMAIL] > > > > > > On Fri, Feb 28, 2020 at 12:21 AM wrote: > > Thanks Gotham, > > > > We are talking about this feature *after U release*. > > Cannot get it done in recently. > > Just do some prepare first. > > > > Great, thanks for confirming. We'll hash out the design on the > specification, and if necessary, we can work through it during the Open > Infra Project Technical Gathering in June [8][9] > > > > [8] https://www.openstack.org/ptg/ > > [9] https://etherpad.openstack.org/p/vancouver-ptg-manila-planning > > > > > > BR, > > Ding Dong > > > > *From:* Goutham Pacha Ravi > *Sent:* Friday, February 28, 2020 7:10 AM > *To:* Ding, Dong > *Cc:* OpenStack Discuss > *Subject:* Re: [manila] share group replication spike/questions > > > > [EXTERNAL EMAIL] > > > > > > > > > > On Tue, Feb 25, 2020 at 12:53 AM wrote: > > Hi, guys, > > As we talked about the topic in a virtual PTG few months ago. > > https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual (*Support > promoting several shares in group (DELL EMC: dingdong) * > > > > I’m trying to write a manila-spec for it. > > > > Hi, thank you for working on this, and for submitting a specification [0]. > We're targeting this for the Victoria release, correct? I like working on > these major changes as soon as possible giving us enough air time for > testing and hardening. > > > > It’s my first experience to implement such feature in framework. > > I need to double check with you something, and hope you can give me some > guides like: > > 1. Where is the extra-spec defined for group/group type, > it’s in Manila repo, right? (like manila.db.sqlalchemy.models….) > > Group type extra specs are added as storage capabilities first, you begin > by modifying the driver interface to report this group type capability. > When share drivers report their support for group replication, operators > can use the corresponding string in their group type extra-specs to > schedule appropriately. I suggest taking a look at an existing share group > type capability called "consistent_snapshot_support". [1] and [2] are > reviews that added it. > > 2. The command cli should be implemented for > ‘python-manilaclinet’ repo, right? (I have never touched this repo before) > > Yes. python-manilaclient encompasses > > - a python SDK to version 2 of the manila API > > - two shell implementations: manila and openstack client (actively being > developed) > > > > Group type extra-specs are passed transparently through the SDK and CLI, > you may probably add some documentation or shell hint text (like [3] if > needed). > > > > > > 3. Where is the rest-api should be implemented? > > The rest API is in the openstack/manila repository. [4][5] contain some > documentation regarding how to change the manila API. > > > > 4. And more tips you have? like any other related project > should be changed? > > For any new feature, we need these additional things besides working code: > > - A first party driver implementation where possible so we can test this > feature in the upstream CI (if no first party driver can support this > feature, you'll need to make the best approximation of this feature through > the Dummy/Fake driver [6]) > > - The feature must be tested with adequate test cases in > manila-tempest-plugin > > - Documentation must be added to the manila documentation [7] > > Just list what I know, and more details questions will be raised when > implementing, I think. > > FYI > > Thanks, > > Ding Dong > > > > Happy to answer any more questions, here or on your specification [0] > > > > Thanks, > > Goutham > > > > [0] https://review.opendev.org/#/c/710166/ > > [1] https://review.opendev.org/#/c/446044/ > > [2] https://review.opendev.org/#/c/447474/ > > [3] > https://opendev.org/openstack/python-manilaclient/src/commit/ac5ca461e8c8dd11fe737de7b90ab5c33366ab35/manilaclient/v2/shell.py#L4543 > > [4] > https://docs.openstack.org/manila/latest/contributor/addmethod.openstackapi.html > > [5] > https://docs.openstack.org/manila/latest/contributor/api_microversion_dev.html > > [6] > https://opendev.org/openstack/manila/src/commit/68a18f49472ac7686ceab15e9788dcef05764822/manila/tests/share/drivers/dummy.py > > [7] > https://docs.openstack.org/manila/latest/contributor/documenting_your_work.html > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sundar.nadathur at intel.com Wed Mar 11 06:22:23 2020 From: sundar.nadathur at intel.com (Nadathur, Sundar) Date: Wed, 11 Mar 2020 06:22:23 +0000 Subject: [nova][ptl] Temporary Nova PTL until election In-Reply-To: <1583482276.12170.14@est.tech> References: <1583482276.12170.14@est.tech> Message-ID: > -----Original Message----- > From: Balázs Gibizer > Sent: Friday, March 6, 2020 12:11 AM > To: OpenStack Discuss > Subject: [nova][ptl] Temporary Nova PTL until election > > Hi, > > Since Eric announced that he has to leave us [1] I have been working > internally with my employee to be able to take over the Nova PTL position. > Now I've got the necessary approvals. The official PTL election is close [2] and > I'm ready to fill the PTL gap until the proper PTL election in April. > > Is this a workable solution for the community? > > Cheers, > gibi Definitely a +1. Thanks a lot, gibi. Regards, Sundar From mike.carden at gmail.com Wed Mar 11 07:06:53 2020 From: mike.carden at gmail.com (Mike Carden) Date: Wed, 11 Mar 2020 18:06:53 +1100 Subject: [all] Guides for newbies to OpenStack Message-ID: Our small team at ${DAYJOB} has built a handful of OpenStack clusters based on Red Hat OpenStack 13 (aka Queens) over the last couple of years. We now find ourselves in the position of being 'gifted' human resources in the shape of mid-level 'IT people' who are sent to join our team for a short time to 'Learn OpenStack'. These tend to be people for whom, "Here's a Horizon URL and some creds - go log in and launch a VM"[1]... is a bit much. I've done a wee bit of web searching (enough to find the dead links) trying to find some newbie friendly tutorials on OpenStack basics. Before I attempt to re-invent the wheel, can anyone suggest some public resources I might point people to? Deity help us if we have to explain Tripleo's Undercloud, Overcloud, partially containered, partially pacemakered, fully flaky... underpinnings. Thanks, MC [1] Even with a step by step guide -------------- next part -------------- An HTML attachment was scrubbed... URL: From licanwei_cn at 163.com Wed Mar 11 07:31:50 2020 From: licanwei_cn at 163.com (licanwei) Date: Wed, 11 Mar 2020 15:31:50 +0800 (GMT+08:00) Subject: [Watcher]no topics and cancelling the irc meeting today Message-ID: <66e7638d.951c.170c881bba6.Coremail.licanwei_cn@163.com> | | licanwei_cn | | 邮箱：licanwei_cn at 163.com | 签名由 网易邮箱大师 定制 -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcin.juszkiewicz at linaro.org Wed Mar 11 08:05:48 2020 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Wed, 11 Mar 2020 09:05:48 +0100 Subject: [horizon][kolla] pyscss failure on newest setuptools Message-ID: <6295fd26-d983-75c5-4ead-a36823034d2c@linaro.org> One of Horizon's requirements is pyscss package. Which had last release over 4 years ago... Two days ago setuptools v46 got released. One of changes was drop of Features feature. Today Kolla builds started to fail: INFO:kolla.common.utils.horizon:Collecting pyScss===1.3.4 INFO:kolla.common.utils.horizon: Downloading http://mirror.ord.rax.opendev.org:8080/pypifiles/packages/1d/4a/221ae7561c8f51c4f28b2b172366ccd0820b14bb947350df82428dfce381/pyScss-1.3.4.tar.gz (120 kB) INFO:kolla.common.utils.horizon: ERROR: Command errored out with exit status 1: INFO:kolla.common.utils.horizon: command: /var/lib/kolla/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rr0db3qs/pyScss/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rr0db3qs/pyScss/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-rr0db3qs/pyScss/pip-egg-info INFO:kolla.common.utils.horizon: cwd: /tmp/pip-install-rr0db3qs/pyScss/ INFO:kolla.common.utils.horizon: Complete output (5 lines): INFO:kolla.common.utils.horizon: Traceback (most recent call last): INFO:kolla.common.utils.horizon: File "", line 1, in INFO:kolla.common.utils.horizon: File "/tmp/pip-install-rr0db3qs/pyScss/setup.py", line 9, in INFO:kolla.common.utils.horizon: from setuptools import setup, Extension, Feature INFO:kolla.common.utils.horizon: ImportError: cannot import name 'Feature' Are there any plans to fix it? pyscss project got issue: https://github.com/Kronuz/pyScss/issues/385 In Kolla I made ugly workaround: https://paste.centos.org/view/2e29d284 What are plans of Horizon team? From dtantsur at redhat.com Wed Mar 11 09:17:09 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Wed, 11 Mar 2020 10:17:09 +0100 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: Hi, Ironic did, I think, 3-4 virtual midcycles, and I think they were quite successful. The most positive outcome was to reach out to folks who usually cannot travel. They really appreciated that. We used SIP the first couple of times, but it caused problems for a big share of participants. Also lack of moderation was problematic. We switched to Bluejeans later, and that was, IMO, a big improvement: 1) A participant list with information who is speaking. 2) An option for a moderator to mute a person or everyone. 3) Screen sharing. Dmitry On Tue, Mar 10, 2020 at 1:29 AM Kendall Nelson wrote: > Hello Everyone! > > I wanted to collect best practices and pitfalls to avoid wrt projects > experiences with virtual midcycles. I know of a few projects that have done > them in the past and with how travel is hard for a lot of people right now, > I expect more projects to have midcycles. I think it would be helpful to > have all of the data we can collect in one place for those not just new to > virtual midcycles but the whole community. > > I threw some categories into this etherpad[1] and filled in some options. > Please add to it :) > > -Kendall (diablo_rojo) > > [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Wed Mar 11 09:42:46 2020 From: mark at stackhpc.com (Mark Goddard) Date: Wed, 11 Mar 2020 09:42:46 +0000 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: On Tue, 10 Mar 2020 at 21:19, Kendall Nelson wrote: > > Thanks for sharing Mark! I think there is a lot of good information in there. > > How many people were joining approximately? How did you coordinate the when you would do it? > > Would you mind adding some of that to the etherpad[1] I am collecting info into? No problem, added some things to the etherpad. > > -Kendall (diablo_rojo) > > [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices > > On Tue, Mar 10, 2020 at 2:47 AM Mark Goddard wrote: >> >> On Tue, 10 Mar 2020 at 09:17, Thierry Carrez wrote: >> > >> > Kendall Nelson wrote: >> > > I wanted to collect best practices and pitfalls to avoid wrt projects >> > > experiences with virtual midcycles. I know of a few projects that have >> > > done them in the past and with how travel is hard for a lot of people >> > > right now, I expect more projects to have midcycles. I think it would be >> > > helpful to have all of the data we can collect in one place for those >> > > not just new to virtual midcycles but the whole community. >> > > [...] >> > >> > Also interested in feedback from teams that had virtual PTGs in the past >> > (keeping all possibilities on the table). I think Kolla, Telemetry and a >> > few others did that. >> >> Kolla has now had two virtual PTGs. Overall I think they went fairly >> well, particularly the most recent one. We tried Zoom, then moved to >> Google Meet. I forget the problems with Zoom. There were inevitably a >> few teething problems with the video, but I think we worked it out >> after 15-20 minutes. Etherpad for Ussuri vPTG here: >> https://etherpad.openstack.org/p/kolla-ussuri-ptg. >> >> Without seeing people's faces it can be hard to ensure everyone keeps >> focussed. It's quite rare for the whole room to be focussed at >> physical discussions though. >> >> Going around the room giving short intros helps to get people talking, >> and it may be better to do these ~1 hour in as people may miss the >> start. Directing questions at non-cores can help overcome that pesky >> imposter syndrome. Keeping video on definitely helps with engagement, >> up to the point where it impacts audio quality. >> >> There was also the Denver PTG where the PTL and a number of cores were >> remote where we struggled to make any progress. I think there were a >> few reasons for this. The fixed time of the PTG was not optimal for >> many remote attendees living in Europe or Asia. When there are a >> number of participants in one location, it can be easy to forget to >> direct speech at the microphone, allow time for remote callers to ask >> questions/respond etc. This makes it difficult and frustrating for >> them to join in, making it easier to get distracted and drop off. >> >> Not too much hard data in there, but hopefully a feel for how it went for us. >> >> > >> > -- >> > Thierry Carrez (ttx) >> > >> From thierry at openstack.org Wed Mar 11 11:37:35 2020 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 11 Mar 2020 12:37:35 +0100 Subject: [largescale-sig] Meeting summary and next actions Message-ID: <84525104-d146-1741-a32c-f4580e585b33@openstack.org> Hi everyone, The Large Scale SIG held a meeting today. You can catch up with the summary and logs of the meeting at: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-03-11-09.00.html No progress on "Documenting large scale operations" this week. masahito just posted a new revision for the oslo.metrics spec: https://review.opendev.org/#/c/704733/ belmoreira asks for input/comment on the proposed "Large Scale operations" Opendev track content: https://etherpad.openstack.org/p/LargeScaleOps_OpenDev Standing TODOs: - amorin to create a wiki page for large scale documentation - amorin to propose patch against Nova doc - all to check/comment on https://etherpad.openstack.org/p/LargeScaleOps_OpenDev - all to review new patchset of oslo.metrics spec https://review.opendev.org/#/c/704733/ - oneswig to contribute a scaling story on bare metal cluster scaling The next meeting will happen on March 25, at 9:00 UTC on #openstack-meeting. Cheers, -- Thierry Carrez (ttx) From kklimonda at syntaxhighlighted.com Wed Mar 11 13:29:58 2020 From: kklimonda at syntaxhighlighted.com (Krzysztof Klimonda) Date: Wed, 11 Mar 2020 14:29:58 +0100 Subject: [neutron][largescale-sig] Debugging and tracking missing flows with l2pop Message-ID: <6A0F6E0F-9D6E-4ED2-B4AC-F862885220B4@syntaxhighlighted.com> Hi, (This is stein deployment with 14.0.2 neutron release) I’ve just spent some time debugging a missing connection between two VMs running on OS stein with ovs+l2pop enabled and the direct cause was missing flows in table 20 and a very incomplete flood flow in table 22. Restarting neutron-openvswitch-agent on that host has fixed the issue. Last time we’ve encountered missing flood flows (in another pike-based deployment), we tracked it down to https://review.opendev.org/#/c/600151/ and since then it was stable. My initial thought was that we were hitting the same bug - a couple of VMs are scheduled on the same compute, 3 ports are activated at the same time, and the flood entry is not broadcasted to other computes. However that issue was only affecting one of the computes, and it was the only one missing both MAC entries in table 20 and VXLAN tunnels in table 22. The only other idea I have is that the compute with missing flows have not received them from rabbitmq, but there I see nothing in logs that would suggest that agent was disconnected from rabbitmq. So at this point I have three questions: - what would be a good place to look next to track down those missing flows - for other operators, how stable do you find l2pop in general? and if you have problems with missing flows in your environment, do you try to monitor your deployment for that? -Chris From sundar.nadathur at intel.com Wed Mar 11 14:08:49 2020 From: sundar.nadathur at intel.com (Nadathur, Sundar) Date: Wed, 11 Mar 2020 14:08:49 +0000 Subject: [cyborg] Proposing core reviewers Message-ID: Hello all, Brin Zhang has been actively contributing to Cyborg in various areas, adding new features, improving quality, reviewing patches, and generally helping others in the community. Despite the relatively short time, he has been one of the most prolific contributors, and brings an enthusiastic and active mindset. I would like to thank him and acknowledge his significant contributions by proposing him as a core reviewer for Cyborg. Shogo Saito has been active in Cyborg since Train release. He has been driving the Cyborg client improvements, including its revamp to use OpenStackSDK. Previously he was instrumental in the transition to Python 3, testing and fixing issues in the process. As he has access to real FPGA hardware, he brings a users’ perspective and also tests Cyborg with real hardware. I would like to thank and acknowledge him for his steady valuable contributions, and propose him as a core reviewer for Cyborg. Some of the currently listed core reviewers have not been participating for a lengthy period of time. It is proposed that those who have had no contributions for the past 18 months – i.e. no participation in meetings, no code contributions and no reviews – be removed from the list of core reviewers. If no objections are made known by March 20, I will make the changes proposed above. Thanks. Regards, Sundar From Dong.Ding at dell.com Wed Mar 11 06:17:57 2020 From: Dong.Ding at dell.com (Dong.Ding at dell.com) Date: Wed, 11 Mar 2020 06:17:57 +0000 Subject: [manila] share group replication spike/questions In-Reply-To: References: <55d84e2e29cb4758aaff0b8c07aaa0bd@KULX13MDC124.APAC.DELL.COM> <3b103cb9a3894762a8664815fff5771c@KULX13MDC124.APAC.DELL.COM> Message-ID: <5ca39398c52f4c0ab7e77616eabc76fe@KULX13MDC124.APAC.DELL.COM> Sure. I can list it in manila-spec. I’m wondering if we need the share-group-instances-xxx APIs also at the same time？ Like share-instance-xxx: [cid:image001.jpg at 01D5F7AF.A32AB440] Thanks, Ding Dong From: Goutham Pacha Ravi Sent: Wednesday, March 11, 2020 2:10 PM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Mon, Mar 9, 2020 at 6:55 PM > wrote: Hi, Gotham, After checked the manila DB, I noticed there is table called ‘share_instances’ which was added for share replication and snapshot. Now, for group replication, do you think we also need a new table like ‘share_group_instances’ ? Agree, I think that's a sane approach to capture source and destination replicas adequately. Could you please discuss this through your specification? Thanks, Ding Dong From: Goutham Pacha Ravi > Sent: Saturday, February 29, 2020 7:43 AM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Fri, Feb 28, 2020 at 12:21 AM > wrote: Thanks Gotham, We are talking about this feature after U release. Cannot get it done in recently. Just do some prepare first. Great, thanks for confirming. We'll hash out the design on the specification, and if necessary, we can work through it during the Open Infra Project Technical Gathering in June [8][9] [8] https://www.openstack.org/ptg/ [9] https://etherpad.openstack.org/p/vancouver-ptg-manila-planning BR, Ding Dong From: Goutham Pacha Ravi > Sent: Friday, February 28, 2020 7:10 AM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Tue, Feb 25, 2020 at 12:53 AM > wrote: Hi, guys, As we talked about the topic in a virtual PTG few months ago. https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual (Support promoting several shares in group (DELL EMC: dingdong) I’m trying to write a manila-spec for it. Hi, thank you for working on this, and for submitting a specification [0]. We're targeting this for the Victoria release, correct? I like working on these major changes as soon as possible giving us enough air time for testing and hardening. It’s my first experience to implement such feature in framework. I need to double check with you something, and hope you can give me some guides like: 1. Where is the extra-spec defined for group/group type, it’s in Manila repo, right? (like manila.db.sqlalchemy.models….) Group type extra specs are added as storage capabilities first, you begin by modifying the driver interface to report this group type capability. When share drivers report their support for group replication, operators can use the corresponding string in their group type extra-specs to schedule appropriately. I suggest taking a look at an existing share group type capability called "consistent_snapshot_support". [1] and [2] are reviews that added it. 2. The command cli should be implemented for ‘python-manilaclinet’ repo, right? (I have never touched this repo before) Yes. python-manilaclient encompasses - a python SDK to version 2 of the manila API - two shell implementations: manila and openstack client (actively being developed) Group type extra-specs are passed transparently through the SDK and CLI, you may probably add some documentation or shell hint text (like [3] if needed). 3. Where is the rest-api should be implemented? The rest API is in the openstack/manila repository. [4][5] contain some documentation regarding how to change the manila API. 4. And more tips you have? like any other related project should be changed? For any new feature, we need these additional things besides working code: - A first party driver implementation where possible so we can test this feature in the upstream CI (if no first party driver can support this feature, you'll need to make the best approximation of this feature through the Dummy/Fake driver [6]) - The feature must be tested with adequate test cases in manila-tempest-plugin - Documentation must be added to the manila documentation [7] Just list what I know, and more details questions will be raised when implementing, I think. FYI Thanks, Ding Dong Happy to answer any more questions, here or on your specification [0] Thanks, Goutham [0] https://review.opendev.org/#/c/710166/ [1] https://review.opendev.org/#/c/446044/ [2] https://review.opendev.org/#/c/447474/ [3] https://opendev.org/openstack/python-manilaclient/src/commit/ac5ca461e8c8dd11fe737de7b90ab5c33366ab35/manilaclient/v2/shell.py#L4543 [4] https://docs.openstack.org/manila/latest/contributor/addmethod.openstackapi.html [5] https://docs.openstack.org/manila/latest/contributor/api_microversion_dev.html [6] https://opendev.org/openstack/manila/src/commit/68a18f49472ac7686ceab15e9788dcef05764822/manila/tests/share/drivers/dummy.py [7] https://docs.openstack.org/manila/latest/contributor/documenting_your_work.html -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 24640 bytes Desc: image001.jpg URL: From Dong.Ding at dell.com Wed Mar 11 08:30:42 2020 From: Dong.Ding at dell.com (Dong.Ding at dell.com) Date: Wed, 11 Mar 2020 08:30:42 +0000 Subject: [manila] share group replication spike/questions In-Reply-To: <5ca39398c52f4c0ab7e77616eabc76fe@KULX13MDC124.APAC.DELL.COM> References: <55d84e2e29cb4758aaff0b8c07aaa0bd@KULX13MDC124.APAC.DELL.COM> <3b103cb9a3894762a8664815fff5771c@KULX13MDC124.APAC.DELL.COM> <5ca39398c52f4c0ab7e77616eabc76fe@KULX13MDC124.APAC.DELL.COM> Message-ID: Sorry, send again to involve my colleague. * Sure. I can list the DB change in manila-spec. * Because I don’t use the instance API too much, I’m wondering if we need share-group-instances-xxx APIs if add such table？ Like share-instance-xxx: [cid:image002.jpg at 01D5F7C1.2E887020] Thanks, Ding Dong From: Goutham Pacha Ravi > Sent: Wednesday, March 11, 2020 3:40 PM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Mon, Mar 9, 2020 at 6:55 PM > wrote: Hi, Gotham, After checked the manila DB, I noticed there is table called ‘share_instances’ which was added for share replication and snapshot. Now, for group replication, do you think we also need a new table like ‘share_group_instances’ ? Agree, I think that's a sane approach to capture source and destination replicas adequately. Could you please discuss this through your specification? Thanks, Ding Dong From: Goutham Pacha Ravi > Sent: Saturday, February 29, 2020 7:43 AM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Fri, Feb 28, 2020 at 12:21 AM > wrote: Thanks Gotham, We are talking about this feature after U release. Cannot get it done in recently. Just do some prepare first. Great, thanks for confirming. We'll hash out the design on the specification, and if necessary, we can work through it during the Open Infra Project Technical Gathering in June [8][9] [8] https://www.openstack.org/ptg/ [9] https://etherpad.openstack.org/p/vancouver-ptg-manila-planning BR, Ding Dong From: Goutham Pacha Ravi > Sent: Friday, February 28, 2020 7:10 AM To: Ding, Dong Cc: OpenStack Discuss Subject: Re: [manila] share group replication spike/questions [EXTERNAL EMAIL] On Tue, Feb 25, 2020 at 12:53 AM > wrote: Hi, guys, As we talked about the topic in a virtual PTG few months ago. https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual (Support promoting several shares in group (DELL EMC: dingdong) I’m trying to write a manila-spec for it. Hi, thank you for working on this, and for submitting a specification [0]. We're targeting this for the Victoria release, correct? I like working on these major changes as soon as possible giving us enough air time for testing and hardening. It’s my first experience to implement such feature in framework. I need to double check with you something, and hope you can give me some guides like: 1. Where is the extra-spec defined for group/group type, it’s in Manila repo, right? (like manila.db.sqlalchemy.models….) Group type extra specs are added as storage capabilities first, you begin by modifying the driver interface to report this group type capability. When share drivers report their support for group replication, operators can use the corresponding string in their group type extra-specs to schedule appropriately. I suggest taking a look at an existing share group type capability called "consistent_snapshot_support". [1] and [2] are reviews that added it. 2. The command cli should be implemented for ‘python-manilaclinet’ repo, right? (I have never touched this repo before) Yes. python-manilaclient encompasses - a python SDK to version 2 of the manila API - two shell implementations: manila and openstack client (actively being developed) Group type extra-specs are passed transparently through the SDK and CLI, you may probably add some documentation or shell hint text (like [3] if needed). 3. Where is the rest-api should be implemented? The rest API is in the openstack/manila repository. [4][5] contain some documentation regarding how to change the manila API. 4. And more tips you have? like any other related project should be changed? For any new feature, we need these additional things besides working code: - A first party driver implementation where possible so we can test this feature in the upstream CI (if no first party driver can support this feature, you'll need to make the best approximation of this feature through the Dummy/Fake driver [6]) - The feature must be tested with adequate test cases in manila-tempest-plugin - Documentation must be added to the manila documentation [7] Just list what I know, and more details questions will be raised when implementing, I think. FYI Thanks, Ding Dong Happy to answer any more questions, here or on your specification [0] Thanks, Goutham [0] https://review.opendev.org/#/c/710166/ [1] https://review.opendev.org/#/c/446044/ [2] https://review.opendev.org/#/c/447474/ [3] https://opendev.org/openstack/python-manilaclient/src/commit/ac5ca461e8c8dd11fe737de7b90ab5c33366ab35/manilaclient/v2/shell.py#L4543 [4] https://docs.openstack.org/manila/latest/contributor/addmethod.openstackapi.html [5] https://docs.openstack.org/manila/latest/contributor/api_microversion_dev.html [6] https://opendev.org/openstack/manila/src/commit/68a18f49472ac7686ceab15e9788dcef05764822/manila/tests/share/drivers/dummy.py [7] https://docs.openstack.org/manila/latest/contributor/documenting_your_work.html -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 24613 bytes Desc: image002.jpg URL: From yangyi01 at inspur.com Wed Mar 11 14:33:50 2020 From: yangyi01 at inspur.com (=?utf-8?B?WWkgWWFuZyAo5p2o54eaKS3kupHmnI3liqHpm4blm6I=?=) Date: Wed, 11 Mar 2020 14:33:50 +0000 Subject: =?utf-8?B?562U5aSNOiBbbmV1dHJvbl0gV2h5IG5ldHdvcmsgcGVyZm9ybWFuY2UgaXMg?= =?utf-8?B?ZXh0cmVtZWx5IGJhZCBhbmQgbGluZWFybHkgcmVsYXRlZCB3aXRoIG51bWJl?= =?utf-8?Q?r_of_VMs=3F?= In-Reply-To: <52E415A7-AF3E-4798-803C-391123729345@gmail.com> References: <52E415A7-AF3E-4798-803C-391123729345@gmail.com> Message-ID: <40e10b0a56b140f2aab1a50a1018c9c9@inspur.com> Just let you guys know, this is indeed a common bug of openstack, https://bugs.launchpad.net/neutron/+bug/1732067 has more info, but there isn’t a real fix patch, current merged fix patch is just a workaround, it doesn’t fix MAC learn issue. 发件人: Satish Patel [mailto:satish.txt at gmail.com] 发送时间: 2020年2月24日 7:10 收件人: Donny Davis 抄送: Yi Yang (杨燚)-云服务集团 ; openstack-discuss at lists.openstack.org 主题: Re: [neutron] Why network performance is extremely bad and linearly related with number of VMs? What is max age time in Linux bridge? If it’s zero then it won’t learn Mac and flush arp table. Sent from my iPhone On Feb 23, 2020, at 12:52 AM, Donny Davis > wrote:  So I am curious as to what your question is. Are you asking about ovs bridges learning MAC's of other compute nodes or why network performance is affected when you run more than one instance per node. I have not observed this behaviour in my experience. Could you tell us more about the configuration of your deployment? I understand you are currently using linux bridges that are connected to openvswitch bridges? Why not just use ovs? OVS can handle security groups. On Fri, Feb 21, 2020 at 9:48 AM Yi Yang (杨燚)-云服务集团 > wrote: Hi, All Anybody has noticed network performance between VMs is extremely bad, it is basically linearly related with numbers of VMs in same compute node. In my case, if I launch one VM per compute node and run iperf3 tcp and udp, performance is good, it is about 4Gbps and 1.7Gbps, for 16 bytes small UDP packets, it can reach 180000 pps (packets per second), but if I launch two VMs per compute node (note: they are in the same subnet) and only run pps test case, that will be decrease to about 90000 pps, if I launch 3 VMs per compute node, that will be about 50000 pps, I tried to find out the root cause, other VMs in this subnet (they are in the same compute node as iperf3 client) can receive all the packets iperf3 client VM sent out although destination MAC isn’t broadcast MAC or multicast MAC, actually it is MAC of iperf3 server VM in another compute node, by further check, I did find qemu instances of these VMs have higher CPU utilization and corresponding vhost kernel threads also also higher CPU utilization, to be importantly, I did find ovs was broadcasting these packets because all the ovs bridges didn’t learn this destination MAC. I tried this in Queens and Rocky, the same issue is there. By the way, we’re using linux bridge for security group, so VM tap interface is attached into linux bridge which is connected to br-int by veth pair. Here is output of “ovs-appctl dpif/dump-flows br-int” after I launched many VMs: recirc_id(0),in_port(12),eth(src=fa:16:3e:49:26:51,dst=fa:16:3e:a7:0a:3a),et h_type(0x0800),ipv4(tos=0/0x3,frag=no), packets:11012944, bytes:726983412, used:0.000s, flags:SP., actions:push_vlan(vid=1,pcp=0),2,set(tunnel(tun_id=0x49,src=10.3.2.17,dst=10 .3.2.16,ttl=64,tp_dst=4789,flags(df|key))),pop_vlan,9,8,11,13,14,15,16,17,18 ,19 $ sudo ovs-appctl fdb/show br-floating | grep fa:16:3e:49:26:51 $ sudo ovs-appctl fdb/show br-tun | grep fa:16:3e:49:26:51 $ sudo ovs-appctl fdb/show br-bond1 | grep fa:16:3e:49:26:51 $ sudo ovs-appctl fdb/show br-int | grep fa:16:3e:49:26:51 All the bridges can’t learn this MAC. My question is why ovs bridges can’t learn MACs of other compute nodes, is this common issue of all the Openstack versions? Is there any known existing way to fix it? Look forward to hearing your insights and solutions, thank you in advance and have a good day. -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3600 bytes Desc: not available URL: From smooney at redhat.com Wed Mar 11 14:37:35 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 11 Mar 2020 14:37:35 +0000 Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 In-Reply-To: References: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> Message-ID: <21add71a504fa8f9a0d32b2181f2660076a6d356.camel@redhat.com> On Tue, 2020-03-10 at 17:04 +0000, Arkady.Kanevsky at dell.com wrote: > Thank Lee. Sound approach. > A few questions/comments. > 1. Assume that we have unwritten assumption that all nova nodes have access to volumes on the backend. > So we rely on it except for ephemeral storage. well the job deploys the storage backend so its not an asusmtion we deploy it that way intentionally. we also set up ssh keys so we can rsync the qcow files between host when we do block migration. > 2. What need to be done for volumes that use FC not iSCSI? we dont test FC in the migration job currently so i think that is out of scope of this refactor. the goal is to move it to zuulv3 while testing all the existing cases not to add more cases in this phase. > 3. You have one for Ceph. Does that mean that we need an analog for other cinder back ends? no. the ceph backend is tested seperatly as there are basicaly 3 storage backend to the libvirt driver. local file which is tested as part of block migration with qcow2 local block device which is tested via cinder/lvm with the block device mounted on the host vi isci (FC would be the same form a qemu point of view) and finally ceph is used to test the qemu nataive network block device support. so we are not trying to test different cinder backends but rahter the different image backends/qemu storage types supprot in nova > 4. Do we need to anything analogous for Manila? maybe but again that seams like its out of scope so intally i would say no > 5. How do we address multi-attach volumes and multipathing? Expect that if we have multipthaing on origin node we laso > have multipathing at destination at the end. multi attach is already tested in the job i belive so we would continue that. i think both cinder lvm and ceph support multi attach. i dont think we test multipath in the gate in the current jobs so i would not imediatly assume we woudl add it as part of this refactor. > > > Thanks, > Arkady > > > -----Original Message----- > From: Lee Yarwood > Sent: Tuesday, March 10, 2020 10:22 AM > To: openstack-discuss at lists.openstack.org > Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 > > Hello all, > > I've started PoC'ing some ideas around $subject in the topic below and wanted to ask the wider team for feedback on > the approach I'm taking: > > https://review.opendev.org/#/q/topic:nova-live-migration-zuulv3 > > My initial idea is to break the job up into the following smaller multinode jobs that are hopefully easier to > understand and maintain. > > * nova-multinode-live-migration-py3 > > A simple LM job using the qcow2 imagebackend and LVM/iSCSI c-vol. > > * nova-multinode-live-migration-ceph-py3 this would be replaceing our existing devstack-plugin-ceph-tempest-py3 job runing all the same test but in a multinode config with live migration tests enabled in the tempest config. > > A ceph based LM job using rbd for both imagebackend and c-vol. > > * nova-multinode-evacuate-py3 so this would be the only new job although i am not sure it should be seperated out. we likely want to test evacuate with file,block and network storage so i think it makes sense to do this as a post playbook in the other two jobs. > > A separate evacuation job using qcow2 imagebackend and LVM/iSCSI c-vol. > The existing script *could* then be ported to an Ansible role as part of the migration to Zuulv3. > > Hopefully this is pretty straight forward but I'd appreciate any feedback on this all the same. > > Cheers, > > -- > Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 From gmann at ghanshyammann.com Wed Mar 11 14:38:57 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 11 Mar 2020 09:38:57 -0500 Subject: Not running for TC next election In-Reply-To: References: Message-ID: <170ca08c61f.f20f108159019.296868179124784200@ghanshyammann.com> Thanks, Jim for your contribution as TC and hope to see you back :) -gmann ---- On Tue, 10 Mar 2020 15:59:48 -0500 Jim Rollenhagen wrote ---- > Hi all, > I won't be running for TC next election. As you probably noticed, I don't really have enough time these days to meaningfully contribute, so leaving it open for someone new. It's been fun and a great learning experience, so I highly encourage others in the community to run! > I'll still be around to heckle in the background, don't worry. :) > > // jim From thierry at openstack.org Wed Mar 11 15:15:32 2020 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 11 Mar 2020 16:15:32 +0100 Subject: [all] A call for consolidation and simplification Message-ID: Hi all, I'd like to issue a call for consolidation and simplification for OpenStack development. In the early years of the project, we faced a lot of challenges. We had to spread the development load across manageable-size groups, so we encouraged the creation of a lot of project teams. We wanted to capture all the energy that was sent towards the project, so we passed project structure reforms (like the big tent) that would aggressively include new community groups in the "official" OpenStack community. We needed to remove bottlenecks, so we encouraged decentralized decision making. And we had to answer unique challenges, so we created software to match them (Zuul). In summary, we had a lot of people, and not enough systems to organize them, so we created those. Fast-forward to 2020, and our challenges are different. The many systems that we created in the early days have created silos, with very small groups of people working in isolation, making cross-project work more difficult than it should be. The many systems that we created generate a lot of fragmentation. Like we have too many meetings (76, in case you were wondering), too much energy spent running them, too much frustration when nobody joins. Finally, the many systems that we created represent a lot of complexity for newcomers to handle. We have 180 IRC channels, most of them ghost towns where by the time someone answers, the person asking the question is long gone. So I think it's time to generally think about simplifying and consolidating things. It's not as easy as it sounds. Our successful decentralization efforts make it difficult to make the centralized decision to regroup. It's hard to justify time and energy spent to /remove/ things, especially those that we spent time creating in the first place. But we now have too many systems and not enough people, so we need to consolidate and simplify. Back around Havana, when we had around the same number of active contributors as today, we used to have 36 meetings and 20 teams. Do we really need 180 IRC channels, 76 meetings, 63 project teams (not even counting SIGs)? Yes, we all specialized over time, so it's hard to merge for example Oslo + Requirements, or QA + Infrastructure, or Stable + Release Management, or Monasca + Telemetry. We are all overextended so it's hard to learn new tricks or codebases. And yet, while I'm not really sure what the best approach is, I think it's necessary. Comments, thoughts? -- Thierry Carrez (ttx) From zhipengh512 at gmail.com Wed Mar 11 16:17:59 2020 From: zhipengh512 at gmail.com (Zhipeng Huang) Date: Thu, 12 Mar 2020 00:17:59 +0800 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: Message-ID: Big +1 for Brin and shogo's nomination and well deserved :) I'm a little bit concerned over the 18 months period. The original rule we setup is volunteer step down, since this is a small team we want to acknowledge everyone that has made significant contributions. Some of the inactive core reviewers like Justin Kilpatrick have moved on a long time ago, and I don't see people like him could do any harm to the project. But if the core reviewer has a size limit in the system, that would be reasonable to replace the inactive ones with the new recruits :) Just my two cents On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar wrote: > Hello all, > Brin Zhang has been actively contributing to Cyborg in various areas, > adding new features, improving quality, reviewing patches, and generally > helping others in the community. Despite the relatively short time, he has > been one of the most prolific contributors, and brings an enthusiastic and > active mindset. I would like to thank him and acknowledge his significant > contributions by proposing him as a core reviewer for Cyborg. > > Shogo Saito has been active in Cyborg since Train release. He has been > driving the Cyborg client improvements, including its revamp to use > OpenStackSDK. Previously he was instrumental in the transition to Python 3, > testing and fixing issues in the process. As he has access to real FPGA > hardware, he brings a users’ perspective and also tests Cyborg with real > hardware. I would like to thank and acknowledge him for his steady valuable > contributions, and propose him as a core reviewer for Cyborg. > > Some of the currently listed core reviewers have not been participating > for a lengthy period of time. It is proposed that those who have had no > contributions for the past 18 months – i.e. no participation in meetings, > no code contributions and no reviews – be removed from the list of core > reviewers. > > If no objections are made known by March 20, I will make the changes > proposed above. > > Thanks. > > Regards, > Sundar > -- Zhipeng (Howard) Huang Principle Engineer OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, DMTF, W3C -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Wed Mar 11 16:37:06 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 11 Mar 2020 16:37:06 +0000 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: Message-ID: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > Big +1 for Brin and shogo's nomination and well deserved :) > > I'm a little bit concerned over the 18 months period. The original rule we > setup is volunteer step down, since this is a small team we want to > acknowledge everyone that has made significant contributions. Some of the > inactive core reviewers like Justin Kilpatrick have moved on a long time > ago, and I don't see people like him could do any harm to the project. > > But if the core reviewer has a size limit in the system, that would be > reasonable to replace the inactive ones with the new recruits :) it is generally considerd best pratice to maintian the core team adding or removing people based on there activity. if a core is removed due to in activity and they come back they can always be restored but it give a bad perception if a project has like 20 core but only 2 are active. as a new contibutor you dont know which ones are active and it can be frustrating to reach out to them and get no responce. also just form a project healt point of view it make the project look like its more diverse or more active then it actully is which is also not generally a good thing. that said core can step down if they feel like they can contribute time anymore when ever they like so and if a core is steping a way for a few months but intends to come back they can also say that in advance and there is no harm in leaving them for a cycle or two but in general after a period of in activity (usally more then a full release/6months) i think its good to reduce back down the core team. > > Just my two cents > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > wrote: > > > Hello all, > > Brin Zhang has been actively contributing to Cyborg in various areas, > > adding new features, improving quality, reviewing patches, and generally > > helping others in the community. Despite the relatively short time, he has > > been one of the most prolific contributors, and brings an enthusiastic and > > active mindset. I would like to thank him and acknowledge his significant > > contributions by proposing him as a core reviewer for Cyborg. > > > > Shogo Saito has been active in Cyborg since Train release. He has been > > driving the Cyborg client improvements, including its revamp to use > > OpenStackSDK. Previously he was instrumental in the transition to Python 3, > > testing and fixing issues in the process. As he has access to real FPGA > > hardware, he brings a users’ perspective and also tests Cyborg with real > > hardware. I would like to thank and acknowledge him for his steady valuable > > contributions, and propose him as a core reviewer for Cyborg. > > > > Some of the currently listed core reviewers have not been participating > > for a lengthy period of time. It is proposed that those who have had no > > contributions for the past 18 months – i.e. no participation in meetings, > > no code contributions and no reviews – be removed from the list of core > > reviewers. > > > > If no objections are made known by March 20, I will make the changes > > proposed above. > > > > Thanks. > > > > Regards, > > Sundar > > > > From balazs.gibizer at est.tech Wed Mar 11 17:17:13 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Wed, 11 Mar 2020 18:17:13 +0100 Subject: [nova][ptg] PTG participation Message-ID: <1583947033.12170.37@est.tech> Hi, I've just got the news from my employer that due to COVID19 I cannot travel to Vancouver in June. cheers, gibi From smooney at redhat.com Wed Mar 11 17:36:20 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 11 Mar 2020 17:36:20 +0000 Subject: [nova][ptg] PTG participation In-Reply-To: <1583947033.12170.37@est.tech> References: <1583947033.12170.37@est.tech> Message-ID: On Wed, 2020-03-11 at 18:17 +0100, Balázs Gibizer wrote: > Hi, > > I've just got the news from my employer that due to COVID19 I cannot > travel to Vancouver in June. >From a redhat perspective i dont think a decision has been made on if we should attend or not. last time i spoke to my manager we were still waiting to see how thing progress with the assumtion we would attend but we might want to plan for a virtual PTG (via video conference, etherpad and email) in the event many cant travel or that things escalate to a point where the PTG event could be canceled. i dont think the foundation has indicated that that is likely to happen but im sure they are monitoring things closely as our employers will be so having a plan b might now be a bad thing in either case. if there isnt a diverse quoram physically at the ptg it would limit our ability to make desisions as happend to some extent in china. it would be still good to get operator feedback but they may also be under similar travel restrictions. > > cheers, > gibi > > > From smooney at redhat.com Wed Mar 11 18:01:29 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 11 Mar 2020 18:01:29 +0000 Subject: [nova][ptg] PTG participation In-Reply-To: References: <1583947033.12170.37@est.tech> Message-ID: <064cda0d6d2511f3cbccd26fbf1bb5460797fbef.camel@redhat.com> On Wed, 2020-03-11 at 17:36 +0000, Sean Mooney wrote: > On Wed, 2020-03-11 at 18:17 +0100, Balázs Gibizer wrote: > > Hi, > > > > I've just got the news from my employer that due to COVID19 I cannot > > travel to Vancouver in June. > > From a redhat perspective i dont think a decision has been made on if we should attend or not. ill clarify that slightly in that we do have guidence that "Red Hatters may not travel to attend external events or conferences with 1000+ attendees, even within their home country." in the past when the ptg and summit were combinined and we had the devsumit have ment that travel to the openstack even would not be allowed. At its current size its kind of in a gray zone where its is not banned as a public event but if it was an internal event the number of redhat employee that would be attending woudl be over the limit we have and the physical event would be canceled and converted to a virtual only event. so its tbd if i will be attending too although i have not heard a definitive No at this point but i also cant really book tickets and flight yet either however the guidance we have been given is to try and default to virtual attendance were possible. > last time i spoke to my manager we were still waiting to see how thing progress with the assumtion we would attend > but we might want to plan for a virtual PTG (via video conference, etherpad and email) in the event many cant travel > or > that things escalate to a point where the PTG event could be canceled. > > i dont think the foundation has indicated that that is likely to happen but im sure they are monitoring > things closely as our employers will be so having a plan b might now be a bad thing in either case. > if there isnt a diverse quoram physically at the ptg it would limit our ability to make desisions as happend to > some extent in china. it would be still good to get operator feedback but they may also be under similar travel > restrictions. > > > > cheers, > > gibi > > > > > > > > From lyarwood at redhat.com Wed Mar 11 18:34:53 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Wed, 11 Mar 2020 18:34:53 +0000 Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 In-Reply-To: <21add71a504fa8f9a0d32b2181f2660076a6d356.camel@redhat.com> References: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> <21add71a504fa8f9a0d32b2181f2660076a6d356.camel@redhat.com> Message-ID: <20200311183453.iabtv3gxkn5i43jj@lyarwood.usersys.redhat.com> On 11-03-20 14:37:35, Sean Mooney wrote: > On Tue, 2020-03-10 at 17:04 +0000, Arkady.Kanevsky at dell.com wrote: > > Thank Lee. Sound approach. > > A few questions/comments. > > 1. Assume that we have unwritten assumption that all nova nodes have > > access to volumes on the backend. > > So we rely on it except for ephemeral storage. > well the job deploys the storage backend so its not an asusmtion we > deploy it that way intentionally. we also set up ssh keys so we can > rsync the qcow files between host when we do block migration. Correct, the jobs are simple multinode deployments of one main controller/compute and a smaller subnode compute. > > 2. What need to be done for volumes that use FC not iSCSI? > we dont test FC in the migration job currently so i think that is out > of scope of this refactor. the goal is to move it to zuulv3 while > testing all the existing cases not to add more cases in this phase. Yes, apologies if that wasn't clear from my initial post. That said I'd argue that FC testing of any kind would be out of scope for our jobs in openstack/nova. Specific backends and interconnects being better tested by openstack/cinder and openstack/os-brick IMHO. > > 3. You have one for Ceph. Does that mean that we need an analog for > > other cinder back ends? > no. the ceph backend is tested seperatly as there are basicaly 3 > storage backend to the libvirt driver. local file which is tested as > part of block migration with qcow2 local block device which is tested > via cinder/lvm with the block device mounted on the host vi isci (FC > would be the same form a qemu point of view) and finally ceph is used > to test the qemu nataive network block device support. > > so we are not trying to test different cinder backends but rahter the > different image backends/qemu storage types supprot in nova Correct. > > 4. Do we need to anything analogous for Manila? > maybe but again that seams like its out of scope so intally i would > say no Correct, we don't have any coverage for this at the moment. > > 5. How do we address multi-attach volumes and multipathing? Expect > > that if we have multipthaing on origin node we laso have > > multipathing at destination at the end. > multi attach is already tested in the job i belive so we would > continue that. i think both cinder lvm and ceph support I'm actually not sure if we do have any multiattach LM coverage, something to potentially add with this refactor. > multi attach. i dont think we test multipath in the gate in the > current jobs so i would not imediatly assume we woudl add it as part > of this refactor. As with FC I don't think this should live in our jobs tbh. > > -----Original Message----- > > From: Lee Yarwood > > Sent: Tuesday, March 10, 2020 10:22 AM > > To: openstack-discuss at lists.openstack.org > > Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 > > > > Hello all, > > > > I've started PoC'ing some ideas around $subject in the topic below > > and wanted to ask the wider team for feedback on the approach I'm > > taking: > > > > https://review.opendev.org/#/q/topic:nova-live-migration-zuulv3 > > > > My initial idea is to break the job up into the following smaller > > multinode jobs that are hopefully easier to understand and maintain. > > > > * nova-multinode-live-migration-py3 > > > > A simple LM job using the qcow2 imagebackend and LVM/iSCSI c-vol. > > > > * nova-multinode-live-migration-ceph-py3 > > this would be replaceing our existing devstack-plugin-ceph-tempest-py3 > job runing all the same test but in a multinode config with live > migration tests enabled in the tempest config. If we want to merge the evacuation tests back into this I was going to limit it to live migration tests only and continue running devstack-plugin-ceph-tempest-py3 for everything else. FWIW devstack-plugin-ceph-tempest-py3 is still NV even when we've been gating on the success of ceph live migration in the original nova-live-migration job. > > A ceph based LM job using rbd for both imagebackend and c-vol. > > > > * nova-multinode-evacuate-py3 > so this would be the only new job although i am not sure it should be > seperated out. we likely want to test evacuate with file,block and > network storage so i think it makes sense to do this as a post > playbook in the other two jobs. Yeah that's fair, I might start with this broken out just to work on that playbook/role before merging it back into the above jobs tbh. > > A separate evacuation job using qcow2 imagebackend and LVM/iSCSI > > c-vol. The existing script *could* then be ported to an Ansible > > role as part of the migration to Zuulv3. > > > > Hopefully this is pretty straight forward but I'd appreciate any > > feedback on this all the same. -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From lyarwood at redhat.com Wed Mar 11 18:36:48 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Wed, 11 Mar 2020 18:36:48 +0000 Subject: [nova] Breaking up and migrating the nova-live-migration job to Zuulv3 In-Reply-To: <00363f3f-ed3d-488d-98d4-c3025b7e179f@www.fastmail.com> References: <20200310152203.6fv57e5qyqhxdgep@lyarwood.usersys.redhat.com> <00363f3f-ed3d-488d-98d4-c3025b7e179f@www.fastmail.com> Message-ID: <20200311183648.c4dyz3r2upv5zyrd@lyarwood.usersys.redhat.com> On 10-03-20 08:25:38, Clark Boylan wrote: > On Tue, Mar 10, 2020, at 8:22 AM, Lee Yarwood wrote: > > Hello all, > > > > I've started PoC'ing some ideas around $subject in the topic below and > > wanted to ask the wider team for feedback on the approach I'm taking: > > > > https://review.opendev.org/#/q/topic:nova-live-migration-zuulv3 > > > > My initial idea is to break the job up into the following smaller > > multinode jobs that are hopefully easier to understand and maintain. > > > > * nova-multinode-live-migration-py3 > > > > A simple LM job using the qcow2 imagebackend and LVM/iSCSI c-vol. > > > > * nova-multinode-live-migration-ceph-py3 > > > > A ceph based LM job using rbd for both imagebackend and c-vol. > > > > * nova-multinode-evacuate-py3 > > > > A separate evacuation job using qcow2 imagebackend and LVM/iSCSI c-vol. > > The existing script *could* then be ported to an Ansible role as part of > > the migration to Zuulv3. > > > > Hopefully this is pretty straight forward but I'd appreciate any > > feedback on this all the same. > > Just a note that you can probably drop the -py3 suffix as I imagine that is assumed at this point? Gah, of course, thanks Clark. -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From juliaashleykreger at gmail.com Wed Mar 11 18:54:20 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 11 Mar 2020 11:54:20 -0700 Subject: [ironic] proposing Iury Gregory for bifrost-core, ironic-inspector-core, sushy-core Message-ID: Iury has been working hard across the ironic community and has been quite active in changing and improving our CI, as well as reviewing code contributions and helpfully pointing out issues or items that need to be fixed. I feel that he is on track to join ironic-core in the next few months, but first I propose we add him to bifrost-core, ironic-inspector-core, and sushy-core. Any objections? From opetrenko at mirantis.com Wed Mar 11 19:01:42 2020 From: opetrenko at mirantis.com (Oleksii Petrenko) Date: Wed, 11 Mar 2020 21:01:42 +0200 Subject: Add pytest, pytest-django and pytest-html to global requirements Message-ID: Adding pytest, pytest-django and pytest-html allows import of tests results in xml, html formats for openstack-horizon. What do you think about this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mthode at mthode.org Wed Mar 11 19:06:23 2020 From: mthode at mthode.org (Matthew Thode) Date: Wed, 11 Mar 2020 14:06:23 -0500 Subject: Add pytest, pytest-django and pytest-html to global requirements In-Reply-To: References: Message-ID: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> On 20-03-11 21:01:42, Oleksii Petrenko wrote: > Adding pytest, pytest-django and pytest-html allows import of tests results > in xml, html formats for openstack-horizon. What do you think about this? I think the question is in refrence to using something that is already included in global-requirements (like stestr or something else). Also, here's the review https://review.opendev.org/712315 Starting with stestr, could you explain why it was not good enough for your use case? -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From smooney at redhat.com Wed Mar 11 19:42:47 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 11 Mar 2020 19:42:47 +0000 Subject: Add pytest, pytest-django and pytest-html to global requirements In-Reply-To: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> References: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> Message-ID: <2f761df2a3db8c080f5cab75710b825242ed79f9.camel@redhat.com> On Wed, 2020-03-11 at 14:06 -0500, Matthew Thode wrote: > On 20-03-11 21:01:42, Oleksii Petrenko wrote: > > Adding pytest, pytest-django and pytest-html allows import of tests results > > in xml, html formats for openstack-horizon. What do you think about this? > > I think the question is in refrence to using something that is already > included in global-requirements (like stestr or something else). > > Also, here's the review https://review.opendev.org/712315 > > Starting with stestr, could you explain why it was not good enough for > your use case? more of a general question if they are test only deps that wont be used at runtime which i think is the case in all of the above do they enven need to be in Global-requirements? ignoring the fact that devstack installes all test- requireemtes when it in stall packages which is a different topic if this is only used for generating html report for tests then it seams liek we would not need to corrdiate the software version. that said i am also curious why the normal html report that gets generated form our tox runs in the ci is not sufficent. what does pytest-html add? is it just the ablity to do a local html report when you run tox manually? or do you plan to use pytest-django and pytest-html to do some other testing that you cant currently do? > From smooney at redhat.com Wed Mar 11 19:54:36 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 11 Mar 2020 19:54:36 +0000 Subject: Add pytest, pytest-django and pytest-html to global requirements In-Reply-To: <2f761df2a3db8c080f5cab75710b825242ed79f9.camel@redhat.com> References: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> <2f761df2a3db8c080f5cab75710b825242ed79f9.camel@redhat.com> Message-ID: <4636a78af2b78b8d316ff5cd3d2b76a1a2173cd7.camel@redhat.com> On Wed, 2020-03-11 at 19:42 +0000, Sean Mooney wrote: > On Wed, 2020-03-11 at 14:06 -0500, Matthew Thode wrote: > > On 20-03-11 21:01:42, Oleksii Petrenko wrote: > > > Adding pytest, pytest-django and pytest-html allows import of tests results > > > in xml, html formats for openstack-horizon. What do you think about this? > > > > I think the question is in refrence to using something that is already > > included in global-requirements (like stestr or something else). > > > > Also, here's the review https://review.opendev.org/712315 > > > > Starting with stestr, could you explain why it was not good enough for > > your use case? > > more of a general question if they are test only deps that wont be used at runtime which i think is the case in all of > the above do they enven need to be in Global-requirements? ignoring the fact that devstack installes all test- > requireemtes when it in stall packages which is a different topic if this is only used for generating html report for > tests then it seams liek we would not need to corrdiate the software version. > > that said i am also curious why the normal html report that gets generated form our tox runs in the ci > is not sufficent. what does pytest-html add? is it just the ablity to do a local html report when you run tox > manually? > or do you plan to use pytest-django and pytest-html to do some other testing that you cant currently do? ah i see that pytest-django will allow the removal of django test which is presuable to adress https://bugs.launchpad.net/horizon/+bug/1866666 based on https://review.opendev.org/#/c/711195/ if hoizong is thinking of moveing away form its current custome test runner https://github.com/openstack/horizon/blob/stable/pike/manage.py then stestr should at least be considerd but if there is a valid technical reason to go with pytest instead. that said as someone that does not work on horizon im supriced its not already using stestr. give it still has a testr.conf meanign at somepoint it moved a way form testrepostry and to its current custom runner when the other project moved to os-testr and then to stestr. that said i kind of like pytest as a test runner and it would solve some other issues with subunit in nova so im not against adding it. > > > > From opetrenko at mirantis.com Wed Mar 11 20:32:32 2020 From: opetrenko at mirantis.com (Oleksii Petrenko) Date: Wed, 11 Mar 2020 22:32:32 +0200 Subject: Add pytest, pytest-django and pytest-html to global requirements In-Reply-To: <4636a78af2b78b8d316ff5cd3d2b76a1a2173cd7.camel@redhat.com> References: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> <2f761df2a3db8c080f5cab75710b825242ed79f9.camel@redhat.com> <4636a78af2b78b8d316ff5cd3d2b76a1a2173cd7.camel@redhat.com> Message-ID: > > > Starting with stestr, could you explain why it was not good enough for > > > your use case? > > Stestr will not provide us with fixtures for django (for future use), also with the help of pytest, we probably would be able to unify html creation all across our projects. Also, xml exporting in different formats can help users with aggregating test statistics. > > more of a general question if they are test only deps that wont be used at runtime which i think is the case in all of > > the above do they enven need to be in Global-requirements? ignoring the fact that devstack installes all test- > > requireemtes when it in stall packages which is a different topic if this is only used for generating html report for > > tests then it seams liek we would not need to corrdiate the software version. pytest is needed to generate coverage reports. > > > > > > > > > From mtreinish at kortar.org Wed Mar 11 21:08:27 2020 From: mtreinish at kortar.org (Matthew Treinish) Date: Wed, 11 Mar 2020 17:08:27 -0400 Subject: Add pytest, pytest-django and pytest-html to global requirements In-Reply-To: References: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> <2f761df2a3db8c080f5cab75710b825242ed79f9.camel@redhat.com> <4636a78af2b78b8d316ff5cd3d2b76a1a2173cd7.camel@redhat.com> Message-ID: <20200311210827.GA90029@sinanju> On Wed, Mar 11, 2020 at 10:32:32PM +0200, Oleksii Petrenko wrote: > > > > Starting with stestr, could you explain why it was not good enough for > > > > your use case? > > > > Stestr will not provide us with fixtures for django (for future use), > also with the help of pytest, we probably would be able to unify html > creation all across our projects. Also, xml exporting in different > formats can help users with aggregating test statistics. The aggregated data view already exists: http://status.openstack.org/openstack-health/#/ We also have 2 different html views of a test run depending on the level of detail you want: https://7dd927a4891851ac968e-517bfbb0b76f5445108257ba8a306671.ssl.cf5.rackcdn.com/712315/2/check/tempest-full-py3/c20f9f1/testr_results.html and https://7dd927a4891851ac968e-517bfbb0b76f5445108257ba8a306671.ssl.cf5.rackcdn.com/712315/2/check/tempest-full-py3/c20f9f1/controller/logs/stackviz/index.html#/stdin/timeline As for "xml exporting" I assume you're talking about xunitxml. There are several limitations around it, especially for parallel test execution which is why stestr is built around and uses subunit. But, if you want to generate xunitxml from subunit for any reason this is straightforward to do, it's built into subunit: https://github.com/testing-cabal/subunit/blob/master/filters/subunit2junitxml > > > > more of a general question if they are test only deps that wont be used at runtime which i think is the case in all of > > > the above do they enven need to be in Global-requirements? ignoring the fact that devstack installes all test- > > > requireemtes when it in stall packages which is a different topic if this is only used for generating html report for > > > tests then it seams liek we would not need to corrdiate the software version. > pytest is needed to generate coverage reports. I don't understand this either, we have coverage jobs already running on most projects. The reports get published as part of the job artifacts: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fd1/706509/2/check/openstack-tox-cover/fd1df05/cover/index.html Also as I pointed out on the review, this is not the first time we've discussed this. Since I started working on OpenStack why not runner or framework X (at one point it was nose, then it switched to pytest) has been brought up by someone. We tried to write it down in the project testing interface: https://governance.openstack.org/tc/reference/pti/python.html#python-test-running Basically, by using a unittest based runner anybody can use their preferred test runner locally. stestr is used for CI because of the parallel execution and subunit integration to leverage all the infra tooling built around it. That being said horizon has always been an exception because django has special requirements for testing (mainly they publish their testing framework as an extension for a test frameworks other than stdlib unittest). In the past it was needed a nose extension and now it looks like that has been updated to be a pytest exception. I don't see a problem to just morph the old exception that horizon uses nose to horizon uses pytest if it's really necessary to test django. If you do end up using pytest because there is no other choice for django testing, you can convert the xunitxml to subunit to integrate it into all those existing tools I mentioned before with either: https://github.com/mtreinish/health-helm/blob/master/junitxml2subunit.py or https://github.com/mtreinish/junitxml2subunit (do note stackviz and subunit2sql/openstack-health won't be really useful with xunitxml to subunit conversion because xunitxml doesn't track execution timestamps) -Matt Treinish -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From gouthampravi at gmail.com Wed Mar 11 21:17:53 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Wed, 11 Mar 2020 14:17:53 -0700 Subject: [OSSA-2020-002] Manila: Unprivileged users can retrieve, use and manipulate share networks (CVE-2020-9543) Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 ================================================================================= OSSA-2020-002: Unprivileged users can retrieve, use and manipulate share networks ================================================================================= :Date: March 10, 2020 :CVE: CVE-2020-9543 Affects ~~~~~~~ - - Manila: <7.4.1, >=8.0.0 <8.1.1, >=9.0.0 <9.1.1 Description ~~~~~~~~~~~ Tobias Rydberg from City Network Hosting AB reported a vulnerability with the manila's share network APIs. An attacker can retrieve and manipulate share networks that do not belong to them if they possess the share network ID. By exploiting this vulnerability, they can view and manipulate share network subnets and use the share network to create resources such as shares and share groups. Patches ~~~~~~~ - - https://review.opendev.org/712167 (Pike) - - https://review.opendev.org/712166 (Queens) - - https://review.opendev.org/712165 (Rocky) - - https://review.opendev.org/712164 (Stein) - - https://review.opendev.org/712163 (Train) - - https://review.opendev.org/712158 (Ussuri) Credits ~~~~~~~ - - Tobias Rydberg from City Network Hosting AB (CVE-2020-9543) References ~~~~~~~~~~ - - https://launchpad.net/bugs/1861485 - - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-9543 Notes ~~~~~ - - The stable/queens and stable/pike branches are under extended maintenance and will receive no new point releases, but patches for them are provided as a courtesy. - -- Goutham Pacha Ravi PTL, OpenStack Manila -----BEGIN PGP SIGNATURE----- wsFcBAEBCAAGBQJeaVVWAAoJEDEySBmyuw9i8c0P/Rjkr4mxbDi7GzDCLdvC 4SK31LaF92uop/t2XXnm/p2Lui/4nG6ss46ajnmsplN2D//f+/NhBC+Oa/+R 3rwEl1YFFO8NoNcpjWS+6oE66HNPEPTxSMheyfWJTjl8bmH4wL0ZGnQ+cNWM q1XhO5Qjwv58epa0IK5vRA6lfWEmZQ69/+7nf6Tyha8vuLFOpStWXj7sV0SZ j/AxvTeCu/30EH9U4E10VQ/GpHz00WuueEYUCJgOZw4jGk32238yXmuF1fBU il4PR53ZPFqb20It56t/rrr0sGB8lLui7KiBhaHFmjRK8YqwD1pqz9XAaxNq CsgbkMnR8+WsheAgMr49NeYsQ1PD6SCLBXPQGVNus/pl5bzctIaqmswPN1ey p23tREpTEjOxg9mQJLkTCKICvi0alx3Nlk9EsrSapovJk/v8BJGrjkIj8iH0 a1pAMzjcHfGpCTGO2dHBOfJs7BXL9B6Jdba9bdRTt5BRI4NHKwvM9SP9yBb6 F7UNoo8cd+pQp0EV6i8CPUTF/qWU5rqOyIr9tGTAOPm0lg8+uIOot7oZzJcu QBaKyEZu9X4OV1o5mZ68KokiVP7RWYGMGz94NV4ZMNNfmgpsxP/h2+MZCUQJ +lmMPInx5abdwMtqiyhrSQxdgLCOKlWMYXgrs7w225sjv2+LpuVltIPXGPEJ tJq+ =tXeN -----END PGP SIGNATURE----- From emilien at redhat.com Wed Mar 11 22:46:13 2020 From: emilien at redhat.com (Emilien Macchi) Date: Wed, 11 Mar 2020 18:46:13 -0400 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: <1583528201.853712216@emailsrvr.com> References: <1583528201.853712216@emailsrvr.com> Message-ID: Hi Mark, Thanks for the transparency, as usual. I have a few thoughts, please read inline. On Fri, Mar 6, 2020 at 4:04 PM Mark Collier wrote: > upcoming event in Vancouver is no exception. The OpenDev tracks > > each morning will be programmed by volunteers from the community, and > the project > > teams will be organizing their own conversations as well each afternoon > M-W, and > > all day Thursday. > > > > But the larger question is here: should the show go on? > > > > The short answer is that as of now, the Vancouver and Berlin events are > still > > scheduled to happen in June (8-11) and October (19-23), respectively. > > > > However, we are willing to cancel or approach the events in a different > way (i.e. > > virtual) if the facts indicate that is the best path, and we know the > facts are > > changing rapidly. One of the most critical inputs we need is to hear > from each of > > you. We know that many of you rely on the twice-annual events to get > together and > > make rapid progress on the software, which is one reason we are not > making any > > decisions in haste. We also know that many of you may be unable or > unwilling to > > travel in June, and that is critical information to hear as we get > closer to the > > event so that we can make the most informed decision. > I believe that we, as a community should show the example and our strengths by cancelling the Vancouver event and organize a virtual event like some other big events are doing. There is an opportunity for the OSF to show leadership in Software communities and acknowledge the risk of spread during that meeting; not only for the people attending it but for also those in contact with these people later. I'm not a doctor nor I know much about the virus; but I'm not interested to travel and take the risk to 1) catch the virus and 2) spread it at home and in my country; and as a community member, I feel like our responsibility is also to maintain ourselves healthy. In my opinion, the sooner we cancel, the better we can focus on organizing the virtual meetings, and also we can influence more communities to take that kind of decisions. Thanks Mark for starting that discussion, it's a perfect sign of how healthy is our community; and hopefully it will continue to be. -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From miguel at mlavalle.com Wed Mar 11 23:36:51 2020 From: miguel at mlavalle.com (Miguel Lavalle) Date: Wed, 11 Mar 2020 18:36:51 -0500 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> Message-ID: Hi, At Verizon Media we haven't been told specifically we won't attend the Vancouver event. However, all international travel is cancelled and in-country trips are highly restricted Regards Miguel On Wed, Mar 11, 2020 at 5:47 PM Emilien Macchi wrote: > Hi Mark, > > Thanks for the transparency, as usual. I have a few thoughts, please read > inline. > > On Fri, Mar 6, 2020 at 4:04 PM Mark Collier wrote: > >> upcoming event in Vancouver is no exception. The OpenDev tracks >> > each morning will be programmed by volunteers from the community, and >> the project >> > teams will be organizing their own conversations as well each afternoon >> M-W, and >> > all day Thursday. >> > >> > But the larger question is here: should the show go on? >> > >> > The short answer is that as of now, the Vancouver and Berlin events are >> still >> > scheduled to happen in June (8-11) and October (19-23), respectively. >> > >> > However, we are willing to cancel or approach the events in a different >> way (i.e. >> > virtual) if the facts indicate that is the best path, and we know the >> facts are >> > changing rapidly. One of the most critical inputs we need is to hear >> from each of >> > you. We know that many of you rely on the twice-annual events to get >> together and >> > make rapid progress on the software, which is one reason we are not >> making any >> > decisions in haste. We also know that many of you may be unable or >> unwilling to >> > travel in June, and that is critical information to hear as we get >> closer to the >> > event so that we can make the most informed decision. >> > > I believe that we, as a community should show the example and our > strengths by cancelling the Vancouver event and organize a virtual event > like some other big events are doing. > There is an opportunity for the OSF to show leadership in Software > communities and acknowledge the risk of spread during that meeting; not > only for the people attending it but for also those in contact with these > people later. > > I'm not a doctor nor I know much about the virus; but I'm not interested > to travel and take the risk to 1) catch the virus and 2) spread it at home > and in my country; and as a community member, I feel like our > responsibility is also to maintain ourselves healthy. > > In my opinion, the sooner we cancel, the better we can focus on organizing > the virtual meetings, and also we can influence more communities to take > that kind of decisions. > > Thanks Mark for starting that discussion, it's a perfect sign of how > healthy is our community; and hopefully it will continue to be. > -- > Emilien Macchi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhipengh512 at gmail.com Thu Mar 12 00:00:37 2020 From: zhipengh512 at gmail.com (Zhipeng Huang) Date: Thu, 12 Mar 2020 08:00:37 +0800 Subject: [cyborg] Proposing core reviewers In-Reply-To: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: Hi Sean, This is a good point on making clarity on the contributor side, wasn't thinking about that. Re "be restored" my thoughts were that if anyone comes back, and not as core member, they should follow the process and be nominated/elected again. If a previous inactive core being deleted and then come back just restored, it is also problematic :) That is also why I suggested not to presumptively remove inactive cores. On Thu, Mar 12, 2020 at 12:37 AM Sean Mooney wrote: > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > I'm a little bit concerned over the 18 months period. The original rule > we > > setup is volunteer step down, since this is a small team we want to > > acknowledge everyone that has made significant contributions. Some of the > > inactive core reviewers like Justin Kilpatrick have moved on a long time > > ago, and I don't see people like him could do any harm to the project. > > > > But if the core reviewer has a size limit in the system, that would be > > reasonable to replace the inactive ones with the new recruits :) > it is generally considerd best pratice to maintian the core team adding or > removing > people based on there activity. if a core is removed due to in activity > and they > come back they can always be restored but it give a bad perception if a > project has > like 20 core but only 2 are active. as a new contibutor you dont know > which ones are > active and it can be frustrating to reach out to them and get no responce. > also just form a project healt point of view it make the project look like > its more diverse > or more active then it actully is which is also not generally a good thing. > > that said core can step down if they feel like they can contribute time > anymore > when ever they like so and if a core is steping a way for a few months but > intends to > come back they can also say that in advance and there is no harm in > leaving them > for a cycle or two but in general after a period of in activity (usally > more then a full release/6months) > i think its good to reduce back down the core team. > > > > Just my two cents > > > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar < > sundar.nadathur at intel.com> > > wrote: > > > > > Hello all, > > > Brin Zhang has been actively contributing to Cyborg in various > areas, > > > adding new features, improving quality, reviewing patches, and > generally > > > helping others in the community. Despite the relatively short time, he > has > > > been one of the most prolific contributors, and brings an enthusiastic > and > > > active mindset. I would like to thank him and acknowledge his > significant > > > contributions by proposing him as a core reviewer for Cyborg. > > > > > > Shogo Saito has been active in Cyborg since Train release. He has been > > > driving the Cyborg client improvements, including its revamp to use > > > OpenStackSDK. Previously he was instrumental in the transition to > Python 3, > > > testing and fixing issues in the process. As he has access to real FPGA > > > hardware, he brings a users’ perspective and also tests Cyborg with > real > > > hardware. I would like to thank and acknowledge him for his steady > valuable > > > contributions, and propose him as a core reviewer for Cyborg. > > > > > > Some of the currently listed core reviewers have not been participating > > > for a lengthy period of time. It is proposed that those who have had no > > > contributions for the past 18 months – i.e. no participation in > meetings, > > > no code contributions and no reviews – be removed from the list of core > > > reviewers. > > > > > > If no objections are made known by March 20, I will make the changes > > > proposed above. > > > > > > Thanks. > > > > > > Regards, > > > Sundar > > > > > > > > > -- Zhipeng (Howard) Huang Principle Engineer OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, DMTF, W3C -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 12 00:13:32 2020 From: satish.txt at gmail.com (Satish Patel) Date: Wed, 11 Mar 2020 20:13:32 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> Message-ID: I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. This is my favorite video, not sure you have seen this before or not but anyway posting here - https://www.youtube.com/watch?v=bpmgxrPOrZw On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: > > Hi all, > > We are currently experiencing some fairly major issues with our > OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We > are seeing a lot of time out messages in responses to replies and > because of this instance creation or anything to do with instances and > networking is broken. > > We are running OpenStack Queens. > > We have already tuned Rabbit for Neutron by doing the following on neutron: > > heartbeat_timeout_threshold = 0 > rpc_conn_pool_size = 300 > rpc_thread_pool_size = 2048 > rpc_response_timeout = 3600 > rpc_poll_timeout = 60 > > ## Rpc all > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > What we are seeing in the error logs for neutron for all services > (l3-agent, dhcp, linux-bridge etc ) are these timeouts: > > https://pastebin.com/Fjh23A5a > > We have manually tried to get everything in sync by forcing fail-over of > the networking which seems to get routers in sync. > > We are also seeing that there are a lot of "unacknowledged" messages in > RabbitMQ for 'q-plugin' in the neutron queues. > > Some times restarting of the services on neutron gets these back > acknowledged again, however the timeouts come back. > > The RabbitMQ servers themselves are not loaded at all. All memory, file > descriptors and errlang processes have plenty of resources available. > > We are also seeing a lot of rpc issues: > > Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds before > next attempt. If the server is not down, consider increasing the > rpc_response_timeout option as Neutron server(s) may be overloaded and > unable to respond quickly enough.: MessagingTimeout: Timed out waiting > for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 > 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc > [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC > method release_dhcp_port. Waiting for 3347 seconds before next attempt. > If the server is not down, consider increasing the rpc_response_timeout > option as Neutron server(s) may be overloaded and unable to respond > quickly enough.: MessagingTimeout: Timed out waiting for a reply to > message ID 7937465f15634fbfa443fe1758a12a9c > > Does anyone know if there is anymore tuning to be done at all? Upgrading > for us at the moment to a newer version isn't really an option > unfortunately. > > Because of our setup, we also have roughly 800 routers enabled and I > know that will be putting a load on the system. However these problems > have only started to happen roughly 1 week ago and have steadily got worse. > > If anyone has any use cases for this or any more recommendations that > would be great. > > Many thanks, > > From sundar.nadathur at intel.com Thu Mar 12 00:40:42 2020 From: sundar.nadathur at intel.com (Nadathur, Sundar) Date: Thu, 12 Mar 2020 00:40:42 +0000 Subject: [cyborg] Proposing core reviewers In-Reply-To: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: > From: Sean Mooney > Sent: Wednesday, March 11, 2020 9:37 AM > > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > I'm a little bit concerned over the 18 months period. The original > > rule we setup is volunteer step down, since this is a small team we > > want to acknowledge everyone that has made significant contributions. > > Some of the inactive core reviewers like Justin Kilpatrick have moved > > on a long time ago, and I don't see people like him could do any harm to > the project. > > > > But if the core reviewer has a size limit in the system, that would be > > reasonable to replace the inactive ones with the new recruits :) > it is generally considerd best pratice to maintian the core team adding or > removing people based on there activity. if a core is removed due to in > activity and they come back they can always be restored but it give a bad > perception if a project has like 20 core but only 2 are active. as a new > contibutor you dont know which ones are active and it can be frustrating to > reach out to them and get no responce. > also just form a project healt point of view it make the project look like its > more diverse or more active then it actully is which is also not generally a > good thing. > > that said core can step down if they feel like they can contribute time > anymore when ever they like so and if a core is steping a way for a few > months but intends to come back they can also say that in advance and there > is no harm in leaving them for a cycle or two but in general after a period of > in activity (usally more then a full release/6months) i think its good to reduce > back down the core team. > > > > Just my two cents As of now, Cyborg core team officially has 12 members [1]. That is hardly small. Justin Kilpatrick seems to be gone for good; he didn't respond to my emails. Rushil Chugh confirmed that he is not working on OpenStack anymore and asked to step down as core. With due thanks to him for his contributions, I'll go ahead. Those are the two cores I had in mind. Agree with Sean that it is better to keep the list of core reviewers up to date. With all the changes in Cyborg over the past 18 months, it will be tough for a person to jump in after a long hiatus and resume as a core reviewer. Even if they want to come back, it is better for them to come up to speed first. Given this background, if there is any objection to the removal of these two cores, please let me know. [1] https://review.opendev.org/#/admin/groups/1243,members Regards, Sundar > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > > > > wrote: > > > > > Hello all, > > > Brin Zhang has been actively contributing to Cyborg in various > > > areas, adding new features, improving quality, reviewing patches, > > > and generally helping others in the community. Despite the > > > relatively short time, he has been one of the most prolific > > > contributors, and brings an enthusiastic and active mindset. I would > > > like to thank him and acknowledge his significant contributions by > proposing him as a core reviewer for Cyborg. > > > > > > Shogo Saito has been active in Cyborg since Train release. He has > > > been driving the Cyborg client improvements, including its revamp to > > > use OpenStackSDK. Previously he was instrumental in the transition > > > to Python 3, testing and fixing issues in the process. As he has > > > access to real FPGA hardware, he brings a users’ perspective and > > > also tests Cyborg with real hardware. I would like to thank and > > > acknowledge him for his steady valuable contributions, and propose him > as a core reviewer for Cyborg. > > > > > > Some of the currently listed core reviewers have not been > > > participating for a lengthy period of time. It is proposed that > > > those who have had no contributions for the past 18 months – i.e. no > > > participation in meetings, no code contributions and no reviews – be > > > removed from the list of core reviewers. > > > > > > If no objections are made known by March 20, I will make the changes > > > proposed above. > > > > > > Thanks. > > > > > > Regards, > > > Sundar From fungi at yuggoth.org Thu Mar 12 00:43:26 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 12 Mar 2020 00:43:26 +0000 Subject: [all] IRC channel cleanup (was: A call for consolidation and simplification) In-Reply-To: References: Message-ID: <20200312004325.p7zwr7pkc6aeuezc@yuggoth.org> On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: [...] > Do we really need 180 IRC channels [...] There are around 110 we currently seem to deem worth logging with the "openstack" meetbot (note that not all are OpenStack community channels): A quick survey of logs suggests these have seen no comment from a human in 6 months (all are OpenStack-related): #scientific-wg #openstack-women #openstack-sprint #openstack-net-bgpvpn #openstack-heat-translator #openstack-forum #openstack-dragonflow #congress And these have averaged fewer than one comment from a human per month since September: #openstack-outreachy #murano #openstack-tricircle #openstack-performance #openstack-ec2api #openstack-browbeat Does anyone object to us ceasing logging of the above 14 channels? -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From amy at demarco.com Thu Mar 12 00:49:42 2020 From: amy at demarco.com (Amy Marrich) Date: Wed, 11 Mar 2020 19:49:42 -0500 Subject: [all] IRC channel cleanup (was: A call for consolidation and simplification) In-Reply-To: <20200312004325.p7zwr7pkc6aeuezc@yuggoth.org> References: <20200312004325.p7zwr7pkc6aeuezc@yuggoth.org> Message-ID: You can go ahead and archive #openstack-women as anyone should now be on #openstack-diversity. Thanks, Amy (spotz) On Wed, Mar 11, 2020 at 7:45 PM Jeremy Stanley wrote: > On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: > [...] > > Do we really need 180 IRC channels > [...] > > There are around 110 we currently seem to deem worth logging with > the "openstack" meetbot (note that not all are OpenStack community > channels): > > https://opendev.org/opendev/system-config/src/commit/c24853076ddc59932a0760ddc2dcafdc6958340e/hiera/common.yaml#L102-L214 > > > > A quick survey of logs suggests these have seen no comment from a > human in 6 months (all are OpenStack-related): > > #scientific-wg > #openstack-women > #openstack-sprint > #openstack-net-bgpvpn > #openstack-heat-translator > #openstack-forum > #openstack-dragonflow > #congress > > And these have averaged fewer than one comment from a human per > month since September: > > #openstack-outreachy > #murano > #openstack-tricircle > #openstack-performance > #openstack-ec2api > #openstack-browbeat > > Does anyone object to us ceasing logging of the above 14 channels? > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Thu Mar 12 01:03:33 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 12 Mar 2020 01:03:33 +0000 Subject: [all] Removing defunct meeting records (was: A call for consolidation and simplification) In-Reply-To: References: Message-ID: <20200312010333.auxhcao54e6gbf42@yuggoth.org> On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: [...] > we have too many meetings (76, in case you were wondering), too > much energy spent running them, too much frustration when nobody > joins. [...] Here's a list of 25 currently defined meetings which have not been held in 2020 (though it's possible some are being held with a different meeting_id passed to #startmeeting than is listed in the meeting record): CloudKitty Team Meeting Congress Team Meeting Containers Team Meeting Documentation Team Meeting First Contact SIG Meeting Freezer Meeting Glance Bug Squad Meeting Group Based Policy Team Meeting Heat (Orchestration) Team Meeting I18N Team Meeting Interop Working Group Meeting Kuryr Project Office Hours LOCI Development Meeting Mistral Meeting Networking VPP team meeting OpenStack Charms Placement Team Office Hour PowerVM Driver Meeting Public Cloud SIG Searchlight Team Meeting Telemetry Team Meeting Trove (DBaaS) Team Meeting Upgrades SIG Vitrage Team Meeting Zaqar Team Meeting I recommend at least correcting inaccurate meeting_id entries in the YAML files here: https://opendev.org/opendev/irc-meetings/src/branch/master/meetings/ If there are meetings you know are not being held, please submit changes to remove their corresponding YAML files. I'll set myself a reminder to rerun this query again sometime soon and we can discuss bulk removing any which are presumed defunct at that time. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From eandersson at blizzard.com Thu Mar 12 01:05:21 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Thu, 12 Mar 2020 01:05:21 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> Message-ID: We are hitting something awfully similar. We have basically been hitting a few pretty serious bugs with RabbitMQ. The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 I wrote two quick scripts to audit these issues. http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. Best Regards, Erik Olof Gunnar Andersson -----Original Message----- From: Satish Patel Sent: Wednesday, March 11, 2020 5:14 PM To: Grant Morley Cc: openstack-discuss at lists.openstack.org Subject: Re: Neutron RabbitMQ issues I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: > > Hi all, > > We are currently experiencing some fairly major issues with our > OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We > are seeing a lot of time out messages in responses to replies and > because of this instance creation or anything to do with instances and > networking is broken. > > We are running OpenStack Queens. > > We have already tuned Rabbit for Neutron by doing the following on neutron: > > heartbeat_timeout_threshold = 0 > rpc_conn_pool_size = 300 > rpc_thread_pool_size = 2048 > rpc_response_timeout = 3600 > rpc_poll_timeout = 60 > > ## Rpc all > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > What we are seeing in the error logs for neutron for all services > (l3-agent, dhcp, linux-bridge etc ) are these timeouts: > > https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n > 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK > 9aOA$ > > We have manually tried to get everything in sync by forcing fail-over > of the networking which seems to get routers in sync. > > We are also seeing that there are a lot of "unacknowledged" messages > in RabbitMQ for 'q-plugin' in the neutron queues. > > Some times restarting of the services on neutron gets these back > acknowledged again, however the timeouts come back. > > The RabbitMQ servers themselves are not loaded at all. All memory, > file descriptors and errlang processes have plenty of resources available. > > We are also seeing a lot of rpc issues: > > Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds > before next attempt. If the server is not down, consider increasing > the rpc_response_timeout option as Neutron server(s) may be overloaded > and unable to respond quickly enough.: MessagingTimeout: Timed out > waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 > 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc > [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC > method release_dhcp_port. Waiting for 3347 seconds before next attempt. > If the server is not down, consider increasing the > rpc_response_timeout option as Neutron server(s) may be overloaded and > unable to respond quickly enough.: MessagingTimeout: Timed out waiting > for a reply to message ID 7937465f15634fbfa443fe1758a12a9c > > Does anyone know if there is anymore tuning to be done at all? > Upgrading for us at the moment to a newer version isn't really an > option unfortunately. > > Because of our setup, we also have roughly 800 routers enabled and I > know that will be putting a load on the system. However these problems > have only started to happen roughly 1 week ago and have steadily got worse. > > If anyone has any use cases for this or any more recommendations that > would be great. > > Many thanks, > > From whayutin at redhat.com Thu Mar 12 02:42:52 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Wed, 11 Mar 2020 20:42:52 -0600 Subject: [tripleo] Missing tag in cron container image - no recheck please In-Reply-To: References: Message-ID: On Wed, Mar 11, 2020 at 8:12 AM Emilien Macchi wrote: > Hi folks, > > We seem to have an issue with container images, where one (at least) has a > missing tag: > https://bugs.launchpad.net/tripleo/+bug/1866927 > > It is causing most of our jobs to go red and fail on: > tripleo_common.image.exception.ImageNotFoundException: Not found image: > docker:// > docker.io/tripleomaster/centos-binary-cron:3621159be13b41f8ead2e873b357f4a5 > > Please refrain from approving or rechecking patches until we have > sorted this out. > > Thanks and stay tuned. > -- > Emilien Macchi > This issue has been resolved.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 12 03:29:39 2020 From: satish.txt at gmail.com (Satish Patel) Date: Wed, 11 Mar 2020 23:29:39 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> Message-ID: Totally agreed with you, I had similar issue when my cluster got split and not able to recover from that state then finally i have to re-build it from scratch to make it functional. There isn't any good guideline about rabbitmq capacity planning, every deployment is unique. Anyway thanks for those script i will hook them up with my monitoring system. On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: > > We are hitting something awfully similar. > > We have basically been hitting a few pretty serious bugs with RabbitMQ. > > The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. > > e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 > > I wrote two quick scripts to audit these issues. > http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). > http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. > > The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. > > Best Regards, Erik Olof Gunnar Andersson > > -----Original Message----- > From: Satish Patel > Sent: Wednesday, March 11, 2020 5:14 PM > To: Grant Morley > Cc: openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. > > This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ > > On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: > > > > Hi all, > > > > We are currently experiencing some fairly major issues with our > > OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We > > are seeing a lot of time out messages in responses to replies and > > because of this instance creation or anything to do with instances and > > networking is broken. > > > > We are running OpenStack Queens. > > > > We have already tuned Rabbit for Neutron by doing the following on neutron: > > > > heartbeat_timeout_threshold = 0 > > rpc_conn_pool_size = 300 > > rpc_thread_pool_size = 2048 > > rpc_response_timeout = 3600 > > rpc_poll_timeout = 60 > > > > ## Rpc all > > executor_thread_pool_size = 64 > > rpc_response_timeout = 3600 > > > > What we are seeing in the error logs for neutron for all services > > (l3-agent, dhcp, linux-bridge etc ) are these timeouts: > > > > https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n > > 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK > > 9aOA$ > > > > We have manually tried to get everything in sync by forcing fail-over > > of the networking which seems to get routers in sync. > > > > We are also seeing that there are a lot of "unacknowledged" messages > > in RabbitMQ for 'q-plugin' in the neutron queues. > > > > Some times restarting of the services on neutron gets these back > > acknowledged again, however the timeouts come back. > > > > The RabbitMQ servers themselves are not loaded at all. All memory, > > file descriptors and errlang processes have plenty of resources available. > > > > We are also seeing a lot of rpc issues: > > > > Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds > > before next attempt. If the server is not down, consider increasing > > the rpc_response_timeout option as Neutron server(s) may be overloaded > > and unable to respond quickly enough.: MessagingTimeout: Timed out > > waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 > > 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc > > [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC > > method release_dhcp_port. Waiting for 3347 seconds before next attempt. > > If the server is not down, consider increasing the > > rpc_response_timeout option as Neutron server(s) may be overloaded and > > unable to respond quickly enough.: MessagingTimeout: Timed out waiting > > for a reply to message ID 7937465f15634fbfa443fe1758a12a9c > > > > Does anyone know if there is anymore tuning to be done at all? > > Upgrading for us at the moment to a newer version isn't really an > > option unfortunately. > > > > Because of our setup, we also have roughly 800 routers enabled and I > > know that will be putting a load on the system. However these problems > > have only started to happen roughly 1 week ago and have steadily got worse. > > > > If anyone has any use cases for this or any more recommendations that > > would be great. > > > > Many thanks, > > > > > From amotoki at gmail.com Thu Mar 12 03:48:54 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Thu, 12 Mar 2020 12:48:54 +0900 Subject: Add pytest, pytest-django and pytest-html to global requirements In-Reply-To: <20200311210827.GA90029@sinanju> References: <20200311190623.rrezlwz6kfoiuf4o@mthode.org> <2f761df2a3db8c080f5cab75710b825242ed79f9.camel@redhat.com> <4636a78af2b78b8d316ff5cd3d2b76a1a2173cd7.camel@redhat.com> <20200311210827.GA90029@sinanju> Message-ID: I am commenting only on horizon specific cases. I believe it helps understanding the situation and discussing the direction. > That being said horizon has always been an exception because django > has special requirements for testing (mainly they publish their testing > framework as an extension for a test frameworks other than stdlib unittest). In > the past it was needed a nose extension and now it looks like that has been > updated to be a pytest exception. I don't see a problem to just morph the old > exception that horizon uses nose to horizon uses pytest if it's really > necessary to test django. Previously we used nose for the test runner, but the Django (default) test runner is used now. It happens because Django support in nose looks unmaintained and the migration to the Django test runner is simplest and we are confident that it is maintained as long as the Django project is live. Only reason we did/could not choose stestr is that there is no integration support for Django applications in stestr. IIUC, roughly speaking, what Django testing framework provides are: (1) convenient methods for writing tests (for example, custom assertion, sending requests to Django framework and so on) (2) setup Django for testing mainly including loading Django settings (3) Fixtures for Django Database integration (which is not used in horizon) (4) provide Django test runner (1) is useful for test writers and (2) is required as Django settings module is common in Django projects. (1) and (2) are things re-implemented by individual projects like horizon, and (2) is what pytest-django and nose Django plugin do. (2) is missing in stestr. We already have test runners for Django: the Django default test runner and pytest (with pytest-django) and they are maintained well, so from the horizon team perspective it is better to use an existing one. The downside of the Django default test runner is there seems no good way to handle test results. It has no subunit support. It looks like that only stdout is a place to see test results. I think this is the reason Oleksii proposed pytest usage. The above is my understanding around Django testing. I have no preference on a test runner in horizon. If someone can work on stestr Django integration, it would be great. It provides more consistency with other OpenStack projects. If pytest(-django) is adopted I would like to see zuul role(s) along with the proposal. Akihiro Motoki (irc: amotoki) On Thu, Mar 12, 2020 at 6:10 AM Matthew Treinish wrote: > > On Wed, Mar 11, 2020 at 10:32:32PM +0200, Oleksii Petrenko wrote: > > > > > Starting with stestr, could you explain why it was not good enough for > > > > > your use case? > > > > > > Stestr will not provide us with fixtures for django (for future use), > > also with the help of pytest, we probably would be able to unify html > > creation all across our projects. Also, xml exporting in different > > formats can help users with aggregating test statistics. > > The aggregated data view already exists: > > http://status.openstack.org/openstack-health/#/ > > We also have 2 different html views of a test run depending on the level of > detail you want: > > https://7dd927a4891851ac968e-517bfbb0b76f5445108257ba8a306671.ssl.cf5.rackcdn.com/712315/2/check/tempest-full-py3/c20f9f1/testr_results.html > and > https://7dd927a4891851ac968e-517bfbb0b76f5445108257ba8a306671.ssl.cf5.rackcdn.com/712315/2/check/tempest-full-py3/c20f9f1/controller/logs/stackviz/index.html#/stdin/timeline > > As for "xml exporting" I assume you're talking about xunitxml. There are > several limitations around it, especially for parallel test execution > which is why stestr is built around and uses subunit. But, if you want to > generate xunitxml from subunit for any reason this is straightforward to > do, it's built into subunit: > > https://github.com/testing-cabal/subunit/blob/master/filters/subunit2junitxml > > > > > > > more of a general question if they are test only deps that wont be used at runtime which i think is the case in all of > > > > the above do they enven need to be in Global-requirements? ignoring the fact that devstack installes all test- > > > > requireemtes when it in stall packages which is a different topic if this is only used for generating html report for > > > > tests then it seams liek we would not need to corrdiate the software version. > > pytest is needed to generate coverage reports. > > I don't understand this either, we have coverage jobs already running on most > projects. The reports get published as part of the job artifacts: > > https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fd1/706509/2/check/openstack-tox-cover/fd1df05/cover/index.html > > > Also as I pointed out on the review, this is not the first time we've > discussed this. Since I started working on OpenStack why not runner or > framework X (at one point it was nose, then it switched to pytest) has > been brought up by someone. We tried to write it down in the project > testing interface: > > https://governance.openstack.org/tc/reference/pti/python.html#python-test-running > > Basically, by using a unittest based runner anybody can use their preferred > test runner locally. stestr is used for CI because of the parallel execution > and subunit integration to leverage all the infra tooling built around it. > > That being said horizon has always been an exception because django > has special requirements for testing (mainly they publish their testing > framework as an extension for a test frameworks other than stdlib unittest). In > the past it was needed a nose extension and now it looks like that has been > updated to be a pytest exception. I don't see a problem to just morph the old > exception that horizon uses nose to horizon uses pytest if it's really > necessary to test django. > > If you do end up using pytest because there is no other choice for django > testing, you can convert the xunitxml to subunit to integrate it into all > those existing tools I mentioned before with either: > > https://github.com/mtreinish/health-helm/blob/master/junitxml2subunit.py > or > https://github.com/mtreinish/junitxml2subunit > > (do note stackviz and subunit2sql/openstack-health won't be really useful > with xunitxml to subunit conversion because xunitxml doesn't track > execution timestamps) > > -Matt Treinish From arnaud.morin at gmail.com Thu Mar 12 06:46:49 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Thu, 12 Mar 2020 06:46:49 +0000 Subject: [neutron][largescale-sig] Debugging and tracking missing flows with l2pop In-Reply-To: <6A0F6E0F-9D6E-4ED2-B4AC-F862885220B4@syntaxhighlighted.com> References: <6A0F6E0F-9D6E-4ED2-B4AC-F862885220B4@syntaxhighlighted.com> Message-ID: <20200312064649.GI29109@sync> Hey Krzysztof, In my company we dont use l2pop, I remember that it has some downsides when scaling a lot (more that 1k computes in a region) but I dont remember the details. Anyway, our agent is based on an OVS Agent, which is also using OpenFlow rules. We do monitor the openflow rules out of neutron with custom tools. We do that mainly for 2 reasons: - we want to make sure that neutron wont leak any rule, this could be very harmful - we want to make sure that neutron did not miss any rule when configuring a specific port, which could lead a broken network connection for our clients. We track the missing openflow rules on the compute itself, because we dont want to rely on a centralized system for that. So, to do that, we found a way to pull information about ports on the compute itself, from neutron server and database. Cheers, -- Arnaud Morin On 11.03.20 - 14:29, Krzysztof Klimonda wrote: > Hi, > > (This is stein deployment with 14.0.2 neutron release) > > I’ve just spent some time debugging a missing connection between two VMs running on OS stein with ovs+l2pop enabled and the direct cause was missing flows in table 20 and a very incomplete flood flow in table 22. Restarting neutron-openvswitch-agent on that host has fixed the issue. > > Last time we’ve encountered missing flood flows (in another pike-based deployment), we tracked it down to https://review.opendev.org/#/c/600151/ and since then it was stable. > > My initial thought was that we were hitting the same bug - a couple of VMs are scheduled on the same compute, 3 ports are activated at the same time, and the flood entry is not broadcasted to other computes. However that issue was only affecting one of the computes, and it was the only one missing both MAC entries in table 20 and VXLAN tunnels in table 22. > > The only other idea I have is that the compute with missing flows have not received them from rabbitmq, but there I see nothing in logs that would suggest that agent was disconnected from rabbitmq. > > So at this point I have three questions: > > - what would be a good place to look next to track down those missing flows > - for other operators, how stable do you find l2pop in general? and if you have problems with missing flows in your environment, do you try to monitor your deployment for that? > > -Chris From zhipengh512 at gmail.com Thu Mar 12 07:55:07 2020 From: zhipengh512 at gmail.com (Zhipeng Huang) Date: Thu, 12 Mar 2020 15:55:07 +0800 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: I have no particular objection about removing these particular two previous active cores, however I do concern that when we start to build a new precedence, we should do it right which means we should have an agreed set of metrics that provides the objective qualification of the "core removal" process. The original proposed qualification is "18 months no participation in meetings, no code contributions and no reviews", I would like that we could make the clarification that: - Is it a consecutive 18 months period with the construed "absence criteria" met ? - For the "absence criteria", could we settle upon a set of exhaustive metrics: no meeting, no code contribution, no review, no email discussion participation, anything more ? - If there were a set of agreed "absence criteria"s, what are the logical connection between these pre-conditions ? Is it an "AND" (all of the pre-conditions shall be satisfied) or just "OR" (only one of the pre-conditions satisfies) Once we have a concrete rule setup, we are good to go with a current core reviewer vote for the record of removing, as far as I understand :) Due process is very important. On Thu, Mar 12, 2020 at 8:40 AM Nadathur, Sundar wrote: > > > From: Sean Mooney > > Sent: Wednesday, March 11, 2020 9:37 AM > > > > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > > > I'm a little bit concerned over the 18 months period. The original > > > rule we setup is volunteer step down, since this is a small team we > > > want to acknowledge everyone that has made significant contributions. > > > Some of the inactive core reviewers like Justin Kilpatrick have moved > > > on a long time ago, and I don't see people like him could do any harm > to > > the project. > > > > > > But if the core reviewer has a size limit in the system, that would be > > > reasonable to replace the inactive ones with the new recruits :) > > it is generally considerd best pratice to maintian the core team adding > or > > removing people based on there activity. if a core is removed due to in > > activity and they come back they can always be restored but it give a bad > > perception if a project has like 20 core but only 2 are active. as a new > > contibutor you dont know which ones are active and it can be frustrating > to > > reach out to them and get no responce. > > also just form a project healt point of view it make the project look > like its > > more diverse or more active then it actully is which is also not > generally a > > good thing. > > > > that said core can step down if they feel like they can contribute time > > anymore when ever they like so and if a core is steping a way for a few > > months but intends to come back they can also say that in advance and > there > > is no harm in leaving them for a cycle or two but in general after a > period of > > in activity (usally more then a full release/6months) i think its good > to reduce > > back down the core team. > > > > > > Just my two cents > > As of now, Cyborg core team officially has 12 members [1]. That is hardly > small. > > Justin Kilpatrick seems to be gone for good; he didn't respond to my > emails. Rushil Chugh confirmed that he is not working on OpenStack anymore > and asked to step down as core. With due thanks to him for his > contributions, I'll go ahead. > > Those are the two cores I had in mind. Agree with Sean that it is better > to keep the list of core reviewers up to date. With all the changes in > Cyborg over the past 18 months, it will be tough for a person to jump in > after a long hiatus and resume as a core reviewer. Even if they want to > come back, it is better for them to come up to speed first. > > Given this background, if there is any objection to the removal of these > two cores, please let me know. > > [1] https://review.opendev.org/#/admin/groups/1243,members > > Regards, > Sundar > > > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > > > > > > wrote: > > > > > > > Hello all, > > > > Brin Zhang has been actively contributing to Cyborg in various > > > > areas, adding new features, improving quality, reviewing patches, > > > > and generally helping others in the community. Despite the > > > > relatively short time, he has been one of the most prolific > > > > contributors, and brings an enthusiastic and active mindset. I would > > > > like to thank him and acknowledge his significant contributions by > > proposing him as a core reviewer for Cyborg. > > > > > > > > Shogo Saito has been active in Cyborg since Train release. He has > > > > been driving the Cyborg client improvements, including its revamp to > > > > use OpenStackSDK. Previously he was instrumental in the transition > > > > to Python 3, testing and fixing issues in the process. As he has > > > > access to real FPGA hardware, he brings a users’ perspective and > > > > also tests Cyborg with real hardware. I would like to thank and > > > > acknowledge him for his steady valuable contributions, and propose > him > > as a core reviewer for Cyborg. > > > > > > > > Some of the currently listed core reviewers have not been > > > > participating for a lengthy period of time. It is proposed that > > > > those who have had no contributions for the past 18 months – i.e. no > > > > participation in meetings, no code contributions and no reviews – be > > > > removed from the list of core reviewers. > > > > > > > > If no objections are made known by March 20, I will make the changes > > > > proposed above. > > > > > > > > Thanks. > > > > > > > > Regards, > > > > Sundar > > -- Zhipeng (Howard) Huang Principle Engineer OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, DMTF, W3C -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Thu Mar 12 09:54:44 2020 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 12 Mar 2020 10:54:44 +0100 Subject: [all] Removing defunct meeting records In-Reply-To: <20200312010333.auxhcao54e6gbf42@yuggoth.org> References: <20200312010333.auxhcao54e6gbf42@yuggoth.org> Message-ID: Jeremy Stanley wrote: > On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: > [...] >> we have too many meetings (76, in case you were wondering), too >> much energy spent running them, too much frustration when nobody >> joins. > [...] > > Here's a list of 25 currently defined meetings which have not been > held in 2020 (though it's possible some are being held with a > different meeting_id passed to #startmeeting than is listed in the > meeting record): > > CloudKitty Team Meeting > Congress Team Meeting > Containers Team Meeting > Documentation Team Meeting > First Contact SIG Meeting > Freezer Meeting > Glance Bug Squad Meeting > Group Based Policy Team Meeting > Heat (Orchestration) Team Meeting > I18N Team Meeting > Interop Working Group Meeting > Kuryr Project Office Hours > LOCI Development Meeting > Mistral Meeting > Networking VPP team meeting > OpenStack Charms > Placement Team Office Hour > PowerVM Driver Meeting > Public Cloud SIG > Searchlight Team Meeting > Telemetry Team Meeting > Trove (DBaaS) Team Meeting > Upgrades SIG > Vitrage Team Meeting > Zaqar Team Meeting Note that I already filed for removal of those which did not happen for over a year: https://review.opendev.org/#/q/topic:abandoned-meetings-q1-2020 -- Thierry Carrez (ttx) From ignaziocassano at gmail.com Thu Mar 12 10:38:44 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 12 Mar 2020 11:38:44 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch Message-ID: Hello All, I am facing some problems migrating from iptables_hybrid frirewall to openvswitch firewall on centos 7 queens, I am doing this because I want enable security groups logs which require openvswitch firewall. I would like to migrate without restarting my instances. I startded moving all instances from compute node 1. Then I configured openvswitch firewall on compute node 1, Instances migrated from compute node 2 to compute node 1 without problems. Once the compute node 2 was empty, I migrated it to openvswitch. But now instances does not migrate from node 1 to node 2 because it requires the presence of qbr bridge on node 2 This happened because migrating instances from node2 with iptables_hybrid to compute node 1 with openvswitch, does not put the tap under br-int as requested by openvswich firewall, but qbr is still present on compute node 1. Once I enabled openvswitch on compute node 2, migration from compute node 1 fails because it exprects qbr on compute node 2 . So I think I should moving on the fly tap interfaces from qbr to br-int on compute node 1 before migrating to compute node 2 but it is a huge work on a lot of instances. Any workaround, please ? Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Mar 12 11:48:26 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 12 Mar 2020 11:48:26 +0000 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: On Thu, 2020-03-12 at 15:55 +0800, Zhipeng Huang wrote: > I have no particular objection about removing these particular two previous > active cores, however I do concern that when we start to build a new > precedence, we should do it right which means we should have an agreed set > of metrics that provides the objective qualification of the "core removal" > process. > > The original proposed qualification is "18 months no participation in > meetings, no code contributions and no reviews", I would like that we could > make the clarification that: > > - Is it a consecutive 18 months period with the construed "absence > criteria" met ? i would think 18 months is slightly too long, it certely should not be longer then that. between 12 and 18 feels right to me. after about 2 cycle things can have changed significantly after 3 even more so. 6 monts feels way to sort but 12 to 18 i think is about right. > - For the "absence criteria", could we settle upon a set of exhaustive > metrics: no meeting, no code contribution, no review, no email discussion > participation, anything more ? the only metric for being a core rerviewer for being a core review should be based on well code reviews. code contibution without review should not be a consideration to keep core reviewer status. meeting, irc and email is also not really relevent with one exception. if cyborg was to do a virtual pre-ptg where specs desigin was disucssed and review on the mailing list via eamil, placmenet and nova have tried this in the last cycle or two, then i would consider that the same as gerrit review. i should qulify this that the metric should not be based on the number of reviews alone but rather how how detailed and well reasoned the comments are should be a large factor. a large number of +1 with no comment is generally an anti patteren fro considering some as a core. asking questions to clarify the design choices and confrim the authors intent and your understanding are in sync and make sense is perfectly valid to do while also +1 because you belive in your view the patch is correct and should be encurraged over +1 and no comment. the +1/-1 ratio should also be a factor. its if someone always +1s and never -1 they likely are not reviewing correctly other factors such as email participation, meeting attendence, irc presence or other community partisatpation are supporting factors that suggest a good candiate for becomeing a core but on there own should not be a vaild critia for granting or retaining core reviewer role in a project. my understanding of the above is derived form the definition of what a core review is https://docs.openstack.org/project-team-guide/open-development.html#core-reviewers the review critia https://docs.openstack.org/project-team-guide/open-development.html#reviews-guidelines https://docs.openstack.org/project-team-guide/review-the-openstack-way.html and my general experience with contributing to different openstack project. The core reviewer role withing a project while similar in some repects to a maintianer role in other opensouce models is not the same. a maintainers role tends to focus more on code authorship in addtion to review which is not a factor in the core reviewer role in openstack. if you never write a singel line of code but make detail and technically correct reviews in openstack that makes you an amazing core reviewer. conversly closing 100 bugs in a release with commits and doing no code review would make you a good maintainer but a bad core reviewer, you would be an invaluable contibutor for all your bug fixing work but no a good reviewer which s the focus of the core reviewer role. > - If there were a set of agreed "absence criteria"s, what are the logical > connection between these pre-conditions ? Is it an "AND" (all of the > pre-conditions shall be satisfied) or just "OR" (only one of the > pre-conditions satisfies) > > Once we have a concrete rule setup, we are good to go with a current core > reviewer vote for the record of removing, as far as I understand :) well any core can step down without any kind of vote at any time. they just need to go to https://review.opendev.org/#/admin/groups/1243,members tick there name and remove them selvs and well tell the rest of the team so they know. unless the person being removed object or there is an object to the 2 people being proposed by one of the core team i don't think there is a reason to wait in this case but that is up to the core team to decide. > > Due process is very important. actually i would think that haveing concreate rules like this probably are not useful but if you do write them down you should stick with them. > > On Thu, Mar 12, 2020 at 8:40 AM Nadathur, Sundar > wrote: > > > > > > From: Sean Mooney > > > Sent: Wednesday, March 11, 2020 9:37 AM > > > > > > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > > > > > I'm a little bit concerned over the 18 months period. The original > > > > rule we setup is volunteer step down, since this is a small team we > > > > want to acknowledge everyone that has made significant contributions. > > > > Some of the inactive core reviewers like Justin Kilpatrick have moved > > > > on a long time ago, and I don't see people like him could do any harm > > > > to > > > the project. > > > > > > > > But if the core reviewer has a size limit in the system, that would be > > > > reasonable to replace the inactive ones with the new recruits :) > > > > > > it is generally considerd best pratice to maintian the core team adding > > > > or > > > removing people based on there activity. if a core is removed due to in > > > activity and they come back they can always be restored but it give a bad > > > perception if a project has like 20 core but only 2 are active. as a new > > > contibutor you dont know which ones are active and it can be frustrating > > > > to > > > reach out to them and get no responce. > > > also just form a project healt point of view it make the project look > > > > like its > > > more diverse or more active then it actully is which is also not > > > > generally a > > > good thing. > > > > > > that said core can step down if they feel like they can contribute time > > > anymore when ever they like so and if a core is steping a way for a few > > > months but intends to come back they can also say that in advance and > > > > there > > > is no harm in leaving them for a cycle or two but in general after a > > > > period of > > > in activity (usally more then a full release/6months) i think its good > > > > to reduce > > > back down the core team. > > > > > > > > Just my two cents > > > > As of now, Cyborg core team officially has 12 members [1]. That is hardly > > small. > > > > Justin Kilpatrick seems to be gone for good; he didn't respond to my > > emails. Rushil Chugh confirmed that he is not working on OpenStack anymore > > and asked to step down as core. With due thanks to him for his > > contributions, I'll go ahead. > > > > Those are the two cores I had in mind. Agree with Sean that it is better > > to keep the list of core reviewers up to date. With all the changes in > > Cyborg over the past 18 months, it will be tough for a person to jump in > > after a long hiatus and resume as a core reviewer. Even if they want to > > come back, it is better for them to come up to speed first. > > > > Given this background, if there is any objection to the removal of these > > two cores, please let me know. > > > > [1] https://review.opendev.org/#/admin/groups/1243,members > > > > Regards, > > Sundar > > > > > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > > > > > > > > wrote: > > > > > > > > > Hello all, > > > > > Brin Zhang has been actively contributing to Cyborg in various > > > > > areas, adding new features, improving quality, reviewing patches, > > > > > and generally helping others in the community. Despite the > > > > > relatively short time, he has been one of the most prolific > > > > > contributors, and brings an enthusiastic and active mindset. I would > > > > > like to thank him and acknowledge his significant contributions by > > > > > > proposing him as a core reviewer for Cyborg. > > > > > > > > > > Shogo Saito has been active in Cyborg since Train release. He has > > > > > been driving the Cyborg client improvements, including its revamp to > > > > > use OpenStackSDK. Previously he was instrumental in the transition > > > > > to Python 3, testing and fixing issues in the process. As he has > > > > > access to real FPGA hardware, he brings a users’ perspective and > > > > > also tests Cyborg with real hardware. I would like to thank and > > > > > acknowledge him for his steady valuable contributions, and propose > > > > him > > > as a core reviewer for Cyborg. > > > > > > > > > > Some of the currently listed core reviewers have not been > > > > > participating for a lengthy period of time. It is proposed that > > > > > those who have had no contributions for the past 18 months – i.e. no > > > > > participation in meetings, no code contributions and no reviews – be > > > > > removed from the list of core reviewers. > > > > > > > > > > If no objections are made known by March 20, I will make the changes > > > > > proposed above. > > > > > > > > > > Thanks. > > > > > > > > > > Regards, > > > > > Sundar > > > > > > From radoslaw.piliszek at gmail.com Thu Mar 12 11:55:55 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 12 Mar 2020 12:55:55 +0100 Subject: [all][dev][qa] cirros 0.5.1 Message-ID: Hiya Folks, as you might have noticed, cinder 0.5.1 has been released. This build seems to be passing the current devstack gate. [1] Big thanks to hrw and smoser for letting cirros 0.5.1 happen (and cirros having bright future yet again). Also thanks to mordred for cleaning up SDK testing to pass. :-) I think it would be nice to merge this in Ussuri still, preferably before April. On the other hand, we all know that devstack gate is not super comprehensive and hence I would like to ask teams whose tests depend on interaction with guest OS to test their gates on this patch (or help me help you do that). I deliberately marked it W-1 to avoid merging too early. Let the discussion begin. :-) [1] https://review.opendev.org/711492 -yoctozepto From mats.karlsson at apistraining.com Thu Mar 12 12:12:38 2020 From: mats.karlsson at apistraining.com (Mats Karlsson) Date: Thu, 12 Mar 2020 12:12:38 +0000 Subject: Missing OVA in GitHub Message-ID: Hi, I’m new in OpenStack and found out that there is a an VM with OpenStack at https://github.com/openstack/upstream-institute-virtual-environment But it look like that the VM (http://bit.ly/vm-2019-shanghai-v1) is missing and I can’t file a support issue in that repo so that’s why I’m asking here. Is this a known issue ? Regards Mats Karlsson Trainer [cid:image002.jpg at 01D4C2DE.FE359640] Rosenlundsgatan 54 SE-118 63 Stockholm, Sweden M: +46 766 967 835 T: +46 8 555 105 15 E:mats.karlsson at apistraining.com W: www.apistraining.com Connect with us: LinkedIn Youtube Facebook Twitter Instagram -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 1657 bytes Desc: image002.jpg URL: From james.denton at rackspace.com Thu Mar 12 12:30:42 2020 From: james.denton at rackspace.com (James Denton) Date: Thu, 12 Mar 2020 12:30:42 +0000 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: Message-ID: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Hi Ignazio, I tested a process that converted iptables_hybrid to openvswitch in-place, but not without a hard reboot of the VM and some massaging of the existing bridges/veths. Since you are live-migrating, though, you might be able to get around that. Regardless, to make this work, I had to update the port’s vif_details in the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > use neutron; > update ml2_port_bindings set vif_details='{"port_filter": true, "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; So, perhaps making that change prior to moving the VM back to the other compute node will do the trick. Good luck! James From: Ignazio Cassano Date: Thursday, March 12, 2020 at 6:41 AM To: openstack-discuss Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello All, I am facing some problems migrating from iptables_hybrid frirewall to openvswitch firewall on centos 7 queens, I am doing this because I want enable security groups logs which require openvswitch firewall. I would like to migrate without restarting my instances. I startded moving all instances from compute node 1. Then I configured openvswitch firewall on compute node 1, Instances migrated from compute node 2 to compute node 1 without problems. Once the compute node 2 was empty, I migrated it to openvswitch. But now instances does not migrate from node 1 to node 2 because it requires the presence of qbr bridge on node 2 This happened because migrating instances from node2 with iptables_hybrid to compute node 1 with openvswitch, does not put the tap under br-int as requested by openvswich firewall, but qbr is still present on compute node 1. Once I enabled openvswitch on compute node 2, migration from compute node 1 fails because it exprects qbr on compute node 2 . So I think I should moving on the fly tap interfaces from qbr to br-int on compute node 1 before migrating to compute node 2 but it is a huge work on a lot of instances. Any workaround, please ? Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Thu Mar 12 12:54:00 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 12 Mar 2020 13:54:00 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Message-ID: Hello James, I will try that. Many thanks Ignazio Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < james.denton at rackspace.com> ha scritto: > Hi Ignazio, > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > use neutron; > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > Good luck! > > > > James > > > > *From: *Ignazio Cassano > *Date: *Thursday, March 12, 2020 at 6:41 AM > *To: *openstack-discuss > *Subject: *[qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > *CAUTION:* This message originated externally, please use caution when > clicking on links or opening attachments! > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > to compute node 1 with openvswitch, does not put the tap under br-int as > requested by openvswich firewall, but qbr is still present on compute node > 1. > > Once I enabled openvswitch on compute node 2, migration from compute node > 1 fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int on > compute node 1 before migrating to compute node 2 but it is a huge work on > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kklimonda at syntaxhighlighted.com Thu Mar 12 13:12:30 2020 From: kklimonda at syntaxhighlighted.com (Krzysztof Klimonda) Date: Thu, 12 Mar 2020 14:12:30 +0100 Subject: [neutron][largescale-sig] Debugging and tracking missing flows with l2pop In-Reply-To: <20200312064649.GI29109@sync> References: <6A0F6E0F-9D6E-4ED2-B4AC-F862885220B4@syntaxhighlighted.com> <20200312064649.GI29109@sync> Message-ID: <2319D79E-5C11-4E7F-A710-977807B894B9@syntaxhighlighted.com> Thanks. Do your tools query neutron for ports, or do you query the database directly? I’m a bit concerned about having ~100 nodes query neutron for a list of ports and flows every minute or so, and how much extra load will that add on our neutron-server. What do you mean by neutron leaking rules? Is it security group rules that you are concerned about? -Chris > On 12 Mar 2020, at 07:46, Arnaud Morin wrote: > > Hey Krzysztof, > > In my company we dont use l2pop, I remember that it has some downsides > when scaling a lot (more that 1k computes in a region) but I dont > remember the details. > > Anyway, our agent is based on an OVS Agent, which is also using OpenFlow > rules. > We do monitor the openflow rules out of neutron with custom tools. > We do that mainly for 2 reasons: > - we want to make sure that neutron wont leak any rule, this could be > very harmful > - we want to make sure that neutron did not miss any rule when > configuring a specific port, which could lead a broken network > connection for our clients. > > We track the missing openflow rules on the compute itself, because we > dont want to rely on a centralized system for that. So, to do that, we > found a way to pull information about ports on the compute itself, from > neutron server and database. > > Cheers, > > -- > Arnaud Morin > > On 11.03.20 - 14:29, Krzysztof Klimonda wrote: >> Hi, >> >> (This is stein deployment with 14.0.2 neutron release) >> >> I’ve just spent some time debugging a missing connection between two VMs running on OS stein with ovs+l2pop enabled and the direct cause was missing flows in table 20 and a very incomplete flood flow in table 22. Restarting neutron-openvswitch-agent on that host has fixed the issue. >> >> Last time we’ve encountered missing flood flows (in another pike-based deployment), we tracked it down to https://review.opendev.org/#/c/600151/ and since then it was stable. >> >> My initial thought was that we were hitting the same bug - a couple of VMs are scheduled on the same compute, 3 ports are activated at the same time, and the flood entry is not broadcasted to other computes. However that issue was only affecting one of the computes, and it was the only one missing both MAC entries in table 20 and VXLAN tunnels in table 22. >> >> The only other idea I have is that the compute with missing flows have not received them from rabbitmq, but there I see nothing in logs that would suggest that agent was disconnected from rabbitmq. >> >> So at this point I have three questions: >> >> - what would be a good place to look next to track down those missing flows >> - for other operators, how stable do you find l2pop in general? and if you have problems with missing flows in your environment, do you try to monitor your deployment for that? >> >> -Chris From zhipengh512 at gmail.com Thu Mar 12 13:20:56 2020 From: zhipengh512 at gmail.com (Zhipeng Huang) Date: Thu, 12 Mar 2020 21:20:56 +0800 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: I like what Sean proposed, and a cycle bound time criteria (2 cycles or 12 months) would be very good, and if we center the quality criteria on meaningful reviews would largely reduced the burden of unnecessary computations. I agree that we should document this and stick to it. For me "12 months + no meaningful review" would be a good enough concrete criteria, for removing the non-active core reviewer in a non-voluntarily step down fashion. On Thu, Mar 12, 2020 at 7:48 PM Sean Mooney wrote: > On Thu, 2020-03-12 at 15:55 +0800, Zhipeng Huang wrote: > > I have no particular objection about removing these particular two > previous > > active cores, however I do concern that when we start to build a new > > precedence, we should do it right which means we should have an agreed > set > > of metrics that provides the objective qualification of the "core > removal" > > process. > > > > The original proposed qualification is "18 months no participation in > > meetings, no code contributions and no reviews", I would like that we > could > > make the clarification that: > > > > - Is it a consecutive 18 months period with the construed "absence > > criteria" met ? > i would think 18 months is slightly too long, it certely should not be > longer then that. > between 12 and 18 feels right to me. after about 2 cycle things can have > changed significantly > after 3 even more so. 6 monts feels way to sort but 12 to 18 i think is > about right. > > - For the "absence criteria", could we settle upon a set of exhaustive > > metrics: no meeting, no code contribution, no review, no email discussion > > participation, anything more ? > the only metric for being a core rerviewer for being a core review should > be based on well code reviews. > code contibution without review should not be a consideration to keep core > reviewer status. > meeting, irc and email is also not really relevent with one exception. if > cyborg was to do a virtual pre-ptg where specs > desigin was disucssed and review on the mailing list via eamil, placmenet > and nova have tried this in the last cycle or > two, then i would consider that the same as gerrit review. > > i should qulify this that the metric should not be based on the number of > reviews alone but rather how how detailed > and well reasoned the comments are should be a large factor. a large > number of +1 with no comment is generally an anti > patteren fro considering some as a core. asking questions to clarify the > design choices and confrim the authors intent > and your understanding are in sync and make sense is perfectly valid to do > while also +1 because you belive in your view > the patch is correct and should be encurraged over +1 and no comment. > > the +1/-1 ratio should also be a factor. its if someone always +1s and > never -1 they likely are not reviewing correctly > > other factors such as email participation, meeting attendence, irc > presence or other community partisatpation are > supporting factors that suggest a good candiate for becomeing a core but > on there own should not be a vaild critia > for granting or retaining core reviewer role in a project. > > my understanding of the above is derived form the definition of what a > core review is > > https://docs.openstack.org/project-team-guide/open-development.html#core-reviewers > the review critia > https://docs.openstack.org/project-team-guide/open-development.html#reviews-guidelines > https://docs.openstack.org/project-team-guide/review-the-openstack-way.html > and my general experience with contributing to different openstack project. > The core reviewer role withing a project while similar in some repects to > a maintianer role in other opensouce models > is not the same. a maintainers role tends to focus more on code authorship > in addtion to review which is not a factor in > the core reviewer role in openstack. if you never write a singel line of > code but make detail and technically correct > reviews in openstack that makes you an amazing core reviewer. conversly > closing 100 bugs in a release with commits and > doing no code review would make you a good maintainer but a bad core > reviewer, you would be an invaluable contibutor for > all your bug fixing work but no a good reviewer which s the focus of the > core reviewer role. > > > - If there were a set of agreed "absence criteria"s, what are the logical > > connection between these pre-conditions ? Is it an "AND" (all of the > > pre-conditions shall be satisfied) or just "OR" (only one of the > > pre-conditions satisfies) > > > > Once we have a concrete rule setup, we are good to go with a current core > > reviewer vote for the record of removing, as far as I understand :) > well any core can step down without any kind of vote at any time. > they just need to go to > https://review.opendev.org/#/admin/groups/1243,members > tick there name and remove them selvs and well tell the rest of the team > so they know. > > unless the person being removed object or there is an object to the 2 > people being proposed by one > of the core team i don't think there is a reason to wait in this case but > that is up to the core team to decide. > > > > Due process is very important. > actually i would think that haveing concreate rules like this probably are > not useful but if you > do write them down you should stick with them. > > > > On Thu, Mar 12, 2020 at 8:40 AM Nadathur, Sundar < > sundar.nadathur at intel.com> > > wrote: > > > > > > > > > From: Sean Mooney > > > > Sent: Wednesday, March 11, 2020 9:37 AM > > > > > > > > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > > > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > > > > > > > I'm a little bit concerned over the 18 months period. The original > > > > > rule we setup is volunteer step down, since this is a small team we > > > > > want to acknowledge everyone that has made significant > contributions. > > > > > Some of the inactive core reviewers like Justin Kilpatrick have > moved > > > > > on a long time ago, and I don't see people like him could do any > harm > > > > > > to > > > > the project. > > > > > > > > > > But if the core reviewer has a size limit in the system, that > would be > > > > > reasonable to replace the inactive ones with the new recruits :) > > > > > > > > it is generally considerd best pratice to maintian the core team > adding > > > > > > or > > > > removing people based on there activity. if a core is removed due to > in > > > > activity and they come back they can always be restored but it give > a bad > > > > perception if a project has like 20 core but only 2 are active. as > a new > > > > contibutor you dont know which ones are active and it can be > frustrating > > > > > > to > > > > reach out to them and get no responce. > > > > also just form a project healt point of view it make the project look > > > > > > like its > > > > more diverse or more active then it actully is which is also not > > > > > > generally a > > > > good thing. > > > > > > > > that said core can step down if they feel like they can contribute > time > > > > anymore when ever they like so and if a core is steping a way for a > few > > > > months but intends to come back they can also say that in advance and > > > > > > there > > > > is no harm in leaving them for a cycle or two but in general after a > > > > > > period of > > > > in activity (usally more then a full release/6months) i think its > good > > > > > > to reduce > > > > back down the core team. > > > > > > > > > > Just my two cents > > > > > > As of now, Cyborg core team officially has 12 members [1]. That is > hardly > > > small. > > > > > > Justin Kilpatrick seems to be gone for good; he didn't respond to my > > > emails. Rushil Chugh confirmed that he is not working on OpenStack > anymore > > > and asked to step down as core. With due thanks to him for his > > > contributions, I'll go ahead. > > > > > > Those are the two cores I had in mind. Agree with Sean that it is > better > > > to keep the list of core reviewers up to date. With all the changes in > > > Cyborg over the past 18 months, it will be tough for a person to jump > in > > > after a long hiatus and resume as a core reviewer. Even if they want to > > > come back, it is better for them to come up to speed first. > > > > > > Given this background, if there is any objection to the removal of > these > > > two cores, please let me know. > > > > > > [1] https://review.opendev.org/#/admin/groups/1243,members > > > > > > Regards, > > > Sundar > > > > > > > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > > > > > > > > > > wrote: > > > > > > > > > > > Hello all, > > > > > > Brin Zhang has been actively contributing to Cyborg in > various > > > > > > areas, adding new features, improving quality, reviewing patches, > > > > > > and generally helping others in the community. Despite the > > > > > > relatively short time, he has been one of the most prolific > > > > > > contributors, and brings an enthusiastic and active mindset. I > would > > > > > > like to thank him and acknowledge his significant contributions > by > > > > > > > > proposing him as a core reviewer for Cyborg. > > > > > > > > > > > > Shogo Saito has been active in Cyborg since Train release. He has > > > > > > been driving the Cyborg client improvements, including its > revamp to > > > > > > use OpenStackSDK. Previously he was instrumental in the > transition > > > > > > to Python 3, testing and fixing issues in the process. As he has > > > > > > access to real FPGA hardware, he brings a users’ perspective and > > > > > > also tests Cyborg with real hardware. I would like to thank and > > > > > > acknowledge him for his steady valuable contributions, and > propose > > > > > > him > > > > as a core reviewer for Cyborg. > > > > > > > > > > > > Some of the currently listed core reviewers have not been > > > > > > participating for a lengthy period of time. It is proposed that > > > > > > those who have had no contributions for the past 18 months – > i.e. no > > > > > > participation in meetings, no code contributions and no reviews > – be > > > > > > removed from the list of core reviewers. > > > > > > > > > > > > If no objections are made known by March 20, I will make the > changes > > > > > > proposed above. > > > > > > > > > > > > Thanks. > > > > > > > > > > > > Regards, > > > > > > Sundar > > > > > > > > > > > > -- Zhipeng (Howard) Huang Principle Engineer OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, DMTF, W3C -------------- next part -------------- An HTML attachment was scrubbed... URL: From tdecacqu at redhat.com Thu Mar 12 13:23:41 2020 From: tdecacqu at redhat.com (Tristan Cacqueray) Date: Thu, 12 Mar 2020 13:23:41 +0000 Subject: [tripleo] In-Reply-To: References: <1B1A2143-A018-408D-9515-A367CEA952B5@inaugust.com> Message-ID: <87h7yt7llu.tristanC@fedora> On Tue, Mar 10, 2020 at 19:38 Emilien Macchi wrote: > On Tue, Mar 10, 2020 at 10:41 AM Monty Taylor wrote: > >> Yay! >> >> When you have brainspace after firefighting (always fun) - maybe we should >> find a time to talk about whether our image building and publishing >> automation could help you out here. No rush - this is one of those “we’ve >> got some tools we might be able to leverage to help” - just ping me >> whenever. >> > > Hey Monty, > > The CI team is presently busy with CentOS 8 fires but I would be happy to > help and work together on convergence. > Maybe I can start by explaining how our process works, then you can do the > same and we see where we can collaborate. > > The TL;DR is that we have built TripleO CLI and Ansible roles to consume > Kolla tooling and build our images. > For what its worth, we, the software factory project team, would like to investigate using zuul pipeline to periodically update, test and promote a collection of images. Note that the goal is to update and promote only valid layers (instead of a doing a full rebuild each time). We actually plan to work on that story[0] in the upcoming weeks, it seems like zuul-jobs already feature most of the image building roles we would need, but we might require some modifications to be able to detect if a layer needs to be tested (e.g. looks for "Nothing to do." in stdout) Perhaps we can adapt the zuul-jobs role in such a way that it would support the update use-case as well as using TripleO CLI and roles. Cheers, -Tristan [0] https://tree.taiga.io/project/morucci-software-factory/us/3419 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From Arkady.Kanevsky at dell.com Thu Mar 12 14:05:25 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Thu, 12 Mar 2020 14:05:25 +0000 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> Message-ID: <168c8eed885645daaa82340403231e65@AUSX13MPS308.AMER.DELL.COM> Agree that going virtual makes most sense given current status From: Emilien Macchi Sent: Wednesday, March 11, 2020 5:46 PM To: Mark Collier Cc: openstack-discuss; Jonathan Bryce Subject: Re: FW: 2020 OSF Events & coronavirus [EXTERNAL EMAIL] Hi Mark, Thanks for the transparency, as usual. I have a few thoughts, please read inline. On Fri, Mar 6, 2020 at 4:04 PM Mark Collier > wrote: upcoming event in Vancouver is no exception. The OpenDev tracks > each morning will be programmed by volunteers from the community, and the project > teams will be organizing their own conversations as well each afternoon M-W, and > all day Thursday. > > But the larger question is here: should the show go on? > > The short answer is that as of now, the Vancouver and Berlin events are still > scheduled to happen in June (8-11) and October (19-23), respectively. > > However, we are willing to cancel or approach the events in a different way (i.e. > virtual) if the facts indicate that is the best path, and we know the facts are > changing rapidly. One of the most critical inputs we need is to hear from each of > you. We know that many of you rely on the twice-annual events to get together and > make rapid progress on the software, which is one reason we are not making any > decisions in haste. We also know that many of you may be unable or unwilling to > travel in June, and that is critical information to hear as we get closer to the > event so that we can make the most informed decision. I believe that we, as a community should show the example and our strengths by cancelling the Vancouver event and organize a virtual event like some other big events are doing. There is an opportunity for the OSF to show leadership in Software communities and acknowledge the risk of spread during that meeting; not only for the people attending it but for also those in contact with these people later. I'm not a doctor nor I know much about the virus; but I'm not interested to travel and take the risk to 1) catch the virus and 2) spread it at home and in my country; and as a community member, I feel like our responsibility is also to maintain ourselves healthy. In my opinion, the sooner we cancel, the better we can focus on organizing the virtual meetings, and also we can influence more communities to take that kind of decisions. Thanks Mark for starting that discussion, it's a perfect sign of how healthy is our community; and hopefully it will continue to be. -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Thu Mar 12 14:37:03 2020 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 12 Mar 2020 14:37:03 +0000 Subject: [ironic] proposing Iury Gregory for bifrost-core, ironic-inspector-core, sushy-core In-Reply-To: References: Message-ID: On Wed, 11 Mar 2020 at 18:55, Julia Kreger wrote: > > Iury has been working hard across the ironic community and has been > quite active in changing and improving our CI, as well as reviewing > code contributions and helpfully pointing out issues or items that > need to be fixed. I feel that he is on track to join ironic-core in > the next few months, but first I propose we add him to bifrost-core, > ironic-inspector-core, and sushy-core. > > Any objections? > +1 From thierry at openstack.org Thu Mar 12 14:37:54 2020 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 12 Mar 2020 15:37:54 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <2e142636-0070-704c-c5f7-1e035bc9d406@openstack.org> References: <2e142636-0070-704c-c5f7-1e035bc9d406@openstack.org> Message-ID: Thierry Carrez wrote: > [...] > So one solution might be: > > - Define multiple roles (release liaison, event liaison, meeting > chair...) and allow them to be filled by the team as they want, for the > duration they want, replaced when they want (would just need +1 from > previous and new holder of the role) > > - Use the TC as a governance safety valve to resolve any conflict > (instead of PTL elections) Proposed as a strawman at: https://review.opendev.org/#/c/712696/ Feel free to comment on how crazy that is there... -- Thierry Carrez (ttx) From sbauza at redhat.com Thu Mar 12 14:45:09 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Thu, 12 Mar 2020 15:45:09 +0100 Subject: [nova][ptg] PTG participation In-Reply-To: <064cda0d6d2511f3cbccd26fbf1bb5460797fbef.camel@redhat.com> References: <1583947033.12170.37@est.tech> <064cda0d6d2511f3cbccd26fbf1bb5460797fbef.camel@redhat.com> Message-ID: On Wed, Mar 11, 2020 at 7:11 PM Sean Mooney wrote: > On Wed, 2020-03-11 at 17:36 +0000, Sean Mooney wrote: > > On Wed, 2020-03-11 at 18:17 +0100, Balázs Gibizer wrote: > > > Hi, > > > > > > I've just got the news from my employer that due to COVID19 I cannot > > > travel to Vancouver in June. > > > > From a redhat perspective i dont think a decision has been made on if we > should attend or not. > ill clarify that slightly in that we do have guidence that "Red Hatters > may not travel to attend external events or > conferences with 1000+ attendees, even within their home country." > > in the past when the ptg and summit were combinined and we had the > devsumit have ment that travel to the openstack even > would not be allowed. At its current size its kind of in a gray zone where > its is not banned as a public event but if it > was an internal event the number of redhat employee that would be > attending woudl be over the limit we have and the > physical event would be canceled and converted to a virtual only event. > > so its tbd if i will be attending too although i have not heard a > definitive No at this point but i also cant really > book tickets and flight yet either however the guidance we have been given > is to try and default to virtual attendance > were possible. > > > last time i spoke to my manager we were still waiting to see how thing > progress with the assumtion we would attend > > but we might want to plan for a virtual PTG (via video conference, > etherpad and email) in the event many cant travel > > or > > that things escalate to a point where the PTG event could be canceled. > > > > i dont think the foundation has indicated that that is likely to happen > but im sure they are monitoring > > things closely as our employers will be so having a plan b might now be > a bad thing in either case. > > if there isnt a diverse quoram physically at the ptg it would limit our > ability to make desisions as happend to > > some extent in china. it would be still good to get operator feedback > but they may also be under similar travel > > restrictions. > > > > I personnally have concerns with virtual events that are large like our PTG : - which timezone should we use for the virtual PTG ? Anyway, Asia, Europe or North America folks could have problems with one. - some folks (like me) have bandwidth issues in their (home) offices. They could be having problems when trying to discuss - how can we be sure that all of us can discuss by a virtual meeting ? Not saying it won't work, but discussions could take more time. Anyway, I'm sad for you, gibi. I'm not -2 about a virtual PTG, I just want to make sure we think about the above concerns before agreeing with any virtual PTG. > > cheers, > > > gibi > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Thu Mar 12 15:50:10 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 12 Mar 2020 16:50:10 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Message-ID: Hello James; I made the db update before live migrating the virtual machine, but the node where I migrated continues to create the qbr bridge also if it is configured with openvswitch firewall. The difference if that if I do not update the database I can ping the migrated vm, while now I cannot Ignazio Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < james.denton at rackspace.com> ha scritto: > Hi Ignazio, > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > use neutron; > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > Good luck! > > > > James > > > > *From: *Ignazio Cassano > *Date: *Thursday, March 12, 2020 at 6:41 AM > *To: *openstack-discuss > *Subject: *[qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > *CAUTION:* This message originated externally, please use caution when > clicking on links or opening attachments! > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > to compute node 1 with openvswitch, does not put the tap under br-int as > requested by openvswich firewall, but qbr is still present on compute node > 1. > > Once I enabled openvswitch on compute node 2, migration from compute node > 1 fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int on > compute node 1 before migrating to compute node 2 but it is a huge work on > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mthode at mthode.org Thu Mar 12 16:00:33 2020 From: mthode at mthode.org (Matthew Thode) Date: Thu, 12 Mar 2020 11:00:33 -0500 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library Message-ID: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> I'd like to suggest, now that we are on modern python, that we stop using the mock library and instead use the built in mock library. unittest.mock has been around since python-3.3 so we should not have a problem with python support. This would allow us to drop a library and also help prevent issues with stuff like https://review.opendev.org/712713 is going to expose (nova/neutron fail tests with the external mock-4 library). -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From dtantsur at redhat.com Thu Mar 12 16:15:40 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Thu, 12 Mar 2020 17:15:40 +0100 Subject: [all][dev][qa] cirros 0.5.1 In-Reply-To: References: Message-ID: Testing ironic: https://review.opendev.org/#/c/712728/ Thanks for the heads-up! Dmitry On Thu, Mar 12, 2020 at 12:57 PM Radosław Piliszek < radoslaw.piliszek at gmail.com> wrote: > Hiya Folks, > > as you might have noticed, cinder 0.5.1 has been released. > This build seems to be passing the current devstack gate. [1] > Big thanks to hrw and smoser for letting cirros 0.5.1 happen (and > cirros having bright future yet again). > Also thanks to mordred for cleaning up SDK testing to pass. :-) > > I think it would be nice to merge this in Ussuri still, preferably before > April. > On the other hand, we all know that devstack gate is not super > comprehensive and hence I would like to ask teams whose tests depend > on interaction with guest OS to test their gates on this patch (or > help me help you do that). > I deliberately marked it W-1 to avoid merging too early. > > Let the discussion begin. :-) > > [1] https://review.opendev.org/711492 > > -yoctozepto > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.denton at rackspace.com Thu Mar 12 16:18:21 2020 From: james.denton at rackspace.com (James Denton) Date: Thu, 12 Mar 2020 16:18:21 +0000 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Message-ID: Hi Ignazio, > but the node where I migrated continues to create the qbr bridge also if it is configured with openvswitch firewall. I assume the neutron-openvswitch-agent has been restarted since making the firewall_driver change? What happens if you create a new VM on that compute? > The difference if that if I do not update the database I can ping the migrated vm, while now I cannot Can you look at br-int and see if your tap interface is connected without a vlan tag? Or is the tap still connected to the qbr bridge? If the latter, were any iptables rules created? Unfortunately, I don’t have the ability to test this w/ live migration. James From: Ignazio Cassano Date: Thursday, March 12, 2020 at 11:50 AM To: James Denton Cc: openstack-discuss Subject: Re: [qeeens][neutron] migrating from iptables_hybrid to openvswitch CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello James; I made the db update before live migrating the virtual machine, but the node where I migrated continues to create the qbr bridge also if it is configured with openvswitch firewall. The difference if that if I do not update the database I can ping the migrated vm, while now I cannot Ignazio Il giorno gio 12 mar 2020 alle ore 13:30 James Denton > ha scritto: Hi Ignazio, I tested a process that converted iptables_hybrid to openvswitch in-place, but not without a hard reboot of the VM and some massaging of the existing bridges/veths. Since you are live-migrating, though, you might be able to get around that. Regardless, to make this work, I had to update the port’s vif_details in the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > use neutron; > update ml2_port_bindings set vif_details='{"port_filter": true, "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; So, perhaps making that change prior to moving the VM back to the other compute node will do the trick. Good luck! James From: Ignazio Cassano > Date: Thursday, March 12, 2020 at 6:41 AM To: openstack-discuss > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello All, I am facing some problems migrating from iptables_hybrid frirewall to openvswitch firewall on centos 7 queens, I am doing this because I want enable security groups logs which require openvswitch firewall. I would like to migrate without restarting my instances. I startded moving all instances from compute node 1. Then I configured openvswitch firewall on compute node 1, Instances migrated from compute node 2 to compute node 1 without problems. Once the compute node 2 was empty, I migrated it to openvswitch. But now instances does not migrate from node 1 to node 2 because it requires the presence of qbr bridge on node 2 This happened because migrating instances from node2 with iptables_hybrid to compute node 1 with openvswitch, does not put the tap under br-int as requested by openvswich firewall, but qbr is still present on compute node 1. Once I enabled openvswitch on compute node 2, migration from compute node 1 fails because it exprects qbr on compute node 2 . So I think I should moving on the fly tap interfaces from qbr to br-int on compute node 1 before migrating to compute node 2 but it is a huge work on a lot of instances. Any workaround, please ? Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Thu Mar 12 16:23:29 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Thu, 12 Mar 2020 16:23:29 +0000 Subject: [neutron][largescale-sig] Debugging and tracking missing flows with l2pop In-Reply-To: <2319D79E-5C11-4E7F-A710-977807B894B9@syntaxhighlighted.com> References: <6A0F6E0F-9D6E-4ED2-B4AC-F862885220B4@syntaxhighlighted.com> <20200312064649.GI29109@sync> <2319D79E-5C11-4E7F-A710-977807B894B9@syntaxhighlighted.com> Message-ID: <20200312162329.GJ29109@sync> Hello, We had the same concerns as you, so that's why, we have another API which is in front of our databases (on read node). This API is designed to do read-only in DB, and bypass openstack API. We do that because sometimes the OpenStack API calls are less performants or does not give the info in a way we would like. So to avoid doing multiples calls to retrieve one info, we built this custom internal API. Our tool to check the OpenFlow rules on the compute is calling this API to get the neutron ports info. By neutron leaking rules, I mean openflow rules. Cheers, -- Arnaud Morin On 12.03.20 - 14:12, Krzysztof Klimonda wrote: > Thanks. > > Do your tools query neutron for ports, or do you query the database directly? I’m a bit concerned about having ~100 nodes query neutron for a list of ports and flows every minute or so, and how much extra load will that add on our neutron-server. > > What do you mean by neutron leaking rules? Is it security group rules that you are concerned about? > > -Chris > > > On 12 Mar 2020, at 07:46, Arnaud Morin wrote: > > > > Hey Krzysztof, > > > > In my company we dont use l2pop, I remember that it has some downsides > > when scaling a lot (more that 1k computes in a region) but I dont > > remember the details. > > > > Anyway, our agent is based on an OVS Agent, which is also using OpenFlow > > rules. > > We do monitor the openflow rules out of neutron with custom tools. > > We do that mainly for 2 reasons: > > - we want to make sure that neutron wont leak any rule, this could be > > very harmful > > - we want to make sure that neutron did not miss any rule when > > configuring a specific port, which could lead a broken network > > connection for our clients. > > > > We track the missing openflow rules on the compute itself, because we > > dont want to rely on a centralized system for that. So, to do that, we > > found a way to pull information about ports on the compute itself, from > > neutron server and database. > > > > Cheers, > > > > -- > > Arnaud Morin > > > > On 11.03.20 - 14:29, Krzysztof Klimonda wrote: > >> Hi, > >> > >> (This is stein deployment with 14.0.2 neutron release) > >> > >> I’ve just spent some time debugging a missing connection between two VMs running on OS stein with ovs+l2pop enabled and the direct cause was missing flows in table 20 and a very incomplete flood flow in table 22. Restarting neutron-openvswitch-agent on that host has fixed the issue. > >> > >> Last time we’ve encountered missing flood flows (in another pike-based deployment), we tracked it down to https://review.opendev.org/#/c/600151/ and since then it was stable. > >> > >> My initial thought was that we were hitting the same bug - a couple of VMs are scheduled on the same compute, 3 ports are activated at the same time, and the flood entry is not broadcasted to other computes. However that issue was only affecting one of the computes, and it was the only one missing both MAC entries in table 20 and VXLAN tunnels in table 22. > >> > >> The only other idea I have is that the compute with missing flows have not received them from rabbitmq, but there I see nothing in logs that would suggest that agent was disconnected from rabbitmq. > >> > >> So at this point I have three questions: > >> > >> - what would be a good place to look next to track down those missing flows > >> - for other operators, how stable do you find l2pop in general? and if you have problems with missing flows in your environment, do you try to monitor your deployment for that? > >> > >> -Chris > From ignaziocassano at gmail.com Thu Mar 12 16:26:43 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 12 Mar 2020 17:26:43 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Message-ID: O yes, I restarted the openvswitch to make the driver change. I did not see into openvswitch I ran: virsh domiflist instancename and I saw the instance interface was under qbr (and this is the problem), I aldo tried on the migrated node to detach the interface from qbr and attach it under br-int like suggested with the method suggested at this link: https://docs.openstack.org/neutron/pike/contributor/internals/openvswitch_firewall.html But virsh does not allow to change the bridge on the fly Ignazio Il giorno gio 12 mar 2020 alle ore 17:18 James Denton < james.denton at rackspace.com> ha scritto: > Hi Ignazio, > > > > > but the node where I migrated continues to create the qbr bridge also if > it is configured with openvswitch firewall. > > > > I assume the neutron-openvswitch-agent has been restarted since making the > firewall_driver change? What happens if you create a new VM on that compute? > > > > > The difference if that if I do not update the database I can ping the > migrated vm, while now I cannot > > > > Can you look at br-int and see if your tap interface is connected without > a vlan tag? Or is the tap still connected to the qbr bridge? If the latter, > were any iptables rules created? > > > > Unfortunately, I don’t have the ability to test this w/ live migration. > > > > James > > > > *From: *Ignazio Cassano > *Date: *Thursday, March 12, 2020 at 11:50 AM > *To: *James Denton > *Cc: *openstack-discuss > *Subject: *Re: [qeeens][neutron] migrating from iptables_hybrid to > openvswitch > > > > *CAUTION:* This message originated externally, please use caution when > clicking on links or opening attachments! > > > > Hello James; I made the db update before live migrating the virtual > machine, but the node where I migrated continues to create the qbr bridge > also if it is configured with openvswitch firewall. > > The difference if that if I do not update the database I can ping the > migrated vm, while now I cannot > > Ignazio > > > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < > james.denton at rackspace.com> ha scritto: > > Hi Ignazio, > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > use neutron; > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > Good luck! > > > > James > > > > *From: *Ignazio Cassano > *Date: *Thursday, March 12, 2020 at 6:41 AM > *To: *openstack-discuss > *Subject: *[qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > *CAUTION:* This message originated externally, please use caution when > clicking on links or opening attachments! > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > to compute node 1 with openvswitch, does not put the tap under br-int as > requested by openvswich firewall, but qbr is still present on compute node > 1. > > Once I enabled openvswitch on compute node 2, migration from compute node > 1 fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int on > compute node 1 before migrating to compute node 2 but it is a huge work on > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From elmiko at redhat.com Thu Mar 12 16:58:02 2020 From: elmiko at redhat.com (Michael McCune) Date: Thu, 12 Mar 2020 12:58:02 -0400 Subject: [API][SDK][SIG] distilling our community wisdom Message-ID: hello all, given the recent trend of discussing how we can condense and streamline various parts of the openstack community, i would like to start socializing the idea of merging the API-SIG, and its artifacts/processes/members, with the SDK group. over the last few years we have seen the activity of the API SIG greatly reduced. although there are a few outstanding guidelines that should be merged, we are having fewer and fewer contributors. additionally, although we migrated from weekly meetings down to weekly office hours, we have seen that aside from facilitating discussions for other groups these office hours are largely unnecessary. i don't have a proposal at the moment, beyond this email, but i would appreciate any and all feedback to help us reach a place where the SIG can still react as needed to the community but also reduce its surface area. the heart of these goals is to sustain and preserve the hard work that has been done over the years, and also to recognize the diminishing availability of our membership. peace o/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From openstack at nemebean.com Thu Mar 12 17:14:02 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 12 Mar 2020 12:14:02 -0500 Subject: [all] A call for consolidation and simplification In-Reply-To: References: Message-ID: <00fdbe27-bd7d-8540-feae-fc17ddeeae3f@nemebean.com> On 3/11/20 10:15 AM, Thierry Carrez wrote: > Hi all, > > I'd like to issue a call for consolidation and simplification for > OpenStack development. > > In the early years of the project, we faced a lot of challenges. We had > to spread the development load across manageable-size groups, so we > encouraged the creation of a lot of project teams. We wanted to capture > all the energy that was sent towards the project, so we passed project > structure reforms (like the big tent) that would aggressively include > new community groups in the "official" OpenStack community. We needed to > remove bottlenecks, so we encouraged decentralized decision making. And > we had to answer unique challenges, so we created software to match them > (Zuul). In summary, we had a lot of people, and not enough systems to > organize them, so we created those. > > Fast-forward to 2020, and our challenges are different. The many systems > that we created in the early days have created silos, with very small > groups of people working in isolation, making cross-project work more > difficult than it should be. The many systems that we created generate a > lot of fragmentation. Like we have too many meetings (76, in case you > were wondering), too much energy spent running them, too much > frustration when nobody joins. Finally, the many systems that we created > represent a lot of complexity for newcomers to handle. We have 180 IRC > channels, most of them ghost towns where by the time someone answers, > the person asking the question is long gone. > > So I think it's time to generally think about simplifying and > consolidating things. It's not as easy as it sounds. Our successful > decentralization efforts make it difficult to make the centralized > decision to regroup. It's hard to justify time and energy spent to > /remove/ things, especially those that we spent time creating in the > first place. But we now have too many systems and not enough people, so > we need to consolidate and simplify. > > Back around Havana, when we had around the same number of active > contributors as today, we used to have 36 meetings and 20 teams. Do we > really need 180 IRC channels, 76 meetings, 63 project teams (not even > counting SIGs)? > > Yes, we all specialized over time, so it's hard to merge for example > Oslo + Requirements, or QA + Infrastructure, or Stable + Release > Management, or Monasca + Telemetry. We are all overextended so it's hard > to learn new tricks or codebases. And yet, while I'm not really sure > what the best approach is, I think it's necessary. We've often had a fair amount of overlap between Oslo and some of the other horizontal teams like releases and requirements, which makes a certain amount of sense since they're all cross-OpenStack efforts. Naturally they tend to attract the same people. That said, would it make sense to merge with any of them? I'm unsure. And that's not a passive-aggressive "unsure", I actually don't know ;-). I will say that I feel pretty good about where the Oslo team is right now. Our meetings are generally well-attended, I would say even better than they were a year ago, and there's good discussion that happens. Many weeks topics are brought up by someone who is not me, which seems like a good sign of engagement. I guess we'll see what happens in the upcoming PTL election, but I'm not feeling like we need to do anything drastic to ensure a positive future for the project. Maybe that's an argument that we should bring another smaller team under our umbrella. We kind of just did that with the docs team not so long ago. I don't know if anyone else has strong opinions about how that has gone - mostly it hasn't changed much for me as PTL other than having a few more projects to review and release from time to time. I'm not sure if there are other projects where a merger would go as smoothly, but I'm open to suggestions. I don't know if any of the above is helpful at all, but I think it's a good summary of my thoughts as I've considered this. > > Comments, thoughts? > From sean.mcginnis at gmx.com Thu Mar 12 17:24:17 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 12 Mar 2020 12:24:17 -0500 Subject: [all] A call for consolidation and simplification In-Reply-To: <00fdbe27-bd7d-8540-feae-fc17ddeeae3f@nemebean.com> References: <00fdbe27-bd7d-8540-feae-fc17ddeeae3f@nemebean.com> Message-ID: <8033b9fc-2817-0daa-d554-4594110908fd@gmx.com> >> Yes, we all specialized over time, so it's hard to merge for example >> Oslo + Requirements, or QA + Infrastructure, or Stable + Release >> Management, or Monasca + Telemetry. We are all overextended so it's >> hard to learn new tricks or codebases. And yet, while I'm not really >> sure what the best approach is, I think it's necessary. > > We've often had a fair amount of overlap between Oslo and some of the > other horizontal teams like releases and requirements, which makes a > certain amount of sense since they're all cross-OpenStack efforts. > Naturally they tend to attract the same people. > > That said, would it make sense to merge with any of them? I'm unsure. > And that's not a passive-aggressive "unsure", I actually don't know ;-). > > I will say that I feel pretty good about where the Oslo team is right > now. Our meetings are generally well-attended, I would say even better > than they were a year ago, and there's good discussion that happens. > Many weeks topics are brought up by someone who is not me, which seems > like a good sign of engagement. I guess we'll see what happens in the > upcoming PTL election, but I'm not feeling like we need to do anything > drastic to ensure a positive future for the project. > > Maybe that's an argument that we should bring another smaller team > under our umbrella. We kind of just did that with the docs team not so > long ago. I don't know if anyone else has strong opinions about how > that has gone - mostly it hasn't changed much for me as PTL other than > having a few more projects to review and release from time to time. > I'm not sure if there are other projects where a merger would go as > smoothly, but I'm open to suggestions. > > I don't know if any of the above is helpful at all, but I think it's a > good summary of my thoughts as I've considered this. > I agree with this assessment. There are overlaps in people, but I do think that is just due to cross-project interests, not because there is much overlap in the work being done. I know these were just suggestions to get folks thinking, but for the specific idea of oslo+requirements - I think there are much different areas of focus for these and for at least this instance, there doesn't seem to be much benefit in combining teams. If anything, like Ben mentions, Oslo is growing and could maybe benefit from some loose collection of smaller teams. Sean From openstack at fried.cc Thu Mar 12 17:37:35 2020 From: openstack at fried.cc (Eric Fried) Date: Thu, 12 Mar 2020 12:37:35 -0500 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library In-Reply-To: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> References: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> Message-ID: <98e9a3a5-c253-f4ae-db4b-cda0e8c5eb91@fried.cc> > I'd like to suggest, now that we are on modern python, that we stop > using the mock library and instead use the built in mock library. +1. I started working on this a few weeks ago for nova, and came to the conclusion that it's going to be a multi-step process. This is because step 2 (see below) is going to take a long time and be quite difficult. 1) a) i) Remove mock-the-lib from $project's requirements. ii) Make $project explicitly import unittest.mock everywhere: s/import mock/from unittest import mock/ iii) Handle any weird fallout. You can see i) and ii) for nova here: https://review.opendev.org/#/c/708262/ ...but I likely won't get a chance to work on iii), which would need to happen in the same patch. b) Poison the use of `import mock` e.g. via a hacking rule so it doesn't creep back in. Example in nova: https://review.opendev.org/#/c/708768/ 2) Ferret out and eradicate all transitive dependencies. This is the hard part. That said, if we do 1) everywhere, maybe we don't need to do this. 3) Remove mock-the-lib from the requirements repo. We can't reasonably do this until {every requirements-using repo has done 1)} or {2) has happened}. Thanks, efried . From smooney at redhat.com Thu Mar 12 17:59:39 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 12 Mar 2020 17:59:39 +0000 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library In-Reply-To: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> References: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> Message-ID: <4965a29ab626b3193d2552113c684686ea649382.camel@redhat.com> On Thu, 2020-03-12 at 11:00 -0500, Matthew Thode wrote: > I'd like to suggest, now that we are on modern python, that we stop > using the mock library and instead use the built in mock library. yep i have already started doing that in cases where i am intoducing a new unit test file. i have not start chaning over existing usage of the mock lib but im avoiding new usage of it. > > unittest.mock has been around since python-3.3 so we should not have a > problem with python support. This would allow us to drop a library and > also help prevent issues with stuff like > https://review.opendev.org/712713 is going to expose (nova/neutron fail > tests with the external mock-4 library). > From marcin.juszkiewicz at linaro.org Thu Mar 12 18:02:18 2020 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Thu, 12 Mar 2020 19:02:18 +0100 Subject: [all][dev][qa] cirros 0.5.1 In-Reply-To: References: Message-ID: <8f17da78-6516-d8f6-ba08-f55fe08805ae@linaro.org> W dniu 12.03.2020 o 12:55, Radosław Piliszek pisze: > as you might have noticed, cinder 0.5.1 has been released. > This build seems to be passing the current devstack gate. [1] > Big thanks to hrw and smoser for letting cirros 0.5.1 happen (and > cirros having bright future yet again). If you have encounter any issues or would like to have some changes (but small ones) then please open issue on GitHub [1]. 1. https://github.com/cirros-dev/cirros Thanks to Radosław we fixed 'ip -o link' issue that's why 0.5.1 is for tests instead of 0.5.0 one. From ashlee at openstack.org Thu Mar 12 18:04:21 2020 From: ashlee at openstack.org (Ashlee Ferguson) Date: Thu, 12 Mar 2020 13:04:21 -0500 Subject: [OpenDev] We want your discussion ideas! Message-ID: <4BBA2919-CF4C-4493-BD82-16261D081DAA@openstack.org> Hi everyone, We still need your help shaping the discussion content for OpenDev + PTG, June 8-11, 2020 ! Our vision is for the content to be programmed by you-- the community. Programming Committees for each Track have begun meeting to discuss potential topics for the event, but they need your ideas and input to make sure all of the content is relevant and interesting to the community. The Programming Committee will also select moderators who will lead interactive discussions on a particular topic within the Track, so make sure to volunteer through the form below if you’re interested in that as well. Check out this etherpad to see what topics the Committees are currently discussing. If you’re interested in volunteering as an OpenDev discussion moderator, or would like to suggest topics for moderated discussions within a particular Track, please read the details below, and then fill out this form . We’re looking for discussion topics and Moderators for the following OpenDev Tracks: - Hardware Automation - Large-scale Usage of Open Source Infrastructure Software - Containers in Production - Key Challenges for Open Source in 2020 OpenDev Discussion Moderators will: - Be appointed by the Programming Committees - Facilitate discussions within a particular Track - Have adequate knowledge and experience to lead and moderate discussion around certain topics during the event - Work with Programming Committee to decide focal point of discussion Moderators need to be available to attend OpenDev, June 8 - 10, 2020. Volunteer to be a moderator or suggest discussion topics here before March 20: https://openstackfoundation.formstack.com/forms/opendev_vancouver2020_volunteer Cheers, Ashlee Ashlee Ferguson Community & Events Coordinator OpenStack Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Thu Mar 12 18:09:30 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 12 Mar 2020 19:09:30 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Message-ID: James, I checked again with your method. While live migration phase, the informations on neutron db are changed automatically and returns with "system", "ovs_hybrid_plug": True} ...... This is because the instance migrated has got interface under qbr. Ignazio Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < james.denton at rackspace.com> ha scritto: > Hi Ignazio, > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > use neutron; > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > Good luck! > > > > James > > > > *From: *Ignazio Cassano > *Date: *Thursday, March 12, 2020 at 6:41 AM > *To: *openstack-discuss > *Subject: *[qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > *CAUTION:* This message originated externally, please use caution when > clicking on links or opening attachments! > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > to compute node 1 with openvswitch, does not put the tap under br-int as > requested by openvswich firewall, but qbr is still present on compute node > 1. > > Once I enabled openvswitch on compute node 2, migration from compute node > 1 fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int on > compute node 1 before migrating to compute node 2 but it is a huge work on > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Thu Mar 12 18:18:43 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 12 Mar 2020 18:18:43 +0000 Subject: [all] Removing defunct meeting records In-Reply-To: References: <20200312010333.auxhcao54e6gbf42@yuggoth.org> Message-ID: <20200312181843.ogbuhw3sm4ezaadc@yuggoth.org> On 2020-03-12 10:54:44 +0100 (+0100), Thierry Carrez wrote: > Jeremy Stanley wrote: > > On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: > > [...] > > > we have too many meetings (76, in case you were wondering), too > > > much energy spent running them, too much frustration when nobody > > > joins. > > [...] > > > > Here's a list of 25 currently defined meetings which have not been > > held in 2020 (though it's possible some are being held with a > > different meeting_id passed to #startmeeting than is listed in the > > meeting record): [...] > Note that I already filed for removal of those which did not happen for over > a year: [...] Thanks! That's brought the remaining list of those which haven't had a meeting this year down to 18: CloudKitty Team Meeting Congress Team Meeting Containers Team Meeting First Contact SIG Meeting Freezer Meeting Heat (Orchestration) Team Meeting I18N Team Meeting Interop Working Group Meeting Kuryr Project Office Hours LOCI Development Meeting Mistral Meeting OpenStack Charms Placement Team Office Hour PowerVM Driver Meeting Public Cloud SIG Telemetry Team Meeting Vitrage Team Meeting Zaqar Team Meeting -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From radoslaw.piliszek at gmail.com Thu Mar 12 18:32:19 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 12 Mar 2020 19:32:19 +0100 Subject: [all][dev][qa] cirros 0.5.1 In-Reply-To: References: Message-ID: Thanks, Dmitry, for your quick response. I left 2 comments on it, rather important. @all: Based on this experience, please ensure you actually get to use devstack's cirros 0.5.1 :-) -yoctozepto czw., 12 mar 2020 o 17:18 Dmitry Tantsur napisał(a): > > Testing ironic: https://review.opendev.org/#/c/712728/ > > Thanks for the heads-up! > > Dmitry > > On Thu, Mar 12, 2020 at 12:57 PM Radosław Piliszek wrote: >> >> Hiya Folks, >> >> as you might have noticed, cinder 0.5.1 has been released. >> This build seems to be passing the current devstack gate. [1] >> Big thanks to hrw and smoser for letting cirros 0.5.1 happen (and >> cirros having bright future yet again). >> Also thanks to mordred for cleaning up SDK testing to pass. :-) >> >> I think it would be nice to merge this in Ussuri still, preferably before April. >> On the other hand, we all know that devstack gate is not super >> comprehensive and hence I would like to ask teams whose tests depend >> on interaction with guest OS to test their gates on this patch (or >> help me help you do that). >> I deliberately marked it W-1 to avoid merging too early. >> >> Let the discussion begin. :-) >> >> [1] https://review.opendev.org/711492 >> >> -yoctozepto >> From whayutin at redhat.com Thu Mar 12 18:34:44 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Thu, 12 Mar 2020 12:34:44 -0600 Subject: [tripleo] Message-ID: Greetings, So, an update on centos-8 jobs for master / ussuri. The tooling that tags containers w/ "current-tripleo" is almost available. Until that time one can derive the hash used to tag containers by navigating to [1] For example [2] you'll see the centos-binary-base is tagged w/ [1] I'm retagging the current containers in current-tripleo w/ the tag current-tripleo to resolve this issue for now. The tooling will be back up shortly :) I appreciate your patience. Thanks [1] https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md5 3621159be13b41f8ead2e873b357f4a5 [2] https://hub.docker.com/layers/tripleomaster/centos-binary-base/3621159be13b41f8ead2e873b357f4a5/images/sha256-b1e8b1acb774d9a299539d3063222e54f16f4673d5b1283f83f5b7f791c68d29?context=explore -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Thu Mar 12 19:45:05 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Thu, 12 Mar 2020 12:45:05 -0700 Subject: [all] A call for consolidation and simplification In-Reply-To: References: Message-ID: On Wed, Mar 11, 2020 at 8:18 AM Thierry Carrez wrote: > > Hi all, > > I'd like to issue a call for consolidation and simplification for > OpenStack development. > [trim] > So I think it's time to generally think about simplifying and > consolidating things. It's not as easy as it sounds. Our successful > decentralization efforts make it difficult to make the centralized > decision to regroup. It's hard to justify time and energy spent to > /remove/ things, especially those that we spent time creating in the > first place. But we now have too many systems and not enough people, so > we need to consolidate and simplify. > > Back around Havana, when we had around the same number of active > contributors as today, we used to have 36 meetings and 20 teams. Do we > really need 180 IRC channels, 76 meetings, 63 project teams (not even > counting SIGs)? I highly doubt we need that much. Part of me wonders about how we record the uses and successes, and also how we broadcast that out. I feel like in us trying to do everything and continue doing everything possible as a community, we've never stopped to ask ourselves why, nor even leave ourselves time to breath and be able to share the story of our successes as well as why we are motivated. > > Yes, we all specialized over time, so it's hard to merge for example > Oslo + Requirements, or QA + Infrastructure, or Stable + Release > Management, or Monasca + Telemetry. We are all overextended so it's hard > to learn new tricks or codebases. And yet, while I'm not really sure > what the best approach is, I think it's necessary. > > Comments, thoughts? > > -- > Thierry Carrez (ttx) > I suspect some of the divisions in specific focus areas or projects is a place that we likely can't come back from, and that ?might? be a good thing. We have specific projects that have niche or focused use cases that those smaller communities will continue to focus on and support because they are working to solve or support a specific problem space. We also have lots of code, and not everyone can be generalists. Nor can we ask people to try and maintain code that we have no idea if anyone is using or finds that it fills one of their problem spaces. So I think it is important to focus on the projects that are actually in use and have maintainers. I distinctly note "and have maintainers" because we cannot ask people to stretch themselves any more than they already have to support yet another thing, another effort. And ultimately trying to force consolidation is going to be viewed as being asked to give more by contributors. So I do think we as a community need to scale back some of our complexity. Some of the disperse areas of information. But we also need to know where we should be focusing our time and energy, and if our work matters only to ourselves, to our employers, or to the community at large. And I'm sure the immediate thought is that the user survey has this data. I'm not convinced this is the case. The user survey is only a slice of a user-base which maintains a close communication with the community. It gets filled out by those that get the emails, those that see the social media, but users are out there that are not easily reachable via those avenues because they live in different circles. So a few thoughts off the top of my head: * If projects don't have maintainers nor anyone who wishes to be elected as a PTL, the TC should not try and continue to drive the project along by naming a leader of some sort. Those smaller communities can and should coalescence if they are still active through self organization to figure out their next step. * If projects can't coalescence, then we need to consider "maintenance" or some other similar word to indicate inactive. * All non-security releases should stop for the such projects, including no release for the cycle. No horizontal contributor should feel obligated to keep it maintained. * We need to accept that a piece of software in the community may stop working at some point in time for some set of users. This is okay and should actually help bring fixes in that we would not normally receive which becomes an indicator of use, and thus a means to revive! * We need to consolidate our sources of truth. i.e. Sunset and decommission wiki.openstack.org. * We need to learn to trust, and not keep power of the core reviewer. Keeping reviewers to a small select group just hurts projects. Does that mean mistakes may be made? absolutely! Can they be fixed? Most likely and at worst there is revert! Do core reviewers make mistakes too? Definitely! * We need to accept change, and move forward. * We need to abandon the desire for perfection for that blocks forward movement. * We need data to make decisions to move forward, so if we had a simple instrumentation service that allowed operators to post statistics, then that could be amazingly powerful! It would be even better to gamify, but I'm not sure that is entirely possible. For that data/statistics idea, I'm thinking a `-survey` command, which includes a couple questions, and maybe polls some statistical data from the deployment at the time of submission. Phone home sounds worse, but operator invoked functionally that is what I'm thinking on some level. Anyway, I agree we are over extended. I think the only way we can even begin to move away from being over extended is to begin to scale certain things back, but I also feel strongly we have lots of hidden users out there based upon discussions I've had in hallway tracks. Regardless, I hope some of these thoughts and ideas resonate, and nobody decides to get out a flame thrower. Meanwhile, I'll dream of a day when I can +2 nova/virt/ironic code, and cinder folks can +2 ironic/common/cinder code. From whayutin at redhat.com Thu Mar 12 20:04:56 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Thu, 12 Mar 2020 14:04:56 -0600 Subject: [tripleo] In-Reply-To: References: Message-ID: On Thu, Mar 12, 2020 at 12:34 PM Wesley Hayutin wrote: > Greetings, > > So, an update on centos-8 jobs for master / ussuri. > The tooling that tags containers w/ "current-tripleo" is almost > available. Until that time one can derive the hash used to tag containers > by navigating to [1] > > For example [2] you'll see the centos-binary-base is tagged w/ [1] > > I'm retagging the current containers in current-tripleo w/ the tag > current-tripleo to resolve this issue for now. The tooling will be back up > shortly :) I appreciate your patience. > > Thanks > > > [1] > https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md5 > > 3621159be13b41f8ead2e873b357f4a5 > > [2] > https://hub.docker.com/layers/tripleomaster/centos-binary-base/3621159be13b41f8ead2e873b357f4a5/images/sha256-b1e8b1acb774d9a299539d3063222e54f16f4673d5b1283f83f5b7f791c68d29?context=explore > > > Follow up, I don't recommend trying to use centos-7 at this time.. but if *need* here is the following information you will need. https://trunk.rdoproject.org/centos7-master/current-tripleo/delorean.repo name=delorean-openstack-tripleo-heat-templates-b5ef03c9c939db551b03e9490edc6981ff582035 hash identifier is b5ef03c9c939db551b03e9490edc6981ff582035 https://hub.docker.com/layers/tripleomaster/centos-binary-base/b5ef03c9c939db551b03e9490edc6981ff582035_76ebc465_manifest/images/sha256-2b8a4fd7760fc39e82bb2859c021fe1adf216415056da60c16eee31a697bb2b5?context=explore You have to update your containers-prepare info to: parameter_defaults: ContainerImagePrepare: - set: tag: b5ef03c9c939db551b03e9490edc6981ff582035 Hope this helps, Any remaining centos-7 jobs will be removed as we get to them and will be considered unsupported. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stig.openstack at telfer.org Thu Mar 12 21:17:38 2020 From: stig.openstack at telfer.org (Stig Telfer) Date: Thu, 12 Mar 2020 21:17:38 +0000 Subject: [all] IRC channel cleanup (was: A call for consolidation and simplification) In-Reply-To: <20200312004325.p7zwr7pkc6aeuezc@yuggoth.org> References: <20200312004325.p7zwr7pkc6aeuezc@yuggoth.org> Message-ID: <45A0631A-B253-44C7-9FA8-4499133196F5@telfer.org> Hi Jeremy - Feel free to delete the scientific-wg IRC channel. The group discussion has moved to a Slack channel, os-scientific-sig.slack.com - we still go to IRC for meetings though... Thanks, Stig > On 12 Mar 2020, at 00:43, Jeremy Stanley wrote: > > On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: > [...] >> Do we really need 180 IRC channels > [...] > > There are around 110 we currently seem to deem worth logging with > the "openstack" meetbot (note that not all are OpenStack community > channels): > > > > A quick survey of logs suggests these have seen no comment from a > human in 6 months (all are OpenStack-related): > > #scientific-wg > #openstack-women > #openstack-sprint > #openstack-net-bgpvpn > #openstack-heat-translator > #openstack-forum > #openstack-dragonflow > #congress > > And these have averaged fewer than one comment from a human per > month since September: > > #openstack-outreachy > #murano > #openstack-tricircle > #openstack-performance > #openstack-ec2api > #openstack-browbeat > > Does anyone object to us ceasing logging of the above 14 channels? > -- > Jeremy Stanley From skaplons at redhat.com Thu Mar 12 21:22:28 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 12 Mar 2020 22:22:28 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> Message-ID: <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> Hi, IIRC, if You want to manually change Your database to force nova to not use hybrid connection anymore and to not require qbr bridge You may need to update also one of the tables in Nova’s db. It’s called “instance_info_network_cache” or something similar. But TBH I’m not sure if live migration then will work or not as I’m not sure if instance’s libvirt.xml file isn’t going from src to dest node during the live migration. If You don’t need to do live migration, You can switch firewall_driver in the L2 agent’s config file and restart it. Even instances which has got hybrid connectivity (so are plugged through qbr bridge) will have SG working in new way. It shouldn’t be problem that those instances are plugged through qbr bridge as it finally ends up in br-int and there SG rules will be applied. You will need to manually clean iptables rules for such ports as it will not be cleaned automatically. New instances on such host should works fine and will be plugged in “new way”, directly to br-int. The only problem with this approach is that You will not be able to do live-migration for those old vms. If You want to do it properly, You should do “nova interface-detach” and then “nova interface-attach” for each of such instances. Then new ports plugged to the instances will be bound in new way and plugged directly to br-int. > On 12 Mar 2020, at 19:09, Ignazio Cassano wrote: > > James, I checked again with your method. While live migration phase, the informations on neutron db are changed automatically and returns with "system", "ovs_hybrid_plug": True} ...... > This is because the instance migrated has got interface under qbr. > Ignazio > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton ha scritto: > Hi Ignazio, > > > > I tested a process that converted iptables_hybrid to openvswitch in-place, but not without a hard reboot of the VM and some massaging of the existing bridges/veths. Since you are live-migrating, though, you might be able to get around that. > > > > Regardless, to make this work, I had to update the port’s vif_details in the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > use neutron; > > > update ml2_port_bindings set vif_details='{"port_filter": true, "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > So, perhaps making that change prior to moving the VM back to the other compute node will do the trick. > > > > Good luck! > > > > James > > > > From: Ignazio Cassano > Date: Thursday, March 12, 2020 at 6:41 AM > To: openstack-discuss > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! > > > > Hello All, I am facing some problems migrating from iptables_hybrid frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid to compute node 1 with openvswitch, does not put the tap under br-int as requested by openvswich firewall, but qbr is still present on compute node 1. > > Once I enabled openvswitch on compute node 2, migration from compute node 1 fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int on compute node 1 before migrating to compute node 2 but it is a huge work on a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > — Slawek Kaplonski Senior software engineer Red Hat From kennelson11 at gmail.com Thu Mar 12 21:23:09 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 12 Mar 2020 14:23:09 -0700 Subject: [all] Removing defunct meeting records (was: A call for consolidation and simplification) In-Reply-To: <20200312010333.auxhcao54e6gbf42@yuggoth.org> References: <20200312010333.auxhcao54e6gbf42@yuggoth.org> Message-ID: The First Contact SIG is still active, we just haven't had much to talk about lately so we've been holding off on meeting. -Kendall (diablo_rojo) On Wed, Mar 11, 2020 at 6:05 PM Jeremy Stanley wrote: > On 2020-03-11 16:15:32 +0100 (+0100), Thierry Carrez wrote: > [...] > > we have too many meetings (76, in case you were wondering), too > > much energy spent running them, too much frustration when nobody > > joins. > [...] > > Here's a list of 25 currently defined meetings which have not been > held in 2020 (though it's possible some are being held with a > different meeting_id passed to #startmeeting than is listed in the > meeting record): > > CloudKitty Team Meeting > Congress Team Meeting > Containers Team Meeting > Documentation Team Meeting > First Contact SIG Meeting > Freezer Meeting > Glance Bug Squad Meeting > Group Based Policy Team Meeting > Heat (Orchestration) Team Meeting > I18N Team Meeting > Interop Working Group Meeting > Kuryr Project Office Hours > LOCI Development Meeting > Mistral Meeting > Networking VPP team meeting > OpenStack Charms > Placement Team Office Hour > PowerVM Driver Meeting > Public Cloud SIG > Searchlight Team Meeting > Telemetry Team Meeting > Trove (DBaaS) Team Meeting > Upgrades SIG > Vitrage Team Meeting > Zaqar Team Meeting > > I recommend at least correcting inaccurate meeting_id entries in the > YAML files here: > > https://opendev.org/opendev/irc-meetings/src/branch/master/meetings/ > > If there are meetings you know are not being held, please submit > changes to remove their corresponding YAML files. I'll set myself a > reminder to rerun this query again sometime soon and we can discuss > bulk removing any which are presumed defunct at that time. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Thu Mar 12 21:25:59 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 12 Mar 2020 22:25:59 +0100 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library In-Reply-To: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> References: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> Message-ID: Hi, > On 12 Mar 2020, at 17:00, Matthew Thode wrote: > > I'd like to suggest, now that we are on modern python, that we stop > using the mock library and instead use the built in mock library. It sounds like good candidate for community goal for the next cycle for me :) > > unittest.mock has been around since python-3.3 so we should not have a > problem with python support. This would allow us to drop a library and > also help prevent issues with stuff like > https://review.opendev.org/712713 is going to expose (nova/neutron fail > tests with the external mock-4 library). > > -- > Matthew Thode — Slawek Kaplonski Senior software engineer Red Hat From fungi at yuggoth.org Thu Mar 12 22:03:15 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 12 Mar 2020 22:03:15 +0000 Subject: [all] Removing defunct meeting records (was: A call for consolidation and simplification) In-Reply-To: References: <20200312010333.auxhcao54e6gbf42@yuggoth.org> Message-ID: <20200312220314.dcybkwprh7b2snwu@yuggoth.org> On 2020-03-12 14:23:09 -0700 (-0700), Kendall Nelson wrote: > The First Contact SIG is still active, we just haven't had much to > talk about lately so we've been holding off on meeting. [...] This was more intended to trigger discussion about whether some of these groups find meetings worthwhile, in cases where they're still active but not running scheduled meetings very often. It looks like the FC SIG has had two meetings in the past six months. Since it's scheduled to meet every two weeks that means it's only holding ~15% of its meetings. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From rosmaita.fossdev at gmail.com Thu Mar 12 22:09:31 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Thu, 12 Mar 2020 18:09:31 -0400 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <2e142636-0070-704c-c5f7-1e035bc9d406@openstack.org> Message-ID: <24850ca7-a0a3-769e-aecb-15e824538a84@gmail.com> On 3/12/20 10:37 AM, Thierry Carrez wrote: > Thierry Carrez wrote: >> [...] >> So one solution might be: >> >> - Define multiple roles (release liaison, event liaison, meeting >> chair...) and allow them to be filled by the team as they want, for >> the duration they want, replaced when they want (would just need +1 >> from previous and new holder of the role) >> >> - Use the TC as a governance safety valve to resolve any conflict >> (instead of PTL elections) > > Proposed as a strawman at: https://review.opendev.org/#/c/712696/ > Feel free to comment on how crazy that is there... > Maybe the proposal itself is crazy, maybe not, but the timing of this is certainly crazy. This is not a good time to be having this discussion. We're just about to start week R-8. The final release for non-client libraries is at R-6, the final release for client libraries and feature freeze is at R-5. So there is (should be) a lot of stuff happening in the projects right now, that may preclude people from being as active in this discussion as they might wish. Plus, I don't know about y'all, but the current pandemic is making simple day-to-day living more stressful on me than it was a few weeks ago. What I'm saying is that I'd appreciate clarification on the timeline for this change. This is the kind of thing that could use discussion at the PTG. If the point of Thierry's patch is to prepare for discussion at the PTG, then that's great, I personally will revisit it at R-1, and be ready to have a productive discussion. But I think this is not a good time to make the usual "silence is assent" inference to a proposal floated on the mailing list. cheers, brian From jlibosva at redhat.com Thu Mar 12 19:50:31 2020 From: jlibosva at redhat.com (Jakub Libosvar) Date: Thu, 12 Mar 2020 20:50:31 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: Message-ID: On 12/03/2020 11:38, Ignazio Cassano wrote: > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > I am doing this because I want enable security groups logs which require > openvswitch firewall. > I would like to migrate without restarting my instances. > I startded moving all instances from compute node 1. > Then I configured openvswitch firewall on compute node 1, > Instances migrated from compute node 2 to compute node 1 without problems. > Once the compute node 2 was empty, I migrated it to openvswitch. > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > This happened because migrating instances from node2 with iptables_hybrid > to compute node 1 with openvswitch, does not put the tap under br-int as > requested by openvswich firewall, but qbr is still present on compute node > 1. > Once I enabled openvswitch on compute node 2, migration from compute node 1 > fails because it exprects qbr on compute node 2 . > So I think I should moving on the fly tap interfaces from qbr to br-int on > compute node 1 before migrating to compute node 2 but it is a huge work on > a lot of instances. > > Any workaround, please ? > > Ignazio > I may be a little outdated here but to the best of my knowledge there are two ways how to migrate from iptables to openvswitch. 1) If you don't mind the intermediate linux bridge and you care about logs, you can just change the config file on compute node to start using openvswitch firewall and restart the ovs agent. That should trigger a mechanism that deletes iptables rules and starts using openflow rules. It will leave the intermediate bridge there but except the extra hop in networking stack, it doesn't mind. 2) With multiple-port binding feature, what you described above should be working. I know Miguel spent some time working on that so perhaps he has more information about which release it should be functional at, I think it was Queens. Not sure if any Nova work was required to make it work. Hope that helps. Kuba From smooney at redhat.com Thu Mar 12 23:04:46 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 12 Mar 2020 23:04:46 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <24850ca7-a0a3-769e-aecb-15e824538a84@gmail.com> References: <2e142636-0070-704c-c5f7-1e035bc9d406@openstack.org> <24850ca7-a0a3-769e-aecb-15e824538a84@gmail.com> Message-ID: <7cbd432168715df7511e184ea909986b50bcc764.camel@redhat.com> On Thu, 2020-03-12 at 18:09 -0400, Brian Rosmaita wrote: > On 3/12/20 10:37 AM, Thierry Carrez wrote: > > Thierry Carrez wrote: > > > [...] > > > So one solution might be: > > > > > > - Define multiple roles (release liaison, event liaison, meeting > > > chair...) and allow them to be filled by the team as they want, for > > > the duration they want, replaced when they want (would just need +1 > > > from previous and new holder of the role) > > > > > > - Use the TC as a governance safety valve to resolve any conflict > > > (instead of PTL elections) > > > > Proposed as a strawman at: https://review.opendev.org/#/c/712696/ > > Feel free to comment on how crazy that is there... > > > > Maybe the proposal itself is crazy, maybe not, but the timing of this is > certainly crazy. at least form a nova perspecitve this is an appriate time as we were in a postion where due to changes in errics employment he would not be able to continue with his role as novas ptl. with as you indicate later ther are many factors going on currenlty on the world at larage and in people personal and internal commitments so we were in a postion where we might not have a candiate to take over until the next election and we may or may not have candidates for that election. fortunetly gibi has been able to find time to become novas interim ptl for the rest of the ussuri cycle. i am not going to speak on behalf of nova but i suspect that if the core team had the option of moving to a ptl less model then they might just do that as from my experince nova ptl have never or very really been in a postion where they must be a tie breaker between other cores out side of there normal role as a core themeses. so for nova i dont think that aspect of the ptl role is partically relevent as the core team general reaches consensious without requiring a ptl to step in and make a dessions. given the lack of ptl candiages over the last few cycle i think removing hte need to have a candiate run in a election would encurage decentralistion of the set of jobs that defualt to the ptl. if gibi or someone decied to run as ptl for victoria have the flexablity to not require a ptl does not perculde operationg as we do today, it just takes the pressure off contiutors as we wont have to involve the tc if noone puts there name forward for the election. we can fall back to a ptl less model and just ensure that each of the function are performed by someone in a timely manner. so if this was to proceed i was hoping that it could take effect form the victoria cycle. that said i dont dislike the ptl role, i think it has served use well and could continue too but this is not the first time in the last few release where i fell like the requirement for there to be a ptl has been more harmful then helpful as i think without a nominated ptl the work would still have got done more organically with less process. > > This is not a good time to be having this discussion. We're just about > to start week R-8. The final release for non-client libraries is at > R-6, the final release for client libraries and feature freeze is at > R-5. So there is (should be) a lot of stuff happening in the projects > right now, that may preclude people from being as active in this > discussion as they might wish. Plus, I don't know about y'all, but the > current pandemic is making simple day-to-day living more stressful on me > than it was a few weeks ago. > > What I'm saying is that I'd appreciate clarification on the timeline for > this change. This is the kind of thing that could use discussion at the > PTG. If the point of Thierry's patch is to prepare for discussion at > the PTG, then that's great, I personally will revisit it at R-1, and be > ready to have a productive discussion. But I think this is not a good > time to make the usual "silence is assent" inference to a proposal > floated on the mailing list. > > cheers, > brian > From sundar.nadathur at intel.com Thu Mar 12 23:48:54 2020 From: sundar.nadathur at intel.com (Nadathur, Sundar) Date: Thu, 12 Mar 2020 23:48:54 +0000 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: From: Zhipeng Huang Sent: Thursday, March 12, 2020 6:21 AM > I like what Sean proposed, and a cycle bound time criteria (2 cycles or 12 months) would be very good, and if we center the quality criteria on > meaningful reviews would largely reduced the burden of unnecessary computations. > I agree that we should document this and stick to it. For me "12 months + no meaningful review" would be a good enough concrete criteria, > for removing the non-active core reviewer in a non-voluntarily step down fashion. Apart from a specific period for reviewing the, um, reviews, I would suggest talking to the reviewer in question to understand any exigent circumstances and their intentions, esp. if there are other factors like meanigful patch contributions and active attendance in meetings/PTG/etc. After all, a person may have spent months preparing to be a core reviewer, and it would be a shame to let all that drop. If the person is unreachable or explicitly indicates a desire to step down, as has happened in this situation, there would be solid grounds to take their name off. Regards, Sundar On Thu, Mar 12, 2020 at 7:48 PM Sean Mooney > wrote: On Thu, 2020-03-12 at 15:55 +0800, Zhipeng Huang wrote: > I have no particular objection about removing these particular two previous > active cores, however I do concern that when we start to build a new > precedence, we should do it right which means we should have an agreed set > of metrics that provides the objective qualification of the "core removal" > process. > > The original proposed qualification is "18 months no participation in > meetings, no code contributions and no reviews", I would like that we could > make the clarification that: > > - Is it a consecutive 18 months period with the construed "absence > criteria" met ? i would think 18 months is slightly too long, it certely should not be longer then that. between 12 and 18 feels right to me. after about 2 cycle things can have changed significantly after 3 even more so. 6 monts feels way to sort but 12 to 18 i think is about right. > - For the "absence criteria", could we settle upon a set of exhaustive > metrics: no meeting, no code contribution, no review, no email discussion > participation, anything more ? the only metric for being a core rerviewer for being a core review should be based on well code reviews. code contibution without review should not be a consideration to keep core reviewer status. meeting, irc and email is also not really relevent with one exception. if cyborg was to do a virtual pre-ptg where specs desigin was disucssed and review on the mailing list via eamil, placmenet and nova have tried this in the last cycle or two, then i would consider that the same as gerrit review. i should qulify this that the metric should not be based on the number of reviews alone but rather how how detailed and well reasoned the comments are should be a large factor. a large number of +1 with no comment is generally an anti patteren fro considering some as a core. asking questions to clarify the design choices and confrim the authors intent and your understanding are in sync and make sense is perfectly valid to do while also +1 because you belive in your view the patch is correct and should be encurraged over +1 and no comment. the +1/-1 ratio should also be a factor. its if someone always +1s and never -1 they likely are not reviewing correctly other factors such as email participation, meeting attendence, irc presence or other community partisatpation are supporting factors that suggest a good candiate for becomeing a core but on there own should not be a vaild critia for granting or retaining core reviewer role in a project. my understanding of the above is derived form the definition of what a core review is https://docs.openstack.org/project-team-guide/open-development.html#core-reviewers the review critia https://docs.openstack.org/project-team-guide/open-development.html#reviews-guidelines https://docs.openstack.org/project-team-guide/review-the-openstack-way.html and my general experience with contributing to different openstack project. The core reviewer role withing a project while similar in some repects to a maintianer role in other opensouce models is not the same. a maintainers role tends to focus more on code authorship in addtion to review which is not a factor in the core reviewer role in openstack. if you never write a singel line of code but make detail and technically correct reviews in openstack that makes you an amazing core reviewer. conversly closing 100 bugs in a release with commits and doing no code review would make you a good maintainer but a bad core reviewer, you would be an invaluable contibutor for all your bug fixing work but no a good reviewer which s the focus of the core reviewer role. > - If there were a set of agreed "absence criteria"s, what are the logical > connection between these pre-conditions ? Is it an "AND" (all of the > pre-conditions shall be satisfied) or just "OR" (only one of the > pre-conditions satisfies) > > Once we have a concrete rule setup, we are good to go with a current core > reviewer vote for the record of removing, as far as I understand :) well any core can step down without any kind of vote at any time. they just need to go to https://review.opendev.org/#/admin/groups/1243,members tick there name and remove them selvs and well tell the rest of the team so they know. unless the person being removed object or there is an object to the 2 people being proposed by one of the core team i don't think there is a reason to wait in this case but that is up to the core team to decide. > > Due process is very important. actually i would think that haveing concreate rules like this probably are not useful but if you do write them down you should stick with them. > > On Thu, Mar 12, 2020 at 8:40 AM Nadathur, Sundar > > wrote: > > > > > > From: Sean Mooney > > > > Sent: Wednesday, March 11, 2020 9:37 AM > > > > > > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > > > > > I'm a little bit concerned over the 18 months period. The original > > > > rule we setup is volunteer step down, since this is a small team we > > > > want to acknowledge everyone that has made significant contributions. > > > > Some of the inactive core reviewers like Justin Kilpatrick have moved > > > > on a long time ago, and I don't see people like him could do any harm > > > > to > > > the project. > > > > > > > > But if the core reviewer has a size limit in the system, that would be > > > > reasonable to replace the inactive ones with the new recruits :) > > > > > > it is generally considerd best pratice to maintian the core team adding > > > > or > > > removing people based on there activity. if a core is removed due to in > > > activity and they come back they can always be restored but it give a bad > > > perception if a project has like 20 core but only 2 are active. as a new > > > contibutor you dont know which ones are active and it can be frustrating > > > > to > > > reach out to them and get no responce. > > > also just form a project healt point of view it make the project look > > > > like its > > > more diverse or more active then it actully is which is also not > > > > generally a > > > good thing. > > > > > > that said core can step down if they feel like they can contribute time > > > anymore when ever they like so and if a core is steping a way for a few > > > months but intends to come back they can also say that in advance and > > > > there > > > is no harm in leaving them for a cycle or two but in general after a > > > > period of > > > in activity (usally more then a full release/6months) i think its good > > > > to reduce > > > back down the core team. > > > > > > > > Just my two cents > > > > As of now, Cyborg core team officially has 12 members [1]. That is hardly > > small. > > > > Justin Kilpatrick seems to be gone for good; he didn't respond to my > > emails. Rushil Chugh confirmed that he is not working on OpenStack anymore > > and asked to step down as core. With due thanks to him for his > > contributions, I'll go ahead. > > > > Those are the two cores I had in mind. Agree with Sean that it is better > > to keep the list of core reviewers up to date. With all the changes in > > Cyborg over the past 18 months, it will be tough for a person to jump in > > after a long hiatus and resume as a core reviewer. Even if they want to > > come back, it is better for them to come up to speed first. > > > > Given this background, if there is any objection to the removal of these > > two cores, please let me know. > > > > [1] https://review.opendev.org/#/admin/groups/1243,members > > > > Regards, > > Sundar > > > > > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > > > > > > > > > wrote: > > > > > > > > > Hello all, > > > > > Brin Zhang has been actively contributing to Cyborg in various > > > > > areas, adding new features, improving quality, reviewing patches, > > > > > and generally helping others in the community. Despite the > > > > > relatively short time, he has been one of the most prolific > > > > > contributors, and brings an enthusiastic and active mindset. I would > > > > > like to thank him and acknowledge his significant contributions by > > > > > > proposing him as a core reviewer for Cyborg. > > > > > > > > > > Shogo Saito has been active in Cyborg since Train release. He has > > > > > been driving the Cyborg client improvements, including its revamp to > > > > > use OpenStackSDK. Previously he was instrumental in the transition > > > > > to Python 3, testing and fixing issues in the process. As he has > > > > > access to real FPGA hardware, he brings a users’ perspective and > > > > > also tests Cyborg with real hardware. I would like to thank and > > > > > acknowledge him for his steady valuable contributions, and propose > > > > him > > > as a core reviewer for Cyborg. > > > > > > > > > > Some of the currently listed core reviewers have not been > > > > > participating for a lengthy period of time. It is proposed that > > > > > those who have had no contributions for the past 18 months – i.e. no > > > > > participation in meetings, no code contributions and no reviews – be > > > > > removed from the list of core reviewers. > > > > > > > > > > If no objections are made known by March 20, I will make the changes > > > > > proposed above. > > > > > > > > > > Thanks. > > > > > > > > > > Regards, > > > > > Sundar > > > > > > -- Zhipeng (Howard) Huang Principle Engineer OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, DMTF, W3C -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Fri Mar 13 01:39:50 2020 From: smooney at redhat.com (Sean Mooney) Date: Fri, 13 Mar 2020 01:39:50 +0000 Subject: [cyborg] Proposing core reviewers In-Reply-To: References: <8232aae9fd2fcd78bbcf039dc1cc680cba417ca0.camel@redhat.com> Message-ID: <47f04f2cd4f882ec7933e8a3a5e2bd52d5a4f67f.camel@redhat.com> On Thu, 2020-03-12 at 23:48 +0000, Nadathur, Sundar wrote: > > From: Zhipeng Huang > Sent: Thursday, March 12, 2020 6:21 AM > > > I like what Sean proposed, and a cycle bound time criteria (2 cycles or 12 months) would be very good, and if we > > center the quality criteria on > > meaningful reviews would largely reduced the burden of unnecessary computations. > > I agree that we should document this and stick to it. For me "12 months + no meaningful review" would be a good > > enough concrete criteria, > > for removing the non-active core reviewer in a non-voluntarily step down fashion. > > Apart from a specific period for reviewing the, um, reviews, I would suggest talking to the reviewer in question to > understand any exigent circumstances and their intentions, esp. if there are other factors like meanigful patch > contributions and active attendance in meetings/PTG/etc. After all, a person may have spent months preparing to be a > core reviewer, and it would be a shame to let all that drop. > > If the person is unreachable or explicitly indicates a desire to step down, as has happened in this situation, there > would be solid grounds to take their name off. yep reaching out to them before doing anything else is both politte and pragmatic. we reach out before propsoeing some one as a core to ensure they are ok with it it makes total sense to reach out to people before propoesing reomving someone and ya or factor in life migth affect why a person has to be inactive although if it gets to the point of 2-3 cylces or 12-18 months where they cant partisapte then its likely they will continue to be unable to participate and review. > > Regards, > Sundar > > > > > On Thu, Mar 12, 2020 at 7:48 PM Sean Mooney > wrote: > On Thu, 2020-03-12 at 15:55 +0800, Zhipeng Huang wrote: > > I have no particular objection about removing these particular two previous > > active cores, however I do concern that when we start to build a new > > precedence, we should do it right which means we should have an agreed set > > of metrics that provides the objective qualification of the "core removal" > > process. > > > > The original proposed qualification is "18 months no participation in > > meetings, no code contributions and no reviews", I would like that we could > > make the clarification that: > > > > - Is it a consecutive 18 months period with the construed "absence > > criteria" met ? > > i would think 18 months is slightly too long, it certely should not be longer then that. > between 12 and 18 feels right to me. after about 2 cycle things can have changed significantly > after 3 even more so. 6 monts feels way to sort but 12 to 18 i think is about right. > > - For the "absence criteria", could we settle upon a set of exhaustive > > metrics: no meeting, no code contribution, no review, no email discussion > > participation, anything more ? > > the only metric for being a core rerviewer for being a core review should be based on well code reviews. > code contibution without review should not be a consideration to keep core reviewer status. > meeting, irc and email is also not really relevent with one exception. if cyborg was to do a virtual pre-ptg where > specs > desigin was disucssed and review on the mailing list via eamil, placmenet and nova have tried this in the last cycle > or > two, then i would consider that the same as gerrit review. > > i should qulify this that the metric should not be based on the number of reviews alone but rather how how detailed > and well reasoned the comments are should be a large factor. a large number of +1 with no comment is generally an anti > patteren fro considering some as a core. asking questions to clarify the design choices and confrim the authors intent > and your understanding are in sync and make sense is perfectly valid to do while also +1 because you belive in your > view > the patch is correct and should be encurraged over +1 and no comment. > > the +1/-1 ratio should also be a factor. its if someone always +1s and never -1 they likely are not reviewing > correctly > > other factors such as email participation, meeting attendence, irc presence or other community partisatpation are > supporting factors that suggest a good candiate for becomeing a core but on there own should not be a vaild critia > for granting or retaining core reviewer role in a project. > > my understanding of the above is derived form the definition of what a core review is > https://docs.openstack.org/project-team-guide/open-development.html#core-reviewers > the review critia https://docs.openstack.org/project-team-guide/open-development.html#reviews-guidelines > https://docs.openstack.org/project-team-guide/review-the-openstack-way.html > and my general experience with contributing to different openstack project. > The core reviewer role withing a project while similar in some repects to a maintianer role in other opensouce models > is not the same. a maintainers role tends to focus more on code authorship in addtion to review which is not a factor > in > the core reviewer role in openstack. if you never write a singel line of code but make detail and technically correct > reviews in openstack that makes you an amazing core reviewer. conversly closing 100 bugs in a release with commits and > doing no code review would make you a good maintainer but a bad core reviewer, you would be an invaluable contibutor > for > all your bug fixing work but no a good reviewer which s the focus of the core reviewer role. > > > - If there were a set of agreed "absence criteria"s, what are the logical > > connection between these pre-conditions ? Is it an "AND" (all of the > > pre-conditions shall be satisfied) or just "OR" (only one of the > > pre-conditions satisfies) > > > > Once we have a concrete rule setup, we are good to go with a current core > > reviewer vote for the record of removing, as far as I understand :) > > well any core can step down without any kind of vote at any time. > they just need to go to https://review.opendev.org/#/admin/groups/1243,members > tick there name and remove them selvs and well tell the rest of the team so they know. > > unless the person being removed object or there is an object to the 2 people being proposed by one > of the core team i don't think there is a reason to wait in this case but that is up to the core team to decide. > > > > Due process is very important. > > actually i would think that haveing concreate rules like this probably are not useful but if you > do write them down you should stick with them. > > > > On Thu, Mar 12, 2020 at 8:40 AM Nadathur, Sundar > > > wrote: > > > > > > > > > From: Sean Mooney > > > > > Sent: Wednesday, March 11, 2020 9:37 AM > > > > > > > > On Thu, 2020-03-12 at 00:17 +0800, Zhipeng Huang wrote: > > > > > Big +1 for Brin and shogo's nomination and well deserved :) > > > > > > > > > > I'm a little bit concerned over the 18 months period. The original > > > > > rule we setup is volunteer step down, since this is a small team we > > > > > want to acknowledge everyone that has made significant contributions. > > > > > Some of the inactive core reviewers like Justin Kilpatrick have moved > > > > > on a long time ago, and I don't see people like him could do any harm > > > > > > to > > > > the project. > > > > > > > > > > But if the core reviewer has a size limit in the system, that would be > > > > > reasonable to replace the inactive ones with the new recruits :) > > > > > > > > it is generally considerd best pratice to maintian the core team adding > > > > > > or > > > > removing people based on there activity. if a core is removed due to in > > > > activity and they come back they can always be restored but it give a bad > > > > perception if a project has like 20 core but only 2 are active. as a new > > > > contibutor you dont know which ones are active and it can be frustrating > > > > > > to > > > > reach out to them and get no responce. > > > > also just form a project healt point of view it make the project look > > > > > > like its > > > > more diverse or more active then it actully is which is also not > > > > > > generally a > > > > good thing. > > > > > > > > that said core can step down if they feel like they can contribute time > > > > anymore when ever they like so and if a core is steping a way for a few > > > > months but intends to come back they can also say that in advance and > > > > > > there > > > > is no harm in leaving them for a cycle or two but in general after a > > > > > > period of > > > > in activity (usally more then a full release/6months) i think its good > > > > > > to reduce > > > > back down the core team. > > > > > > > > > > Just my two cents > > > > > > As of now, Cyborg core team officially has 12 members [1]. That is hardly > > > small. > > > > > > Justin Kilpatrick seems to be gone for good; he didn't respond to my > > > emails. Rushil Chugh confirmed that he is not working on OpenStack anymore > > > and asked to step down as core. With due thanks to him for his > > > contributions, I'll go ahead. > > > > > > Those are the two cores I had in mind. Agree with Sean that it is better > > > to keep the list of core reviewers up to date. With all the changes in > > > Cyborg over the past 18 months, it will be tough for a person to jump in > > > after a long hiatus and resume as a core reviewer. Even if they want to > > > come back, it is better for them to come up to speed first. > > > > > > Given this background, if there is any objection to the removal of these > > > two cores, please let me know. > > > > > > [1] https://review.opendev.org/#/admin/groups/1243,members > > > > > > Regards, > > > Sundar > > > > > > > > On Wed, Mar 11, 2020 at 10:19 PM Nadathur, Sundar > > > > > > > > > > > wrote: > > > > > > > > > > > Hello all, > > > > > > Brin Zhang has been actively contributing to Cyborg in various > > > > > > areas, adding new features, improving quality, reviewing patches, > > > > > > and generally helping others in the community. Despite the > > > > > > relatively short time, he has been one of the most prolific > > > > > > contributors, and brings an enthusiastic and active mindset. I would > > > > > > like to thank him and acknowledge his significant contributions by > > > > > > > > proposing him as a core reviewer for Cyborg. > > > > > > > > > > > > Shogo Saito has been active in Cyborg since Train release. He has > > > > > > been driving the Cyborg client improvements, including its revamp to > > > > > > use OpenStackSDK. Previously he was instrumental in the transition > > > > > > to Python 3, testing and fixing issues in the process. As he has > > > > > > access to real FPGA hardware, he brings a users’ perspective and > > > > > > also tests Cyborg with real hardware. I would like to thank and > > > > > > acknowledge him for his steady valuable contributions, and propose > > > > > > him > > > > as a core reviewer for Cyborg. > > > > > > > > > > > > Some of the currently listed core reviewers have not been > > > > > > participating for a lengthy period of time. It is proposed that > > > > > > those who have had no contributions for the past 18 months – i.e. no > > > > > > participation in meetings, no code contributions and no reviews – be > > > > > > removed from the list of core reviewers. > > > > > > > > > > > > If no objections are made known by March 20, I will make the changes > > > > > > proposed above. > > > > > > > > > > > > Thanks. > > > > > > > > > > > > Regards, > > > > > > Sundar > > > > > > > > > > > > > -- > Zhipeng (Howard) Huang > > Principle Engineer > OpenStack, Kubernetes, CNCF, LF Edge, ONNX, Kubeflow, OpenSDS, Open Service Broker API, OCP, Hyperledger, ETSI, SNIA, > DMTF, W3C > From kennelson11 at gmail.com Fri Mar 13 02:02:48 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 12 Mar 2020 19:02:48 -0700 Subject: [all] Removing defunct meeting records (was: A call for consolidation and simplification) In-Reply-To: <20200312220314.dcybkwprh7b2snwu@yuggoth.org> References: <20200312010333.auxhcao54e6gbf42@yuggoth.org> <20200312220314.dcybkwprh7b2snwu@yuggoth.org> Message-ID: I suppose we could switch to just be monthly.. that might be better. I guess we will hold a meeting at our next regularly scheduled slot and decide that :) -Kendall (diablo_rojo) On Thu, 12 Mar 2020, 3:04 pm Jeremy Stanley, wrote: > On 2020-03-12 14:23:09 -0700 (-0700), Kendall Nelson wrote: > > The First Contact SIG is still active, we just haven't had much to > > talk about lately so we've been holding off on meeting. > [...] > > This was more intended to trigger discussion about whether some of > these groups find meetings worthwhile, in cases where they're still > active but not running scheduled meetings very often. It looks like > the FC SIG has had two meetings in the past six months. Since it's > scheduled to meet every two weeks that means it's only holding ~15% > of its meetings. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Fri Mar 13 03:46:23 2020 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Fri, 13 Mar 2020 11:46:23 +0800 Subject: [heat][magnum][tripleo] Official build and test plan for heat container agent Message-ID: Dear all tl;dr: Shall we build, publish, and test heat container agents on heat-agents directly?:) Please help to review https://review.opendev.org/#/q/topic:build-container-agent+(status:open) Current I think we encounter with following issues: ** No functional/scenario test for heat-agents but only unit tests* Right now, whatever patch people send up in heat-agents, we only vote against unit tests. This makes no sense for long term maintenance. Which makes me thinking of the second issue. ** No enable scenario tests for software deployment:* We have tests defined, but since we didn't build any customized image. We didn't enable it for a long time. To build a container image for heat-agents (aka heat container agent) is (IMO) the best way for us to enable these teats. Which we can also use it to cross gating on heat and heat-agents (or even heat-template). The no much to change for the test itself but only update the config script to enable the container. ** Can't guarantee cross-project usage:* We currently provide no guarantee for any usage of heat agents from other projects, especially Magnum and TripleO (I didn't personally check if TripleO still using it, but since they still have experiment CI running) who uses heat container agents. It will be nice if we provide basic images for other teams can build their additional requirements. To resolve the above issues, I propose [1]. Which a lot of logic was copied from the current magnum heat container agent scripts [2] so we can start with a higher possibility to allow magnum to reuse our works. For a more detail approach in [1]. The patch defines a build job in `check` pipeline. And a publish job in `post-check`. Uses zuul secrets to save docker authN information. As for the account, I created a new `openstackheat` account [3] for it (I specifically recall there ware discussion about having an official image hub for OpenStack, but not quite sure what happened since). The current approach also can support multiple architectures (like amd64 arm arm64 ppc64le s390x, etc). But for now, I think we should have tested before we publish for other architectures. You can run the build or publish job easily by running `make build-images-all-archs` to build images for all architectures or `make upload-images-all-archs` to build and publish images for all architectures. [1] https://review.opendev.org/#/q/topic:build-container-agent+(status:open) [2] https://opendev.org/openstack/magnum/src/branch/master/dockerfiles/heat-container-agent [3] https://hub.docker.com/r/openstackheat -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From renat.akhmerov at gmail.com Fri Mar 13 05:32:29 2020 From: renat.akhmerov at gmail.com (Renat Akhmerov) Date: Fri, 13 Mar 2020 12:32:29 +0700 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> Message-ID: <29a571a5-6081-448e-ba09-740e83200341@Spark> On 5 Mar 2020, 02:57 +0700, Jay Bryant , wrote: > Zane's input sums up how I feel as well.  I think that having consistent > leadership structure across projects is important and helps keep us > aware of the health of projects. > > Perhaps we can help return interest in the PTL role by providing > examples of teams that share the work and have the PTL to help make > final decisions.  I know that the Cinder team has been doing this for > quite some time successfully. I really fail to understand the issue of an overwhelming PTLship. From my personal 5+ experience of being a PTL, I can say I’ve never tried to do all stuff like releases, fixing CI, back porting etc. etc. myself. Especially given that I really like solving technical problems, I always try to offload some of these duties to others and make sure I have time for development. And IMO it’s totally fine. As a PTL I try to make sure that everyone in the team (at least most active members) has this balance between problem solving and necessary procedures. I absolutely agree though that as the PTL I have to know what’s going on with the project and after all TC or someone else can ask me about its current state and progress. Of course, I realise that my project is somewhat not really typical for OpenStack: it is relatively small and has not so many connections with other projects. But I believe this principle is universal enough. As far as the PTL role, I think for PTLs it’s important to focus on the big picture, ideas and directions and keep reminding everyone about that. All team members, even active ones, often can’t afford thinking about this too much. This contradicts with lots of what I heard before from my former managers and colleagues, and also some PTLs I know. They claimed: “PTLs just need to maintain Launchpad (or Storyboard), keep an eye on the release process and that’s basically it. Plus reviewing a little bit.” I’ve always shrugged when hearing this.. If so, let’s remove “L” from “PTL” and replace it with “A”, so that it’s “PTA” - Project Technical Administrator. Something that can legally exist, no issue. And it’s more honest. What I’m going to is, from my perspective, it probably doesn’t make any difference if a project leader is an official role or not. I guess there will always be someone who naturally gains trust of others and influences the direction of a project. As far as “having a final word on a deadlocked issue” I thought this is something really important but, in fact, it’s a very rare thing and may not be needed at all. Usually, we have to make a decision anyway, since “not making a decision is more expensive than making even bad decision”. So I believe some leadership is always needed. The most high quality techs I’ve ever seen have been all made with a very strong leadership, I don’t believe it works the other way. Whether it’s official or not, I think is not important at all. But “administrative duties” that often assign to PTLs can be easily split between people. Thanks Renat Akhmerov @Nokia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Fri Mar 13 07:16:08 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 13 Mar 2020 08:16:08 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> Message-ID: Thanks Slawek, I am going to check nova tables as well. Ignazio Il Gio 12 Mar 2020, 22:22 Slawek Kaplonski ha scritto: > Hi, > > IIRC, if You want to manually change Your database to force nova to not > use hybrid connection anymore and to not require qbr bridge You may need to > update also one of the tables in Nova’s db. It’s called > “instance_info_network_cache” or something similar. > But TBH I’m not sure if live migration then will work or not as I’m not > sure if instance’s libvirt.xml file isn’t going from src to dest node > during the live migration. > > If You don’t need to do live migration, You can switch firewall_driver in > the L2 agent’s config file and restart it. Even instances which has got > hybrid connectivity (so are plugged through qbr bridge) will have SG > working in new way. It shouldn’t be problem that those instances are > plugged through qbr bridge as it finally ends up in br-int and there SG > rules will be applied. You will need to manually clean iptables rules for > such ports as it will not be cleaned automatically. > New instances on such host should works fine and will be plugged in “new > way”, directly to br-int. > The only problem with this approach is that You will not be able to do > live-migration for those old vms. > > If You want to do it properly, You should do “nova interface-detach” and > then “nova interface-attach” for each of such instances. Then new ports > plugged to the instances will be bound in new way and plugged directly to > br-int. > > > On 12 Mar 2020, at 19:09, Ignazio Cassano > wrote: > > > > James, I checked again with your method. While live migration phase, the > informations on neutron db are changed automatically and returns with > "system", "ovs_hybrid_plug": True} ...... > > This is because the instance migrated has got interface under qbr. > > Ignazio > > > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < > james.denton at rackspace.com> ha scritto: > > Hi Ignazio, > > > > > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > > > > > use neutron; > > > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > > > > > Good luck! > > > > > > > > James > > > > > > > > From: Ignazio Cassano > > Date: Thursday, March 12, 2020 at 6:41 AM > > To: openstack-discuss > > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > > > > > CAUTION: This message originated externally, please use caution when > clicking on links or opening attachments! > > > > > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > > > I would like to migrate without restarting my instances. > > > > I startded moving all instances from compute node 1. > > > > Then I configured openvswitch firewall on compute node 1, > > > > Instances migrated from compute node 2 to compute node 1 without > problems. > > > > Once the compute node 2 was empty, I migrated it to openvswitch. > > > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > > > > > This happened because migrating instances from node2 with > iptables_hybrid to compute node 1 with openvswitch, does not put the tap > under br-int as requested by openvswich firewall, but qbr is still present > on compute node 1. > > > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 fails because it exprects qbr on compute node 2 . > > > > So I think I should moving on the fly tap interfaces from qbr to br-int > on compute node 1 before migrating to compute node 2 but it is a huge work > on a lot of instances. > > > > > > > > Any workaround, please ? > > > > > > > > Ignazio > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Fri Mar 13 07:24:08 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 13 Mar 2020 08:24:08 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: Message-ID: Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node switched on openvswitch works fine . The problem is this migration create the qbr on the mode switched to openvswitch. But when I switch another compute node to openvswitch and I try to live migrate the same vm (openvswitch to qopenswitch) it does not work because the qbr presence. I verified on nova logs. Ignazio Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha scritto: > On 12/03/2020 11:38, Ignazio Cassano wrote: > > Hello All, I am facing some problems migrating from iptables_hybrid > > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without > problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > > to compute node 1 with openvswitch, does not put the tap under br-int as > > requested by openvswich firewall, but qbr is still present on compute > node > > 1. > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 > > fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int > on > > compute node 1 before migrating to compute node 2 but it is a huge work > on > > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > > > > I may be a little outdated here but to the best of my knowledge there > are two ways how to migrate from iptables to openvswitch. > > 1) If you don't mind the intermediate linux bridge and you care about > logs, you can just change the config file on compute node to start using > openvswitch firewall and restart the ovs agent. That should trigger a > mechanism that deletes iptables rules and starts using openflow rules. > It will leave the intermediate bridge there but except the extra hop in > networking stack, it doesn't mind. > > 2) With multiple-port binding feature, what you described above should > be working. I know Miguel spent some time working on that so perhaps he > has more information about which release it should be functional at, I > think it was Queens. Not sure if any Nova work was required to make it > work. > > Hope that helps. > Kuba > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevinzs2048 at gmail.com Fri Mar 13 07:45:35 2020 From: kevinzs2048 at gmail.com (kevinz) Date: Fri, 13 Mar 2020 15:45:35 +0800 Subject: [Multi-Arch-SIG][devstack][nova] Implement Arm64 CI now for Nova and DevStack Message-ID: Hi OpenStacker, Linaro has donated a cluster for OpenStack CI on Arm64. Now the cluster is ready and some jobs(Kolla and disk builder, thanks hrw's hard-working) are running here, https://opendev.org/openstack/project-config/src/branch/master/nodepool/nl03.openstack.org.yaml#L414 Thanks for the help from Infra team and nova team, I am now working on Devstack tempest jobs running on arm64: https://review.opendev.org/708317 Some changes from Nova also need to be covered in order to make devstack jobs successfully. https://review.opendev.org/712607 https://review.opendev.org/709494 Yet, there are still a lot of tasks need to fix for arm64 CI work. If you are interested in this, any forms of help will be really welcomed! let's get together to make arm64 CI happens. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-philippe at evrard.me Fri Mar 13 09:00:57 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Fri, 13 Mar 2020 10:00:57 +0100 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library In-Reply-To: References: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> Message-ID: <3a378a14059936392c5c5b8e240dd91566bc8e23.camel@evrard.me> On Thu, 2020-03-12 at 22:25 +0100, Slawek Kaplonski wrote: > Hi, > > > On 12 Mar 2020, at 17:00, Matthew Thode wrote: > > > > I'd like to suggest, now that we are on modern python, that we stop > > using the mock library and instead use the built in mock library. > > It sounds like good candidate for community goal for the next cycle > for me :) > > > Agreed. Matt, can you propose it? Proposing it doesn't mean that it will be selected, nor that you sign up for work, it just makes sure it's available in the pool of goals to select it from. You seem to have a definitive idea, might be worth documenting it! Regards, JP From thierry at openstack.org Fri Mar 13 09:41:40 2020 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 13 Mar 2020 10:41:40 +0100 Subject: [API][SDK][SIG] distilling our community wisdom In-Reply-To: References: Message-ID: <6f01e0b1-f86d-cf34-d9d0-aa2571403454@openstack.org> Michael McCune wrote: > given the recent trend of discussing how we can condense and streamline > various parts of the openstack community, i would like to start > socializing the idea of merging the API-SIG, and its > artifacts/processes/members, with the SDK group. > > over the last few years we have seen the activity of the API SIG greatly > reduced. although there are a few outstanding guidelines that should be > merged, we are having fewer and fewer contributors. additionally, > although we migrated from weekly meetings down to weekly office hours, > we have seen that aside from facilitating discussions for other groups > these office hours are largely unnecessary. > > i don't have a proposal at the moment, beyond this email, but i would > appreciate any and all feedback to help us reach a place where the SIG > can still react as needed to the community but also reduce its surface > area. the heart of these goals is to sustain and preserve the hard work > that has been done over the years, and also to recognize the diminishing > availability of our membership. As I mentioned in another email, I welcome consolidation: teams with 3-6 people are more motivating that teams of 1-2. Having API guidelines designed in close discussion with the primary API consumers makes a lot of sense too. -- Thierry Carrez (ttx) From ignaziocassano at gmail.com Fri Mar 13 11:10:26 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 13 Mar 2020 12:10:26 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> Message-ID: Hello Slawek, I tried with nova detach and attach on a running vm after switch to openvswitch driver but it stops to receive packets also if put the interface under br-int as you said. The result is virsh domiflist domainname list two interfaces: one under qbr ad one under br-int Ignazio Il giorno gio 12 mar 2020 alle ore 22:22 Slawek Kaplonski < skaplons at redhat.com> ha scritto: > Hi, > > IIRC, if You want to manually change Your database to force nova to not > use hybrid connection anymore and to not require qbr bridge You may need to > update also one of the tables in Nova’s db. It’s called > “instance_info_network_cache” or something similar. > But TBH I’m not sure if live migration then will work or not as I’m not > sure if instance’s libvirt.xml file isn’t going from src to dest node > during the live migration. > > If You don’t need to do live migration, You can switch firewall_driver in > the L2 agent’s config file and restart it. Even instances which has got > hybrid connectivity (so are plugged through qbr bridge) will have SG > working in new way. It shouldn’t be problem that those instances are > plugged through qbr bridge as it finally ends up in br-int and there SG > rules will be applied. You will need to manually clean iptables rules for > such ports as it will not be cleaned automatically. > New instances on such host should works fine and will be plugged in “new > way”, directly to br-int. > The only problem with this approach is that You will not be able to do > live-migration for those old vms. > > If You want to do it properly, You should do “nova interface-detach” and > then “nova interface-attach” for each of such instances. Then new ports > plugged to the instances will be bound in new way and plugged directly to > br-int. > > > On 12 Mar 2020, at 19:09, Ignazio Cassano > wrote: > > > > James, I checked again with your method. While live migration phase, the > informations on neutron db are changed automatically and returns with > "system", "ovs_hybrid_plug": True} ...... > > This is because the instance migrated has got interface under qbr. > > Ignazio > > > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < > james.denton at rackspace.com> ha scritto: > > Hi Ignazio, > > > > > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > > > > > use neutron; > > > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > > > > > Good luck! > > > > > > > > James > > > > > > > > From: Ignazio Cassano > > Date: Thursday, March 12, 2020 at 6:41 AM > > To: openstack-discuss > > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > > > > > CAUTION: This message originated externally, please use caution when > clicking on links or opening attachments! > > > > > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > > > I would like to migrate without restarting my instances. > > > > I startded moving all instances from compute node 1. > > > > Then I configured openvswitch firewall on compute node 1, > > > > Instances migrated from compute node 2 to compute node 1 without > problems. > > > > Once the compute node 2 was empty, I migrated it to openvswitch. > > > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > > > > > This happened because migrating instances from node2 with > iptables_hybrid to compute node 1 with openvswitch, does not put the tap > under br-int as requested by openvswich firewall, but qbr is still present > on compute node 1. > > > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 fails because it exprects qbr on compute node 2 . > > > > So I think I should moving on the fly tap interfaces from qbr to br-int > on compute node 1 before migrating to compute node 2 but it is a huge work > on a lot of instances. > > > > > > > > Any workaround, please ? > > > > > > > > Ignazio > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Fri Mar 13 11:12:54 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 13 Mar 2020 12:12:54 +0100 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> Message-ID: <761AE9BD-D262-4F66-A97F-515F8DCA0DC0@redhat.com> Hi, As it’s already a week since nomination and I see only very positive feedback about it, I think we can now say: welcome in the Neutron core team Lajos :) > On 6 Mar 2020, at 16:28, Slawek Kaplonski wrote: > > Hi neutrinos, > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of the Neutron core team. > Lajos is Neutron contributor Neutron since around Queens cycle and now he is one of the most active reviewers in the Neutron group projects. > He was one of the key contributors in cooperation with Nova and Placement teams to deliver guaranteed minimum bandwidth feature in OpenStack. > He is very active and helpful with triaging and fixing Neutron bugs and issues in our CI. > > During last few cycles he proved that he has wide knowledge about Neutron code base. He is currently also a maintainer of some neutron stadium projects which shows that he has knowledge about code base not only about neutron but also Neutron stadium. > > The quality and number of his reviews are comparable to other members of the Neutron core team: https://www.stackalytics.com/?release=ussuri&module=neutron-group and are higher every cycle :) > I think he will be great addition to our core team. > > I will keep this nomination open for a week or until all current cores will respond. > > — > Slawek Kaplonski > Senior software engineer > Red Hat > — Slawek Kaplonski Senior software engineer Red Hat From katonalala at gmail.com Fri Mar 13 12:43:36 2020 From: katonalala at gmail.com (Lajos Katona) Date: Fri, 13 Mar 2020 13:43:36 +0100 Subject: [neutron] Propose Lajos Katona for Neutron core team In-Reply-To: <761AE9BD-D262-4F66-A97F-515F8DCA0DC0@redhat.com> References: <478F5503-6507-419E-88A6-24B0BFBE0BE1@redhat.com> <761AE9BD-D262-4F66-A97F-515F8DCA0DC0@redhat.com> Message-ID: Thank you all for all the support. I will do my best :-) Lajos Slawek Kaplonski ezt írta (időpont: 2020. márc. 13., P, 12:17): > Hi, > > As it’s already a week since nomination and I see only very positive > feedback about it, I think we can now say: welcome in the Neutron core team > Lajos :) > > > On 6 Mar 2020, at 16:28, Slawek Kaplonski wrote: > > > > Hi neutrinos, > > > > I would like to propose Lajos Katona (irc: lajoskatona) as a member of > the Neutron core team. > > Lajos is Neutron contributor Neutron since around Queens cycle and now > he is one of the most active reviewers in the Neutron group projects. > > He was one of the key contributors in cooperation with Nova and > Placement teams to deliver guaranteed minimum bandwidth feature in > OpenStack. > > He is very active and helpful with triaging and fixing Neutron bugs and > issues in our CI. > > > > During last few cycles he proved that he has wide knowledge about > Neutron code base. He is currently also a maintainer of some neutron > stadium projects which shows that he has knowledge about code base not only > about neutron but also Neutron stadium. > > > > The quality and number of his reviews are comparable to other members of > the Neutron core team: > https://www.stackalytics.com/?release=ussuri&module=neutron-group and are > higher every cycle :) > > I think he will be great addition to our core team. > > > > I will keep this nomination open for a week or until all current cores > will respond. > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mthode at mthode.org Fri Mar 13 15:14:32 2020 From: mthode at mthode.org (Matthew Thode) Date: Fri, 13 Mar 2020 10:14:32 -0500 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library In-Reply-To: <3a378a14059936392c5c5b8e240dd91566bc8e23.camel@evrard.me> References: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> <3a378a14059936392c5c5b8e240dd91566bc8e23.camel@evrard.me> Message-ID: <20200313151432.73yufkbqkpyku3dh@mthode.org> On 20-03-13 10:00:57, Jean-Philippe Evrard wrote: > On Thu, 2020-03-12 at 22:25 +0100, Slawek Kaplonski wrote: > > Hi, > > > > > On 12 Mar 2020, at 17:00, Matthew Thode wrote: > > > > > > I'd like to suggest, now that we are on modern python, that we stop > > > using the mock library and instead use the built in mock library. > > > > It sounds like good candidate for community goal for the next cycle > > for me :) > > > > > > > Agreed. Matt, can you propose it? Proposing it doesn't mean that it > will be selected, nor that you sign up for work, it just makes sure > it's available in the pool of goals to select it from. You seem to have > a definitive idea, might be worth documenting it! > > > Regards, > JP > > Sure, will add it to todo list for today. -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From gmann at ghanshyammann.com Fri Mar 13 16:08:48 2020 From: gmann at ghanshyammann.com (gmann at ghanshyammann.com) Date: Fri, 13 Mar 2020 11:08:48 -0500 Subject: [all] Removing defunct meeting records (was: A call for consolidation and simplification) In-Reply-To: References: <20200312010333.auxhcao54e6gbf42@yuggoth.org> <20200312220314.dcybkwprh7b2snwu@yuggoth.org> Message-ID: <170d4a7c1cb.bf75921f226192.7129344523614130824@ghanshyammann.com> ---- On Thu, 12 Mar 2020 21:02:48 -0500 Kendall Nelson wrote ---- > I suppose we could switch to just be monthly.. that might be better. I guess we will hold a meeting at our next regularly scheduled slot and decide that :) Or I will say we make the daily/weekly (say spend 1 hour on any preferred week day) practice of doing planned homework to monitor new contributors instead of waiting for the meeting at least current active members in FC SIG. Monthly meetings for check and other planning is no harm too. We can report our monthly observations there. -gmann > -Kendall (diablo_rojo) > On Thu, 12 Mar 2020, 3:04 pm Jeremy Stanley, wrote: > On 2020-03-12 14:23:09 -0700 (-0700), Kendall Nelson wrote: > > The First Contact SIG is still active, we just haven't had much to > > talk about lately so we've been holding off on meeting. > [...] > > This was more intended to trigger discussion about whether some of > these groups find meetings worthwhile, in cases where they're still > active but not running scheduled meetings very often. It looks like > the FC SIG has had two meetings in the past six months. Since it's > scheduled to meet every two weeks that means it's only holding ~15% > of its meetings. > -- > Jeremy Stanley > From e0ne at e0ne.info Fri Mar 13 16:29:06 2020 From: e0ne at e0ne.info (Ivan Kolodyazhny) Date: Fri, 13 Mar 2020 18:29:06 +0200 Subject: [tc][requirements][horizon] Broken gates due to the new setuptools and pyScss package Message-ID: Hi team, I'm sorry for being too noisy, but I decided to tag TC to get more attention to the current Horizon situation. We've got a bug reported by Kolla team two days ago [1]. We merged some workaround [2] and [3] yesterday. Thanks a lot to the Requirements team for the really quick merge! I appreciate it. For now, we've got horizon gates broken because of hose patches fix only devstack but not unit tests jobs. The root cause of this situation is pyScss package which is not maintained for the last two years. It's not a surprise to me that it doesn't work with the new setuptools. I'm really surprised that we've found this kind of bugs only now. Since I don't believe we can block new setuptools forever, I decided to fork pyScss [4] and django-pyscss [5] projects. I'm still not sure that I've done everything right with licensing and versioning, but it works now on my environment. Any help on these areas would be much appreciated. I proposed patches to requirements and horizon repos [6] to use new libraries. The reason I've tagged TC in the mail thread is described below. Horizon has a lot of too old and unmaintained libraries. I'm pretty sure that this only one of the first issues with outdated dependencies which blocks horizon and other gates. I do understand why we've got this situation. Unfortunately, we don't have any full-time horizon developers in the community. Horizon is mostly in maintenance phrase but not in active development. I would like to get more attention on this issue because we have to update all dependencies not because they are new, have new features and/or security fixes. We have to take care of our dependencies asap to avoid usage of unmaintained libraries to have the whole OpenStack and Horizon healthy. P.S. I'm sorry if this message is too rude or emotional, I really don't want to make it such one. [1] https://bugs.launchpad.net/kolla/+bug/1866961 [2] https://review.opendev.org/#/c/711930/ [3] https://review.opendev.org/#/c/712777/ [4] https://github.com/e0ne/pyScss/ [5] https://github.com/e0ne/django-pyscss [6] https://review.opendev.org/#/q/status:open+topic:fix-pyscss Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Fri Mar 13 17:38:27 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 13 Mar 2020 18:38:27 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> Message-ID: Hi All, first of all I want to thank everyone for help me. I I tried all your suggestions but without rebooting instances does not work. The following steps do the job well: - evacuate node A and switch to openvswitch - migrate (no live) instances from node B - switch node B to openvswitch but before activating the agent, clean Its openswitch entries otherwhise live migration from node A does not work - and so on for all remaing nodes Ignazio Il Gio 12 Mar 2020, 22:22 Slawek Kaplonski ha scritto: > Hi, > > IIRC, if You want to manually change Your database to force nova to not > use hybrid connection anymore and to not require qbr bridge You may need to > update also one of the tables in Nova’s db. It’s called > “instance_info_network_cache” or something similar. > But TBH I’m not sure if live migration then will work or not as I’m not > sure if instance’s libvirt.xml file isn’t going from src to dest node > during the live migration. > > If You don’t need to do live migration, You can switch firewall_driver in > the L2 agent’s config file and restart it. Even instances which has got > hybrid connectivity (so are plugged through qbr bridge) will have SG > working in new way. It shouldn’t be problem that those instances are > plugged through qbr bridge as it finally ends up in br-int and there SG > rules will be applied. You will need to manually clean iptables rules for > such ports as it will not be cleaned automatically. > New instances on such host should works fine and will be plugged in “new > way”, directly to br-int. > The only problem with this approach is that You will not be able to do > live-migration for those old vms. > > If You want to do it properly, You should do “nova interface-detach” and > then “nova interface-attach” for each of such instances. Then new ports > plugged to the instances will be bound in new way and plugged directly to > br-int. > > > On 12 Mar 2020, at 19:09, Ignazio Cassano > wrote: > > > > James, I checked again with your method. While live migration phase, the > informations on neutron db are changed automatically and returns with > "system", "ovs_hybrid_plug": True} ...... > > This is because the instance migrated has got interface under qbr. > > Ignazio > > > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < > james.denton at rackspace.com> ha scritto: > > Hi Ignazio, > > > > > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > > > > > use neutron; > > > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > > > > > Good luck! > > > > > > > > James > > > > > > > > From: Ignazio Cassano > > Date: Thursday, March 12, 2020 at 6:41 AM > > To: openstack-discuss > > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > > > > > CAUTION: This message originated externally, please use caution when > clicking on links or opening attachments! > > > > > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > > > I would like to migrate without restarting my instances. > > > > I startded moving all instances from compute node 1. > > > > Then I configured openvswitch firewall on compute node 1, > > > > Instances migrated from compute node 2 to compute node 1 without > problems. > > > > Once the compute node 2 was empty, I migrated it to openvswitch. > > > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > > > > > This happened because migrating instances from node2 with > iptables_hybrid to compute node 1 with openvswitch, does not put the tap > under br-int as requested by openvswich firewall, but qbr is still present > on compute node 1. > > > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 fails because it exprects qbr on compute node 2 . > > > > So I think I should moving on the fly tap interfaces from qbr to br-int > on compute node 1 before migrating to compute node 2 but it is a huge work > on a lot of instances. > > > > > > > > Any workaround, please ? > > > > > > > > Ignazio > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.denton at rackspace.com Fri Mar 13 17:59:00 2020 From: james.denton at rackspace.com (James Denton) Date: Fri, 13 Mar 2020 17:59:00 +0000 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> Message-ID: <90666E22-8FCE-415A-AD22-C2E662862E8A@rackspace.com> Thanks for the update! If you can work out the exact process for someone else to follow, I’m sure the docs team would appreciate your input! James From: Ignazio Cassano Date: Friday, March 13, 2020 at 1:38 PM To: Slawek Kaplonski Cc: James Denton , openstack-discuss Subject: Re: [qeeens][neutron] migrating from iptables_hybrid to openvswitch CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hi All, first of all I want to thank everyone for help me. I I tried all your suggestions but without rebooting instances does not work. The following steps do the job well: - evacuate node A and switch to openvswitch - migrate (no live) instances from node B - switch node B to openvswitch but before activating the agent, clean Its openswitch entries otherwhise live migration from node A does not work - and so on for all remaing nodes Ignazio Il Gio 12 Mar 2020, 22:22 Slawek Kaplonski > ha scritto: Hi, IIRC, if You want to manually change Your database to force nova to not use hybrid connection anymore and to not require qbr bridge You may need to update also one of the tables in Nova’s db. It’s called “instance_info_network_cache” or something similar. But TBH I’m not sure if live migration then will work or not as I’m not sure if instance’s libvirt.xml file isn’t going from src to dest node during the live migration. If You don’t need to do live migration, You can switch firewall_driver in the L2 agent’s config file and restart it. Even instances which has got hybrid connectivity (so are plugged through qbr bridge) will have SG working in new way. It shouldn’t be problem that those instances are plugged through qbr bridge as it finally ends up in br-int and there SG rules will be applied. You will need to manually clean iptables rules for such ports as it will not be cleaned automatically. New instances on such host should works fine and will be plugged in “new way”, directly to br-int. The only problem with this approach is that You will not be able to do live-migration for those old vms. If You want to do it properly, You should do “nova interface-detach” and then “nova interface-attach” for each of such instances. Then new ports plugged to the instances will be bound in new way and plugged directly to br-int. > On 12 Mar 2020, at 19:09, Ignazio Cassano > wrote: > > James, I checked again with your method. While live migration phase, the informations on neutron db are changed automatically and returns with "system", "ovs_hybrid_plug": True} ...... > This is because the instance migrated has got interface under qbr. > Ignazio > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton > ha scritto: > Hi Ignazio, > > > > I tested a process that converted iptables_hybrid to openvswitch in-place, but not without a hard reboot of the VM and some massaging of the existing bridges/veths. Since you are live-migrating, though, you might be able to get around that. > > > > Regardless, to make this work, I had to update the port’s vif_details in the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > use neutron; > > > update ml2_port_bindings set vif_details='{"port_filter": true, "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > So, perhaps making that change prior to moving the VM back to the other compute node will do the trick. > > > > Good luck! > > > > James > > > > From: Ignazio Cassano > > Date: Thursday, March 12, 2020 at 6:41 AM > To: openstack-discuss > > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! > > > > Hello All, I am facing some problems migrating from iptables_hybrid frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid to compute node 1 with openvswitch, does not put the tap under br-int as requested by openvswich firewall, but qbr is still present on compute node 1. > > Once I enabled openvswitch on compute node 2, migration from compute node 1 fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int on compute node 1 before migrating to compute node 2 but it is a huge work on a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > — Slawek Kaplonski Senior software engineer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Fri Mar 13 18:01:56 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 13 Mar 2020 19:01:56 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <90666E22-8FCE-415A-AD22-C2E662862E8A@rackspace.com> References: <803A7B19-7B9E-423C-9358-B0138332A105@rackspace.com> <6BD78B63-EDFF-4D23-9B4C-853016C64CED@redhat.com> <90666E22-8FCE-415A-AD22-C2E662862E8A@rackspace.com> Message-ID: How can I do that? Where I must write the exact process? Ignazio Il Ven 13 Mar 2020, 18:59 James Denton ha scritto: > Thanks for the update! If you can work out the exact process for someone > else to follow, I’m sure the docs team would appreciate your input! > > > > James > > > > *From: *Ignazio Cassano > *Date: *Friday, March 13, 2020 at 1:38 PM > *To: *Slawek Kaplonski > *Cc: *James Denton , openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject: *Re: [qeeens][neutron] migrating from iptables_hybrid to > openvswitch > > > > *CAUTION:* This message originated externally, please use caution when > clicking on links or opening attachments! > > > > Hi All, first of all I want to thank everyone for help me. > > I I tried all your suggestions but without rebooting instances does not > work. > > The following steps do the job well: > > - evacuate node A and switch to openvswitch > > - migrate (no live) instances from node B > > - switch node B to openvswitch but before activating the agent, clean Its > openswitch entries otherwhise live migration from node A does not work > > - and so on for all remaing nodes > > Ignazio > > > > Il Gio 12 Mar 2020, 22:22 Slawek Kaplonski ha > scritto: > > Hi, > > IIRC, if You want to manually change Your database to force nova to not > use hybrid connection anymore and to not require qbr bridge You may need to > update also one of the tables in Nova’s db. It’s called > “instance_info_network_cache” or something similar. > But TBH I’m not sure if live migration then will work or not as I’m not > sure if instance’s libvirt.xml file isn’t going from src to dest node > during the live migration. > > If You don’t need to do live migration, You can switch firewall_driver in > the L2 agent’s config file and restart it. Even instances which has got > hybrid connectivity (so are plugged through qbr bridge) will have SG > working in new way. It shouldn’t be problem that those instances are > plugged through qbr bridge as it finally ends up in br-int and there SG > rules will be applied. You will need to manually clean iptables rules for > such ports as it will not be cleaned automatically. > New instances on such host should works fine and will be plugged in “new > way”, directly to br-int. > The only problem with this approach is that You will not be able to do > live-migration for those old vms. > > If You want to do it properly, You should do “nova interface-detach” and > then “nova interface-attach” for each of such instances. Then new ports > plugged to the instances will be bound in new way and plugged directly to > br-int. > > > On 12 Mar 2020, at 19:09, Ignazio Cassano > wrote: > > > > James, I checked again with your method. While live migration phase, the > informations on neutron db are changed automatically and returns with > "system", "ovs_hybrid_plug": True} ...... > > This is because the instance migrated has got interface under qbr. > > Ignazio > > > > Il giorno gio 12 mar 2020 alle ore 13:30 James Denton < > james.denton at rackspace.com> ha scritto: > > Hi Ignazio, > > > > > > > > I tested a process that converted iptables_hybrid to openvswitch > in-place, but not without a hard reboot of the VM and some massaging of the > existing bridges/veths. Since you are live-migrating, though, you might be > able to get around that. > > > > > > > > Regardless, to make this work, I had to update the port’s vif_details in > the Neutron DB and set ‘ovs_hybrid_plug’ to false. Something like this: > > > > > > > > > use neutron; > > > > > update ml2_port_bindings set vif_details='{"port_filter": true, > "bridge_name": "br-int", "datapath_type": "system", "ovs_hybrid_plug": > false}' where port_id='3d88982a-6b39-4f7e-8772-69367c442939' limit 1; > > > > > > > > So, perhaps making that change prior to moving the VM back to the other > compute node will do the trick. > > > > > > > > Good luck! > > > > > > > > James > > > > > > > > From: Ignazio Cassano > > Date: Thursday, March 12, 2020 at 6:41 AM > > To: openstack-discuss > > Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch > > > > > > > > CAUTION: This message originated externally, please use caution when > clicking on links or opening attachments! > > > > > > > > Hello All, I am facing some problems migrating from iptables_hybrid > frirewall to openvswitch firewall on centos 7 queens, > > > > I am doing this because I want enable security groups logs which require > openvswitch firewall. > > > > I would like to migrate without restarting my instances. > > > > I startded moving all instances from compute node 1. > > > > Then I configured openvswitch firewall on compute node 1, > > > > Instances migrated from compute node 2 to compute node 1 without > problems. > > > > Once the compute node 2 was empty, I migrated it to openvswitch. > > > > But now instances does not migrate from node 1 to node 2 because it > requires the presence of qbr bridge on node 2 > > > > > > > > This happened because migrating instances from node2 with > iptables_hybrid to compute node 1 with openvswitch, does not put the tap > under br-int as requested by openvswich firewall, but qbr is still present > on compute node 1. > > > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 fails because it exprects qbr on compute node 2 . > > > > So I think I should moving on the fly tap interfaces from qbr to br-int > on compute node 1 before migrating to compute node 2 but it is a huge work > on a lot of instances. > > > > > > > > Any workaround, please ? > > > > > > > > Ignazio > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kecarter at redhat.com Fri Mar 13 19:24:46 2020 From: kecarter at redhat.com (Kevin Carter) Date: Fri, 13 Mar 2020 14:24:46 -0500 Subject: [tripleo][operators] Removal of mistral from the TripleO Undercloud Message-ID: Hello stackers, In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4]. As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud. So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases? Although it would be perfectly acceptable to keep Mistral around a little while longer, it would also be a bummer if this one capability was the only reason we were not able to remove Mistral from the Undercloud. Any and all feedback from operators, deployers, developers, former developers, etc would be greatly appreciated. Thank you in advance. [0] https://opendev.org/openstack/python-tripleoclient/src/commit/af719b795bc9ba2b2c3dbcec855990ef815bfc2c/tripleoclient/workflows/parameters.py#L33-L74 [1] https://opendev.org/openstack/python-tripleoclient/src/commit/af719b795bc9ba2b2c3dbcec855990ef815bfc2c/tripleoclient/v1/overcloud_deploy.py#L221-L224 [2] https://review.opendev.org/#/c/457874 [3] https://specs.openstack.org/openstack/tripleo-specs/specs/pike/tripleo-derive-parameters.html [4] https://opendev.org/openstack/tripleo-heat-templates/src/commit/48da6a139367f21c4dfcd4afcbce1090cfc5f329/plan-samples/plan-environment-derived-params.yaml#L9 [5] https://opendev.org/openstack/tripleo-common/src/commit/cc4f9c57b708259e9e946c6d0c45709b3ffaabe2/workbooks/derive_params.yaml Kevin Carter IRC: kecarter -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Fri Mar 13 19:33:03 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Fri, 13 Mar 2020 15:33:03 -0400 Subject: [cinder] final reminder: ussuri virtual mid-cycle 16 march at 12:00 UTC Message-ID: <95b7c6f8-c146-912d-543b-596ac4232062@gmail.com> For people in the New York timezone, this event starts at 8:00 am on Monday. (Feedback from the last virtual meet-up was that it needed more promotion, so if you know anyone who might be (or should be) interested, please tell them, in addition to marking your own calendar.) Session Two of the Cinder Ussuri virtual mid-cycle will be held: DATE: Monday, 16 March 2020 TIME: 1200-1400 UTC LOCATION: https://bluejeans.com/3228528973 The meeting will be recorded. Please add topics to the planning etherpad: https://etherpad.openstack.org/p/cinder-ussuri-mid-cycle-planning cheers, brian From jimmy at openstack.org Fri Mar 13 20:27:58 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Fri, 13 Mar 2020 15:27:58 -0500 Subject: [openstack.org] SEO Improvements Message-ID: <5E6BECCE.2070404@openstack.org> Hi all - We've contracted a professional SEO firm to help improve search placement for all OSF projects. I'd like to crowd source this on each of the project mailing lists, so the community is able to be involved. Could you all help out in providing the below: “Wish List” of keywords: 8-12 big terms you think fit your domain (if you only have 4-5 to share, fewer is okay) - open source ci - ? At least 3 competitors - Jenkins - ? Any other input on positioning, offerings, etc. that you think will help best filter for relevance - ? Cheers, Jimmy From johfulto at redhat.com Fri Mar 13 20:36:45 2020 From: johfulto at redhat.com (John Fulton) Date: Fri, 13 Mar 2020 16:36:45 -0400 Subject: [tripleo][operators] Removal of mistral from the TripleO Undercloud In-Reply-To: References: Message-ID: On Fri, Mar 13, 2020 at 3:27 PM Kevin Carter wrote: > > Hello stackers, > > In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4]. As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud. > > So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases? Although it would be perfectly acceptable to keep Mistral around a little while longer, it would also be a bummer if this one capability was the only reason we were not able to remove Mistral from the Undercloud. > > Any and all feedback from operators, deployers, developers, former developers, etc would be greatly appreciated. > > Thank you in advance. Hey Kevin, +Saravanan +Jaganathan +Alan and I collaborated on the derive parameters feature. We used Mistral because that was the recommended method at the time. I think the same functionality could be implemented in Ansible. Derive parameters is used for both HCI and NFV deployments which benefit from specific tuning relative to the hardware used. Though a deployer can determine the appropriate tuning and put those values in their environment files to override the defaults, determining them can be error prone or tedious. The feature uses information from introspection and some formulas recommended by performance engineers to determine the value automatically and then dynamically populate the tuned values in the environment file. The Mistral based workflow took advantage of the deployment plan which was stored in Swift on the undercloud. My understanding is that too is going away. At the moment there's a number of parameter transformations which happen along this path: mistral --> deployment plan --> heat --> ansible The Heat to Ansible layer is one place to move the translation. E.g. we could have a Heat parameter to trigger the feature and then have new ansible roles which compute the desired values during the execution of config-download. They'd need to have access to the introspection data as a prerequisite. I'm not sure this email exchange will be sufficient though to redesign the feature. I'm curious what Saravanan thinks. Do we need a blueprint or spec to change it and do the work during Victoria? John > > > [0] https://opendev.org/openstack/python-tripleoclient/src/commit/af719b795bc9ba2b2c3dbcec855990ef815bfc2c/tripleoclient/workflows/parameters.py#L33-L74 > [1] https://opendev.org/openstack/python-tripleoclient/src/commit/af719b795bc9ba2b2c3dbcec855990ef815bfc2c/tripleoclient/v1/overcloud_deploy.py#L221-L224 > [2] https://review.opendev.org/#/c/457874 > [3] https://specs.openstack.org/openstack/tripleo-specs/specs/pike/tripleo-derive-parameters.html > [4] https://opendev.org/openstack/tripleo-heat-templates/src/commit/48da6a139367f21c4dfcd4afcbce1090cfc5f329/plan-samples/plan-environment-derived-params.yaml#L9 > [5] https://opendev.org/openstack/tripleo-common/src/commit/cc4f9c57b708259e9e946c6d0c45709b3ffaabe2/workbooks/derive_params.yaml > > > Kevin Carter > IRC: kecarter From jimmy at openstack.org Fri Mar 13 20:46:24 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Fri, 13 Mar 2020 15:46:24 -0500 Subject: [openstack.org] SEO Improvements In-Reply-To: <5E6BECCE.2070404@openstack.org> References: <5E6BECCE.2070404@openstack.org> Message-ID: <5E6BF120.9040103@openstack.org> Sorry - I accidentally left Zuul keywords and examples in there. Fixed below: > Jimmy McArthur > March 13, 2020 at 3:27 PM > Hi all - > > We've contracted a professional SEO firm to help improve search > placement for all OSF projects. I'd like to crowd source this on each > of the project mailing lists, so the community is able to be > involved. Could you all help out in providing the below: > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if > you only have 4-5 to share, fewer is okay) > - open infrastructure > - ? > > At least 3 competitors > - AWS > - ? > > Any other input on positioning, offerings, etc. that you think will > help best filter for relevance > - ? > > Cheers, > Jimmy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Fri Mar 13 20:59:30 2020 From: smooney at redhat.com (Sean Mooney) Date: Fri, 13 Mar 2020 20:59:30 +0000 Subject: [openstack.org] SEO Improvements In-Reply-To: <5E6BF120.9040103@openstack.org> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> Message-ID: On Fri, 2020-03-13 at 15:46 -0500, Jimmy McArthur wrote: > Sorry - I accidentally left Zuul keywords and examples in there. Fixed > below: > > > Jimmy McArthur > > March 13, 2020 at 3:27 PM > > Hi all - > > > > We've contracted a professional SEO firm to help improve search > > placement for all OSF projects. I'd like to crowd source this on each > > of the project mailing lists, so the community is able to be > > involved. Could you all help out in providing the below: > > > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if > > you only have 4-5 to share, fewer is okay) > > - open infrastructure > > - ? > > > > At least 3 competitors > > - AWS > > - ? > > > > Any other input on positioning, offerings, etc. that you think will > > help best filter for relevance > > - ? honestly the only think i would like to see is fixing the search result so that the 'latest' version of all our doc are at the top of the list instead of pike. the docs for our older release always come up first and its hard fo fine the direct link to the 'latest' version which tracks master. granted i have it save in my broswer history but when users are looking for docs on things it would be nice if they got the more recent docs. > > > > Cheers, > > Jimmy > > > > From fungi at yuggoth.org Fri Mar 13 21:33:41 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 13 Mar 2020 21:33:41 +0000 Subject: [openstack.org] SEO Improvements In-Reply-To: References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> Message-ID: <20200313213341.ivhagnfygtfce5vm@yuggoth.org> On 2020-03-13 20:59:30 +0000 (+0000), Sean Mooney wrote: [...] > honestly the only think i would like to see is fixing the search > result so that the 'latest' version of all our doc are at the top > of the list instead of pike. the docs for our older release always > come up first and its hard fo fine the direct link to the 'latest' > version which tracks master. > > granted i have it save in my broswer history but when users are > looking for docs on things it would be nice if they got the more > recent docs. Yep, this trips me up fairly often as well. The pattern of serving multiple versions of documentation is a fairly widespread one, far beyond just OpenStack circles, so maybe we should look at some other examples and see if we can reverse engineer how they manage to direct search results to their latest versions. For example, why do searches for Python module names return results under https://docs.python.org/3/ before they return results for https://docs.python.org/3.5/ ? I briefly skimmed the page sources for some examples but nothing jumped out at me, nor did the site's robots.txt provide any insight. Perhaps SEO specialists know what trick is at play there? -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From jimmy at openstack.org Fri Mar 13 21:49:24 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Fri, 13 Mar 2020 16:49:24 -0500 Subject: [openstack.org] SEO Improvements In-Reply-To: <20200313213341.ivhagnfygtfce5vm@yuggoth.org> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> Message-ID: <5E6BFFE4.8090905@openstack.org> > Jeremy Stanley > March 13, 2020 at 4:33 PM > On 2020-03-13 20:59:30 +0000 (+0000), Sean Mooney wrote: > [...] > > Yep, this trips me up fairly often as well. The pattern of serving > multiple versions of documentation is a fairly widespread one, far > beyond just OpenStack circles, so maybe we should look at some other > examples and see if we can reverse engineer how they manage to > direct search results to their latest versions. For example, why do > searches for Python module names return results under > https://docs.python.org/3/ before they return results for > https://docs.python.org/3.5/ ? I briefly skimmed the page sources > for some examples but nothing jumped out at me, nor did the site's > robots.txt provide any insight. Perhaps SEO specialists know what > trick is at play there? I will add it to my list of questions. Normally you'd do it with redirects to the latest, but that doesn't help if you're trying to keep archived documentation. > Sean Mooney > March 13, 2020 at 3:59 PM > On Fri, 2020-03-13 at 15:46 -0500, Jimmy McArthur wrote: >> Sorry - I accidentally left Zuul keywords and examples in there. Fixed >> below: >> >>> Jimmy McArthur >>> March 13, 2020 at 3:27 PM >>> Hi all - >>> >>> We've contracted a professional SEO firm to help improve search >>> placement for all OSF projects. I'd like to crowd source this on each >>> of the project mailing lists, so the community is able to be >>> involved. Could you all help out in providing the below: >>> >>> “Wish List” of keywords: 8-12 big terms you think fit your domain (if >>> you only have 4-5 to share, fewer is okay) >>> - open infrastructure >>> - ? >>> >>> At least 3 competitors >>> - AWS >>> - ? >>> >>> Any other input on positioning, offerings, etc. that you think will >>> help best filter for relevance >>> - ? > > honestly the only think i would like to see is fixing the search result so that the 'latest' version of all our doc > are at the top of the list instead of pike. > the docs for our older release always come up first and its hard fo fine the direct link to the 'latest' version which > tracks master. > > granted i have it save in my broswer history but when users are looking for docs on things it would be nice if they got > the more recent docs. >>> Cheers, >>> Jimmy >>> > > Jimmy McArthur > March 13, 2020 at 3:46 PM > Sorry - I accidentally left Zuul keywords and examples in there. Fixed > below: > > > Jimmy McArthur > March 13, 2020 at 3:27 PM > Hi all - > > We've contracted a professional SEO firm to help improve search > placement for all OSF projects. I'd like to crowd source this on each > of the project mailing lists, so the community is able to be > involved. Could you all help out in providing the below: > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if > you only have 4-5 to share, fewer is okay) > - open source ci > - ? > > At least 3 competitors > - Jenkins > - ? > > Any other input on positioning, offerings, etc. that you think will > help best filter for relevance > - ? > > Cheers, > Jimmy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bholeprashant.oss at gmail.com Sat Mar 14 01:10:55 2020 From: bholeprashant.oss at gmail.com (Prashant Bhole) Date: Sat, 14 Mar 2020 10:10:55 +0900 Subject: [requirements] add sqlalchemy-filters Message-ID: Hello requirements people, I wanted to discuss about addition of this new requirement: sqlalchemy-filters https://pypi.org/project/sqlalchemy-filters/ Why it is needed: This library is used in Tacker project for the implementation of Attribute Filtering of VNF Packages defined in ETSI NFV_SOL005 section 9.4.2.3.2. It is used for adding filter criterion to SQL alchemy queries. These criterion are based on attribute values of VNF Packages passed in the GET request to list VNF Packages. This library can be reused for similar cases in future. Matthew Thode had raised a concern about this project being stale as there are pending PRs. On contacting with the developer of the library the developer said that he will will work on pending PRs. The project is being actively maintained. Please review it. https://review.opendev.org/#/c/710555/ Thanks, Prashant From mthode at mthode.org Sat Mar 14 03:52:58 2020 From: mthode at mthode.org (Matthew Thode) Date: Fri, 13 Mar 2020 22:52:58 -0500 Subject: [requirements] add sqlalchemy-filters In-Reply-To: References: Message-ID: <20200314035258.dd3yprigppaifmo4@mthode.org> On 20-03-14 10:10:55, Prashant Bhole wrote: > Hello requirements people, > I wanted to discuss about addition of this new requirement: > sqlalchemy-filters > https://pypi.org/project/sqlalchemy-filters/ > > Why it is needed: > This library is used in Tacker project for the implementation > of Attribute Filtering of VNF Packages defined in ETSI > NFV_SOL005 section 9.4.2.3.2. It is used for adding filter > criterion to SQL alchemy queries. These criterion are based > on attribute values of VNF Packages passed in the GET > request to list VNF Packages. This library can be reused > for similar cases in future. > > Matthew Thode had raised a concern about this project > being stale as there are pending PRs. On contacting with > the developer of the library the developer said that he will > will work on pending PRs. The project is being actively > maintained. > > Please review it. > https://review.opendev.org/#/c/710555/ > > Thanks, > Prashant > Good to hear about the dev becoming more active again (and more importantly responding upon being contacted). I'll go over the review again this weekend. -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From mthode at mthode.org Sat Mar 14 04:00:02 2020 From: mthode at mthode.org (Matthew Thode) Date: Fri, 13 Mar 2020 23:00:02 -0500 Subject: [requirements][all] migration OFF of mock to the unittest.mock built-in library In-Reply-To: <3a378a14059936392c5c5b8e240dd91566bc8e23.camel@evrard.me> References: <20200312160033.xmtffmxv5zvqqvs6@mthode.org> <3a378a14059936392c5c5b8e240dd91566bc8e23.camel@evrard.me> Message-ID: <20200314040002.spx22yq3i2klltbr@mthode.org> On 20-03-13 10:00:57, Jean-Philippe Evrard wrote: > On Thu, 2020-03-12 at 22:25 +0100, Slawek Kaplonski wrote: > > Hi, > > > > > On 12 Mar 2020, at 17:00, Matthew Thode wrote: > > > > > > I'd like to suggest, now that we are on modern python, that we stop > > > using the mock library and instead use the built in mock library. > > > > It sounds like good candidate for community goal for the next cycle > > for me :) > > > > > > > Agreed. Matt, can you propose it? Proposing it doesn't mean that it > will be selected, nor that you sign up for work, it just makes sure > it's available in the pool of goals to select it from. You seem to have > a definitive idea, might be worth documenting it! > > > Regards, > JP > > https://etherpad.openstack.org/p/community-goals seemed like the right first place to put it, so that's where it goes (at the bottom). -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ramishra at redhat.com Sat Mar 14 12:06:20 2020 From: ramishra at redhat.com (Rabi Mishra) Date: Sat, 14 Mar 2020 17:36:20 +0530 Subject: [tripleo][operators] Removal of mistral from the TripleO Undercloud In-Reply-To: References: Message-ID: On Sat, Mar 14, 2020 at 2:10 AM John Fulton wrote: > On Fri, Mar 13, 2020 at 3:27 PM Kevin Carter wrote: > > > > Hello stackers, > > > > In the pursuit to remove Mistral from the TripleO undercloud, we've > discovered an old capability that we need to figure out how best to handle. > Currently, we provide the ability for an end-user (operator / deployer) to > pass in "N" Mistral workflows as part of a given deployment plan which is > processed by python-tripleoclient at runtime [0][1]. From what we have > documented, and what we can find within the code-base, we're not using this > feature by default. That said, we do not remove something if it is valuable > in the field without an adequate replacement. The ability to run arbitrary > Mistral workflows at deployment time was first created in 2017 [2] and > while present all this time, its documented [3] and intra-code-base uses > are still limited to samples [4]. As it stands now, we're on track to making Mistral inert this cycle and if > our progress holds over the next couple of weeks the capability to run > arbitrary Mistral workflows will be the only thing left within our codebase > that relies on Mistral running on the Undercloud. > > > > So the question is what do we do with functionality. Do we remove this > ability out right, do we convert the example workflow [5] into a > stand-alone Ansible playbook and change the workflow runner to an arbitrary > playbook runner, or do we simply leave everything as-is and deprecate it to > be removed within the next two releases? Yeah, as John mentioned, tripleo.derive_params.v1.derive_parameters workflow is surely being used for NFV (DPDK/SR-IOV) and HCI use cases and can't be deprecated or dropped. Though we've a generic interface in tripleoclient to run any workflow in plan-environment, I have not seen it being used for anything other than the mentioned workflow. In the scope of 'mistral-ansible' work, we seem to have two options. 1. Convert the workflow to ansible playbook 'as-is' i.e calculate and merge the derived parameters in plan-environment and as you've mentioned, change tripleoclient code to call any playbook in plan-environment.yaml and the parameters/vars. 2. Move the functionality further down the component chain in TripleO to have the required ansible host/group_vars set for them to be used by config-download playbooks/ansible/puppet. I guess option 1 would be easier within the timelines. I've done some preliminary work to move some of the functionality in relevant mistral actions to utils modules[1], so that they can be called from ansible modules without depending on mistral/mistral-lib and use those in a playbook that kinda replicate the tasks in the mistral workflow. Having said that, it would be good to know what the DFG:NFV folks think, as they are the original authors/maintainers of that workflow. The Mistral based workflow took advantage of the deployment plan which > was stored in Swift on the undercloud. My understanding is that too is > going away. I'm not sure that would be in the scope of 'mstral-to-ansible' work. Dropping swift would probably be a bit more complex, as we use it to store templates, plan-environment, plan backups (possibly missing few more) etc and would require significant design rework (may be possible when we get rid of heat in undercloud). In spite of heat using the templates from swift and merging environments on the client side, we've had already bumped heat's REST API json body size limit (max_json_body_size) on the undercloud to 4MB[2] from the default 1MB and sending all required templates as part of API request would not be a good idea from undercloud scalability pov. [1] https://review.opendev.org/#/c/709546/ [2] https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml#L109 -- Regards, Rabi Mishra -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Sat Mar 14 13:36:49 2020 From: donny at fortnebula.com (Donny Davis) Date: Sat, 14 Mar 2020 09:36:49 -0400 Subject: [openstack.org] SEO Improvements In-Reply-To: <5E6BFFE4.8090905@openstack.org> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> Message-ID: On Fri, Mar 13, 2020 at 5:53 PM Jimmy McArthur wrote: > > Jeremy Stanley > March 13, 2020 at 4:33 PM > On 2020-03-13 20:59:30 +0000 (+0000), Sean Mooney wrote: > [...] > > Yep, this trips me up fairly often as well. The pattern of serving > multiple versions of documentation is a fairly widespread one, far > beyond just OpenStack circles, so maybe we should look at some other > examples and see if we can reverse engineer how they manage to > direct search results to their latest versions. For example, why do > searches for Python module names return results under > https://docs.python.org/3/ before they return results for > https://docs.python.org/3.5/ ? I briefly skimmed the page sources > for some examples but nothing jumped out at me, nor did the site's > robots.txt provide any insight. Perhaps SEO specialists know what > trick is at play there? > > I will add it to my list of questions. Normally you'd do it with > redirects to the latest, but that doesn't help if you're trying to keep > archived documentation. > > Sean Mooney > March 13, 2020 at 3:59 PM > > On Fri, 2020-03-13 at 15:46 -0500, Jimmy McArthur wrote: > > Sorry - I accidentally left Zuul keywords and examples in there. Fixed > below: > > > Jimmy McArthur > March 13, 2020 at 3:27 PM > Hi all - > > We've contracted a professional SEO firm to help improve search > placement for all OSF projects. I'd like to crowd source this on each > of the project mailing lists, so the community is able to be > involved. Could you all help out in providing the below: > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if > you only have 4-5 to share, fewer is okay) > - open infrastructure > - ? > > At least 3 competitors > - AWS > - ? > > Any other input on positioning, offerings, etc. that you think will > help best filter for relevance > - ? > > honestly the only think i would like to see is fixing the search result so that the 'latest' version of all our doc > are at the top of the list instead of pike. > the docs for our older release always come up first and its hard fo fine the direct link to the 'latest' version which > tracks master. > > granted i have it save in my broswer history but when users are looking for docs on things it would be nice if they got > the more recent docs. > > Cheers, > Jimmy > > > Jimmy McArthur > March 13, 2020 at 3:46 PM > Sorry - I accidentally left Zuul keywords and examples in there. Fixed > below: > > > Jimmy McArthur > March 13, 2020 at 3:27 PM > Hi all - > > We've contracted a professional SEO firm to help improve search placement > for all OSF projects. I'd like to crowd source this on each of the project > mailing lists, so the community is able to be involved. Could you all help > out in providing the below: > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if you > only have 4-5 to share, fewer is okay) > - open source ci > - ? > > At least 3 competitors > - Jenkins > - ? > > Any other input on positioning, offerings, etc. that you think will help > best filter for relevance > - ? > > Cheers, > Jimmy > > > It would be really great if when you click the current release is X button at the top of the page it would reload the same doc and just replace the release instead of directing you back to the home page of the doc release. [image: image.png] -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6459 bytes Desc: not available URL: From smooney at redhat.com Sat Mar 14 14:05:32 2020 From: smooney at redhat.com (Sean Mooney) Date: Sat, 14 Mar 2020 14:05:32 +0000 Subject: [openstack.org] SEO Improvements In-Reply-To: References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> Message-ID: <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> On Sat, 2020-03-14 at 09:36 -0400, Donny Davis wrote: > On Fri, Mar 13, 2020 at 5:53 PM Jimmy McArthur wrote: > > > > > Jeremy Stanley > > March 13, 2020 at 4:33 PM > > On 2020-03-13 20:59:30 +0000 (+0000), Sean Mooney wrote: > > [...] > > > > Yep, this trips me up fairly often as well. The pattern of serving > > multiple versions of documentation is a fairly widespread one, far > > beyond just OpenStack circles, so maybe we should look at some other > > examples and see if we can reverse engineer how they manage to > > direct search results to their latest versions. For example, why do > > searches for Python module names return results under > > https://docs.python.org/3/ before they return results for > > https://docs.python.org/3.5/ ? I briefly skimmed the page sources > > for some examples but nothing jumped out at me, nor did the site's > > robots.txt provide any insight. Perhaps SEO specialists know what > > trick is at play there? > > > > I will add it to my list of questions. Normally you'd do it with > > redirects to the latest, but that doesn't help if you're trying to keep > > archived documentation. > > > > Sean Mooney > > March 13, 2020 at 3:59 PM > > > > On Fri, 2020-03-13 at 15:46 -0500, Jimmy McArthur wrote: > > > > Sorry - I accidentally left Zuul keywords and examples in there. Fixed > > below: > > > > > > Jimmy McArthur > > March 13, 2020 at 3:27 PM > > Hi all - > > > > We've contracted a professional SEO firm to help improve search > > placement for all OSF projects. I'd like to crowd source this on each > > of the project mailing lists, so the community is able to be > > involved. Could you all help out in providing the below: > > > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if > > you only have 4-5 to share, fewer is okay) > > - open infrastructure > > - ? > > > > At least 3 competitors > > - AWS > > - ? > > > > Any other input on positioning, offerings, etc. that you think will > > help best filter for relevance > > - ? > > > > honestly the only think i would like to see is fixing the search result so that the 'latest' version of all our doc > > are at the top of the list instead of pike. > > the docs for our older release always come up first and its hard fo fine the direct link to the 'latest' version > > which > > tracks master. > > > > granted i have it save in my broswer history but when users are looking for docs on things it would be nice if they > > got > > the more recent docs. > > > > Cheers, > > Jimmy > > > > > > Jimmy McArthur > > March 13, 2020 at 3:46 PM > > Sorry - I accidentally left Zuul keywords and examples in there. Fixed > > below: > > > > > > Jimmy McArthur > > March 13, 2020 at 3:27 PM > > Hi all - > > > > We've contracted a professional SEO firm to help improve search placement > > for all OSF projects. I'd like to crowd source this on each of the project > > mailing lists, so the community is able to be involved. Could you all help > > out in providing the below: > > > > “Wish List” of keywords: 8-12 big terms you think fit your domain (if you > > only have 4-5 to share, fewer is okay) > > - open source ci > > - ? > > > > At least 3 competitors > > - Jenkins > > - ? > > > > Any other input on positioning, offerings, etc. that you think will help > > best filter for relevance > > - ? > > > > Cheers, > > Jimmy > > > > > > > > It would be really great if when you click the current release is X button > at the top of the page it would reload the same doc and just replace the > release instead of directing you back to the home page of the doc release. yes althought that is not a SEO thing i think we need to change how that baner is create but i basically always ignore it because it does not do what i want so removing it entirly or more helpflly making it link to the latest verion of the current doc would both be greate improvemnts. > [image: image.png] From fungi at yuggoth.org Sat Mar 14 14:20:29 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Sat, 14 Mar 2020 14:20:29 +0000 Subject: [openstack.org] SEO Improvements In-Reply-To: <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> Message-ID: <20200314142029.lf55ksds2avnzdbx@yuggoth.org> On 2020-03-14 14:05:32 +0000 (+0000), Sean Mooney wrote: > On Sat, 2020-03-14 at 09:36 -0400, Donny Davis wrote: [...] > > It would be really great if when you click the current release > > is X button at the top of the page it would reload the same doc > > and just replace the release instead of directing you back to > > the home page of the doc release. [...] > yes althought that is not a SEO thing i think we need to change > how that baner is create but i basically always ignore it because > it does not do what i want so removing it entirly or more helpflly > making it link to the latest verion of the current doc would both > be greate improvemnts. The solution to the technical problem is straightforward (even a simple HTML form element would to the trick), the bigger challenge is logistical: Content moves around between releases, it's not just renamed but often gets split or combined too, and lots of times folks don't think to include redirects from old filenames when they make major edits like that. All too often when I'm looking something up in the published HTML copies of our documentation I'll land on the wrong version, edit the URL to a later release, and get a 404 for that file. Someone (really multiple someones) will need to do a thorough audit to work out what redirects we missed between previous releases as well as coming up with ways to better remind folks to add redirects and maybe leave other breadcrumb trails when making future changes. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From zigo at debian.org Sat Mar 14 21:52:45 2020 From: zigo at debian.org (Thomas Goirand) Date: Sat, 14 Mar 2020 22:52:45 +0100 Subject: OpenStack support for msgpack 1.0.0 Message-ID: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> Hi, msgpack 1.0.0 was released last Sept. I'm being pressed by the rest of Debian people to upload 1.0.0 to Sid. I'm concern by it: will this break any of the OpenStack package, like oslo.messaging? Is there any plan to have support for 1.0.0 for Ussuri? Cheers, Thomas Goirand (zigo) From mthode at mthode.org Sat Mar 14 22:23:45 2020 From: mthode at mthode.org (Matthew Thode) Date: Sat, 14 Mar 2020 17:23:45 -0500 Subject: OpenStack support for msgpack 1.0.0 In-Reply-To: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> References: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> Message-ID: <20200314222345.xrcz2e3275wtu6bb@mthode.org> On 20-03-14 22:52:45, Thomas Goirand wrote: > Hi, > > msgpack 1.0.0 was released last Sept. I'm being pressed by the rest of > Debian people to upload 1.0.0 to Sid. I'm concern by it: will this break > any of the OpenStack package, like oslo.messaging? Is there any plan to > have support for 1.0.0 for Ussuri? > > Cheers, > > Thomas Goirand (zigo) > Openstack depends on salt Salt capped msgpack to not allow 1.0.0 https://github.com/saltstack/salt/pull/56009 Openstack can't use msgpack-1.0.0 this makes me sad iirc openstack tests passed with it, I just made a review to test https://review.opendev.org/713113 When I mentioned this to upstream salt people they said it'd be in a future release, I'd project after the openstack release (it's not part of 3000.1 which is being preparred now). -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From fungi at yuggoth.org Sat Mar 14 22:31:19 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Sat, 14 Mar 2020 22:31:19 +0000 Subject: OpenStack support for msgpack 1.0.0 In-Reply-To: <20200314222345.xrcz2e3275wtu6bb@mthode.org> References: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> <20200314222345.xrcz2e3275wtu6bb@mthode.org> Message-ID: <20200314223119.7aody6vswaoh5kzi@yuggoth.org> On 2020-03-14 17:23:45 -0500 (-0500), Matthew Thode wrote: [...] > Openstack depends on salt [...] Out of curiosity, what in OpenStack depends on Salt these days? It's not just a holdover from back when we had an official deployment team for that, is it? -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From mnaser at vexxhost.com Sat Mar 14 22:42:13 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Sat, 14 Mar 2020 18:42:13 -0400 Subject: OpenStack support for msgpack 1.0.0 In-Reply-To: <20200314223119.7aody6vswaoh5kzi@yuggoth.org> References: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> <20200314222345.xrcz2e3275wtu6bb@mthode.org> <20200314223119.7aody6vswaoh5kzi@yuggoth.org> Message-ID: On Sat, Mar 14, 2020 at 6:34 PM Jeremy Stanley wrote: > > On 2020-03-14 17:23:45 -0500 (-0500), Matthew Thode wrote: > [...] > > Openstack depends on salt > [...] > > Out of curiosity, what in OpenStack depends on Salt these days? It's > not just a holdover from back when we had an official deployment > team for that, is it? +1 I'm also curious why we're depending on it :> > -- > Jeremy Stanley -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From mthode at mthode.org Sat Mar 14 22:45:59 2020 From: mthode at mthode.org (Matthew Thode) Date: Sat, 14 Mar 2020 17:45:59 -0500 Subject: [heat][requirements] OpenStack support for msgpack 1.0.0 In-Reply-To: <20200314223119.7aody6vswaoh5kzi@yuggoth.org> References: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> <20200314222345.xrcz2e3275wtu6bb@mthode.org> <20200314223119.7aody6vswaoh5kzi@yuggoth.org> Message-ID: <20200314224559.ngvwxd6rhhagevob@mthode.org> On 20-03-14 22:31:19, Jeremy Stanley wrote: > On 2020-03-14 17:23:45 -0500 (-0500), Matthew Thode wrote: > [...] > > Openstack depends on salt > [...] > > Out of curiosity, what in OpenStack depends on Salt these days? It's > not just a holdover from back when we had an official deployment > team for that, is it? > -- > Jeremy Stanley It's heat-agents, if we could get rid of the dep that'd be awesome, but based on the discussion when it was added they said they didn't want to shell out, not sure what's up with how they use ansible and the rest. It's only used for tests though, so maybe it's not as needed now? | openstack/heat-agents | test-requirements.txt | 11 | salt>=2017.7.4 # Apache-2.0 | -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ignaziocassano at gmail.com Sun Mar 15 10:58:58 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 15 Mar 2020 11:58:58 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: Message-ID: Hello Jakub, to be honest I did not understand what multiple-port binding feature does. I guess it can help me to specify how port are created on destination host, for example during live migration and I wonder if this feature can help me to migrate from iptables-hybrid to openvswitch without restarting instances. Could you provide any example for setting multiple-port bindings, please ? Ignazio Il giorno gio 12 mar 2020 alle ore 23:15 Jakub Libosvar ha scritto: > On 12/03/2020 11:38, Ignazio Cassano wrote: > > Hello All, I am facing some problems migrating from iptables_hybrid > > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without > problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > > to compute node 1 with openvswitch, does not put the tap under br-int as > > requested by openvswich firewall, but qbr is still present on compute > node > > 1. > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 > > fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int > on > > compute node 1 before migrating to compute node 2 but it is a huge work > on > > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > > > > I may be a little outdated here but to the best of my knowledge there > are two ways how to migrate from iptables to openvswitch. > > 1) If you don't mind the intermediate linux bridge and you care about > logs, you can just change the config file on compute node to start using > openvswitch firewall and restart the ovs agent. That should trigger a > mechanism that deletes iptables rules and starts using openflow rules. > It will leave the intermediate bridge there but except the extra hop in > networking stack, it doesn't mind. > > 2) With multiple-port binding feature, what you described above should > be working. I know Miguel spent some time working on that so perhaps he > has more information about which release it should be functional at, I > think it was Queens. Not sure if any Nova work was required to make it > work. > > Hope that helps. > Kuba > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Sun Mar 15 17:19:07 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sun, 15 Mar 2020 12:19:07 -0500 Subject: [Tempest] OpenSack Powered * vs OpeStack Compatible In-Reply-To: <316549280.827334.1584259022111@mail.yahoo.com> References: <2130957943.152281.1582365846726.ref@mail.yahoo.com> <2130957943.152281.1582365846726@mail.yahoo.com> <775D37EA-DC9B-4B80-A002-51C89A3A9E62@vmware.com> <1706963754.79322.1582614973577@mail.yahoo.com> <1707cdd49b2.ddd44e5d119585.1167564760858241265@ghanshyammann.com> <934A54A9-2E2D-4086-94FA-198E4421031E@vmware.com> <316549280.827334.1584259022111@mail.yahoo.com> Message-ID: <170df34dad4.10d43d2ae172196.1744031088269733049@ghanshyammann.com> ---- On Sun, 15 Mar 2020 02:57:02 -0500 prakash RAMCHANDRAN wrote ---- > Mark, > it took me some time to get up to speed and a thank you for your thoroughful brief on the subject. > Like to add few more questions before we get on call to see where we go from here. > 5th Area for Interoperability: Should we add latest kubernetes API testing to Tempest or adapt third party Open Source tool to achieve K8S API compliance. Tempest provides the plugin framework where you can write the k8s API testing. I remember that there was a proposal for a container testing tempest plugin. Or I will say check the existing tooling if that works for example: e2e testing[1]. As mentioned earlier, If you want to tests OpenStack APIs then how it is deployed ( either on k8s cluster) does not matter and Tempest existing test will work as it is. > Motivation : > A. Would Like Open Infrastructure project to align in OpenSrack to kubernetes Cluster API for Containers and containerized workloads. > B. The k8s API must have tests to support kubernetes infra on Baremetal and VM. > C. Offer or ask each of the Core services using Container (Docker latest) to an internal test for CRI, CNI and CSI certifications. > D. Level of Branding will be OpenStack and Kubernetes Dual-Stack certification based o. each of the service modules like Nova, Neutron, Cinder, Ceph or any other of projects within or third party k8s tools that will use to get OpebStack trusted interoperability for kubernetes Infrastructure under Open Infrastructure initiative. > This us just the beginning to improve Branding efforts which other community's trust us to be the guides. Let's see if this out of box thinking needs shooting down or should we revive the trailing indicators for adoption and interoperability. > Please advice if thusdditiin should be at end user level or can include kubeadmin APIs too?Should we support RBAC options of keystone or strictly start with K8s RBAC support in Keystone to switch between keystone v3 for VMs vs RBAC v? for Containers? > Think through as a team and let's see if this helps our concept if Dual-Stack support. > Sure we have a bunch of Container projects and they may suggest brownfield options, but stick to Greenfield first before addressing them to fall in line with commo. interoperability area 5 - dual-stack using k8s. > How do we setup a Call for Tempest or DefCore team to review this. Let me have your reply to see if we can get TC members to bring this as response to ask from Thiery for rebooting DefCore Branding efforts. You can reach out to the Tempest team on #openstack-qa IRC channel or let me know if any other communication channel you would like to discuss. [1] https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md -gmann > ThanksPrakash > > > > > Sent from Yahoo Mail on Android > On Wed, Feb 26, 2020 at 6:32 AM, Mark Voelker wrote: > On Feb 25, 2020, at 10:00 AM, Ghanshyam Mann wrote: > ---- On Tue, 25 Feb 2020 01:16:13 -0600 prakash RAMCHANDRAN wrote ---- > Mark, > Glad you pointed to right code. Reviewed and stand corrected. My mis-underdtanding was I considered 20919.01 as first release and 2019.02 as second release of the year. However based on your comments and reference I understand that it is year.month of release , thus 2019.11 includes 'usuri' and previous 3 releases as listed pointed by you. > "os_trademark_approval": { > "target_approval": "2019.11", > "replaces": "2019.06", > "releases": ["rocky", "stein", "train", "ussuri"], > "status": "approved" > } > }, > > > That clears that I should have asked for 2019.11. > Few more questions on Tempedt tests. > I read some where that we have about 1500 Tempest tests overall. Is that correct? > > Yeah, it might be little more or less but around 1500 in-tree tests in Tempest. > > The interop code lines have gone down from 3836 lines to 3232 in train to usuri. > Looks contrary to growth, any comments? > > As Thierry pointed out, a chunk of this was due to the removal of the volumes-v2 API from the required list. A few other tests have been removed over time for various other reasons (changes to or removal of tests in Tempest, etc). Since the test lists are kept in git, you can actually walk through the complete history yourself to see why some tests were added or removed if you’d like: > https://opendev.org/openstack/interop/commits/branch/master > It’s also important to note that just because we’ve added more tests over time to Tempest, more projects over time to OpenStack, or more API’s to existing projects, that doesn’t mean there will be a corresponding increase in the number of tests required for the trademark programs. The OpenStack Powered program is more a trailing indicator of adoption than an attempt to force commercial products to support any and all capabilities. Each API is considered for inclusion in the program against a set of criteria detailed here: > https://opendev.org/openstack/interop/src/branch/master/doc/source/process/CoreCriteria.rst > So, for example: if a project introduced a new API in Usuri, it’s highly unlikely that it would appear in the very next Guideline because it would fail to meet several criteria. It would be unlikely to be “widely deployed” since many deployments would still be using Stein or Train (also covered in the same Guideline). It might not yet be supported in many third-party SDK’s or tools, so it might not yet meet the “used by tools” criteria. It might be only supported by one or two plugins/drivers in it’s first release. Etc, etc, etc. Over time that API might gain adoption, meet sufficient criteria, and be added to the required list--or it might not. > If you’re curious about the history of all this and the process, you might have a look at this slightly old but still mostly relevant deck: > https://www.slideshare.net/markvoelker/interopwg-intro-vertical-programs-jan-2017 > > Question then is 60 compute and 40 storage lines I see in test cases, do we have stats for Tempest tests what's the distribution of 1500 tests across Platform, compute, storage etc. Where and how can I get that. information as documented report. > > Those should be counted from interop guidelines where you have mapping of capabilities with test cases. > Few or most of the capabilities have one test to verifying it or a few more than one test. > For example, "compute-flavors-list" capability is verified by two tests[2]. > This way you can count and identify the exact number of tests per compute, storage etc. > > If you would like to know about the Tempest test categorization, you can find it from the directory > structure. We have structured the tests service wise directory, for example, all compute tests under > tempest/api/compute [2]. > > > Based on above should we expect decrease or increase for say 2020.05 Vancouver release? > > Anecdotally, the Guidelines haven’t been changing very much in recent times as the “core” capabilities that have met with a lot of adoption seem fairly well established (though there have been a few new ones added and a few removed). I wouldn’t expect dramatic changes. > How does one certify a kubernetes cluster running openstak modules, one module per docker container in a kubrrnetes cluster using tempest say like in Airship Control Plane on k8s worker node, which is a OpenStack over kubernetes cluster. > Is this possible and if so what test we need to modify to test and certify a Containerized OpenStack in Airship as OpenStack Platform? > > I should be verified same way via Tempest. Tempest does not reply on how OpenStack is deployed it interacts via the public > interface (where interop needs only user-facing API excluding admin API) of each service which should be accessible on > k8s cluster also. Tests are not required to modify in this case. > > NOTE: we can always extend (writing new tests for current 6 services tests) Tempest for tests required for > interop capabilities either that are new or improved in the verification of existing ones. I remember > about the discussion on covering the API microversion. We keep adding new tests to cover the > new microverison where we need integration tests but we can add API tests also if interop requires those. > > Right on. In fact, the capabilities in the interop programs are intended to be usable independent of how a cloud is deployed (including whether it’s containerized or on bare metal, whether it uses KVM/OVS/Ceph or vSphere/NSX/vSAN, whether it’s highly available or a single-box appliance, etc). > > Can we even certify if for say 2019.11? > > The OpenStack Foundation’s interoperability programs allow you to use either of two most recently approved Guidelines for your testing (which as of right now are 2019.11 and 2019.06). Once the Board of Directors approves the next guideline, 2019.06 will rotate out. I should note though: the Foundation’s interoperability programs are primarily targeted at commercial products (distributions, public clouds, appliances, managed services, etc). There’s no real reason an open source product couldn’t use these same tests of course! > At Your Service, > Mark T. Voelker > This should open up exciting possibilities if practical to extend OpeStack powered platform to Airship. > Like to hear anyone who has insight to educate us on that. > ThanksPrakash > Sent from Yahoo Mail on Android > On Mon, Feb 24, 2020 at 6:09 AM, Mark Voelker wrote: Hi Prakash, > > I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02. > Hm, there actually isn’t a 2019.02 guideline--were you perhaps referring to 2019.06 or 2019.11? 2019.06 does cover Train but not Usuri [1], 2019.11 covers both [2]. As an FYI, the OpenStack Marketplace does list which guideline a particular product was most recently tested against (refer to https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=K9zbzRvFzs3cxVnCocij2Sjy43MNyejDKuRN7ivQkMk%3D&reserved=0 for example, and look for the green “TESTED” checkmark and accompanying guideline version), though this obviously doesn’t tell you what testing might be currently in flight. > [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.06.json%23L75%5B2&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=Vcy8iL%2F8L93m0%2B3PIwikD5vN%2BaREJigrncz7cjd3Zyc%3D&reserved=0] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.11.json%23L75&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=pTrl8xMypZJAPFu2vzQwFFOWnJu51k44oxn2PrrQI2Y%3D&reserved=0 > At Your Service, > Mark T. Voelker > > > On Feb 22, 2020, at 5:04 AM, prakash RAMCHANDRAN wrote: > Hi all, > I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02. > Similarly does any Hardware Driver of Software application seeking OpenStack compatibility Logo? > Finally does anyone think that Open Infra Distro like Airship or StarlingX should promote Open Infra Airship or Open Infra StarlingX powered as a new way to promote eco system surrounding them similar to OpenStack compatible drivers and software. > Will then Argo, customize,Metal3.io or Ironic be qualified as Open Infra Airship compatible? > If so how tempest can help in testing the above comments? > Refer to this market place below as how Distos and Products leverage OpenStack logos and branding programs. > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=rABzR1tWXUHinSG4jpe46ayuELsini4fbKcJhsxrN9g%3D&reserved=0 > Discussions and feedback are welcome. A healthy debate as how k8s modules used in Open Infra can be certified will be a good start. > ThanksPrakash > > Sent from Yahoo Mail on Android > > [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fcommit%2F8f2e82b7db54cfff9315e5647bd2ba3dd6aacaad%2F2019.11.json%23L260-L281&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=wncg32ql1Gq%2B%2F7fZPUDdPzSv2Fbay2zCAXPT%2F9%2BQIo4%3D&reserved=0 > [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Ftempest%2Fsrc%2Fbranch%2Fmaster%2Ftempest%2Fapi%2Fcompute&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=mgDV%2Bng445lIrj3CjMwdTeifyGMdXyogQhnVdXk4FS0%3D&reserved=0 > > -gmann > From ignaziocassano at gmail.com Sun Mar 15 18:11:58 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 15 Mar 2020 19:11:58 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: Message-ID: Hello Jakub, in my previous email I asked about multiple port binding but I forgot to discuss about the point 1 of your email (qbr). I do not mind about intermediate bridge but the following is what happen: A) evacuate node 1 and after changing its openvswitch agent configuration and restarting it, I can migrate vm1 form node 2. Intermediate qbr bridge is created for vm1 on node 1 B) evacuate node 2 and after changing its openvswitch agent configuration and restarting it, live migrating vm1 form node 1 to mode 2 does not works because no qbr is created on node 2 (this is reported on nova logs) Probably without restarting instances live migration works from hybrid_iptables to openvswitch but does not work fron openvswitch to openvswitch. If the first migration is not live, qbr is not created and all works fine. Regards Ignazio Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha scritto: > On 12/03/2020 11:38, Ignazio Cassano wrote: > > Hello All, I am facing some problems migrating from iptables_hybrid > > frirewall to openvswitch firewall on centos 7 queens, > > I am doing this because I want enable security groups logs which require > > openvswitch firewall. > > I would like to migrate without restarting my instances. > > I startded moving all instances from compute node 1. > > Then I configured openvswitch firewall on compute node 1, > > Instances migrated from compute node 2 to compute node 1 without > problems. > > Once the compute node 2 was empty, I migrated it to openvswitch. > > But now instances does not migrate from node 1 to node 2 because it > > requires the presence of qbr bridge on node 2 > > > > This happened because migrating instances from node2 with iptables_hybrid > > to compute node 1 with openvswitch, does not put the tap under br-int as > > requested by openvswich firewall, but qbr is still present on compute > node > > 1. > > Once I enabled openvswitch on compute node 2, migration from compute > node 1 > > fails because it exprects qbr on compute node 2 . > > So I think I should moving on the fly tap interfaces from qbr to br-int > on > > compute node 1 before migrating to compute node 2 but it is a huge work > on > > a lot of instances. > > > > Any workaround, please ? > > > > Ignazio > > > > I may be a little outdated here but to the best of my knowledge there > are two ways how to migrate from iptables to openvswitch. > > 1) If you don't mind the intermediate linux bridge and you care about > logs, you can just change the config file on compute node to start using > openvswitch firewall and restart the ovs agent. That should trigger a > mechanism that deletes iptables rules and starts using openflow rules. > It will leave the intermediate bridge there but except the extra hop in > networking stack, it doesn't mind. > > 2) With multiple-port binding feature, what you described above should > be working. I know Miguel spent some time working on that so perhaps he > has more information about which release it should be functional at, I > think it was Queens. Not sure if any Nova work was required to make it > work. > > Hope that helps. > Kuba > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Sun Mar 15 19:03:43 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sun, 15 Mar 2020 14:03:43 -0500 Subject: [goals][Drop Python 2.7 Support] Week R-9 Update Message-ID: <170df949bcb.bcb003e4173400.5451657073093464285@ghanshyammann.com> Hello Everyone, Below is the progress on "Drop Python 2.7 Support" at end of R-9 week. Schedule: https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html#schedule Highlights: ======== * We already passed the deadline but still work is not completed on this. * Few pythonclient and tempest plugins are not merged yet. * I request projects again to merge the passing patches asap. Project wise status and need reviews: ============================ Phase-1 status: All the OpenStack services have dropped the python2.7. Phase-2 status: * Pending Tempest plugins: ** barbican-tempest-plugin: https://review.opendev.org/#/c/704083/ ** cyborg-tempest-plugin: https://review.opendev.org/#/c/704076/ ** kuryr-tempest-plugin: https://review.opendev.org/#/c/704072/ * Pending pythonclient: ** python-barbicanclient: https://review.opendev.org/#/c/699096/2 ** python-zaqarclient: https://review.opendev.org/#/c/692011/4 ** python-tripleoclient: https://review.opendev.org/#/c/703344 * Few more repo patches need to merge: ** masakari-specs: https://review.opendev.org/#/c/698982/ ** cyborg-specs: https://review.opendev.org/#/c/698824/ ** masakari-monitors: https://review.opendev.org/#/c/694552/ ** nova-powervm: https://review.opendev.org/#/c/700683/ ** paunch: https://review.opendev.org/#/c/703344/ ** cloudkitty: https://review.opendev.org/#/c/709585/3 * Started pushing the required updates on deployment projects. ** Completed or no updates required: *** Openstack-Chef - not required *** Packaging-Rpm - Done *** Puppet Openstack- Done ** In progress: *** Openstack Charms - Patches are ready to merge. Need review. *** Openstackansible - In-progress. centos7 jobs are failing on few projects. ** Waiting from projects team to know the status: *** Openstack-Helm (Helm charts for OpenStack services) *** Tripleo (Deployment service) * Open review: https://review.opendev.org/#/q/topic:drop-py27-support+status:open Phase-3 status: This is audit and requirement repo work which is not started yet. I will start this once all the phase-2 work mentioned above is completed. How you can help: ============== - Review the patches. Push the patches if I missed any repo. -gmann From jean-philippe at evrard.me Mon Mar 16 08:48:13 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Mon, 16 Mar 2020 09:48:13 +0100 Subject: [tc][requirements][horizon] Broken gates due to the new setuptools and pyScss package In-Reply-To: References: Message-ID: <9bbd7f0e9fd3badec5f7f193aa8278ad86cd4279.camel@evrard.me> On Fri, 2020-03-13 at 18:29 +0200, Ivan Kolodyazhny wrote: > Hi team, > > I'm sorry for being too noisy, but I decided to tag TC to get more > attention to the current Horizon situation. Don't be sorry, it's good that you raise this point! > > We've got a bug reported by Kolla team two days ago [1]. We > merged some workaround [2] and [3] yesterday. Thanks a lot to the > Requirements team for the really quick merge! I appreciate it. > > For now, we've got horizon gates broken because of hose patches fix > only devstack but not unit tests jobs. > > The root cause of this situation is pyScss package which is not > maintained for the last two years. It's not a surprise to me that it > doesn't work with the new setuptools. I'm really surprised that we've > found this kind of bugs only now. I suppose that we can't easily get rid of the dependency... > > Since I don't believe we can block new setuptools forever, I decided > to fork pyScss [4] and django-pyscss [5] projects. I'm still not sure > that I've done everything right with licensing and versioning, but it > works now on my environment. Any help on these areas would be much > appreciated. I proposed patches to requirements and horizon repos [6] > to use new libraries. I checked your repos, they are BSD and MIT licensed. Which means: - Horizon didn't do anything wrong with using them in the first place - Your fork can be used to use them - We could make "your forks" projects on opendev. I suppose you're not only calling to say that you're now maintainer, but instead want to raise the fact that you're (now) the only maintainer, and we need more folks to step up and help maintain... > > > The reason I've tagged TC in the mail thread is described below. > > Horizon has a lot of too old and unmaintained libraries. I'm pretty > sure that this only one of the first issues with outdated > dependencies which blocks horizon and other gates. Is there a path forward to remove the usage of those dependencies, or to change things? If we have a plan, it's (relatively) less hard to point people to work on said plan to get help. > I do understand why we've got this situation. Unfortunately, we don't > have any full-time horizon > developers in the community. Horizon is mostly in maintenance phrase > but not in active development. Sadly, it can be said of multiple projects. I am definitely hoping that Horizon will get the attention it deserves. Having a plan of "renovation"/"renewal" of horizon might help, as said above. > > I would like to get more attention on this issue because we have to > update all dependencies not because they are new, have new features > and/or security fixes. We have to take care of our dependencies asap > to avoid usage of unmaintained libraries to have the whole OpenStack > and Horizon healthy. Agreed. Do you have other dependencies that are at risk here for horizon? Did you audit this recently? > > > > P.S. I'm sorry if this message is too rude or emotional, I really > don't want to make it such one. It's not rude or emotional. You're raising a good point. > > > [1] https://bugs.launchpad.net/kolla/+bug/1866961 > [2] https://review.opendev.org/#/c/711930/ > [3] https://review.opendev.org/#/c/712777/ > [4] https://github.com/e0ne/pyScss/ > [5] https://github.com/e0ne/django-pyscss > [6] https://review.opendev.org/#/q/status:open+topic:fix-pyscss > > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > I think it might be worth splitting this email in multiple topics: - Should we move pyScss and django-pyscss into opendev? (Do you want to do it, or are you fine with your current fork right now?) - How can we clean up the dependencies in horizon? - Should Horizon be listed in the business opportunities, to have someone to step up? - Should the TC audit all the official projects to avoid usage of unmaintained libraries? Regards, JP From katonalala at gmail.com Mon Mar 16 09:25:25 2020 From: katonalala at gmail.com (Lajos Katona) Date: Mon, 16 Mar 2020 10:25:25 +0100 Subject: [tc][requirements][horizon] Broken gates due to the new setuptools and pyScss package In-Reply-To: <9bbd7f0e9fd3badec5f7f193aa8278ad86cd4279.camel@evrard.me> References: <9bbd7f0e9fd3badec5f7f193aa8278ad86cd4279.camel@evrard.me> Message-ID: Hi, I would like to just add my cent to the question of ownership of these forked repos: I am on the side to have opendev ownership over them as that give long(er) term stability, even now when we have serious shortages in design. But consider the situation when e0ne leave the community, we have to do the fork again and move these repos under opendev umbrella. Regards Lajos Jean-Philippe Evrard ezt írta (időpont: 2020. márc. 16., H, 9:54): > On Fri, 2020-03-13 at 18:29 +0200, Ivan Kolodyazhny wrote: > > Hi team, > > > > I'm sorry for being too noisy, but I decided to tag TC to get more > > attention to the current Horizon situation. > > Don't be sorry, it's good that you raise this point! > > > > > We've got a bug reported by Kolla team two days ago [1]. We > > merged some workaround [2] and [3] yesterday. Thanks a lot to the > > Requirements team for the really quick merge! I appreciate it. > > > > For now, we've got horizon gates broken because of hose patches fix > > only devstack but not unit tests jobs. > > > > The root cause of this situation is pyScss package which is not > > maintained for the last two years. It's not a surprise to me that it > > doesn't work with the new setuptools. I'm really surprised that we've > > found this kind of bugs only now. > > I suppose that we can't easily get rid of the dependency... > > > > > Since I don't believe we can block new setuptools forever, I decided > > to fork pyScss [4] and django-pyscss [5] projects. I'm still not sure > > that I've done everything right with licensing and versioning, but it > > works now on my environment. Any help on these areas would be much > > appreciated. I proposed patches to requirements and horizon repos [6] > > to use new libraries. > > I checked your repos, they are BSD and MIT licensed. Which means: > - Horizon didn't do anything wrong with using them in the first place > - Your fork can be used to use them > - We could make "your forks" projects on opendev. > > I suppose you're not only calling to say that you're now maintainer, > but instead want to raise the fact that you're (now) the only > maintainer, and we need more folks to step up and help maintain... > > > > > > > The reason I've tagged TC in the mail thread is described below. > > > > Horizon has a lot of too old and unmaintained libraries. I'm pretty > > sure that this only one of the first issues with outdated > > dependencies which blocks horizon and other gates. > > Is there a path forward to remove the usage of those dependencies, or > to change things? If we have a plan, it's (relatively) less hard to > point people to work on said plan to get help. > > > I do understand why we've got this situation. Unfortunately, we don't > > have any full-time horizon > > developers in the community. Horizon is mostly in maintenance phrase > > but not in active development. > > Sadly, it can be said of multiple projects. I am definitely hoping that > Horizon will get the attention it deserves. Having a plan of > "renovation"/"renewal" of horizon might help, as said above. > > > > > I would like to get more attention on this issue because we have to > > update all dependencies not because they are new, have new features > > and/or security fixes. We have to take care of our dependencies asap > > to avoid usage of unmaintained libraries to have the whole OpenStack > > and Horizon healthy. > > Agreed. Do you have other dependencies that are at risk here for > horizon? Did you audit this recently? > > > > > > > > > P.S. I'm sorry if this message is too rude or emotional, I really > > don't want to make it such one. > > It's not rude or emotional. You're raising a good point. > > > > > > > [1] https://bugs.launchpad.net/kolla/+bug/1866961 > > [2] https://review.opendev.org/#/c/711930/ > > [3] https://review.opendev.org/#/c/712777/ > > [4] https://github.com/e0ne/pyScss/ > > [5] https://github.com/e0ne/django-pyscss > > [6] https://review.opendev.org/#/q/status:open+topic:fix-pyscss > > > > Regards, > > Ivan Kolodyazhny, > > http://blog.e0ne.info/ > > > > I think it might be worth splitting this email in multiple topics: > - Should we move pyScss and django-pyscss into opendev? (Do you want to > do it, or are you fine with your current fork right now?) > - How can we clean up the dependencies in horizon? > - Should Horizon be listed in the business opportunities, to have > someone to step up? > - Should the TC audit all the official projects to avoid usage of > unmaintained libraries? > > Regards, > JP > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bcafarel at redhat.com Mon Mar 16 09:37:26 2020 From: bcafarel at redhat.com (Bernard Cafarelli) Date: Mon, 16 Mar 2020 10:37:26 +0100 Subject: [neutron] Bug deputy report (week starting on 2020-03-09) Message-ID: Hello neutrinos, last week we started a new bug deputy rotation, opening this round here are the bugs reported in week 11. This was relatively quiet (for new bugs count), and most bugs have active discussion or suggested fix Critical * [security] Add allowed-address-pair 0.0.0.0/0 to one port will open all others' protocol under same security group - https://bugs.launchpad.net/neutron/+bug/1867119 A follow-up to security bug https://bugs.launchpad.net/neutron/+bug/1793029 (which was fixed in documentation) Potential code fix at https://review.opendev.org/712632 - reviews and opinions most welcome Medium * Restart neutron-linuxbridge-agent service led to all ports status changed - https://bugs.launchpad.net/neutron/+bug/1866743 Reported on Pike/Queens, pretty standard configuration, may be l2pop Related gerrit question: https://review.opendev.org/713156 * MTU too large error presented on create but not update - https://bugs.launchpad.net/neutron/+bug/1867214 Suggested fix: https://review.opendev.org/712801 Low * Packets incorrectly marked as martian - https://bugs.launchpad.net/neutron/+bug/1866615 Martian packets logged with some specific setup, VMs are working fine though, switching to ovs firewall workarounds the issue * Deployment has security group with empty tenant id - https://bugs.launchpad.net/neutron/+bug/1867101 Some master devstacks deployments like networking-odl get empty project ID for default security group * Unnecessary network flapping while update floatingip without port or fixed ip changed - https://bugs.launchpad.net/neutron/+bug/1867122 OVN mech driver only, some discussion about relevant use-case for FIP update in LP and patch https://review.opendev.org/712641 Incomplete * router-update for internal networking not correct when restarting ovs-agent - https://bugs.launchpad.net/neutron/+bug/1866635 Missing flows on restart, I asked for more logs - may be missing tunnel during restart Update from previous week * br-int bridge in one compute can't learn MAC addresses of VMs in other compute nodes - https://bugs.launchpad.net/neutron/+bug/1866445 Was closed as duplicate of bug #1732067 but they do not use OVS firewall Patch for iptables_hybrid proposed: https://review.opendev.org/712640 Last bug I triaged is 1867214, handing over the deputy baton to slaweq -- Bernard Cafarelli -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-philippe at evrard.me Mon Mar 16 09:52:31 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Mon, 16 Mar 2020 10:52:31 +0100 Subject: [all] Guides for newbies to OpenStack In-Reply-To: References: Message-ID: <24f6f94cf9eabffd30389ad02718628989c40edb.camel@evrard.me> On Wed, 2020-03-11 at 18:06 +1100, Mike Carden wrote: > Our small team at ${DAYJOB} has built a handful of OpenStack clusters > based on Red Hat OpenStack 13 (aka Queens) over the last couple of > years. > We now find ourselves in the position of being 'gifted' human > resources in the shape of mid-level 'IT people' who are sent to join > our team for a short time to 'Learn OpenStack'. > > These tend to be people for whom, "Here's a Horizon URL and some > creds - go log in and launch a VM"[1]... is a bit much. > > I've done a wee bit of web searching (enough to find the dead links) > trying to find some newbie friendly tutorials on OpenStack basics. > Before I attempt to re-invent the wheel, can anyone suggest some > public resources I might point people to? > > Deity help us if we have to explain Tripleo's Undercloud, Overcloud, > partially containered, partially pacemakered, fully flaky... > underpinnings. > > Thanks, > MC > [1] Even with a step by step guide Hello, Would https://docs.openstack.org/train/user/ work? For example, you can check the horizon guide to get started with Horizon... Regards, JP -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Mon Mar 16 10:21:04 2020 From: donny at fortnebula.com (Donny Davis) Date: Mon, 16 Mar 2020 06:21:04 -0400 Subject: [all] Guides for newbies to OpenStack In-Reply-To: <24f6f94cf9eabffd30389ad02718628989c40edb.camel@evrard.me> References: <24f6f94cf9eabffd30389ad02718628989c40edb.camel@evrard.me> Message-ID: On Mon, Mar 16, 2020 at 5:56 AM Jean-Philippe Evrard < jean-philippe at evrard.me> wrote: > On Wed, 2020-03-11 at 18:06 +1100, Mike Carden wrote: > > Our small team at ${DAYJOB} has built a handful of OpenStack clusters > based on Red Hat OpenStack 13 (aka Queens) over the last couple of years. > > We now find ourselves in the position of being 'gifted' human resources in > the shape of mid-level 'IT people' who are sent to join our team for a > short time to 'Learn OpenStack'. > > These tend to be people for whom, "Here's a Horizon URL and some creds - > go log in and launch a VM"[1]... is a bit much. > > I've done a wee bit of web searching (enough to find the dead links) > trying to find some newbie friendly tutorials on OpenStack basics. Before I > attempt to re-invent the wheel, can anyone suggest some public resources I > might point people to? > > Deity help us if we have to explain Tripleo's Undercloud, Overcloud, > partially containered, partially pacemakered, fully flaky... underpinnings. > > Thanks, > MC > [1] Even with a step by step guide > > > Hello, > > Would https://docs.openstack.org/train/user/ work? > For example, you can check the horizon guide to get started with Horizon... > > Regards, > JP > I have trained many people over the years on how to use Openstack. Giving people horizon as a starting place has never worked out well. Cloud's are not meant to be pointed and clicked at - can you yes ... should you no. I would be starting them at writing an application (in bash maybe) that fires off a couple cloud resources from the CLI - the sooner they stop pointing and clicking - the sooner they can get on with it. Just my 2C -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From amoralej at redhat.com Mon Mar 16 11:41:10 2020 From: amoralej at redhat.com (Alfredo Moralejo Alonso) Date: Mon, 16 Mar 2020 12:41:10 +0100 Subject: [tripleo][RDO] Version of Ansible in RDO CentOS8 repository In-Reply-To: References: Message-ID: Hi On Wed, Feb 26, 2020 at 3:57 PM Alfredo Moralejo Alonso wrote: > > > On Tue, Feb 25, 2020 at 4:32 PM Alfredo Moralejo Alonso < > amoralej at redhat.com> wrote: > >> Hi all, >> >> During CentOS 8 dependencies preparation we've built ansible 2.9 in RDO >> dependencies repo which was released on Oct 2019, >> >> While testing TripleO with CentOS8 it has been discovered that the latest >> release of ceph-ansible does not support ansible 2.9 but only 2.8, so I'm >> opening discussion about the best way to move on in CentOS 8: >> >> - Make ceph-ansible 4.0 to work with ansible 2.8 *and* 2.9 so that the >> same releases can be used in CentOS7 with Stein and Train and CentOS8 Train >> and Ussuri. >> > ceph-ansible team has created a new release 4.0.16 which supports both ansible 2.8 and 2.9. I've started testing it with master and CentOS8 [1] and i've reported an issue in TripleO jobs [2]. I'd appreciate any help from tripleo to work on this. Note that the plan from ceph team is to remove the support of ansible 2.8 in future releases of ceph-ansible in nautilus so we'd need to backport required fixes for 2.9 to all branches supporting nautilus, which means stable/train and potentially stable/stein (although depending on the EOL and plans for Extended Maintenance in this branch may help to not need it). Regards, Alfredo [1] https://review.rdoproject.org/r/#/c/25914/ [2] https://bugs.launchpad.net/tripleo/+bug/1867608 > - Maintain separated ceph-ansible releases and builds for centos7/ansible >> 2.8 and centos8/ansible 2.9 able to deploy Nautilus. >> - Move ansible back to 2.8 in CentOS 8 Ussuri repository. >> >> I wonder if TripleO or other projects using ansible from RDO repositories >> has any requirement or need to move to ansible 2.9 in Ussuri cycle or can >> stay in 2.8 until next release, any thoughts? >> >> > I've proposed moving to 2.8.8 in CentOS8 ussuri > https://review.rdoproject.org/r/#/c/25379/ > > Feedback is appreciated. > > >> Best regards, >> >> Alfredo >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Mon Mar 16 12:24:14 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Mon, 16 Mar 2020 08:24:14 -0400 Subject: [tc][requirements][horizon] Broken gates due to the new setuptools and pyScss package In-Reply-To: References: <9bbd7f0e9fd3badec5f7f193aa8278ad86cd4279.camel@evrard.me> Message-ID: Would a more sustainable workaround of switching the pyscss code to something that uses libsass-python: https://pypi.org/project/django-libsass/ That seems like an alternative and it means maintaining _far_ less code.. From Arkady.Kanevsky at dell.com Mon Mar 16 12:55:08 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Mon, 16 Mar 2020 12:55:08 +0000 Subject: [Tempest] OpenSack Powered * vs OpeStack Compatible In-Reply-To: <170df34dad4.10d43d2ae172196.1744031088269733049@ghanshyammann.com> References: <2130957943.152281.1582365846726.ref@mail.yahoo.com> <2130957943.152281.1582365846726@mail.yahoo.com> <775D37EA-DC9B-4B80-A002-51C89A3A9E62@vmware.com> <1706963754.79322.1582614973577@mail.yahoo.com> <1707cdd49b2.ddd44e5d119585.1167564760858241265@ghanshyammann.com> <934A54A9-2E2D-4086-94FA-198E4421031E@vmware.com> <316549280.827334.1584259022111@mail.yahoo.com> <170df34dad4.10d43d2ae172196.1744031088269733049@ghanshyammann.com> Message-ID: Before trying to create new group I suggest we review what we already have. We have project Zun and Murano that do have Tempest plugin. -----Original Message----- From: Ghanshyam Mann Sent: Sunday, March 15, 2020 12:19 PM To: pramchan at yahoo.com Cc: Voelker, Mark (VMware); Thierry Carrez; Kanevsky, Arkady; openstack-discuss at lists.openstack.org Subject: Re: [Tempest] OpenSack Powered * vs OpeStack Compatible [EXTERNAL EMAIL] ---- On Sun, 15 Mar 2020 02:57:02 -0500 prakash RAMCHANDRAN wrote ---- > Mark, > it took me some time to get up to speed and a thank you for your thoroughful brief on the subject. > Like to add few more questions before we get on call to see where we go from here. > 5th Area for Interoperability: Should we add latest kubernetes API testing to Tempest or adapt third party Open Source tool to achieve K8S API compliance. Tempest provides the plugin framework where you can write the k8s API testing. I remember that there was a proposal for a container testing tempest plugin. Or I will say check the existing tooling if that works for example: e2e testing[1]. As mentioned earlier, If you want to tests OpenStack APIs then how it is deployed ( either on k8s cluster) does not matter and Tempest existing test will work as it is. > Motivation : > A. Would Like Open Infrastructure project to align in OpenSrack to kubernetes Cluster API for Containers and containerized workloads. > B. The k8s API must have tests to support kubernetes infra on Baremetal and VM. > C. Offer or ask each of the Core services using Container (Docker latest) to an internal test for CRI, CNI and CSI certifications. > D. Level of Branding will be OpenStack and Kubernetes Dual-Stack certification based o. each of the service modules like Nova, Neutron, Cinder, Ceph or any other of projects within or third party k8s tools that will use to get OpebStack trusted interoperability for kubernetes Infrastructure under Open Infrastructure initiative. > This us just the beginning to improve Branding efforts which other community's trust us to be the guides. Let's see if this out of box thinking needs shooting down or should we revive the trailing indicators for adoption and interoperability. > Please advice if thusdditiin should be at end user level or can include kubeadmin APIs too?Should we support RBAC options of keystone or strictly start with K8s RBAC support in Keystone to switch between keystone v3 for VMs vs RBAC v? for Containers? > Think through as a team and let's see if this helps our concept if Dual-Stack support. > Sure we have a bunch of Container projects and they may suggest brownfield options, but stick to Greenfield first before addressing them to fall in line with commo. interoperability area 5 - dual-stack using k8s. > How do we setup a Call for Tempest or DefCore team to review this. Let me have your reply to see if we can get TC members to bring this as response to ask from Thiery for rebooting DefCore Branding efforts. You can reach out to the Tempest team on #openstack-qa IRC channel or let me know if any other communication channel you would like to discuss. [1] https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md -gmann > ThanksPrakash > > > > > Sent from Yahoo Mail on Android > On Wed, Feb 26, 2020 at 6:32 AM, Mark Voelker wrote: > On Feb 25, 2020, at 10:00 AM, Ghanshyam Mann wrote: > ---- On Tue, 25 Feb 2020 01:16:13 -0600 prakash RAMCHANDRAN wrote ---- > Mark, > Glad you pointed to right code. Reviewed and stand corrected. My mis-underdtanding was I considered 20919.01 as first release and 2019.02 as second release of the year. However based on your comments and reference I understand that it is year.month of release , thus 2019.11 includes 'usuri' and previous 3 releases as listed pointed by you. > "os_trademark_approval": { > "target_approval": "2019.11", > "replaces": "2019.06", > "releases": ["rocky", "stein", "train", "ussuri"], > "status": "approved" > } > }, > > > That clears that I should have asked for 2019.11. > Few more questions on Tempedt tests. > I read some where that we have about 1500 Tempest tests overall. Is that correct? > > Yeah, it might be little more or less but around 1500 in-tree tests in Tempest. > > The interop code lines have gone down from 3836 lines to 3232 in train to usuri. > Looks contrary to growth, any comments? > > As Thierry pointed out, a chunk of this was due to the removal of the volumes-v2 API from the required list. A few other tests have been removed over time for various other reasons (changes to or removal of tests in Tempest, etc). Since the test lists are kept in git, you can actually walk through the complete history yourself to see why some tests were added or removed if you’d like: > https://opendev.org/openstack/interop/commits/branch/master > It’s also important to note that just because we’ve added more tests over time to Tempest, more projects over time to OpenStack, or more API’s to existing projects, that doesn’t mean there will be a corresponding increase in the number of tests required for the trademark programs. The OpenStack Powered program is more a trailing indicator of adoption than an attempt to force commercial products to support any and all capabilities. Each API is considered for inclusion in the program against a set of criteria detailed here: > https://opendev.org/openstack/interop/src/branch/master/doc/source/process/CoreCriteria.rst > So, for example: if a project introduced a new API in Usuri, it’s highly unlikely that it would appear in the very next Guideline because it would fail to meet several criteria. It would be unlikely to be “widely deployed” since many deployments would still be using Stein or Train (also covered in the same Guideline). It might not yet be supported in many third-party SDK’s or tools, so it might not yet meet the “used by tools” criteria. It might be only supported by one or two plugins/drivers in it’s first release. Etc, etc, etc. Over time that API might gain adoption, meet sufficient criteria, and be added to the required list--or it might not. > If you’re curious about the history of all this and the process, you might have a look at this slightly old but still mostly relevant deck: > https://www.slideshare.net/markvoelker/interopwg-intro-vertical-programs-jan-2017 > > Question then is 60 compute and 40 storage lines I see in test cases, do we have stats for Tempest tests what's the distribution of 1500 tests across Platform, compute, storage etc. Where and how can I get that. information as documented report. > > Those should be counted from interop guidelines where you have mapping of capabilities with test cases. > Few or most of the capabilities have one test to verifying it or a few more than one test. > For example, "compute-flavors-list" capability is verified by two tests[2]. > This way you can count and identify the exact number of tests per compute, storage etc. > > If you would like to know about the Tempest test categorization, you can find it from the directory > structure. We have structured the tests service wise directory, for example, all compute tests under > tempest/api/compute [2]. > > > Based on above should we expect decrease or increase for say 2020.05 Vancouver release? > > Anecdotally, the Guidelines haven’t been changing very much in recent times as the “core” capabilities that have met with a lot of adoption seem fairly well established (though there have been a few new ones added and a few removed). I wouldn’t expect dramatic changes. > How does one certify a kubernetes cluster running openstak modules, one module per docker container in a kubrrnetes cluster using tempest say like in Airship Control Plane on k8s worker node, which is a OpenStack over kubernetes cluster. > Is this possible and if so what test we need to modify to test and certify a Containerized OpenStack in Airship as OpenStack Platform? > > I should be verified same way via Tempest. Tempest does not reply on how OpenStack is deployed it interacts via the public > interface (where interop needs only user-facing API excluding admin API) of each service which should be accessible on > k8s cluster also. Tests are not required to modify in this case. > > NOTE: we can always extend (writing new tests for current 6 services tests) Tempest for tests required for > interop capabilities either that are new or improved in the verification of existing ones. I remember > about the discussion on covering the API microversion. We keep adding new tests to cover the > new microverison where we need integration tests but we can add API tests also if interop requires those. > > Right on. In fact, the capabilities in the interop programs are intended to be usable independent of how a cloud is deployed (including whether it’s containerized or on bare metal, whether it uses KVM/OVS/Ceph or vSphere/NSX/vSAN, whether it’s highly available or a single-box appliance, etc). > > Can we even certify if for say 2019.11? > > The OpenStack Foundation’s interoperability programs allow you to use either of two most recently approved Guidelines for your testing (which as of right now are 2019.11 and 2019.06). Once the Board of Directors approves the next guideline, 2019.06 will rotate out. I should note though: the Foundation’s interoperability programs are primarily targeted at commercial products (distributions, public clouds, appliances, managed services, etc). There’s no real reason an open source product couldn’t use these same tests of course! > At Your Service, > Mark T. Voelker > This should open up exciting possibilities if practical to extend OpeStack powered platform to Airship. > Like to hear anyone who has insight to educate us on that. > ThanksPrakash > Sent from Yahoo Mail on Android > On Mon, Feb 24, 2020 at 6:09 AM, Mark Voelker wrote: Hi Prakash, > > I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02. > Hm, there actually isn’t a 2019.02 guideline--were you perhaps referring to 2019.06 or 2019.11? 2019.06 does cover Train but not Usuri [1], 2019.11 covers both [2]. As an FYI, the OpenStack Marketplace does list which guideline a particular product was most recently tested against (refer to https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=K9zbzRvFzs3cxVnCocij2Sjy43MNyejDKuRN7ivQkMk%3D&reserved=0 for example, and look for the green “TESTED” checkmark and accompanying guideline version), though this obviously doesn’t tell you what testing might be currently in flight. > [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.06.json%23L75%5B2&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=Vcy8iL%2F8L93m0%2B3PIwikD5vN%2BaREJigrncz7cjd3Zyc%3D&reserved=0] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.11.json%23L75&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=pTrl8xMypZJAPFu2vzQwFFOWnJu51k44oxn2PrrQI2Y%3D&reserved=0 > At Your Service, > Mark T. Voelker > > > On Feb 22, 2020, at 5:04 AM, prakash RAMCHANDRAN wrote: > Hi all, > I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02. > Similarly does any Hardware Driver of Software application seeking OpenStack compatibility Logo? > Finally does anyone think that Open Infra Distro like Airship or StarlingX should promote Open Infra Airship or Open Infra StarlingX powered as a new way to promote eco system surrounding them similar to OpenStack compatible drivers and software. > Will then Argo, customize,Metal3.io or Ironic be qualified as Open Infra Airship compatible? > If so how tempest can help in testing the above comments? > Refer to this market place below as how Distos and Products leverage OpenStack logos and branding programs. > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=rABzR1tWXUHinSG4jpe46ayuELsini4fbKcJhsxrN9g%3D&reserved=0 > Discussions and feedback are welcome. A healthy debate as how k8s modules used in Open Infra can be certified will be a good start. > ThanksPrakash > > Sent from Yahoo Mail on Android > > [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fcommit%2F8f2e82b7db54cfff9315e5647bd2ba3dd6aacaad%2F2019.11.json%23L260-L281&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=wncg32ql1Gq%2B%2F7fZPUDdPzSv2Fbay2zCAXPT%2F9%2BQIo4%3D&reserved=0 > [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Ftempest%2Fsrc%2Fbranch%2Fmaster%2Ftempest%2Fapi%2Fcompute&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=mgDV%2Bng445lIrj3CjMwdTeifyGMdXyogQhnVdXk4FS0%3D&reserved=0 > > -gmann > From jimmy at openstack.org Mon Mar 16 13:55:08 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Mon, 16 Mar 2020 08:55:08 -0500 Subject: [all] Guides for newbies to OpenStack In-Reply-To: <24f6f94cf9eabffd30389ad02718628989c40edb.camel@evrard.me> References: <24f6f94cf9eabffd30389ad02718628989c40edb.camel@evrard.me> Message-ID: <5E6F853C.3040000@openstack.org> If you check on this page under Contributor Resources, you'll find a lot of "getting started" content: https://www.openstack.org/community/ Cheers, Jimmy > Jean-Philippe Evrard > March 16, 2020 at 4:52 AM > On Wed, 2020-03-11 at 18:06 +1100, Mike Carden wrote: > > Hello, > > Would https://docs.openstack.org/train/user/ work? > For example, you can check the horizon guide to get started with > Horizon... > > Regards, > JP > Mike Carden > March 11, 2020 at 2:06 AM > Our small team at ${DAYJOB} has built a handful of OpenStack clusters > based on Red Hat OpenStack 13 (aka Queens) over the last couple of years. > > We now find ourselves in the position of being 'gifted' human > resources in the shape of mid-level 'IT people' who are sent to join > our team for a short time to 'Learn OpenStack'. > > These tend to be people for whom, "Here's a Horizon URL and some creds > - go log in and launch a VM"[1]... is a bit much. > > I've done a wee bit of web searching (enough to find the dead links) > trying to find some newbie friendly tutorials on OpenStack basics. > Before I attempt to re-invent the wheel, can anyone suggest some > public resources I might point people to? > > Deity help us if we have to explain Tripleo's Undercloud, Overcloud, > partially containered, partially pacemakered, fully flaky... > underpinnings. > > Thanks, > MC > [1] Even with a step by step guide -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Mon Mar 16 14:36:03 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 16 Mar 2020 15:36:03 +0100 Subject: Not running for TC this round Message-ID: Hi everyone, I've been elected on the TC (or its previous incarnations) since we had community-elected seats in that body (2011). However, as announced during the previous election round (and in the discussion about reducing the number of TC seats) I do not plan to run for the TC in the upcoming election round. There are multiple reasons for this. The main one is that with name recognition and a Condorcet voting system, it's relatively easy for incumbents to be reelected. We only elect 5 seats this time, so I think it's important to make room for new names to step up. The second reason is that I'd like more people with an operational background to be present on the TC. I think we can simplify our governance by having a single elected body handling the open source project, rather than maintain separate representation for users and developers. As we get more people bringing operational perspective to the TC, that simplification will be even more obvious. The final reason is that I've always been saying that you can work on governance issues and voice your opinion on them, without being elected on the TC and formally voting on things. It's time for me to walk that talk. I do not intend to go anywhere, and I'll be as present as always in proposing the OpenStack governance changes that I deem needed in this day and age. I'll finish this email by encouraging anyone interested in open source project governance, or wanting to help shape the next chapter of the OpenStack story, to step up and run for TC election. As for all governance bodies, the experience can be frustrating, as you'd like to fix more things but there is only so much you can do. But it's a great experience, one that I recommend everyone working on open source project should have. The only prerequisite is an interest and care for "OpenStack" as a whole. No specific experience or diploma needed. If you wonder if you can do it, the answer is yes. Nominations[1] open up March 24. Please consider running, especially if you have operational experience with some cycles to spare. If you have any questions on the job, feel free to reach out to me -- you know where to find me. [1] https://governance.openstack.org/election/ -- Thierry Carrez (ttx) From zigo at debian.org Mon Mar 16 15:18:20 2020 From: zigo at debian.org (Thomas Goirand) Date: Mon, 16 Mar 2020 16:18:20 +0100 Subject: [ops] how to setup bgp-to-the-host and Neutron Dynamic Routing ? Message-ID: Hi, In my setup, the bgp-to-the-host part is already operational. We're using IPv6 local link to 2 switches on each host, each of them advertising for itself to both routers. Now, I'm trying to add Neutron stuff to it. In my setup, I don't have any IPv4 on my nics, both have ipv6 local link only. So, where do I connect br-ex? I don't have a bond0 interface to connect to anymore... Then, we do have an FRR instance to do the bgp-to-the-host IPv4 routing on each host. How does this work on the hosts where the dr-agent is running? Can it re-use the FRR that I've setup to do the route advertisements too? Or do I need another instance of a routing daemon? Cheers, Thomas Goirand (zigo) From pramchan at yahoo.com Sun Mar 15 07:57:02 2020 From: pramchan at yahoo.com (prakash RAMCHANDRAN) Date: Sun, 15 Mar 2020 07:57:02 +0000 (UTC) Subject: [Tempest] OpenSack Powered * vs OpeStack Compatible In-Reply-To: <934A54A9-2E2D-4086-94FA-198E4421031E@vmware.com> References: <2130957943.152281.1582365846726.ref@mail.yahoo.com> <2130957943.152281.1582365846726@mail.yahoo.com> <775D37EA-DC9B-4B80-A002-51C89A3A9E62@vmware.com> <1706963754.79322.1582614973577@mail.yahoo.com> <1707cdd49b2.ddd44e5d119585.1167564760858241265@ghanshyammann.com> <934A54A9-2E2D-4086-94FA-198E4421031E@vmware.com> Message-ID: <316549280.827334.1584259022111@mail.yahoo.com> Mark, it took me some time to get up to speed and a thank you for your thoroughful brief on the subject. Like to add few more questions before we get on call to see where we go from here. 5th  Area for Interoperability: Should we add latest kubernetes API testing to Tempest or adapt third party Open Source  tool to achieve K8S API compliance. Motivation : A. Would Like Open Infrastructure project to align in OpenSrack to   kubernetes Cluster API for Containers and containerized workloads. B. The k8s API must have tests to support kubernetes infra on Baremetal and VM. C. Offer or ask each of the Core  services using Container (Docker latest) to an  internal  test for CRI, CNI and CSI certifications. D. Level of Branding will be OpenStack and Kubernetes Dual-Stack certification based o. each of the service modules like Nova, Neutron, Cinder, Ceph or any other of projects within or third party k8s tools that will use to get OpebStack trusted interoperability for kubernetes Infrastructure under Open Infrastructure initiative. This us just the beginning to improve Branding efforts which other community's trust us to be the guides. Let's see if this out of box thinking needs shooting down or should we revive the trailing indicators for adoption and interoperability. Please advice if thusdditiin should be at end user level or can include kubeadmin APIs too?Should we support RBAC options of keystone or strictly start with K8s RBAC support in Keystone to switch between keystone v3 for VMs vs RBAC v? for Containers? Think through as a team and let's see if this helps our concept if Dual-Stack support. Sure we have a bunch of Container projects and they may suggest brownfield options, but stick to Greenfield first before addressing them to fall in line with commo. interoperability area 5 - dual-stack using k8s. How do we setup a Call for Tempest or DefCore team  to review this. Let me have your reply to see if we can get TC members to bring this as response to ask from Thiery for rebooting DefCore Branding efforts. ThanksPrakash Sent from Yahoo Mail on Android On Wed, Feb 26, 2020 at 6:32 AM, Mark Voelker wrote: On Feb 25, 2020, at 10:00 AM, Ghanshyam Mann wrote: ---- On Tue, 25 Feb 2020 01:16:13 -0600 prakash RAMCHANDRAN wrote ---- Mark, Glad you pointed to right code. Reviewed and stand corrected. My mis-underdtanding was I considered 20919.01 as first release and 2019.02 as second release of the year. However based on your comments and reference I understand that it is year.month of release , thus 2019.11 includes 'usuri' and previous 3 releases as listed pointed by you.    "os_trademark_approval": {      "target_approval": "2019.11",      "replaces": "2019.06",      "releases": ["rocky", "stein", "train", "ussuri"],      "status": "approved"    }  }, That clears that I should have asked for 2019.11. Few more questions on Tempedt tests. I read some where that we have about 1500 Tempest tests overall. Is that correct? Yeah, it might be little more or less but around 1500 in-tree tests in Tempest. The interop code lines have gone down from 3836 lines to 3232 in train to usuri. Looks contrary to growth, any comments? As Thierry pointed out, a chunk of this was due to the removal of the volumes-v2 API from the required list.  A few other tests have been removed over time for various other reasons (changes to or removal of tests in Tempest, etc).  Since the test lists are kept in git, you can actually walk through the complete history yourself to see why some tests were added or removed if you’d like: https://opendev.org/openstack/interop/commits/branch/master It’s also important to note that just because we’ve added more tests over time to Tempest, more projects over time to OpenStack, or more API’s to existing projects, that doesn’t mean there will be a corresponding increase in the number of tests required for the trademark programs.  The OpenStack Powered program is more a trailing indicator of adoption than an attempt to force commercial products to support any and all capabilities.  Each API is considered for inclusion in the program against a set of criteria detailed here: https://opendev.org/openstack/interop/src/branch/master/doc/source/process/CoreCriteria.rst So, for example: if a project introduced a new API in Usuri, it’s highly unlikely that it would appear in the very next Guideline because it would fail to meet several criteria.  It would be unlikely to be “widely deployed” since many deployments would still be using Stein or Train (also covered in the same Guideline).  It might not yet be supported in many third-party SDK’s or tools, so it might not yet meet the “used by tools” criteria.  It might be only supported by one or two plugins/drivers in it’s first release.  Etc, etc, etc.  Over time that API might gain adoption, meet sufficient criteria, and be added to the required list--or it might not.  If you’re curious about the history of all this and the process, you might have a look at this slightly old but still mostly relevant deck: https://www.slideshare.net/markvoelker/interopwg-intro-vertical-programs-jan-2017  Question then is  60 compute and 40 storage lines I see in test cases, do we have stats for Tempest tests what's the distribution of 1500 tests across Platform, compute, storage etc. Where and how can I get that. information as documented report. Those should be counted from interop guidelines where you have mapping of capabilities with test cases. Few or most of the capabilities have one test to verifying it or a few more than one test. For example, "compute-flavors-list" capability is verified by two tests[2].  This way you can count and identify the exact number of tests per compute, storage etc. If you would like to know about the Tempest test categorization, you can find it from the directory structure. We have structured the tests service wise directory, for example, all compute tests under tempest/api/compute [2]. Based on above should we expect decrease or increase for say 2020.05 Vancouver release? Anecdotally, the Guidelines haven’t been changing very much in recent times as the “core” capabilities that have met with a lot of adoption seem fairly well established (though there have been a few new ones added and a few removed).  I wouldn’t expect dramatic changes. How does one certify a kubernetes cluster running openstak modules, one module per docker container in a kubrrnetes  cluster using tempest say like in Airship Control Plane on k8s worker node, which is a OpenStack over kubernetes cluster. Is this possible and if so what test we need to modify to test and certify a Containerized OpenStack in Airship as OpenStack Platform? I should be verified same way via Tempest. Tempest does not reply on how OpenStack is deployed it interacts via the public interface (where interop needs only user-facing API excluding admin API) of each service which should be accessible on k8s cluster also. Tests are not required to modify in this case. NOTE: we can always extend (writing new tests for current 6 services tests) Tempest for tests required for interop capabilities either that are new or improved in the verification of existing ones. I remember about the discussion on covering the API microversion. We keep adding new tests to cover the new microverison where we need integration tests but we can add API tests also if interop requires those. Right on.  In fact, the capabilities in the interop programs are intended to be usable independent of how a cloud is deployed (including whether it’s containerized or on bare metal, whether it uses KVM/OVS/Ceph or vSphere/NSX/vSAN, whether it’s highly available or a single-box appliance, etc). Can we even certify if for say 2019.11? The OpenStack Foundation’s interoperability programs allow you to use either of two most recently approved Guidelines for your testing (which as of right now are 2019.11 and 2019.06).  Once the Board of Directors approves the next guideline, 2019.06 will rotate out.  I should note though: the Foundation’s interoperability programs are primarily targeted at commercial products (distributions, public clouds, appliances, managed services, etc).  There’s no real reason an open source product couldn’t use these same tests of course! At Your Service, Mark T. Voelker  This should open up exciting possibilities if practical to extend OpeStack powered platform to Airship. Like to hear anyone who has insight to educate us on that. ThanksPrakash Sent from Yahoo Mail on Android    On Mon, Feb 24, 2020 at 6:09 AM, Mark Voelker wrote:   Hi Prakash, I am curious to find out if any Distribution or Products based on Openstack  Train or Usuri are seeking the latest certifications based on 2019.02. Hm, there actually isn’t a 2019.02 guideline--were you perhaps referring to 2019.06 or 2019.11?  2019.06 does cover Train but not Usuri [1], 2019.11 covers both [2].  As an FYI, the OpenStack Marketplace does list which guideline a particular product was most recently tested against (refer to https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=K9zbzRvFzs3cxVnCocij2Sjy43MNyejDKuRN7ivQkMk%3D&reserved=0 for example, and look for the green “TESTED” checkmark and accompanying guideline version), though this obviously doesn’t tell you what testing might be currently in flight. [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.06.json%23L75%5B2&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=Vcy8iL%2F8L93m0%2B3PIwikD5vN%2BaREJigrncz7cjd3Zyc%3D&reserved=0] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.11.json%23L75&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=pTrl8xMypZJAPFu2vzQwFFOWnJu51k44oxn2PrrQI2Y%3D&reserved=0 At Your Service, Mark T. Voelker On Feb 22, 2020, at 5:04 AM, prakash RAMCHANDRAN wrote: Hi all, I am curious to find out if any Distribution or Products based on Openstack  Train or Usuri are seeking the latest certifications based on 2019.02. Similarly does any Hardware Driver of Software application seeking OpenStack compatibility Logo? Finally does anyone think that Open Infra Distro like Airship or StarlingX should promote Open Infra Airship or Open Infra  StarlingX powered as a new way to promote eco system surrounding them similar to OpenStack compatible drivers and software. Will then Argo, customize,Metal3.io or Ironic be qualified as Open Infra Airship compatible? If so how tempest can help in testing the above comments? Refer to this market place below as how Distos and Products leverage OpenStack logos and branding programs. https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=rABzR1tWXUHinSG4jpe46ayuELsini4fbKcJhsxrN9g%3D&reserved=0 Discussions and feedback are welcome. A healthy debate as how k8s modules used in Open Infra can be certified will be a good start. ThanksPrakash Sent from Yahoo Mail on Android [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fcommit%2F8f2e82b7db54cfff9315e5647bd2ba3dd6aacaad%2F2019.11.json%23L260-L281&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=wncg32ql1Gq%2B%2F7fZPUDdPzSv2Fbay2zCAXPT%2F9%2BQIo4%3D&reserved=0 [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Ftempest%2Fsrc%2Fbranch%2Fmaster%2Ftempest%2Fapi%2Fcompute&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=mgDV%2Bng445lIrj3CjMwdTeifyGMdXyogQhnVdXk4FS0%3D&reserved=0 -gmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin.uhrin at gmail.com Sun Mar 15 21:33:34 2020 From: martin.uhrin at gmail.com (Martin Uhrin) Date: Sun, 15 Mar 2020 22:33:34 +0100 Subject: PyOS Message-ID: Dear OpenStack devs, Just a quick question: I came across your project 'pyos' on pypi after using this name for internally one of my projects. I see that you haven't updated the package for some time and was wondering if it was dead and if so if you would consider transfering the name to me for hosting my project? Thanks for your time. All the best, -Martin From arne.wiebalck at cern.ch Mon Mar 16 17:21:56 2020 From: arne.wiebalck at cern.ch (Arne Wiebalck) Date: Mon, 16 Mar 2020 18:21:56 +0100 Subject: [ironic] proposing Iury Gregory for bifrost-core, ironic-inspector-core, sushy-core In-Reply-To: References: Message-ID: On 11.03.20 19:54, Julia Kreger wrote: > Iury has been working hard across the ironic community and has been > quite active in changing and improving our CI, as well as reviewing > code contributions and helpfully pointing out issues or items that > need to be fixed. I feel that he is on track to join ironic-core in > the next few months, but first I propose we add him to bifrost-core, > ironic-inspector-core, and sushy-core. > > Any objections? > +1 Good work and thanks for all your help, Iury! From mordred at inaugust.com Mon Mar 16 17:23:11 2020 From: mordred at inaugust.com (Monty Taylor) Date: Mon, 16 Mar 2020 12:23:11 -0500 Subject: PyOS In-Reply-To: References: Message-ID: <46313B5E-C331-498E-AF97-BD5A5BD29288@inaugust.com> > On Mar 15, 2020, at 4:33 PM, Martin Uhrin wrote: > > Dear OpenStack devs, > > Just a quick question: I came across your project 'pyos' on pypi after > using this name for internally one of my projects. I see that you > haven't updated the package for some time and was wondering if it was > dead and if so if you would consider transfering the name to me for > hosting my project? Hi - totally dead (or really, that was attempt #1 at a name and we went with something else) Let me know what your pypi username is and I’ll mark you as an owner. From mordred at inaugust.com Mon Mar 16 18:21:02 2020 From: mordred at inaugust.com (Monty Taylor) Date: Mon, 16 Mar 2020 13:21:02 -0500 Subject: PyOS In-Reply-To: References: <46313B5E-C331-498E-AF97-BD5A5BD29288@inaugust.com> Message-ID: > On Mar 16, 2020, at 12:59 PM, Martin Uhrin wrote: > > martin.uhrin I’ve added you as an Owner, so you should be able to upload content and stuff. Make sure you can … then probably a good idea to remove me. :) Enjoy! From mordred at inaugust.com Mon Mar 16 19:11:47 2020 From: mordred at inaugust.com (Monty Taylor) Date: Mon, 16 Mar 2020 14:11:47 -0500 Subject: PyOS In-Reply-To: References: <46313B5E-C331-498E-AF97-BD5A5BD29288@inaugust.com> Message-ID: > On Mar 16, 2020, at 1:25 PM, Martin Uhrin wrote: > > Works perfectly. You want me to keep around 'pyos 288bbf8' for > posterity or can I remove it? I'm happy either way. No need to keep it - it’s total garbage. :) > -Martin > > Le lun. 16 mars 2020 à 19:21, Monty Taylor a écrit : >> >> >> >>> On Mar 16, 2020, at 12:59 PM, Martin Uhrin wrote: >>> >>> martin.uhrin >> >> I’ve added you as an Owner, so you should be able to upload content and stuff. Make sure you can … then probably a good idea to remove me. :) >> >> Enjoy! > From martin.uhrin at gmail.com Mon Mar 16 17:59:48 2020 From: martin.uhrin at gmail.com (Martin Uhrin) Date: Mon, 16 Mar 2020 18:59:48 +0100 Subject: PyOS In-Reply-To: <46313B5E-C331-498E-AF97-BD5A5BD29288@inaugust.com> References: <46313B5E-C331-498E-AF97-BD5A5BD29288@inaugust.com> Message-ID: Wicked, thanks a bunch Monty. Username: martin.uhrin. Le lun. 16 mars 2020 à 18:23, Monty Taylor a écrit : > > > > > On Mar 15, 2020, at 4:33 PM, Martin Uhrin wrote: > > > > Dear OpenStack devs, > > > > Just a quick question: I came across your project 'pyos' on pypi after > > using this name for internally one of my projects. I see that you > > haven't updated the package for some time and was wondering if it was > > dead and if so if you would consider transfering the name to me for > > hosting my project? > > Hi - totally dead (or really, that was attempt #1 at a name and we went with something else) > > Let me know what your pypi username is and I’ll mark you as an owner. > From martin.uhrin at gmail.com Mon Mar 16 18:25:02 2020 From: martin.uhrin at gmail.com (Martin Uhrin) Date: Mon, 16 Mar 2020 19:25:02 +0100 Subject: PyOS In-Reply-To: References: <46313B5E-C331-498E-AF97-BD5A5BD29288@inaugust.com> Message-ID: Works perfectly. You want me to keep around 'pyos 288bbf8' for posterity or can I remove it? I'm happy either way. -Martin Le lun. 16 mars 2020 à 19:21, Monty Taylor a écrit : > > > > > On Mar 16, 2020, at 12:59 PM, Martin Uhrin wrote: > > > > martin.uhrin > > I’ve added you as an Owner, so you should be able to upload content and stuff. Make sure you can … then probably a good idea to remove me. :) > > Enjoy! From openstack at nemebean.com Mon Mar 16 19:50:51 2020 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 16 Mar 2020 14:50:51 -0500 Subject: Not running for TC this round In-Reply-To: References: Message-ID: <3692fa0f-078d-06dc-9509-e352c7b0758b@nemebean.com> On 3/16/20 9:36 AM, Thierry Carrez wrote: > The final reason is that I've always been saying that you can work on > governance issues and voice your opinion on them, without being elected > on the TC and formally voting on things. Can confirm. Have stuck my nose into plenty of TC discussions without ever being a member. For better or worse. ;-) From juliaashleykreger at gmail.com Mon Mar 16 23:25:43 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 16 Mar 2020 16:25:43 -0700 Subject: Summary: Ironic Mid-cycle at CERN Message-ID: A couple weeks ago (February 25-26), the Ironic community convened its first mid-cycle in quite a long time at the invitation of and encouragement of CERN. A special thanks goes to Arne Wiebalck for organizing the gathering. We spent two days discussing, learning, sharing, and working together to form a path forward. As long time contributors, some of us were able to bring context of not just how, but why. Other community members brought questions and requirements, meanwhile one of the hardware vendors brought their context and needs. And CERN was kind enough to show us how our work matters and makes a difference, which was the most inspiring part of all! Special thanks goes to Dmitry Tantsur, Riccardo Pittau, and Iury Gregory for helping me keep momentum moving forward on this summary. --------------------------------------- Deploy Steps ========== Discussed issues related to the actual workflow, concerns out process efficiency and path forward. The issue in question was the advance validation of deploy steps when some of the steps come from the ironic-python-agent ramdisk and are not reflected in the server code. Creating the whole list of steps for validation and execution requires information from the ramdisk, but it’s only available when the ramdisk is booted. We have discussed the following alternatives: * Start ramdisk before deploy step execution. This was ruled out for the following reasons: ** Some steps need to be executed out-of-band before the ramdisk is running. This is already an issue with iDRAC clean steps. ** The first deploy steps validation happens in the node validation API when the ramdisk is clearly not running. * Using cached deploy steps from the previous cleaning run. It was ruled out because: ** Some deployments disable automated cleaning. ** The deploy step list can change in between, e.g. because of hardware changes or any external input. * Accept that we cannot provide early validation of deploy steps and validate them as we go. This involves booting the ramdisk as one of the deploy steps (no special handling), with only out-of-band steps executed before that (ignoring any in-band steps with higher priority). We decided to go with the third option. In a side discussion we decided to file an RFE for a driver_info flag preventing booting the ramdisk during manual cleaning. Solving the cleaning issues completely probably requires making booting the ramdisk a separate clean step, similarly to deploy steps above. No final plan has been made for it, but we have more clarity than we did before. Security ====== Firmware Management -------------------------------- We entered a discussion about creating a possible “meta step”. After some back and forth discussions, we reached a consensus that it is likely not possible given different vendor parameters and requirements. During this topic, we also reached the point of discussing changes to “Active node” configuration as it is related in large part to firmware updates, and is necessary for larger fleet management, and eventually even for attestation process integration. The consensus kind of revolved around that the way to help enable some of this process was to leverage rescue, this is only theory. Hopefully we’ll have operator feedback in the next year on this subject and can make more informed decisions. By then, we should have deploy steps in a form that one doesn't need to be a python developer to leverage, and the team should be bandwidth to explore this further with operators. Attestation --------------- This is a topic that Julia has been raising for a while because there is a logical and legitimate reason to go ahead and implement some sort of integration with an attestation platform to perform system measurement and attestation during cleaning and deployment processes in order to help identify if machines were tampered with. In our case, remote attestation is likely the way forward, and inspiration can come looking at Keylime (TPM based highly boot attestation and run-time integrity measurement solution, and most important, opensource). We’ll need an implementation to cover at least Clean/Deploy steps, to be able to run and validate TPM measurement, and fail the deployment if attestation fails. We still need to figure out what the actual impact to firmware upgrade is, and how to safely ensure that a re-measurement is valid, or not, and when to trust the measurement is actually valid. Ironic’s next step is to begin to talk to the Keylime folks in more depth. Also one of our contributors, Kaifeng, who read our notes etherpad indicated that he is working towards the same area as well, so we may see some interesting and fruitful collaboration because ultimately we all have some of the same needs. Agent Tokens ------------------ Agent tokens was possibly the quickest topic that we visited with Dmitry suggesting we just needed to add a unit test and merge the code. To be further secure, we need the agent ramdisk to begin using TLS. TODO: Julia is to send out an email to the mailing list to serve as notice to operators that ironic intends to break backwards IPA compatibility next cycle by removing support for agents that do not support agent tokens. NOTE: As we type/refine this summary for distribution, the agent token code has largely merged, and should be completely merged before the end of the current development cycle. TLS in virtual media --------------------------- In order to secure Agent token use, we need to secure their transmission to the ironic-python-agent when commands are issued to the agent from the conductor. Ultimately we’re hoping to begin work on this soon in order to better secure interactions and communications with machines in remote “edge” environments. An RFE has been filed to automatically generate certificates and exchange them with the ramdisk: https://storyboard.openstack.org/#!/story/2007214. Implementing it may require downstream consumers to update their USA export control certification. FIPS 140-2 --------------- This was a late addition topic that was added while other discussion was coming up, and largely more for the purposes of community visibility. In short, we know based on some recent bugs that were fixed, that operators are starting to try and deploy Ironic in environments and on hosts that are configured for a FIPS 140-2 operating mode, in short a much more strict cryptography configuration. We ought to make sure that we don’t have any other surprises waiting for us, so the call went out for someone to at some point review the standard, and sanity check Ironic and components. Post-IPMI universe =============== The decline of IPMI is one that we, as a community, need to plan ahead for as some things become a little more difficult. Discovery ------------- Node discovery, as a feature, is anticipated to become a little more complicated. While we should still be able to identify a BMC address,that address may be the in-band communications channel address once vendors are supporting the Redfish host interface specification. This spurred discussion of alternatives, and one of the items that was raised was possibly supporting the discovery of BMCs using the SSDP and UPnP. This raises an interesting possibility in that the UUID of the BMC is retrievable through these means. It seems logical for us to one day consider the detection and enrollment of machines using an operator tool of some sort. This functionality is defined by the Redfish standard and, as everything in Redfish, is optional. The DMTF provided Redfish library contains an example implementation: https://github.com/DMTF/python-redfish-library/blob/master/src/redfish/discovery/discovery.py. Using System ID/GUID ------------------------------- The discovery topic spurred another topic of if we should be using the System UUID or GUID identifier in addition to or instead of a MAC address on the chassis. Ironic Inspector folks have been considering having additional or even plug-able matches for a long time. The system UUID can be discovered in-band via SMBios before launching inspection/discovery. Supporting this would largely be more of a feature matching a physical machine, but some of our code requires network information anyway, so it may not bring a huge benefit upfront beyond trying to match BMC<->Host. IPMI cipher changes ---------------------------- CERN folks were kind enough to raise an issue that has brought themselves some headaches as of recent which is that some BMC vendors have changed cipher logic and keying so they’ve had to put in some workarounds and modified ipmitool builds on their systems. As far as we’re aware as a community, there is really nothing we can directly do to help them remove this work around, but ultimately this headache may cause them to begin looking at Redfish and cause some development on serial console support for Redfish. Making inspector data a time series =========================== One of the challenges in data centers is identifying when the underlying hardware changes. When a disk is replaced, the serial number changes, and if that disk is in a running system, it would traditionally have needed to be re-inspected in order for information about the machine to be updated. But we added the ability to manually execute and submit this data in the last development cycle, so if there was any time series nature to inspection data, then it allows for changes to be identified, new serial numbers recorded, etc. The overwhelming response during the discussion was “Yes Please!”, in that such a feature would help a number of cases. Of course, what we quickly reached was disagreement over meaning. Turns out, the purpose is more about auditing and identifying changes, so even if there are only two copies, the latest and the previous inspection data, then differences could be identified by external tooling. A spec document or some sort of written MVP will ultimately be required, but the overall concept was submitted to Outreachy. DHCP-less deploy ============== In regard to the DHCP-less deploy specification (https://review.opendev.org/#/c/672780/) we touched upon several areas of the specification. We settled on Nova's network metadata format (as implemented by Glean) as the API format for this feature. Ilya has voiced concerns that it will tie us to Glean closer than we may want. Scalability of rebuilding ISO images per node. The CERN folks rightfully expressed concern that a parallel deployment of several hundred nodes can put a significant load on conductors, especially in terms of disk space. * In case of hardware that has more than one usable virtual media slot, we can keep the base ISO intact and use a 2nd slot (e.g. virtual USB) to provide configuration. * The only other option is documenting it as a limitation of our virtual media implementation. To get rid of the MAC address requirements for DHCP-less virtual media deployments, we determined that it will be necessary to return to sending node UUID and other configuration to the ramdisk via boot parameters. This way we can avoid the requirement for MAC addresses, although this has to be navigated carefully and with Operator feedback. An additional concern, beyond parallel deployment load, was “Rescue” support. The consensus seemed to be that we put giant security warnings in the documentation to signal the security risk of the ramdisk being exposed to a potentially un-trusted network. Agent token work _does_ significantly help improve operational security in these cases, but operators must be cognizant of the risks and potentially consider that rescue may be something they might not want to use under normal circumstances with network edge deployments. OOB DHCP-less deploy =================== We briefly touched on OOB dhcp-less deployment. This is HTTPBoot asserted through to the BMC with sufficient configuration details, ultimately looking a lot like DHCP-less deployments. Interest does seem to exist on this topic, but we can revisit once the base DHCP-less deployment work is done and hopefully an ironic contributor has access hardware where this is an explicit feature of the BMC. CI Improvements and Changes ======================== The upstream CI, and ways to make it more stable and just improve its efficiency, is a recurrent main discussion argument not only as part of the meetup. The impact of the CI in the day-to-day work is very high, and that’s why we took our time to talk about different aspects and do a proper analysis of the different jobs involved. The discussion started with the proposal of reducing the usage of the ironic-python-agent images based on TinyCoreLinux (the so-called tinyipa images), and rely more on the images built using diskimage-builder (DIB) and specifically CentOS 8 as base. This proposal is based on the fact that DIB-built images are what we recommend for production usage, while tinyIPA images have known issues on real hardware. Their only real benefit is a much smaller memory footprint (roughly 400MiB vs 2GiB of a CentOS 8 image). We have agreed to switch all jobs that use 1 testing VM to pre-built CentOS 8 images. This covers all jobs, except for the standalone, multi-node and grenade (upgrade testing) ones. While reviewing the current list of jobs, we realized that there is a lot of duplication between them. Essentially, most of the image type - deploy interface combinations are already tested in the standalone job. As these combinations are orthogonal to the management technology, we can use Redfish instead of IPMI for some of the tests. We decided to split the ironic-standalone job since it covers a lot of scenarios from tempest and it has a high failure rate. The idea to have one job testing software raid, manual cleaning and rescue, the other tests that consists in the combination of image type - deploy interface will be split in two jobs (one using IPMI and the other using Redfish). One other point that we reached was some consensus that more exotic, non-openstack focused CI jobs, were likely best to be implemented using Bifrost as opposed to Tempest. Third Party CI/Driver Requirements ----------------------------------------------- The question has been raised with-in the community if we reconsider 3rd Party CI requirements. For those that are unaware, it has been a requirement for drivers to merge into ironic to have Third-Party operated CI. Operating Third Party CI helps exercise drivers to ensure that the driver code is functional, and provides the community information in the event that there is a breaking changer or enhancement made. The community recognizes third party CI is difficult, and can be hard at times to keep working as the entire code base and dependencies evolve as time moves on. We discussed why some of these things are difficult, and what can we, and the larger community do to try and make it easier. As one would expect, a few questions arose: Q: Do we consider “supported = False” and keeping drivers in-tree until we know they no longer work? A: The consensus was that this is acceptable. That the community can keep unit tests working and code looking sane. Q: Do we consider such drivers as essentially frozen? A: The consensus is that drivers without third party CI will be functionally frozen unless changes are required to the driver for the project to move forward. Q: How do we provide visibility into the state of the driver? A: The current thought is to return a field in the /v1/drivers list to signal if the driver has upstream testing. The thought is to use a field named “Tested” or something similar as opposed to the internal name in the driver interface which is “supported” Q: Will we make it easier to merge a driver? A: Consensus was that we basically want to see it work at least once before we merge drivers. It was pointed out that this helped provide visibility with some of the recently proposed driver code which was developed originally against a much older version of ironic. Q: Do third party CI systems need to run on every patch? A: Consensus is No! A number of paths in the repository can be ignored from changes. In other words, there is no reason to trigger an integration test of Third Party CI for a documentation change or an update to a release note, or even a change to other vendor’s drivers. In summary Drivers without Third Party CI are “use at own risk” and removal is moving towards a model of “don’t be brutal”. This leaves us with a number of tasks in the coming months: * Update the contributor documentation with the Questions and Answers above. * Author an explicit exception path for the process of bringing CI back up as it pertains to drivers, essentially focusing on communication between the ironic community and driver maintainers. * Author a policy stating that unsupported drivers shall be removed immediately upon being made aware that the driver is no longer functional and without a clear/easy fix or path to resolution. * Solicit pain points from driver maintainers who have recently setup or do presently maintain Third Party CI and try to aggregate the data and maybe find some ways of improving the situation. “Fishy Politics”: Adapting sushy for Redfish spec versus implementation reality ============================================================ Everyone’s favorite topic is how implementations differ from specification documents. In part, the community is increasingly seeing cases where different vendors have behavior oddities in their Redfish implementations. We discussed various examples, such as https://storyboard.openstack.org/#!/story/2007071, and the current issues we’re observing on two different vendors around setting the machine boot mode and next boot device at the same time. For some of these issues, the idea of having some sort of Redfish flavor indicator suggested so the appropriate plugin could be loaded which might be able to handle larger differences like major field name differences, or possibly endpoint behavior differences like “PUT” instead of “PATCH”. This has not yet been explored but will likely need to be explored moving forward. Another item for the ironic team to be mindful moving forward, is that newer UEFI specific boot setting fields have been created, which we may want to explore using. This could give us a finer level of granularity of control, and at the same time may not be really usable in other vendor’s hardware due to the data in the field and how or what to correspond it back to. Kexec (or "Faster booting, yes?") ========================= This topic concerns using the kexec mechanism instead of rebooting from the ramdisk to the final instance. Additionally, if it is acceptable to run the agent on the user instance, it can be used for rebuilding and tearing down an instance. Potentially saving numerous reboots and "Power On Self Tests" in the process. We have one potential issue: with multi-tenant networking there is a possibility of a race between kexec and switching from the provisioning to the tenant network(s). In a normal deploy we avoid it by powering the node off first, then flipping the networks, then booting the final instance (on success). There is no such opportunity for kexec, meaning that this feature will be restricted to flat network cases. The group has expressed lots of interest in providing this feature as option for advanced operators, in other words “those who run large scale computing farms”. Julia proposed making a demo of super-fast deployment using fast-track and kexec as a goal moving forward and this received lots of positive feedback. Partitioning, What is next? ==================== Dmitry expressed that there seemed to be lots of interest from EU operators in supporting disk partitioning. This has been long sought, but with minimal consensus. We discussed some possible cases how this could be supported and we reached conclusion that the basis is largely just to support Linux Logical Volume Manager in the simplest possible configuration. At the same time the point was raised that parity basically means some mix of software RAID through LVM and UEFI boot. We soon realized we needed more information! So the decision was reached to start by creating a poll, with questions in three different languages to try and identify community requirements using some simple and feasible scenarios. Such as LVM on a single disk, LVM on multiple disks, LVM + image extraction on top of the LVM. The partitioning topic was actually very power in that we covered a number of different topics that we were not otherwise planning to explicitly cover. Network booting ---------------------- One of them being why not use network booting, and the point was made that Network is our legacy and fundamental for iSCSI based boot and ramdisk booting (such as the deployment ramdisk). During this dive into ironic’s history, we did reach an important point of consensus which is that Ironic should switch the default boot mode, as previously planned, and still keep at least one scenario test running in CI which uses network booting. Stated operator wants in terms of Partitioning ------------------------------------------------------------- Dmitry was able to provide us some insight into what the Russian operator community was seeking from Ironic, and Julia confirmed she had heard similar wants from public cloud operators wanting to offer Bare Metal as a Service. Largely these wants revolve around wanting LVM capability of the most basic possible scenario such as a single disk or a partition image with a LVM, or even Software RAID with partition images. Likely what has stalled some of these discussions in the past is the immediate focus on the more complex partitioning scenarios sought by some operators in the past, which has resulted in these discussions stalling due to complexity of requirements. Traits/Scheduling/Flavor Explosion =========================== Arne with CERN raised this topic to bring greater awareness. CERN presently has greater than 100 flavors representing their hardware fleet as each physical machine type gets its own flavor. This has resulted in pain from the lack of flavor specific quotas. What may help in this area is for Resource Class based quotas, but presently the state of that work is unknown. The bottom line: A user does not have clarity into their resource usage. The question then shifted to being able to report utilization since the current quota model is based on cores/RAM/instances but not resource_class consumption as a whole. The question largely being “How many am I allowed to create? [before I eat someone else’s capacity]”. https://github.com/stackhpc/os-capacity was raised as a reporting tool that may help with these sorts of situations with bare metal cloud operators. Another point raised in this discussion was the lack of being able to tie consumer and project ID to consumed resources, but it turns out the allocation list functionality in Placement now has this functionality. In the end of this discussion, there was consensus that this should be brought back to the Nova community radar. Machine Burn-in ============= Burning in machines is a regular topic that comes up, and it looks like we’re getting much closer to being able to support such functionality. Part of the reason behind discussing this was to determine how to move forward what organizations like CERN can offer the community. There are two use cases. The most important thing is to ensure that hardware does not fail. That doesn’t seem like a “cloudy” thing to have to worry about, but when you're building your cloud, you kind of want to make sure that suddenly half your hardware is not going to fail as soon as you put a workload on it. The second use case is nearly just as important, which is to be able to ensure that you are obtaining the performance you expect from the hardware. This brought a bit of discussion because there are fundamentally two different paths that could be taken. The first is to leverage the inspector, whereas the second is to use clean steps. Both have very useful possible configurations, largely there is no sophisticated data collection in terms of performance data. That being said, the consensus seemed to be that actual data collection was less of a problem than flexibility to invoke as part of cleaning and preparation of a machine into the deployment. In other words, the consensus seemed to be clean-steps would be ideal for community adoption and code acceptance. IPv6/Dual Stack ============ TL;DR, we need to remove the ip_version setting field. This is mostly a matter of time to sift through the PXE code, and determine the code paths that need to be taken. I.E. for IPv6, we would likely only want to signal flags for it if the machine is in UEFI mode. The dhcp-less work should provide some of the API side capabilities this will really need in terms of interaction with Neutron. Graphical Console ============== The question was raised to “what would it take to finally get this moving forward again?” This is because there is initial interface code, two proofs of concept, and it should be relatively straightforward to implement redfish support, OEM capability dependent. The answer was functionally “Someone needs to focus on this for a couple of months, keep it rebased, and engage the community”. The community expressed an absolute willingness to mentor. Software RAID - Specifying devices =========================== We have briefly discussed RFE https://storyboard.openstack.org/#!/story/2006369 that proposes a way to define which physical devices participate in software RAID. A similar functionality already exists in the RAID configuration format for hardware RAID, but software RAID always spans all hard drives, no matter how many. The RFE proposes re-using the same dictionary format as used for root device hints in the “physical_disks” field of the RAID configuration. This idea has been accepted by the audience, with Arne proposing to extend supported hints with a new “type” hint with values like “rotational”, “nvme” or “ssd”. Stickers ====== Yes, we really did discuss next steps for stickers. We have ideas. Lots of ideas… and we are all very busy. So we shall see if we’re able to make some awesome and fun stickers appear for the Berlin time frame. From amotoki at gmail.com Tue Mar 17 04:26:31 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Tue, 17 Mar 2020 13:26:31 +0900 Subject: [tc][requirements][horizon] Broken gates due to the new setuptools and pyScss package In-Reply-To: References: <9bbd7f0e9fd3badec5f7f193aa8278ad86cd4279.camel@evrard.me> Message-ID: Hi, I agree with JP that this thread covers multiple topics. I would like to use this thread to focus on the original topic of the horizon gate breakage and cover other topic on further discussions in separate threads. TLDR; the workaround to fork pyScss and django-pyscss is a good compromise as a short-term solution. it allows us to explore alternatives without blocking other patches. On Mon, Mar 16, 2020 at 9:27 PM Mohammed Naser wrote: > > Would a more sustainable workaround of switching the pyscss code to > something that uses libsass-python: > > https://pypi.org/project/django-libsass/ > > That seems like an alternative and it means maintaining _far_ less code.. > Using python-libsass sounds reasonable as libsass is a mature C/C++ SASS engine and python-libsass provides the python bindings for it. I see two libsass support for Django: django-libsass (mnaser mentioned above) and django-sass-processor. I am not sure which is better at the moment. [1] https://pypi.org/project/django-libsass/ [2] https://pypi.org/project/django-sass-processor/ On the other hand, they are not drop-in-replacement, so the migration may take time. We have the gate breakage right now, so I think the workaround to fork pyScss and django-pyscss is a good compromise as a short-term solution. it allows us to explore alternatives without blocking other patches. On Mon, Mar 16, 2020 at 6:27 PM Lajos Katona wrote: > > Hi, > I would like to just add my cent to the question of ownership of these forked repos: > I am on the side to have opendev ownership over them as that give long(er) term stability, > even now when we have serious shortages in design. But consider the situation when > e0ne leave the community, we have to do the fork again and move these repos under > opendev umbrella. I discussed it with e0ne when he forked these repositories. We don't want to depend on them. We are forking these repos but it is just to catch up the recent changes in setuptools and to give us time to explore alternatives. We don't plan to maintain features in pyscss itself in the forked repository. This is a short-time workaround and IMHO it is not a good idea to create and retire repositories under OpenStack governance in a short term. If we decide to continue to use these libraries, then it is time to host these forked repositories in OpenStack world. Thanks, Akihiro From sshnaidm at redhat.com Tue Mar 17 10:53:19 2020 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Tue, 17 Mar 2020 12:53:19 +0200 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: <168c8eed885645daaa82340403231e65@AUSX13MPS308.AMER.DELL.COM> References: <1583528201.853712216@emailsrvr.com> <168c8eed885645daaa82340403231e65@AUSX13MPS308.AMER.DELL.COM> Message-ID: Hi, all I think the Openstack community has more virtual collaboration experience than any other community and maybe it's a good chance for us to establish an efficient way to make a virtual summit. I'm not alone thinking virtual summits/webinars/whatever are much less efficient than in-person meetings and hanging out together. But in current circumstances, it doesn't seem like an option, so why not try brainstorm about how we still can do a virtual summit and get most of it? Main issues with virtual meetings that I can see are: - topics discussions in a big group of people, while virtual meetings often less convenient and sometimes are tough[1] - the important things happen also *between* sessions, not during them - what would replace booths (and swags of course)? - PTG project rooms which are used both for internal discussions and collaboration between various projects - informal mingling I'm not sure all of these and other issues can be easily replaced virtually, but who knows, maybe we can still make it work. Like maybe constant open video sessions which mimic rooms and booths, some of the discussions moved to dedicated IRC channels. On the flip side, the offline summit had restrictions as well, so we can do things in a virtual summit we couldn't (or couldn't do so much) in offline ones. Maybe more presenters will have a chance to make sessions since we are not so limited in time online and all sessions can be recorded as well. And we don't need a projector in every PTG room now :) I'd propose creating a page where people can put their ideas on how to make good things we have in the offline summit to be virtual or add something new, which couldn't be done in the offline summit. Since we created recently "Ideas for OpenStack" page[2], maybe it's a good candidate to hold these ideas in. Just reminding that Openstack is not the only project that cancels offline meetings and efficient virtual collaboration is a very hot topic right now all over the world. I think the Openstack community has all the resources to contribute from its experience to many other projects and companies and be leading in this area. WDYT? [1] https://www.youtube.com/watch?v=DYu_bGbZiiQ [2] https://governance.openstack.org/ideas/index.html On Thu, Mar 12, 2020 at 4:13 PM wrote: > Agree that going virtual makes most sense given current status > > > > *From:* Emilien Macchi > *Sent:* Wednesday, March 11, 2020 5:46 PM > *To:* Mark Collier > *Cc:* openstack-discuss; Jonathan Bryce > *Subject:* Re: FW: 2020 OSF Events & coronavirus > > > > [EXTERNAL EMAIL] > > Hi Mark, > > > > Thanks for the transparency, as usual. I have a few thoughts, please read > inline. > > > > On Fri, Mar 6, 2020 at 4:04 PM Mark Collier wrote: > > upcoming event in Vancouver is no exception. The OpenDev tracks > > each morning will be programmed by volunteers from the community, and > the project > > teams will be organizing their own conversations as well each afternoon > M-W, and > > all day Thursday. > > > > But the larger question is here: should the show go on? > > > > The short answer is that as of now, the Vancouver and Berlin events are > still > > scheduled to happen in June (8-11) and October (19-23), respectively. > > > > However, we are willing to cancel or approach the events in a different > way (i.e. > > virtual) if the facts indicate that is the best path, and we know the > facts are > > changing rapidly. One of the most critical inputs we need is to hear > from each of > > you. We know that many of you rely on the twice-annual events to get > together and > > make rapid progress on the software, which is one reason we are not > making any > > decisions in haste. We also know that many of you may be unable or > unwilling to > > travel in June, and that is critical information to hear as we get > closer to the > > event so that we can make the most informed decision. > > > I believe that we, as a community should show the example and our > strengths by cancelling the Vancouver event and organize a virtual event > like some other big events are doing. > > There is an opportunity for the OSF to show leadership in Software > communities and acknowledge the risk of spread during that meeting; not > only for the people attending it but for also those in contact with these > people later. > > > > I'm not a doctor nor I know much about the virus; but I'm not interested > to travel and take the risk to 1) catch the virus and 2) spread it at home > and in my country; and as a community member, I feel like our > responsibility is also to maintain ourselves healthy. > > > > In my opinion, the sooner we cancel, the better we can focus on organizing > the virtual meetings, and also we can influence more communities to take > that kind of decisions. > > > > Thanks Mark for starting that discussion, it's a perfect sign of how > healthy is our community; and hopefully it will continue to be. > > -- > > Emilien Macchi > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Tue Mar 17 12:27:21 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Tue, 17 Mar 2020 08:27:21 -0400 Subject: [cinder] virtual mid-cycle session 2 summary available Message-ID: <01789df7-8ce1-b32e-6a38-0b69ca72c770@gmail.com> Hello Cinderinos, Thanks for a productive discussion yesterday. I've posted a summary on the wiki so we can remember what happened and for the edification of those who didn't attend: https://wiki.openstack.org/wiki/CinderUssuriMidCycleSummary A link to the recording will be posted on that page when it's available. cheers, brian From mjozefcz at redhat.com Tue Mar 17 12:50:28 2020 From: mjozefcz at redhat.com (Maciej Jozefczyk) Date: Tue, 17 Mar 2020 13:50:28 +0100 Subject: [all][dev][qa] cirros 0.5.1 In-Reply-To: References: Message-ID: Hello, Testing neutron: https://review.opendev.org/#/c/711425/. Looks like it is green. Maciej On Thu, Mar 12, 2020 at 7:36 PM Radosław Piliszek < radoslaw.piliszek at gmail.com> wrote: > Thanks, Dmitry, for your quick response. > I left 2 comments on it, rather important. > > @all: > Based on this experience, please ensure you actually get to use > devstack's cirros 0.5.1 :-) > > -yoctozepto > > czw., 12 mar 2020 o 17:18 Dmitry Tantsur napisał(a): > > > > Testing ironic: https://review.opendev.org/#/c/712728/ > > > > Thanks for the heads-up! > > > > Dmitry > > > > On Thu, Mar 12, 2020 at 12:57 PM Radosław Piliszek < > radoslaw.piliszek at gmail.com> wrote: > >> > >> Hiya Folks, > >> > >> as you might have noticed, cinder 0.5.1 has been released. > >> This build seems to be passing the current devstack gate. [1] > >> Big thanks to hrw and smoser for letting cirros 0.5.1 happen (and > >> cirros having bright future yet again). > >> Also thanks to mordred for cleaning up SDK testing to pass. :-) > >> > >> I think it would be nice to merge this in Ussuri still, preferably > before April. > >> On the other hand, we all know that devstack gate is not super > >> comprehensive and hence I would like to ask teams whose tests depend > >> on interaction with guest OS to test their gates on this patch (or > >> help me help you do that). > >> I deliberately marked it W-1 to avoid merging too early. > >> > >> Let the discussion begin. :-) > >> > >> [1] https://review.opendev.org/711492 > >> > >> -yoctozepto > >> > > -- Best regards, Maciej Józefczyk -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Tue Mar 17 14:12:17 2020 From: katonalala at gmail.com (Lajos Katona) Date: Tue, 17 Mar 2020 15:12:17 +0100 Subject: [tc][requirements][horizon] Broken gates due to the new setuptools and pyScss package In-Reply-To: References: <9bbd7f0e9fd3badec5f7f193aa8278ad86cd4279.camel@evrard.me> Message-ID: Hi, Thanks for the clear way forward Akihiro, I hope that this helps us. Lajos Akihiro Motoki ezt írta (időpont: 2020. márc. 17., K, 5:31): > Hi, > > I agree with JP that this thread covers multiple topics. > I would like to use this thread to focus on the original topic of the > horizon gate breakage > and cover other topic on further discussions in separate threads. > > TLDR; the workaround to fork pyScss and django-pyscss is a good compromise > as a short-term solution. it allows us to explore alternatives without > blocking other patches. > > On Mon, Mar 16, 2020 at 9:27 PM Mohammed Naser > wrote: > > > > Would a more sustainable workaround of switching the pyscss code to > > something that uses libsass-python: > > > > https://pypi.org/project/django-libsass/ > > > > That seems like an alternative and it means maintaining _far_ less code.. > > > > Using python-libsass sounds reasonable as libsass is a mature C/C++ SASS > engine > and python-libsass provides the python bindings for it. > > I see two libsass support for Django: django-libsass (mnaser mentioned > above) and > django-sass-processor. I am not sure which is better at the moment. > > [1] https://pypi.org/project/django-libsass/ > [2] https://pypi.org/project/django-sass-processor/ > > On the other hand, they are not drop-in-replacement, so the migration > may take time. > We have the gate breakage right now, so I think the workaround to fork > pyScss and > django-pyscss is a good compromise as a short-term solution. it allows > us to explore > alternatives without blocking other patches. > > On Mon, Mar 16, 2020 at 6:27 PM Lajos Katona wrote: > > > > Hi, > > I would like to just add my cent to the question of ownership of these > forked repos: > > I am on the side to have opendev ownership over them as that give > long(er) term stability, > > even now when we have serious shortages in design. But consider the > situation when > > e0ne leave the community, we have to do the fork again and move these > repos under > > opendev umbrella. > > I discussed it with e0ne when he forked these repositories. > We don't want to depend on them. We are forking these repos but it is > just to catch up > the recent changes in setuptools and to give us time to explore > alternatives. > We don't plan to maintain features in pyscss itself in the forked > repository. > This is a short-time workaround and IMHO it is not a good idea to > create and retire > repositories under OpenStack governance in a short term. If we decide > to continue to > use these libraries, then it is time to host these forked repositories > in OpenStack world. > > Thanks, > Akihiro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Tue Mar 17 15:18:55 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Tue, 17 Mar 2020 08:18:55 -0700 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> <168c8eed885645daaa82340403231e65@AUSX13MPS308.AMER.DELL.COM> Message-ID: Greetings, I think we need to consider teasing out the reasoning behind events a little more. When we look at summits, they are largely events to broadcast outward our work and provide vendors an avenue to express their new features and functionality, and generate sales leads. Where as the events likely need to be teased apart a little more. What we've traditionally referred to as a summit is largely rooted in broadcasting outward what we as a community are working on and the direction where we are going, while providing a time and space for vendors to to show off their solutions and offerings. To me, this largely seems like a coordinated outreach and marketing Blitz. And this is not one of JUST vendors, but of project teams as well for they need to convey what they have been working on and their focus as a community. If we look the PTG and Forum sorts of events that we have had, they provide more an avenue for collaboration and discussion. That could be as simple as a rough list of topics, and a few predefined "hallways" where random discussion can occur. In a sense, the hallway becomes the un-conference if we empower and enable it. Anyway, before we jump into ideas and implementation scenarios, we need to have a clear motivation behind each "event" that the community and it's partners can rally behind, and I suspect that will only come from the OSF events team as well as track chairs/organizers. -Julia On Tue, Mar 17, 2020 at 3:56 AM Sagi Shnaidman wrote: > > Hi, all > I think the Openstack community has more virtual collaboration experience than any other community and maybe it's a good chance for us to establish an efficient way to make a virtual summit. I'm not alone thinking virtual summits/webinars/whatever are much less efficient than in-person meetings and hanging out together. But in current circumstances, it doesn't seem like an option, so why not try brainstorm about how we still can do a virtual summit and get most of it? > > Main issues with virtual meetings that I can see are: > > - topics discussions in a big group of people, while virtual meetings often less convenient and sometimes are tough[1] > - the important things happen also *between* sessions, not during them > - what would replace booths (and swags of course)? > - PTG project rooms which are used both for internal discussions and collaboration between various projects > - informal mingling > > I'm not sure all of these and other issues can be easily replaced virtually, but who knows, maybe we can still make it work. Like maybe constant open video sessions which mimic rooms and booths, some of the discussions moved to dedicated IRC channels. > On the flip side, the offline summit had restrictions as well, so we can do things in a virtual summit we couldn't (or couldn't do so much) in offline ones. Maybe more presenters will have a chance to make sessions since we are not so limited in time online and all sessions can be recorded as well. And we don't need a projector in every PTG room now :) > I'd propose creating a page where people can put their ideas on how to make good things we have in the offline summit to be virtual or add something new, which couldn't be done in the offline summit. Since we created recently "Ideas for OpenStack" page[2], maybe it's a good candidate to hold these ideas in. > Just reminding that Openstack is not the only project that cancels offline meetings and efficient virtual collaboration is a very hot topic right now all over the world. I think the Openstack community has all the resources to contribute from its experience to many other projects and companies and be leading in this area. > WDYT? > > [1] https://www.youtube.com/watch?v=DYu_bGbZiiQ > [2] https://governance.openstack.org/ideas/index.html > > On Thu, Mar 12, 2020 at 4:13 PM wrote: >> >> Agree that going virtual makes most sense given current status >> >> >> >> From: Emilien Macchi >> Sent: Wednesday, March 11, 2020 5:46 PM >> To: Mark Collier >> Cc: openstack-discuss; Jonathan Bryce >> Subject: Re: FW: 2020 OSF Events & coronavirus >> >> >> >> [EXTERNAL EMAIL] >> >> Hi Mark, >> >> >> >> Thanks for the transparency, as usual. I have a few thoughts, please read inline. >> >> >> >> On Fri, Mar 6, 2020 at 4:04 PM Mark Collier wrote: >> >> upcoming event in Vancouver is no exception. The OpenDev tracks >> > each morning will be programmed by volunteers from the community, and the project >> > teams will be organizing their own conversations as well each afternoon M-W, and >> > all day Thursday. >> > >> > But the larger question is here: should the show go on? >> > >> > The short answer is that as of now, the Vancouver and Berlin events are still >> > scheduled to happen in June (8-11) and October (19-23), respectively. >> > >> > However, we are willing to cancel or approach the events in a different way (i.e. >> > virtual) if the facts indicate that is the best path, and we know the facts are >> > changing rapidly. One of the most critical inputs we need is to hear from each of >> > you. We know that many of you rely on the twice-annual events to get together and >> > make rapid progress on the software, which is one reason we are not making any >> > decisions in haste. We also know that many of you may be unable or unwilling to >> > travel in June, and that is critical information to hear as we get closer to the >> > event so that we can make the most informed decision. >> >> >> I believe that we, as a community should show the example and our strengths by cancelling the Vancouver event and organize a virtual event like some other big events are doing. >> >> There is an opportunity for the OSF to show leadership in Software communities and acknowledge the risk of spread during that meeting; not only for the people attending it but for also those in contact with these people later. >> >> >> >> I'm not a doctor nor I know much about the virus; but I'm not interested to travel and take the risk to 1) catch the virus and 2) spread it at home and in my country; and as a community member, I feel like our responsibility is also to maintain ourselves healthy. >> >> >> >> In my opinion, the sooner we cancel, the better we can focus on organizing the virtual meetings, and also we can influence more communities to take that kind of decisions. >> >> >> >> Thanks Mark for starting that discussion, it's a perfect sign of how healthy is our community; and hopefully it will continue to be. >> >> -- >> >> Emilien Macchi > > > > -- > Best regards > Sagi Shnaidman From openstack at nemebean.com Tue Mar 17 17:48:29 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 17 Mar 2020 12:48:29 -0500 Subject: OpenStack support for msgpack 1.0.0 In-Reply-To: <20200314222345.xrcz2e3275wtu6bb@mthode.org> References: <95326a32-b0d0-9f69-362d-e1c949fa39a9@debian.org> <20200314222345.xrcz2e3275wtu6bb@mthode.org> Message-ID: On 3/14/20 5:23 PM, Matthew Thode wrote: > On 20-03-14 22:52:45, Thomas Goirand wrote: >> Hi, >> >> msgpack 1.0.0 was released last Sept. I'm being pressed by the rest of >> Debian people to upload 1.0.0 to Sid. I'm concern by it: will this break >> any of the OpenStack package, like oslo.messaging? Is there any plan to >> have support for 1.0.0 for Ussuri? >> >> Cheers, >> >> Thomas Goirand (zigo) >> > > Openstack depends on salt > Salt capped msgpack to not allow 1.0.0 > https://github.com/saltstack/salt/pull/56009 > Openstack can't use msgpack-1.0.0 > this makes me sad > iirc openstack tests passed with it, I just made a review to test > https://review.opendev.org/713113 > > When I mentioned this to upstream salt people they said it'd be in a > future release, I'd project after the openstack release (it's not part > of 3000.1 which is being preparred now). > For convenience, here's the search for projects that explicitly depend on msgpack and might also be affected by this: http://codesearch.openstack.org/?q=msgpack&i=nope&files=requirements.txt&repos= For Oslo, it looks like the only changes we need should be in oslo.privsep[0]. oslo.serialization already got updated for it, and tooz has been running tests without upper-constraints all along so we've been using it there since it released. :-/ 0: https://review.opendev.org/713496 From gagehugo at gmail.com Tue Mar 17 18:51:16 2020 From: gagehugo at gmail.com (Gage Hugo) Date: Tue, 17 Mar 2020 13:51:16 -0500 Subject: [openstack-helm] Virtual Midcycle Message-ID: Good afternoon, The openstack-helm team will be hosting their virtual midcycle next Tuesday, March 24th 2020 at 1400 to 1700 UTC. Please check out the etherpad link for more details about the agenda and conferencing tool. Since this meeting is overlapping our normal meeting time, the actual IRC meeting will be canceled for next week. https://etherpad.openstack.org/p/osh-virtual-ptg-2020-03 Hope to see you all there. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihalis68 at gmail.com Tue Mar 17 19:20:53 2020 From: mihalis68 at gmail.com (Chris Morgan) Date: Tue, 17 Mar 2020 15:20:53 -0400 Subject: [all] Guides for newbies to OpenStack In-Reply-To: <5E6F853C.3040000@openstack.org> References: <24f6f94cf9eabffd30389ad02718628989c40edb.camel@evrard.me> <5E6F853C.3040000@openstack.org> Message-ID: I've found using 'microstack' a good way to get a little "plan vanilla" test openstack instance quickly. you can even put it all in a container using 'multipass' https://microstack.run https://multipass.run On Mon, Mar 16, 2020 at 10:04 AM Jimmy McArthur wrote: > If you check on this page under Contributor Resources, you'll find a lot > of "getting started" content: https://www.openstack.org/community/ > > Cheers, > Jimmy > > Jean-Philippe Evrard > March 16, 2020 at 4:52 AM > On Wed, 2020-03-11 at 18:06 +1100, Mike Carden wrote: > > Hello, > > Would https://docs.openstack.org/train/user/ work? > For example, you can check the horizon guide to get started with Horizon... > > Regards, > JP > Mike Carden > March 11, 2020 at 2:06 AM > Our small team at ${DAYJOB} has built a handful of OpenStack clusters > based on Red Hat OpenStack 13 (aka Queens) over the last couple of years. > > We now find ourselves in the position of being 'gifted' human resources in > the shape of mid-level 'IT people' who are sent to join our team for a > short time to 'Learn OpenStack'. > > These tend to be people for whom, "Here's a Horizon URL and some creds - > go log in and launch a VM"[1]... is a bit much. > > I've done a wee bit of web searching (enough to find the dead links) > trying to find some newbie friendly tutorials on OpenStack basics. Before I > attempt to re-invent the wheel, can anyone suggest some public resources I > might point people to? > > Deity help us if we have to explain Tripleo's Undercloud, Overcloud, > partially containered, partially pacemakered, fully flaky... underpinnings. > > Thanks, > MC > [1] Even with a step by step guide > > > -- Chris Morgan -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Tue Mar 17 20:27:20 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 17 Mar 2020 13:27:20 -0700 Subject: Using #opendev on Freenode for OpenDev Infrastructure Discussions Message-ID: <67dfdeb0-e4f3-4315-88b9-d3244607f6be@www.fastmail.com> Hello all, We have reached a point in time where it now makes sense for us to use #opendev on Freenode for OpenDev discussions, rather than #openstack-infra. We are updating our various bots to reflect this change and are asking you to find us in the new channel. We expect this to be a time of transition so don't be afraid of bringing things up in either the old or new channel if you are unsure of where you should go; we'll get you pointed in the right direction. The old channel, #openstack-infra, won't be going away. We expect it to become more OpenStack specific. Topics like how to use Zuul for OpenStack jobs could go here. This likely will have some overlap with the #openstack-qa channel and we may end up collapsing them, but that can happen if we notice the conversations do overlap significantly. We hope this gives non OpenStack users a friendly neutral location to engage us. Sorry, for any disruptions this may cause. Clark From kennelson11 at gmail.com Tue Mar 17 20:29:11 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 17 Mar 2020 13:29:11 -0700 Subject: [all][PTL][tc] U Community Goal: Project PTL & Contrib Docs Update #5 Message-ID: Hello Everyone! First off, I want to say thank you so much to Watcher, Trove, Searchlight, Qinling, Nova, Neutron, Kolla, Ironic, i18n, Glance, Cinder, and Blazar for getting started/completing the goal! For those projects that have not started yet, you can take a look at what the others have done for some inspiration[1], or simply fill in the template[2]. We did our best to make the process as low effort as possible. If you have any questions, please let me know or refer to the goal[3]. Once you get started, please make sure to keep the story up to date[4]. I look forward to reviewing your patches!! If there is anything I can do to help, please let me know. -Kendall (diablo_rojo) [1] Progress: https://review.opendev.org/#/q/topic:project-ptl-and-contrib-docs+(status:open+OR+status:merged) [2] Cookiecutter Template: https://opendev.org/openstack/cookiecutter/src/branch/master/%7b%7bcookiecutter.repo_name%7d%7d/doc/source/contributor/contributing.rst [3] Accepted Goal: https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-contrib-docs.html [4] Task Tracking: https://storyboard.openstack.org/#!/story/2007236 -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Tue Mar 17 21:08:32 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 17 Mar 2020 14:08:32 -0700 Subject: review.opendev.org Downtime March 20, 2020 at 1400UTC Message-ID: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> Hello all, The OpenDev team is planning to take a short downtime March 20, 2020 from 14:00UTC to 15:00UTC. This outage will enable us to rename a few projects and redeploy Gerrit using containers. The container based deployment is an important first step in our longer term plan for upgrading Gerrit to a modern version. We recognize that this will be a disruption during an already disrupted week for many. We feel this is worthwhile as it fits into project release schedules and sets us up well for the future. Thank you for your patience. Feel free to ask us any questions you may have on this thread or in #opendev on Freenode. Clark From mark.kirkwood at catalyst.net.nz Wed Mar 18 04:22:32 2020 From: mark.kirkwood at catalyst.net.nz (Mark Kirkwood) Date: Wed, 18 Mar 2020 17:22:32 +1300 Subject: [swift] Rolling upgrade, any version relationships? In-Reply-To: <5f55bc36-b51c-5c6d-ad0f-63a32fcba2d4@catalyst.net.nz> References: <5f55bc36-b51c-5c6d-ad0f-63a32fcba2d4@catalyst.net.nz> Message-ID: Ping! This question seems to have slipped in the global bitstream unnoticed :-) On 11/03/20 3:27 pm, Mark Kirkwood wrote: > Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past > I've modeled this on a dev system by upgrading storage nodes one by > one (using 2.17 as the target version). This seemed to work well - I > deliberately left the cluster half upgraded for an extended period to > test for any cross version weirdness (didn't see any). However I'm > wanting to check that I have not missed something important. So my > questions are: > > - If upgrading from 2.7.0 is it safe to just grab the latest version > (e.g 2.23)? > > - If not is there a preferred version to jump to first? > > - Is it ok for the upgrade to take extended time (e.g weeks) and > therefore be running with some new and some old storage nodes for that > time? > > regards > > Mark > > From ppiyakk2 at printf.kr Wed Mar 18 05:14:07 2020 From: ppiyakk2 at printf.kr (SeongSoo Cho) Date: Wed, 18 Mar 2020 14:14:07 +0900 Subject: [swift] Rolling upgrade, any version relationships? In-Reply-To: References: <5f55bc36-b51c-5c6d-ad0f-63a32fcba2d4@catalyst.net.nz> Message-ID: <62556FB9-6460-4D2C-B8E8-9FB3F6B2B1DD@printf.kr> Hi In my case, I upgraded my cluster from ocata to train directly, and It was very successful. - Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time? -> I think.. It's ok but It is better to take a short period. Regards Seongsoo > On Mar 18, 2020, at 1:22 PM, Mark Kirkwood wrote: > > Ping! This question seems to have slipped in the global bitstream unnoticed :-) > > On 11/03/20 3:27 pm, Mark Kirkwood wrote: >> Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are: >> >> - If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)? >> >> - If not is there a preferred version to jump to first? >> >> - Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time? >> >> regards >> >> Mark >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr at ham.ie Wed Mar 18 11:19:18 2020 From: gr at ham.ie (Graham Hayes) Date: Wed, 18 Mar 2020 11:19:18 +0000 Subject: Not running for TC this round In-Reply-To: <3692fa0f-078d-06dc-9509-e352c7b0758b@nemebean.com> References: <3692fa0f-078d-06dc-9509-e352c7b0758b@nemebean.com> Message-ID: On 16/03/2020 19:50, Ben Nemec wrote: > > > On 3/16/20 9:36 AM, Thierry Carrez wrote: >> The final reason is that I've always been saying that you can work on >> governance issues and voice your opinion on them, without being >> elected on the TC and formally voting on things. > > Can confirm. Have stuck my nose into plenty of TC discussions without > ever being a member. For better or worse. ;-) > ++ - I caused plenty of TC discussions before I got elected :) From smooney at redhat.com Wed Mar 18 12:16:12 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 18 Mar 2020 12:16:12 +0000 Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: References: <1583528201.853712216@emailsrvr.com> <168c8eed885645daaa82340403231e65@AUSX13MPS308.AMER.DELL.COM> Message-ID: <5e4b4ab444e6e6a0c17f37b3387d6f3e156aa1ec.camel@redhat.com> On Tue, 2020-03-17 at 08:18 -0700, Julia Kreger wrote: > Greetings, > > I think we need to consider teasing out the reasoning behind events a > little more. When we look at summits, they are largely events to > broadcast outward our work and provide vendors an avenue to express > their new features and functionality, and generate sales leads. > > Where as the events likely need to be teased apart a little more. What > we've traditionally referred to as a summit is largely rooted in > broadcasting outward what we as a community are working on and the > direction where we are going, while providing a time and space for > vendors to to show off their solutions and offerings. To me, this > largely seems like a coordinated outreach and marketing Blitz. And > this is not one of JUST vendors, but of project teams as well for they > need to convey what they have been working on and their focus as a > community. > > If we look the PTG and Forum sorts of events that we have had, they > provide more an avenue for collaboration and discussion. That could be > as simple as a rough list of topics, and a few predefined "hallways" > where random discussion can occur. In a sense, the hallway becomes the > un-conference if we empower and enable it. of the two types of events i think the second e.g. PTG/Forum is the one that we as a comunity would be most impacted by if it was not to go ahead either physically or virtually. the summit/marketplace may impact sales to a degree but for those of us that work at a company/vendor that makes money form openstack we have large sales and marketing teams anyway so the sales lead perspective while valid is not really that critical to the same degree as losing the opportunity to collaborate and cross pollinate ideas between different aspect of the comunity. it is very true that a lot of conversation happen in the hallway but those conversatoin can happen on irc or video calls too but the barrier to entry is obviously higher to start them. > > Anyway, before we jump into ideas and implementation scenarios, we > need to have a clear motivation behind each "event" that the community > and it's partners can rally behind, and I suspect that will only come > from the OSF events team as well as track chairs/organizers. > > -Julia > > On Tue, Mar 17, 2020 at 3:56 AM Sagi Shnaidman wrote: > > > > Hi, all > > I think the Openstack community has more virtual collaboration experience than any other community and maybe it's a > > good chance for us to establish an efficient way to make a virtual summit. I'm not alone thinking virtual > > summits/webinars/whatever are much less efficient than in-person meetings and hanging out together. But in current > > circumstances, it doesn't seem like an option, so why not try brainstorm about how we still can do a virtual summit > > and get most of it? > > > > Main issues with virtual meetings that I can see are: > > > > - topics discussions in a big group of people, while virtual meetings often less convenient and sometimes are > > tough[1] > > - the important things happen also *between* sessions, not during them > > - what would replace booths (and swags of course)? > > - PTG project rooms which are used both for internal discussions and collaboration between various projects > > - informal mingling > > > > I'm not sure all of these and other issues can be easily replaced virtually, but who knows, maybe we can still make > > it work. Like maybe constant open video sessions which mimic rooms and booths, some of the discussions moved to > > dedicated IRC channels. > > On the flip side, the offline summit had restrictions as well, so we can do things in a virtual summit we couldn't > > (or couldn't do so much) in offline ones. Maybe more presenters will have a chance to make sessions since we are not > > so limited in time online and all sessions can be recorded as well. And we don't need a projector in every PTG room > > now :) > > I'd propose creating a page where people can put their ideas on how to make good things we have in the offline > > summit to be virtual or add something new, which couldn't be done in the offline summit. Since we created recently > > "Ideas for OpenStack" page[2], maybe it's a good candidate to hold these ideas in. > > Just reminding that Openstack is not the only project that cancels offline meetings and efficient virtual > > collaboration is a very hot topic right now all over the world. I think the Openstack community has all the > > resources to contribute from its experience to many other projects and companies and be leading in this area. > > WDYT? > > > > [1] https://www.youtube.com/watch?v=DYu_bGbZiiQ > > [2] https://governance.openstack.org/ideas/index.html > > > > On Thu, Mar 12, 2020 at 4:13 PM wrote: > > > > > > Agree that going virtual makes most sense given current status > > > > > > > > > > > > From: Emilien Macchi > > > Sent: Wednesday, March 11, 2020 5:46 PM > > > To: Mark Collier > > > Cc: openstack-discuss; Jonathan Bryce > > > Subject: Re: FW: 2020 OSF Events & coronavirus > > > > > > > > > > > > [EXTERNAL EMAIL] > > > > > > Hi Mark, > > > > > > > > > > > > Thanks for the transparency, as usual. I have a few thoughts, please read inline. > > > > > > > > > > > > On Fri, Mar 6, 2020 at 4:04 PM Mark Collier wrote: > > > > > > upcoming event in Vancouver is no exception. The OpenDev tracks > > > > each morning will be programmed by volunteers from the community, and the project > > > > teams will be organizing their own conversations as well each afternoon M-W, and > > > > all day Thursday. > > > > > > > > But the larger question is here: should the show go on? > > > > > > > > The short answer is that as of now, the Vancouver and Berlin events are still > > > > scheduled to happen in June (8-11) and October (19-23), respectively. > > > > > > > > However, we are willing to cancel or approach the events in a different way (i.e. > > > > virtual) if the facts indicate that is the best path, and we know the facts are > > > > changing rapidly. One of the most critical inputs we need is to hear from each of > > > > you. We know that many of you rely on the twice-annual events to get together and > > > > make rapid progress on the software, which is one reason we are not making any > > > > decisions in haste. We also know that many of you may be unable or unwilling to > > > > travel in June, and that is critical information to hear as we get closer to the > > > > event so that we can make the most informed decision. > > > > > > > > > I believe that we, as a community should show the example and our strengths by cancelling the Vancouver event and > > > organize a virtual event like some other big events are doing. > > > > > > There is an opportunity for the OSF to show leadership in Software communities and acknowledge the risk of spread > > > during that meeting; not only for the people attending it but for also those in contact with these people later. > > > > > > > > > > > > I'm not a doctor nor I know much about the virus; but I'm not interested to travel and take the risk to 1) catch > > > the virus and 2) spread it at home and in my country; and as a community member, I feel like our responsibility is > > > also to maintain ourselves healthy. > > > > > > > > > > > > In my opinion, the sooner we cancel, the better we can focus on organizing the virtual meetings, and also we can > > > influence more communities to take that kind of decisions. > > > > > > > > > > > > Thanks Mark for starting that discussion, it's a perfect sign of how healthy is our community; and hopefully it > > > will continue to be. > > > > > > -- > > > > > > Emilien Macchi > > > > > > > > -- > > Best regards > > Sagi Shnaidman > > From mnaser at vexxhost.com Wed Mar 18 13:04:00 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 18 Mar 2020 09:04:00 -0400 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <29a571a5-6081-448e-ba09-740e83200341@Spark> References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> <29a571a5-6081-448e-ba09-740e83200341@Spark> Message-ID: Hi everyone, I see that the discussion has stalled out, I'd like us to seriously explore this because I think come the PTL time, we might run into a lot of projects who don't have folks stepping in. Thanks, Mohammed On Fri, Mar 13, 2020 at 1:37 AM Renat Akhmerov wrote: > > On 5 Mar 2020, 02:57 +0700, Jay Bryant , wrote: > > Zane's input sums up how I feel as well. I think that having consistent > leadership structure across projects is important and helps keep us > aware of the health of projects. > > Perhaps we can help return interest in the PTL role by providing > examples of teams that share the work and have the PTL to help make > final decisions. I know that the Cinder team has been doing this for > quite some time successfully. > > > I really fail to understand the issue of an overwhelming PTLship. From my > personal 5+ experience of being a PTL, I can say I’ve never tried to do all > stuff like releases, fixing CI, back porting etc. etc. myself. Especially given > that I really like solving technical problems, I always try to offload some of > these duties to others and make sure I have time for development. And > IMO it’s totally fine. As a PTL I try to make sure that everyone in the team > (at least most active members) has this balance between problem solving > and necessary procedures. I absolutely agree though that as the PTL I have > to know what’s going on with the project and after all TC or someone else > can ask me about its current state and progress. > Of course, I realise that my project is somewhat not really typical for > OpenStack: it is relatively small and has not so many connections with other > projects. But I believe this principle is universal enough. > As far as the PTL role, I think for PTLs it’s important to focus on the big > picture, ideas and directions and keep reminding everyone about that. > All team members, even active ones, often can’t afford thinking about this > too much. This contradicts with lots of what I heard before from my former > managers and colleagues, and also some PTLs I know. They claimed: > “PTLs just need to maintain Launchpad (or Storyboard), keep an eye > on the release process and that’s basically it. Plus reviewing a little bit.” > I’ve always shrugged when hearing this.. If so, let’s remove “L” from “PTL” > and replace it with “A”, so that it’s “PTA” - Project Technical Administrator. > Something that can legally exist, no issue. And it’s more honest. > What I’m going to is, from my perspective, it probably doesn’t make any > difference if a project leader is an official role or not. I guess there will > always be someone who naturally gains trust of others and influences > the direction of a project. As far as “having a final word on a deadlocked > issue” I thought this is something really important but, in fact, it’s a > very rare thing and may not be needed at all. Usually, we have to make > a decision anyway, since “not making a decision is more expensive than > making even bad decision”. > > So I believe some leadership is always needed. The most high quality > techs I’ve ever seen have been all made with a very strong leadership, > I don’t believe it works the other way. Whether it’s official or not, I think > is not important at all. But “administrative duties” that often assign to > PTLs can be easily split between people. > > > Thanks > > Renat Akhmerov > @Nokia -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From johfulto at redhat.com Wed Mar 18 19:31:59 2020 From: johfulto at redhat.com (John Fulton) Date: Wed, 18 Mar 2020 15:31:59 -0400 Subject: [tripleo][operators] Removal of mistral from the TripleO Undercloud In-Reply-To: References: Message-ID: On Sat, Mar 14, 2020 at 8:06 AM Rabi Mishra wrote: > > On Sat, Mar 14, 2020 at 2:10 AM John Fulton wrote: >> >> On Fri, Mar 13, 2020 at 3:27 PM Kevin Carter wrote: >> > >> > Hello stackers, >> > >> > In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4]. >> >> As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud. >> >> > >> > So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases? > > > Yeah, as John mentioned, tripleo.derive_params.v1.derive_parameters workflow is surely being used for NFV (DPDK/SR-IOV) and HCI use cases and can't be deprecated or dropped. Though we've a generic interface in tripleoclient to run any workflow in plan-environment, I have not seen it being used for anything other than the mentioned workflow. > > In the scope of 'mistral-ansible' work, we seem to have two options. > > 1. Convert the workflow to ansible playbook 'as-is' i.e calculate and merge the derived parameters in plan-environment and as you've mentioned, change tripleoclient code to call any playbook in plan-environment.yaml and the parameters/vars. Nice idea. I hadn't thought of that. If there's a "hello world" example of this (which results in a THT param in the deployment plan being set to "hello world"), then I could try writing an ansible module to derive the HCI parameters and set them in place of the "hello world". John > 2. Move the functionality further down the component chain in TripleO to have the required ansible host/group_vars set for them to be used by config-download playbooks/ansible/puppet. > > I guess option 1 would be easier within the timelines. I've done some preliminary work to move some of the functionality in relevant mistral actions to utils modules[1], so that they can be called from ansible modules without depending on mistral/mistral-lib and use those in a playbook that kinda replicate the tasks in the mistral workflow. > > Having said that, it would be good to know what the DFG:NFV folks think, as they are the original authors/maintainers of that workflow. > > > >> The Mistral based workflow took advantage of the deployment plan which >> was stored in Swift on the undercloud. My understanding is that too is >> going away. > > > I'm not sure that would be in the scope of 'mstral-to-ansible' work. Dropping swift would probably be a bit more complex, as we use it to store templates, plan-environment, plan backups (possibly missing few more) etc and would require significant design rework (may be possible when we get rid of heat in undercloud). In spite of heat using the templates from swift and merging environments on the client side, we've had already bumped heat's REST API json body size limit (max_json_body_size) on the undercloud to 4MB[2] from the default 1MB and sending all required templates as part of API request would not be a good idea from undercloud scalability pov. > > [1] https://review.opendev.org/#/c/709546/ > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml#L109 > > -- > Regards, > Rabi Mishra > From mark at openstack.org Wed Mar 18 20:40:22 2020 From: mark at openstack.org (Mark Collier) Date: Wed, 18 Mar 2020 15:40:22 -0500 (CDT) Subject: FW: 2020 OSF Events & coronavirus In-Reply-To: <5e4b4ab444e6e6a0c17f37b3387d6f3e156aa1ec.camel@redhat.com> References: <1583528201.853712216@emailsrvr.com> <168c8eed885645daaa82340403231e65@AUSX13MPS308.AMER.DELL.COM> <5e4b4ab444e6e6a0c17f37b3387d6f3e156aa1ec.camel@redhat.com> Message-ID: <1584564022.68813793@emailsrvr.com> I wanted to make sure everyone saw this update I just posted to the OSF foundation mailing list: http://lists.openstack.org/pipermail/foundation/2020-March/002854.html The main thing we need now are volunteers to help us plan a virtual PTG (see link in thread above), and it sounds like many people on this thread are already thinking about ways to help :) I also agree with Julia and others who said it's important to clearly define the purpose of each gathering, as we think through the best way to organize them virtually. By working together we can make the first ever virtual PTG event a success. Mark On Wednesday, March 18, 2020 7:16am, "Sean Mooney" said: > On Tue, 2020-03-17 at 08:18 -0700, Julia Kreger wrote: >> Greetings, >> >> I think we need to consider teasing out the reasoning behind events a >> little more. When we look at summits, they are largely events to >> broadcast outward our work and provide vendors an avenue to express >> their new features and functionality, and generate sales leads. >> >> Where as the events likely need to be teased apart a little more. What >> we've traditionally referred to as a summit is largely rooted in >> broadcasting outward what we as a community are working on and the >> direction where we are going, while providing a time and space for >> vendors to to show off their solutions and offerings. To me, this >> largely seems like a coordinated outreach and marketing Blitz. And >> this is not one of JUST vendors, but of project teams as well for they >> need to convey what they have been working on and their focus as a >> community. >> >> If we look the PTG and Forum sorts of events that we have had, they >> provide more an avenue for collaboration and discussion. That could be >> as simple as a rough list of topics, and a few predefined "hallways" >> where random discussion can occur. In a sense, the hallway becomes the >> un-conference if we empower and enable it. > of the two types of events i think the second e.g. PTG/Forum is the one that > we as a comunity would be most impacted by if it was not to go ahead either > physically or virtually. > > the summit/marketplace may impact sales to a degree but for those of us > that work at a company/vendor that makes money form openstack we have large sales > and marketing teams anyway so the sales lead perspective while valid is not > really > that critical to the same degree as losing the opportunity to collaborate and > cross > pollinate ideas between different aspect of the comunity. > > it is very true that a lot of conversation happen in the hallway but those > conversatoin > can happen on irc or video calls too but the barrier to entry is obviously higher > to start them. > >> >> Anyway, before we jump into ideas and implementation scenarios, we >> need to have a clear motivation behind each "event" that the community >> and it's partners can rally behind, and I suspect that will only come >> from the OSF events team as well as track chairs/organizers. >> >> -Julia >> >> On Tue, Mar 17, 2020 at 3:56 AM Sagi Shnaidman wrote: >> > >> > Hi, all >> > I think the Openstack community has more virtual collaboration experience than >> any other community and maybe it's a >> > good chance for us to establish an efficient way to make a virtual summit. I'm >> not alone thinking virtual >> > summits/webinars/whatever are much less efficient than in-person meetings and >> hanging out together. But in current >> > circumstances, it doesn't seem like an option, so why not try brainstorm about >> how we still can do a virtual summit >> > and get most of it? >> > >> > Main issues with virtual meetings that I can see are: >> > >> > - topics discussions in a big group of people, while virtual meetings often >> less convenient and sometimes are >> > tough[1] >> > - the important things happen also *between* sessions, not during them >> > - what would replace booths (and swags of course)? >> > - PTG project rooms which are used both for internal discussions and >> collaboration between various projects >> > - informal mingling >> > >> > I'm not sure all of these and other issues can be easily replaced virtually, >> but who knows, maybe we can still make >> > it work. Like maybe constant open video sessions which mimic rooms and booths, >> some of the discussions moved to >> > dedicated IRC channels. >> > On the flip side, the offline summit had restrictions as well, so we can do >> things in a virtual summit we couldn't >> > (or couldn't do so much) in offline ones. Maybe more presenters will have a >> chance to make sessions since we are not >> > so limited in time online and all sessions can be recorded as well. And we >> don't need a projector in every PTG room >> > now :) >> > I'd propose creating a page where people can put their ideas on how to make >> good things we have in the offline >> > summit to be virtual or add something new, which couldn't be done in the >> offline summit. Since we created recently >> > "Ideas for OpenStack" page[2], maybe it's a good candidate to hold these ideas >> in. >> > Just reminding that Openstack is not the only project that cancels offline >> meetings and efficient virtual >> > collaboration is a very hot topic right now all over the world. I think the >> Openstack community has all the >> > resources to contribute from its experience to many other projects and >> companies and be leading in this area. >> > WDYT? >> > >> > [1] https://www.youtube.com/watch?v=DYu_bGbZiiQ >> > [2] https://governance.openstack.org/ideas/index.html >> > >> > On Thu, Mar 12, 2020 at 4:13 PM wrote: >> > > >> > > Agree that going virtual makes most sense given current status >> > > >> > > >> > > >> > > From: Emilien Macchi >> > > Sent: Wednesday, March 11, 2020 5:46 PM >> > > To: Mark Collier >> > > Cc: openstack-discuss; Jonathan Bryce >> > > Subject: Re: FW: 2020 OSF Events & coronavirus >> > > >> > > >> > > >> > > [EXTERNAL EMAIL] >> > > >> > > Hi Mark, >> > > >> > > >> > > >> > > Thanks for the transparency, as usual. I have a few thoughts, please read >> inline. >> > > >> > > >> > > >> > > On Fri, Mar 6, 2020 at 4:04 PM Mark Collier wrote: >> > > >> > > upcoming event in Vancouver is no exception. The OpenDev tracks >> > > > each morning will be programmed by volunteers from the community, and the >> project >> > > > teams will be organizing their own conversations as well each afternoon >> M-W, and >> > > > all day Thursday. >> > > > >> > > > But the larger question is here: should the show go on? >> > > > >> > > > The short answer is that as of now, the Vancouver and Berlin events are >> still >> > > > scheduled to happen in June (8-11) and October (19-23), respectively. >> > > > >> > > > However, we are willing to cancel or approach the events in a different way >> (i.e. >> > > > virtual) if the facts indicate that is the best path, and we know the facts >> are >> > > > changing rapidly. One of the most critical inputs we need is to hear from >> each of >> > > > you. We know that many of you rely on the twice-annual events to get >> together and >> > > > make rapid progress on the software, which is one reason we are not making >> any >> > > > decisions in haste. We also know that many of you may be unable or >> unwilling to >> > > > travel in June, and that is critical information to hear as we get closer >> to the >> > > > event so that we can make the most informed decision. >> > > >> > > >> > > I believe that we, as a community should show the example and our strengths >> by cancelling the Vancouver event and >> > > organize a virtual event like some other big events are doing. >> > > >> > > There is an opportunity for the OSF to show leadership in Software >> communities and acknowledge the risk of spread >> > > during that meeting; not only for the people attending it but for also those >> in contact with these people later. >> > > >> > > >> > > >> > > I'm not a doctor nor I know much about the virus; but I'm not interested to >> travel and take the risk to 1) catch >> > > the virus and 2) spread it at home and in my country; and as a community >> member, I feel like our responsibility is >> > > also to maintain ourselves healthy. >> > > >> > > >> > > >> > > In my opinion, the sooner we cancel, the better we can focus on organizing >> the virtual meetings, and also we can >> > > influence more communities to take that kind of decisions. >> > > >> > > >> > > >> > > Thanks Mark for starting that discussion, it's a perfect sign of how healthy >> is our community; and hopefully it >> > > will continue to be. >> > > >> > > -- >> > > >> > > Emilien Macchi >> > >> > >> > >> > -- >> > Best regards >> > Sagi Shnaidman >> >> > > From mark.kirkwood at catalyst.net.nz Wed Mar 18 21:41:33 2020 From: mark.kirkwood at catalyst.net.nz (Mark Kirkwood) Date: Thu, 19 Mar 2020 10:41:33 +1300 Subject: [swift] Erasure code hardware offloading Message-ID: <768ec329-a82e-af7f-9b0b-c63e0fdb977e@catalyst.net.nz> Hi, Can Swift make use of an Erasure Code offload card (e.g https://community.mellanox.com/s/article/understanding-erasure-coding-offload)? regards Mark From kennelson11 at gmail.com Thu Mar 19 00:24:34 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 18 Mar 2020 17:24:34 -0700 Subject: [all][elections][ptl][tc] Combined PTL/TC Election Season Message-ID: Hello Everyone! Election details: https://governance.openstack.org/election/ The nomination period officially begins Mar 24, 2020 23:45 UTC. Please read the stipulations and timelines for candidates and electorate contained in this governance documentation. Due to circumstances of timing, PTL and TC elections for the coming cycle will run concurrently; deadlines for their nomination and voting activities are synchronized but will still use separate ballots. Please note, if only one candidate is nominated as PTL for a project team during the PTL nomination period, that candidate will win by acclaim, and there will be no poll. There will only be a poll if there is more than one candidate stepping forward for a project team's PTL position. There will be further announcements posted to the mailing list as action is required from the electorate or candidates. This email is for information purposes only. If you have any questions which you feel affect others please reply to this email thread. If you have any questions that you which to discuss in private please email any of the election officials[1] so that we may address your concerns. Thank you! -Kendall Nelson (diablo_rojo) & the Election Officials [1] https://governance.openstack.org/election/#election-officials -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 19 01:26:01 2020 From: satish.txt at gmail.com (Satish Patel) Date: Wed, 18 Mar 2020 21:26:01 -0400 Subject: upgrade qemu 4.2 on CentOS 7.x for openstack Message-ID: Folk, I am running openstack stein on CentOS 7.5 so default version of qemu-kvm is 2.12.x. as we know current qemu version is 4.2. Question: If i compile/install latest version of qemu 4.2 using source tarball on compute node in that case do i need to compile/install kvm or Libvirt? ~S From melwittt at gmail.com Thu Mar 19 05:50:18 2020 From: melwittt at gmail.com (melanie witt) Date: Wed, 18 Mar 2020 22:50:18 -0700 Subject: [nova][gate] status of some gate bugs Message-ID: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> Hey all, We've been having a tough time lately in the gate hitting various bugs while our patches go through CI. I just wanted to mention a few of them that I've seen often in my gerrit notifications and give a brief status on fix efforts. * http://status.openstack.org/elastic-recheck/#1813789 This one is where the nova-live-migration job fails a server evacuate test with: "Timeout waiting for [('network-vif-plugged', 'e3d3db3f-bce4-4889-b161-4b73648f79be')] for instance with vm_state error and task_state rebuild_spawning.: eventlet.timeout.Timeout: 300 seconds" in the screen-n-cpu.txt log. lyarwood has a WIP patch here: https://review.opendev.org/713674 and sean-k-mooney has a WIP patch here: https://review.opendev.org/713342 * https://launchpad.net/bugs/1867380 This one is where the nova-live-migration or nova-grenade-multinode job fail due to n-cpu restarting slowly after being reconfigured for ceph. The server will fail to build and it's because the test begins before nova-compute has fully come up and we see this error: "Instance spawn was interrupted before instance_claim, setting instance to ERROR state {{(pid=3783) _error_out_instances_whose_build_was_interrupted" in the screen-n-cpu.txt log. lyarwood has a patch approved here that we've been rechecking the heck out of that has yet to merge: https://review.opendev.org/713035 * https://launchpad.net/bugs/1844568 This one is where a job fails with: "Body: b'{"conflictingRequest": {"code": 409, "message": "Multiple possible networks found, use a Network ID to be more specific."}}'" gmann has a patch proposed to fix some of these here: https://review.opendev.org/711049 There might be more test classes that need create_default_network = True. * http://status.openstack.org/elastic-recheck/#1844929 This one is where a job fails and the following error is seen one of the logs, usually screen-n-sch.txt: "Timed out waiting for response from cell 8acfb79b-2e40-4e1c-bc3d-d404dac6db90". The TL;DR on this one is there's no immediate clue why it's happening. This bug used to hit more occasionally on "slow" nodes like nodes from the OVH or INAP providers (and OVH restricts disk iops [1]). Now, it seems like it's hitting much more often (still mostly on OVH nodes). I've been looking at it for about a week now and I've been using a DNM patch to add debug logging, look at dstat --disk-wait output, try mysqld my.cnf settings, etc: https://review.opendev.org/701478 So far, what I find is that when we get into the fail state, we get no rows back from the database server when we query for nova 'services' and 'compute_nodes' records, and we fail with the "Timed out waiting for response" error. Haven't figured out why yet, so far. The disk wait doesn't look high when this happens (or at any time during a run) so it's not seeming like it's related to disk IO. I'm continuing to look into it. Cheers, -melanie [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010505.html From arnaud.morin at gmail.com Thu Mar 19 08:25:48 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Thu, 19 Mar 2020 08:25:48 +0000 Subject: [nova][gate] status of some gate bugs In-Reply-To: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> References: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> Message-ID: <20200319082548.GL29109@sync> Hey Melanie, all, About OVH case (company I work for). We are digging into the issue. First thing, we do not limit anymore the IOPS. I dont remember when we removed this limit, but this is not new. However, the hypervisor are quite old now, and our policy on this old servers was to use some swap. And we think that the host may slow down when overcommitting on RAM (swapping on disk). Anyway, we also know that we can have better latency when upgrading QEMU. We are currently in the middle of testing a new QEMU version, I will push to upgrade your hypervisors first, so we will see if the latency on QEMU side can help the gate. Finally, we plan to change the hardware and stop doing overcommit on RAM (and swapping on disk). However, I have no ETA about that, but for sure, this will improve the IOPS. I'll keep you in touch. Cheers, -- Arnaud Morin On 18.03.20 - 22:50, melanie witt wrote: > Hey all, > > We've been having a tough time lately in the gate hitting various bugs while > our patches go through CI. I just wanted to mention a few of them that I've > seen often in my gerrit notifications and give a brief status on fix > efforts. > > * http://status.openstack.org/elastic-recheck/#1813789 > > This one is where the nova-live-migration job fails a server evacuate test > with: "Timeout waiting for [('network-vif-plugged', > 'e3d3db3f-bce4-4889-b161-4b73648f79be')] for instance with vm_state error > and task_state rebuild_spawning.: eventlet.timeout.Timeout: 300 seconds" in > the screen-n-cpu.txt log. > > lyarwood has a WIP patch here: > > https://review.opendev.org/713674 > > and sean-k-mooney has a WIP patch here: > > https://review.opendev.org/713342 > > * https://launchpad.net/bugs/1867380 > > This one is where the nova-live-migration or nova-grenade-multinode job fail > due to n-cpu restarting slowly after being reconfigured for ceph. The server > will fail to build and it's because the test begins before nova-compute has > fully come up and we see this error: "Instance spawn was interrupted before > instance_claim, setting instance to ERROR state {{(pid=3783) > _error_out_instances_whose_build_was_interrupted" in the screen-n-cpu.txt > log. > > lyarwood has a patch approved here that we've been rechecking the heck out > of that has yet to merge: > > https://review.opendev.org/713035 > > * https://launchpad.net/bugs/1844568 > > This one is where a job fails with: "Body: b'{"conflictingRequest": {"code": > 409, "message": "Multiple possible networks found, use a Network ID to be > more specific."}}'" > > gmann has a patch proposed to fix some of these here: > > https://review.opendev.org/711049 > > There might be more test classes that need create_default_network = True. > > * http://status.openstack.org/elastic-recheck/#1844929 > > This one is where a job fails and the following error is seen one of the > logs, usually screen-n-sch.txt: "Timed out waiting for response from cell > 8acfb79b-2e40-4e1c-bc3d-d404dac6db90". > > The TL;DR on this one is there's no immediate clue why it's happening. This > bug used to hit more occasionally on "slow" nodes like nodes from the OVH or > INAP providers (and OVH restricts disk iops [1]). Now, it seems like it's > hitting much more often (still mostly on OVH nodes). > > I've been looking at it for about a week now and I've been using a DNM patch > to add debug logging, look at dstat --disk-wait output, try mysqld my.cnf > settings, etc: > > https://review.opendev.org/701478 > > So far, what I find is that when we get into the fail state, we get no rows > back from the database server when we query for nova 'services' and > 'compute_nodes' records, and we fail with the "Timed out waiting for > response" error. > > Haven't figured out why yet, so far. The disk wait doesn't look high when > this happens (or at any time during a run) so it's not seeming like it's > related to disk IO. I'm continuing to look into it. > > Cheers, > -melanie > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010505.html > From thierry at openstack.org Thu Mar 19 09:06:46 2020 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 19 Mar 2020 10:06:46 +0100 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> <29a571a5-6081-448e-ba09-740e83200341@Spark> Message-ID: <16e21ebe-74fc-56a2-1156-dc8406b5e2a0@openstack.org> Mohammed Naser wrote: > I see that the discussion has stalled out, I'd like us to seriously > explore this because I think come the PTL time, we might run into a > lot of projects who don't have folks stepping in. The timing for this discussion was far from ideal, so I respect that people ask for longer discussion without the threat of a deadline. Rather than a big bang change at election time, one option would be for the TC to switch leaderless teams after the election, then have opt-in during the Victoria cycle (elected PTLs choosing to move to new system), then discuss how to treat the remaining ones (after all, the proposed change does not eliminate the PTL, it just stops to define it as a mandatory element of our governance). That phased approach is just a lot more difficult to codify in terms of governance changes :) -- Thierry Carrez (ttx) From mark at stackhpc.com Thu Mar 19 09:19:16 2020 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 19 Mar 2020 09:19:16 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> <29a571a5-6081-448e-ba09-740e83200341@Spark> Message-ID: On Wed, 18 Mar 2020 at 13:05, Mohammed Naser wrote: > > Hi everyone, > > I see that the discussion has stalled out, I'd like us to seriously > explore this because I think come the PTL time, we might run into a > lot of projects who don't have folks stepping in. We could let the elections take place and see how many teams this will apply to. Those teams could then decide how to handle the situation, and report results during the cycle. Based on their experience we could change the model for V. It seems to me that a relaxing of some conditions and expectations on project leadership might be more appropriate than a blanket change. > > Thanks, > Mohammed > > On Fri, Mar 13, 2020 at 1:37 AM Renat Akhmerov wrote: > > > > On 5 Mar 2020, 02:57 +0700, Jay Bryant , wrote: > > > > Zane's input sums up how I feel as well. I think that having consistent > > leadership structure across projects is important and helps keep us > > aware of the health of projects. > > > > Perhaps we can help return interest in the PTL role by providing > > examples of teams that share the work and have the PTL to help make > > final decisions. I know that the Cinder team has been doing this for > > quite some time successfully. > > > > > > I really fail to understand the issue of an overwhelming PTLship. From my > > personal 5+ experience of being a PTL, I can say I’ve never tried to do all > > stuff like releases, fixing CI, back porting etc. etc. myself. Especially given > > that I really like solving technical problems, I always try to offload some of > > these duties to others and make sure I have time for development. And > > IMO it’s totally fine. As a PTL I try to make sure that everyone in the team > > (at least most active members) has this balance between problem solving > > and necessary procedures. I absolutely agree though that as the PTL I have > > to know what’s going on with the project and after all TC or someone else > > can ask me about its current state and progress. > > Of course, I realise that my project is somewhat not really typical for > > OpenStack: it is relatively small and has not so many connections with other > > projects. But I believe this principle is universal enough. > > As far as the PTL role, I think for PTLs it’s important to focus on the big > > picture, ideas and directions and keep reminding everyone about that. > > All team members, even active ones, often can’t afford thinking about this > > too much. This contradicts with lots of what I heard before from my former > > managers and colleagues, and also some PTLs I know. They claimed: > > “PTLs just need to maintain Launchpad (or Storyboard), keep an eye > > on the release process and that’s basically it. Plus reviewing a little bit.” > > I’ve always shrugged when hearing this.. If so, let’s remove “L” from “PTL” > > and replace it with “A”, so that it’s “PTA” - Project Technical Administrator. > > Something that can legally exist, no issue. And it’s more honest. > > What I’m going to is, from my perspective, it probably doesn’t make any > > difference if a project leader is an official role or not. I guess there will > > always be someone who naturally gains trust of others and influences > > the direction of a project. As far as “having a final word on a deadlocked > > issue” I thought this is something really important but, in fact, it’s a > > very rare thing and may not be needed at all. Usually, we have to make > > a decision anyway, since “not making a decision is more expensive than > > making even bad decision”. > > > > So I believe some leadership is always needed. The most high quality > > techs I’ve ever seen have been all made with a very strong leadership, > > I don’t believe it works the other way. Whether it’s official or not, I think > > is not important at all. But “administrative duties” that often assign to > > PTLs can be easily split between people. > > > > > > Thanks > > > > Renat Akhmerov > > @Nokia > > > > -- > Mohammed Naser — vexxhost > ----------------------------------------------------- > D. 514-316-8872 > D. 800-910-1726 ext. 200 > E. mnaser at vexxhost.com > W. https://vexxhost.com > From skramaja at redhat.com Thu Mar 19 09:36:18 2020 From: skramaja at redhat.com (Saravanan KR) Date: Thu, 19 Mar 2020 15:06:18 +0530 Subject: [tripleo][operators] Removal of mistral from the TripleO Undercloud In-Reply-To: References: Message-ID: On Thu, Mar 19, 2020 at 1:02 AM John Fulton wrote: > > On Sat, Mar 14, 2020 at 8:06 AM Rabi Mishra wrote: > > > > On Sat, Mar 14, 2020 at 2:10 AM John Fulton wrote: > >> > >> On Fri, Mar 13, 2020 at 3:27 PM Kevin Carter wrote: > >> > > >> > Hello stackers, > >> > > >> > In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4]. > >> > >> As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud. > >> > >> > > >> > So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases? > > > > > > Yeah, as John mentioned, tripleo.derive_params.v1.derive_parameters workflow is surely being used for NFV (DPDK/SR-IOV) and HCI use cases and can't be deprecated or dropped. Though we've a generic interface in tripleoclient to run any workflow in plan-environment, I have not seen it being used for anything other than the mentioned workflow. > > > > In the scope of 'mistral-ansible' work, we seem to have two options. > > > > 1. Convert the workflow to ansible playbook 'as-is' i.e calculate and merge the derived parameters in plan-environment and as you've mentioned, change tripleoclient code to call any playbook in plan-environment.yaml and the parameters/vars. > > Nice idea. I hadn't thought of that. > > If there's a "hello world" example of this (which results in a THT > param in the deployment plan being set to "hello world"), then I could > try writing an ansible module to derive the HCI parameters and set > them in place of the "hello world". > I am fine with the approach, but the only concern is, we have plans to remove Heat in the coming cycles. One of inputs for the Mistral derive parameters is fetched from the heat stack. If we are going to retain it, then it has to be re-worked during the Heat removal. Mistral to ansible could be the first step towards it. Regards, Saravanan KR > John > > > 2. Move the functionality further down the component chain in TripleO to have the required ansible host/group_vars set for them to be used by config-download playbooks/ansible/puppet. > > > > I guess option 1 would be easier within the timelines. I've done some preliminary work to move some of the functionality in relevant mistral actions to utils modules[1], so that they can be called from ansible modules without depending on mistral/mistral-lib and use those in a playbook that kinda replicate the tasks in the mistral workflow. > > > > Having said that, it would be good to know what the DFG:NFV folks think, as they are the original authors/maintainers of that workflow. > > > > > > > >> The Mistral based workflow took advantage of the deployment plan which > >> was stored in Swift on the undercloud. My understanding is that too is > >> going away. > > > > > > I'm not sure that would be in the scope of 'mstral-to-ansible' work. Dropping swift would probably be a bit more complex, as we use it to store templates, plan-environment, plan backups (possibly missing few more) etc and would require significant design rework (may be possible when we get rid of heat in undercloud). In spite of heat using the templates from swift and merging environments on the client side, we've had already bumped heat's REST API json body size limit (max_json_body_size) on the undercloud to 4MB[2] from the default 1MB and sending all required templates as part of API request would not be a good idea from undercloud scalability pov. > > > > [1] https://review.opendev.org/#/c/709546/ > > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml#L109 > > > > -- > > Regards, > > Rabi Mishra > > > From jayadityagupta11 at gmail.com Thu Mar 19 10:49:18 2020 From: jayadityagupta11 at gmail.com (jayaditya gupta) Date: Thu, 19 Mar 2020 16:19:18 +0530 Subject: [python-openstackclient] --force bug Message-ID: Hello all, new contributor/developer here. I have just submitted a bug for python-openstackclient project : https://storyboard.openstack.org/#!/story/2007440 I am not sure how to provide --force option to openstackclient .. I am comparing quota.py files in nova-client and in openstackclient.but can't seem to understand how to provide --force option. Any suggestions? Best Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Thu Mar 19 13:15:15 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Thu, 19 Mar 2020 09:15:15 -0400 Subject: [ops][cinder] deprecation notice: os-reset_status notifications Message-ID: tl;dr: the notifications themselves aren't being deprecated, there's a change in where they're being sent While working on https://launchpad.net/bugs/1863806, Walt discovered that notifications associated with the Block Storage API os-reset_status calls were being sent to nonstandard publisher_ids, namely: * 'volumeStatusUpdate' for volume status resets * 'volumeStatusUpdate' for snapshot status resets * 'backupStatusUpdate' for backup status resets Walt has a patch up to direct these notifications to the same publisher_id as other Cinder resource actions: * 'volume' for volume status resets * 'snapshot' for snapshot status resets * 'backup' for backup status resets Walt's patch continues to send the notifications to the nonstandard publisher_ids, but this behavior is DEPRECATED. In Ussuri, notifications will be sent to *both* publisher_ids. We propose to remove sending notifications to the nonstandard publisher_ids during the Victoria development cycle. This deprecation will be announced in the cinder release notes, but we wanted to give operators advance notice. No one currently working on cinder knows why these notifications were being sent to nonstandard publisher_ids, which makes it difficult to assess what, if any, impact this change will have on operators. If you have concerns about either the nonstandard publisher_ids being deprecated and removed, or about these notifications being sent to the standard publisher_ids, please express them on the following review before Tuesday 24 March: https://review.opendev.org/#/c/708563/ Thank you. From smooney at redhat.com Thu Mar 19 14:04:21 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 14:04:21 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> <29a571a5-6081-448e-ba09-740e83200341@Spark> Message-ID: <6ab83c91a639653d653bad0dbbcde9b48d4a3c2f.camel@redhat.com> On Thu, 2020-03-19 at 09:19 +0000, Mark Goddard wrote: > On Wed, 18 Mar 2020 at 13:05, Mohammed Naser wrote: > > > > Hi everyone, > > > > I see that the discussion has stalled out, I'd like us to seriously > > explore this because I think come the PTL time, we might run into a > > lot of projects who don't have folks stepping in. > > We could let the elections take place and see how many teams this will > apply to. Those teams could then decide how to handle the situation, > and report results during the cycle. Based on their experience we > could change the model for V. It seems to me that a relaxing of some > conditions and expectations on project leadership might be more > appropriate than a blanket change. the issue is that the bylaws do not currently allow that. so what was currently being proposed would allow every project to continue as before electing ptls but it would no longer make it mandatory for the tc to appoint a ptl if none was elected. currently all projects in the govance repo must have a ptl defiend. so if we do nothing the tc will have to intervene for any project that does not elect a ptl. > > > > > Thanks, > > Mohammed > > > > On Fri, Mar 13, 2020 at 1:37 AM Renat Akhmerov wrote: > > > > > > On 5 Mar 2020, 02:57 +0700, Jay Bryant , wrote: > > > > > > Zane's input sums up how I feel as well. I think that having consistent > > > leadership structure across projects is important and helps keep us > > > aware of the health of projects. > > > > > > Perhaps we can help return interest in the PTL role by providing > > > examples of teams that share the work and have the PTL to help make > > > final decisions. I know that the Cinder team has been doing this for > > > quite some time successfully. > > > > > > > > > I really fail to understand the issue of an overwhelming PTLship. From my > > > personal 5+ experience of being a PTL, I can say I’ve never tried to do all > > > stuff like releases, fixing CI, back porting etc. etc. myself. Especially given > > > that I really like solving technical problems, I always try to offload some of > > > these duties to others and make sure I have time for development. And > > > IMO it’s totally fine. As a PTL I try to make sure that everyone in the team > > > (at least most active members) has this balance between problem solving > > > and necessary procedures. I absolutely agree though that as the PTL I have > > > to know what’s going on with the project and after all TC or someone else > > > can ask me about its current state and progress. > > > Of course, I realise that my project is somewhat not really typical for > > > OpenStack: it is relatively small and has not so many connections with other > > > projects. But I believe this principle is universal enough. > > > As far as the PTL role, I think for PTLs it’s important to focus on the big > > > picture, ideas and directions and keep reminding everyone about that. > > > All team members, even active ones, often can’t afford thinking about this > > > too much. This contradicts with lots of what I heard before from my former > > > managers and colleagues, and also some PTLs I know. They claimed: > > > “PTLs just need to maintain Launchpad (or Storyboard), keep an eye > > > on the release process and that’s basically it. Plus reviewing a little bit.” > > > I’ve always shrugged when hearing this.. If so, let’s remove “L” from “PTL” > > > and replace it with “A”, so that it’s “PTA” - Project Technical Administrator. > > > Something that can legally exist, no issue. And it’s more honest. > > > What I’m going to is, from my perspective, it probably doesn’t make any > > > difference if a project leader is an official role or not. I guess there will > > > always be someone who naturally gains trust of others and influences > > > the direction of a project. As far as “having a final word on a deadlocked > > > issue” I thought this is something really important but, in fact, it’s a > > > very rare thing and may not be needed at all. Usually, we have to make > > > a decision anyway, since “not making a decision is more expensive than > > > making even bad decision”. > > > > > > So I believe some leadership is always needed. The most high quality > > > techs I’ve ever seen have been all made with a very strong leadership, > > > I don’t believe it works the other way. Whether it’s official or not, I think > > > is not important at all. But “administrative duties” that often assign to > > > PTLs can be easily split between people. > > > > > > > > > Thanks > > > > > > Renat Akhmerov > > > @Nokia > > > > > > > > -- > > Mohammed Naser — vexxhost > > ----------------------------------------------------- > > D. 514-316-8872 > > D. 800-910-1726 ext. 200 > > E. mnaser at vexxhost.com > > W. https://vexxhost.com > > > > From smooney at redhat.com Thu Mar 19 14:13:19 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 14:13:19 +0000 Subject: [nova][gate] status of some gate bugs In-Reply-To: <20200319082548.GL29109@sync> References: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> <20200319082548.GL29109@sync> Message-ID: <3f3a426c91bec9592d25ebcc819c6e0cd93cae6c.camel@redhat.com> On Thu, 2020-03-19 at 08:25 +0000, Arnaud Morin wrote: > Hey Melanie, all, > > About OVH case (company I work for). > We are digging into the issue. > > First thing, we do not limit anymore the IOPS. I dont remember when we > removed this limit, but this is not new. > > However, the hypervisor are quite old now, and our policy on this old > servers was to use some swap. > And we think that the host may slow down when overcommitting on RAM > (swapping on disk). > > Anyway, we also know that we can have better latency when upgrading > QEMU. We are currently in the middle of testing a new QEMU version, I > will push to upgrade your hypervisors first, so we will see if the > latency on QEMU side can help the gate. > > Finally, we plan to change the hardware and stop doing overcommit on RAM > (and swapping on disk). However, I have no ETA about that, but for sure, > this will improve the IOPS. if you stop doing over commit on ram can i also suggest that you enable hugepages hosting vms. if this is dedicated hardware for the ci then you should not need to do live migration on these vms since they are short lived (max 3-4 hours) so you can just disable the host leave it drain, do your maintance and enable it again. when there is no over subscption anyway enableing hugepags will not affect capasity but it will also give a 30-40% performacne boost. anyway its just something to consider, it wont allow or prevent any testing we do today but in addtion to the new qemu version it should imporve the perfomance of the ovh nodes, not that they are generally slow ovh used to perform quite well even if it is older hardware but hugepages will imporave all guest memory access times, when compinded with not swaping to disk that should result in a marked imporvement in overall perforamce can ci job time. if ye dont want to enable hugepages thats fine too but since ye are considering making changes to the host anyway i tought i would ask. > > I'll keep you in touch. > > Cheers, > From mnaser at vexxhost.com Thu Mar 19 14:25:27 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Thu, 19 Mar 2020 10:25:27 -0400 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <6ab83c91a639653d653bad0dbbcde9b48d4a3c2f.camel@redhat.com> References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> <29a571a5-6081-448e-ba09-740e83200341@Spark> <6ab83c91a639653d653bad0dbbcde9b48d4a3c2f.camel@redhat.com> Message-ID: On Thu, Mar 19, 2020 at 10:04 AM Sean Mooney wrote: > > On Thu, 2020-03-19 at 09:19 +0000, Mark Goddard wrote: > > On Wed, 18 Mar 2020 at 13:05, Mohammed Naser wrote: > > > > > > Hi everyone, > > > > > > I see that the discussion has stalled out, I'd like us to seriously > > > explore this because I think come the PTL time, we might run into a > > > lot of projects who don't have folks stepping in. > > > > We could let the elections take place and see how many teams this will > > apply to. Those teams could then decide how to handle the situation, > > and report results during the cycle. Based on their experience we > > could change the model for V. It seems to me that a relaxing of some > > conditions and expectations on project leadership might be more > > appropriate than a blanket change. > the issue is that the bylaws do not currently allow that. > so what was currently being proposed would allow every project to continue as before > electing ptls but it would no longer make it mandatory for the tc to appoint > a ptl if none was elected. or we can just appoint TC members to fill that spot > currently all projects in the govance repo must have a ptl defiend. > so if we do nothing the tc will have to intervene for any project that does not elect > a ptl. > > > > > > > > Thanks, > > > Mohammed > > > > > > On Fri, Mar 13, 2020 at 1:37 AM Renat Akhmerov wrote: > > > > > > > > On 5 Mar 2020, 02:57 +0700, Jay Bryant , wrote: > > > > > > > > Zane's input sums up how I feel as well. I think that having consistent > > > > leadership structure across projects is important and helps keep us > > > > aware of the health of projects. > > > > > > > > Perhaps we can help return interest in the PTL role by providing > > > > examples of teams that share the work and have the PTL to help make > > > > final decisions. I know that the Cinder team has been doing this for > > > > quite some time successfully. > > > > > > > > > > > > I really fail to understand the issue of an overwhelming PTLship. From my > > > > personal 5+ experience of being a PTL, I can say I’ve never tried to do all > > > > stuff like releases, fixing CI, back porting etc. etc. myself. Especially given > > > > that I really like solving technical problems, I always try to offload some of > > > > these duties to others and make sure I have time for development. And > > > > IMO it’s totally fine. As a PTL I try to make sure that everyone in the team > > > > (at least most active members) has this balance between problem solving > > > > and necessary procedures. I absolutely agree though that as the PTL I have > > > > to know what’s going on with the project and after all TC or someone else > > > > can ask me about its current state and progress. > > > > Of course, I realise that my project is somewhat not really typical for > > > > OpenStack: it is relatively small and has not so many connections with other > > > > projects. But I believe this principle is universal enough. > > > > As far as the PTL role, I think for PTLs it’s important to focus on the big > > > > picture, ideas and directions and keep reminding everyone about that. > > > > All team members, even active ones, often can’t afford thinking about this > > > > too much. This contradicts with lots of what I heard before from my former > > > > managers and colleagues, and also some PTLs I know. They claimed: > > > > “PTLs just need to maintain Launchpad (or Storyboard), keep an eye > > > > on the release process and that’s basically it. Plus reviewing a little bit.” > > > > I’ve always shrugged when hearing this.. If so, let’s remove “L” from “PTL” > > > > and replace it with “A”, so that it’s “PTA” - Project Technical Administrator. > > > > Something that can legally exist, no issue. And it’s more honest. > > > > What I’m going to is, from my perspective, it probably doesn’t make any > > > > difference if a project leader is an official role or not. I guess there will > > > > always be someone who naturally gains trust of others and influences > > > > the direction of a project. As far as “having a final word on a deadlocked > > > > issue” I thought this is something really important but, in fact, it’s a > > > > very rare thing and may not be needed at all. Usually, we have to make > > > > a decision anyway, since “not making a decision is more expensive than > > > > making even bad decision”. > > > > > > > > So I believe some leadership is always needed. The most high quality > > > > techs I’ve ever seen have been all made with a very strong leadership, > > > > I don’t believe it works the other way. Whether it’s official or not, I think > > > > is not important at all. But “administrative duties” that often assign to > > > > PTLs can be easily split between people. > > > > > > > > > > > > Thanks > > > > > > > > Renat Akhmerov > > > > @Nokia > > > > > > > > > > > > -- > > > Mohammed Naser — vexxhost > > > ----------------------------------------------------- > > > D. 514-316-8872 > > > D. 800-910-1726 ext. 200 > > > E. mnaser at vexxhost.com > > > W. https://vexxhost.com > > > > > > > > From lyarwood at redhat.com Thu Mar 19 14:27:31 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Thu, 19 Mar 2020 14:27:31 +0000 Subject: [nova][gate] status of some gate bugs In-Reply-To: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> References: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> Message-ID: <20200319142731.n7e4i3qucfahkusj@lyarwood.usersys.redhat.com> On 18-03-20 22:50:18, melanie witt wrote: > Hey all, > > We've been having a tough time lately in the gate hitting various bugs while > our patches go through CI. I just wanted to mention a few of them that I've > seen often in my gerrit notifications and give a brief status on fix > efforts. Many thanks for writing this up Mel! Comments below on issues I've been working on. > * http://status.openstack.org/elastic-recheck/#1813789 > > This one is where the nova-live-migration job fails a server evacuate test > with: "Timeout waiting for [('network-vif-plugged', > 'e3d3db3f-bce4-4889-b161-4b73648f79be')] for instance with vm_state error > and task_state rebuild_spawning.: eventlet.timeout.Timeout: 300 seconds" in > the screen-n-cpu.txt log. > > lyarwood has a WIP patch here: > > https://review.opendev.org/713674 This looks like it actually fixes it by fencing the destack@* services and running guest domains on the subnode. In the gate now, thanks gibi, sean-k-mooney and stephenfin! > and sean-k-mooney has a WIP patch here: > > https://review.opendev.org/713342 Might be something we still want to look into under another bug? > * https://launchpad.net/bugs/1867380 > > This one is where the nova-live-migration or nova-grenade-multinode job fail > due to n-cpu restarting slowly after being reconfigured for ceph. The server > will fail to build and it's because the test begins before nova-compute has > fully come up and we see this error: "Instance spawn was interrupted before > instance_claim, setting instance to ERROR state {{(pid=3783) > _error_out_instances_whose_build_was_interrupted" in the screen-n-cpu.txt > log. > > lyarwood has a patch approved here that we've been rechecking the heck out > of that has yet to merge: > > https://review.opendev.org/713035 Merged on master and backported all the way back to stable/pike on the following topic: https://review.opendev.org/#/q/topic:bug/1867380+status:open > * https://launchpad.net/bugs/1844568 > > This one is where a job fails with: "Body: b'{"conflictingRequest": {"code": > 409, "message": "Multiple possible networks found, use a Network ID to be > more specific."}}'" > > gmann has a patch proposed to fix some of these here: > > https://review.opendev.org/711049 > > There might be more test classes that need create_default_network = True. > > * http://status.openstack.org/elastic-recheck/#1844929 > > This one is where a job fails and the following error is seen one of the > logs, usually screen-n-sch.txt: "Timed out waiting for response from cell > 8acfb79b-2e40-4e1c-bc3d-d404dac6db90". > > The TL;DR on this one is there's no immediate clue why it's happening. This > bug used to hit more occasionally on "slow" nodes like nodes from the OVH or > INAP providers (and OVH restricts disk iops [1]). Now, it seems like it's > hitting much more often (still mostly on OVH nodes). > > I've been looking at it for about a week now and I've been using a DNM patch > to add debug logging, look at dstat --disk-wait output, try mysqld my.cnf > settings, etc: > > https://review.opendev.org/701478 > > So far, what I find is that when we get into the fail state, we get no rows > back from the database server when we query for nova 'services' and > 'compute_nodes' records, and we fail with the "Timed out waiting for > response" error. > > Haven't figured out why yet, so far. The disk wait doesn't look high when > this happens (or at any time during a run) so it's not seeming like it's > related to disk IO. I'm continuing to look into it. > > Cheers, > -melanie > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010505.html I've also hit the following a few times on master: test_update_delete_extra_route failing due to timeout when creating subnets https://bugs.launchpad.net/neutron/+bug/1867936 I'll try to write up a logstash query for this now and post a review for recheck. Thanks again, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From fungi at yuggoth.org Thu Mar 19 14:46:56 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 19 Mar 2020 14:46:56 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: <6ab83c91a639653d653bad0dbbcde9b48d4a3c2f.camel@redhat.com> References: <5c461b2e-a730-1fd9-aee4-ae0ea0e2eff9@gmail.com> <29a571a5-6081-448e-ba09-740e83200341@Spark> <6ab83c91a639653d653bad0dbbcde9b48d4a3c2f.camel@redhat.com> Message-ID: <20200319144656.g2qmznald4ddocjl@yuggoth.org> On 2020-03-19 14:04:21 +0000 (+0000), Sean Mooney wrote: > On Thu, 2020-03-19 at 09:19 +0000, Mark Goddard wrote: [...] > > We could let the elections take place and see how many teams > > this will apply to. Those teams could then decide how to handle > > the situation, and report results during the cycle. Based on > > their experience we could change the model for V. It seems to me > > that a relaxing of some conditions and expectations on project > > leadership might be more appropriate than a blanket change. > > the issue is that the bylaws do not currently allow that. [...] https://www.openstack.org/legal/bylaws-of-the-openstack-foundation/ https://www.openstack.org/legal/technical-committee-member-policy/ I can't find where they disallow or, in fact, say anything at all about PTLs. Are you perhaps thinking of the OpenStack Technical Committee Charter? https://governance.openstack.org/tc/reference/charter.html -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From smooney at redhat.com Thu Mar 19 14:59:11 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 14:59:11 +0000 Subject: [python-openstackclient] --force bug In-Reply-To: References: Message-ID: <5884eb023abd91f422e46204eacee408460e4193.camel@redhat.com> On Thu, 2020-03-19 at 16:19 +0530, jayaditya gupta wrote: > Hello all, new contributor/developer here. I have just submitted a bug for > python-openstackclient project : > https://storyboard.openstack.org/#!/story/2007440 > > I am not sure how to provide --force option to openstackclient .. > > I am comparing quota.py files in nova-client and in openstackclient.but > can't seem to understand how to provide --force option. Any suggestions? at leas when it comes to some nova action we intentionally do not want to support --force for thing like migration specifcally for live migration we removed --force in microversion 2.68 https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id62 so since osc never supported --force we dont want it to be enabled. for "openstack quota set" i dont think we have specifically disucced if we want to support --force before. we deprecated the proxing of quotas via nova in micro-version 2.36 https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#microversion 2.39 https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id36 2.50 https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id46 2.57 https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id53 also modifed quotas in different ways. non of these microversion remove the force parameter like we did with live migration so it hink it would be valid to add support for --force. however nova and keystone are currently working on removing support for nova based quotas using the unified limits proposal https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/unified-limits-nova.html https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html so i dont know if we have disucsed how that will impact the openstack clint and if we will deprecate "openstack quota set" and intoduce a "openstack limit set" to interact with the new api or if we will use a microverion but maintain the same cli. i dont know if the unified limits api will support a force paramater. i hope its not nessisary to have a force paramter with the new design so while there is no conflict i am awere of wehre exposing --force might encurage usage that the nova team does not want opertors to do, im not sure it would be consitent with the future direction. so tentative +0.5 from me in terms of closing the gap for older openstack releases but i think it would be good to have the unified limit folks also way in on if it should be added and how that will interact with the current planned changes. > > Best Regards From smooney at redhat.com Thu Mar 19 15:08:09 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 15:08:09 +0000 Subject: [nova][gate] status of some gate bugs In-Reply-To: <20200319142731.n7e4i3qucfahkusj@lyarwood.usersys.redhat.com> References: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> <20200319142731.n7e4i3qucfahkusj@lyarwood.usersys.redhat.com> Message-ID: On Thu, 2020-03-19 at 14:27 +0000, Lee Yarwood wrote: > On 18-03-20 22:50:18, melanie witt wrote: > > Hey all, > > > > We've been having a tough time lately in the gate hitting various bugs while > > our patches go through CI. I just wanted to mention a few of them that I've > > seen often in my gerrit notifications and give a brief status on fix > > efforts. > > Many thanks for writing this up Mel! > > Comments below on issues I've been working on. > > > * http://status.openstack.org/elastic-recheck/#1813789 > > > > This one is where the nova-live-migration job fails a server evacuate test > > with: "Timeout waiting for [('network-vif-plugged', > > 'e3d3db3f-bce4-4889-b161-4b73648f79be')] for instance with vm_state error > > and task_state rebuild_spawning.: eventlet.timeout.Timeout: 300 seconds" in > > the screen-n-cpu.txt log. > > > > lyarwood has a WIP patch here: > > > > https://review.opendev.org/713674 > > This looks like it actually fixes it by fencing the destack@* services > and running guest domains on the subnode. In the gate now, thanks gibi, > sean-k-mooney and stephenfin! > > > and sean-k-mooney has a WIP patch here: > > > > https://review.opendev.org/713342 > > Might be something we still want to look into under another bug? ya so i think we can file a seperate bug for the "intermittently fails with network-vif-plugged timeout exception for shelve" we have already fixed it for resize revert but as part of that mriedemann noted that it was broken for shelve https://bugs.launchpad.net/nova/+bug/1813789/comments/3 and its theoretically broken for evacuate. so what i can do is file the new bug and then track https://bugs.launchpad.net/nova/+bug/1813789 as a related bug but rather then the fix above specificaly closing that since i think your patch above fixes it suffienctly. my [WIP] patch seams to have some other side effect that i was not expecting so i need to look into that more closely. > > > * https://launchpad.net/bugs/1867380 > > > > This one is where the nova-live-migration or nova-grenade-multinode job fail > > due to n-cpu restarting slowly after being reconfigured for ceph. The server > > will fail to build and it's because the test begins before nova-compute has > > fully come up and we see this error: "Instance spawn was interrupted before > > instance_claim, setting instance to ERROR state {{(pid=3783) > > _error_out_instances_whose_build_was_interrupted" in the screen-n-cpu.txt > > log. > > > > lyarwood has a patch approved here that we've been rechecking the heck out > > of that has yet to merge: > > > > https://review.opendev.org/713035 > > Merged on master and backported all the way back to stable/pike on the > following topic: > > https://review.opendev.org/#/q/topic:bug/1867380+status:open > > > * https://launchpad.net/bugs/1844568 > > > > This one is where a job fails with: "Body: b'{"conflictingRequest": {"code": > > 409, "message": "Multiple possible networks found, use a Network ID to be > > more specific."}}'" > > > > gmann has a patch proposed to fix some of these here: > > > > https://review.opendev.org/711049 > > > > There might be more test classes that need create_default_network = True. > > > > * http://status.openstack.org/elastic-recheck/#1844929 > > > > This one is where a job fails and the following error is seen one of the > > logs, usually screen-n-sch.txt: "Timed out waiting for response from cell > > 8acfb79b-2e40-4e1c-bc3d-d404dac6db90". > > > > The TL;DR on this one is there's no immediate clue why it's happening. This > > bug used to hit more occasionally on "slow" nodes like nodes from the OVH or > > INAP providers (and OVH restricts disk iops [1]). Now, it seems like it's > > hitting much more often (still mostly on OVH nodes). > > > > I've been looking at it for about a week now and I've been using a DNM patch > > to add debug logging, look at dstat --disk-wait output, try mysqld my.cnf > > settings, etc: > > > > https://review.opendev.org/701478 > > > > So far, what I find is that when we get into the fail state, we get no rows > > back from the database server when we query for nova 'services' and > > 'compute_nodes' records, and we fail with the "Timed out waiting for > > response" error. > > > > Haven't figured out why yet, so far. The disk wait doesn't look high when > > this happens (or at any time during a run) so it's not seeming like it's > > related to disk IO. I'm continuing to look into it. > > > > Cheers, > > -melanie > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010505.html > > I've also hit the following a few times on master: > > test_update_delete_extra_route failing due to timeout when creating subnets > https://bugs.launchpad.net/neutron/+bug/1867936 > > I'll try to write up a logstash query for this now and post a review for > recheck. > > Thanks again, > From neil at tigera.io Thu Mar 19 15:27:27 2020 From: neil at tigera.io (Neil Jerram) Date: Thu, 19 Mar 2020 15:27:27 +0000 Subject: [all][tc] Moving PTL role to "Maintainers" In-Reply-To: References: Message-ID: On Mon, Mar 2, 2020 at 9:46 PM Mohammed Naser wrote: > Hi everyone: > > We're now in a spot where we have an increasing amount of projects > that don't end up with a volunteer as PTL, even if the project has > contributors .. no one wants to hold that responsibility alone for > many reasons. With time, the PTL role has become far more overloaded > with many extra responsibilities than what we define in our charter: > > > https://governance.openstack.org/tc/reference/charter.html#project-team-leads > > I think it's time to re-evaluate the project leadership model that we > have. I am thinking that perhaps it would make a lot of sense to move > from a single PTL model to multiple maintainers. This would leave it > up to the maintainers to decide how they want to sort the different > requirements/liaisons/contact persons between them. > > The above is just a very basic idea, I don't intend to diving much > more in depth for now as I'd like to hear about what the rest of the > community thinks. > > Thanks, > Mohammed > Mine is an outsider's perspective, and I'm not sure if I should get involved in case it is not well received. But I can't get the thoughts out of my head, so here goes; I hope something constructive can be taken from this... I write as someone whose interest over the last 2-3 years has just been to keep a particular networking driver (Calico) working, as OpenStack master moves along. Doing that, my impression has been of an awful lot of churn, requiring minor updates on my part, but delivering questionable benefit to OpenStack users. For example, in Neutron, there was the extended neutron-lib work, and now we have networking-ovn moving into Neutron core. (Which appears to me - possibly insufficiently informed - as the opposite philosophical direction from the neutron-lib and Neutron stadium efforts.) As techies, we all (myself included) like refactoring our work to make it more elegant, but in my experience that kind of activity can take over when there are fewer real external needs to meet. So, there's the proposal in this thread about PTLs, and separately there is Thierry's RFC about some project consolidation. Very broadly speaking, if feels to me that the consolidation is what OpenStack really needs, and in relation to the kind of churn that I've mentioned, - it feels like consolidation would correctly curtail that back, as consolidated projects would - I think - naturally review the real external needs within their new wider scope - it feels like reducing PTL authority would encourage that kind of churn activity, as there wouldn't necessarily be anyone within a project to give a more strategic lead. So for me I guess this thread feels like the wrong answer, and it's disappointing that there hasn't been more engagement with the consolidation idea. Best wishes, Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Thu Mar 19 15:31:22 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 19 Mar 2020 16:31:22 +0100 Subject: [neutron]Drivers meeting cancelled Message-ID: Hi, Due to lack of agenda, lets cancel tomorrows (20.03.2020) drivers meeting and lets focus on reviewing patches related to the already accepted RFEs [1]. [1] https://tinyurl.com/vezk6n6 — Slawek Kaplonski Senior software engineer Red Hat From ralonsoh at redhat.com Thu Mar 19 15:45:23 2020 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Thu, 19 Mar 2020 15:45:23 +0000 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep Message-ID: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> Hello all: With this mail I would like to propose the goal to move away from oslo.rootwrap and migrate to oslo.privsep. The last one offers a superior security model, faster and more secure. During the last cycles and since privsep was released, the Community has encouraged the usage of privsep and the deprecation of any existing code still using rootwrap. For any developer willing to collaborate, there are plenty of code examples, as I’ll provide later, implementing and using privsep for new methods and migrations. If this goal is approved, I'll open a Story (https://storyboard.openstack.org/) and any developer will be able to add a task for each patch or set of them related. This would be the tracker for this common effort. PROJECTS TO MIGRATE. -------------------- Projects that are still using rootwrap: http://codesearch.openstack.org/?q=rootwrap&i=nope&files=.*.py&repos= neutron os-brick designate cinder ironic-inspector neutron-vpnaas nova solum glance_store ironic zun magnum manila networking-bagpipe sahara ceilometer cinderlib freezer ironic-lib monasca-agent tacker tripleo-common USAGE DOCUMENTATION ABOUT PRIVSEP. ---------------------------------- How to create a privsep context, assign privileges and use it as a decorator: https://docs.openstack.org/oslo.privsep/latest/user/index.html HOW TO MIGRATE FROM ROOTWRAP TO PRIVSEP. ---------------------------------------- >From the same link provided previously, in the section “Converting from rootwrap to privsep”: https://docs.openstack.org/oslo.privsep/latest/user/index.html#converting-from-rootwrap-to-privsep oslo.privsep provides a class, PrivContext, that can be used to create a decorator function. The instance created is a context of execution and has defined a list of capabilities, matching the Linux capabilities. The privsep context decorator should contain the minimum needed capabilities to execute the decorated function. For example: default = priv_context.PrivContext( __name__, cfg_section='privsep', pypath=__name__ + '.default', capabilities=[caps.CAP_SYS_ADMIN, caps.CAP_NET_ADMIN, caps.CAP_DAC_OVERRIDE, caps.CAP_DAC_READ_SEARCH, caps.CAP_SYS_PTRACE], ) The function “entrypoint” of this instance can be used as a decorator for another function: @privileged.default.entrypoint def delete_interface(ifname, namespace, **kwargs): _run_iproute_link("del", ifname, namespace, **kwargs) As commented in the given link, a straight 1:1 filter:function replacement generally results in functions that are still too broad for good security. It is better to replace each chmod rootwrap call with a narrow privsep function that will limit it to specific files. MIGRATION EXAMPLES. ------------------- Nova: https://review.opendev.org/#/q/project:openstack/nova+branch:master+topic:my-own-personal-alternative-universe Neutron: https://review.opendev.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bug/1492714 os-vif: https://review.opendev.org/#/c/287725/ Thank you and regards. From moreira.belmiro.email.lists at gmail.com Thu Mar 19 15:45:43 2020 From: moreira.belmiro.email.lists at gmail.com (Belmiro Moreira) Date: Thu, 19 Mar 2020 16:45:43 +0100 Subject: [python-openstackclient] --force bug In-Reply-To: <5884eb023abd91f422e46204eacee408460e4193.camel@redhat.com> References: <5884eb023abd91f422e46204eacee408460e4193.camel@redhat.com> Message-ID: Hi Sean, applying quotas using "--force" is used by operators when it's required to reduce a quota that's been used by a user. Ex: user has 10 instances running in the project but now the operator wants to set the quota 5 without asking the user to delete instances. I see this as important functionality for operators. It's supported by nova client but not by the openstack client. Reducing the feature gap between them it will easy operations. cheers, Belmiro On Thu, Mar 19, 2020 at 4:20 PM Sean Mooney wrote: > On Thu, 2020-03-19 at 16:19 +0530, jayaditya gupta wrote: > > Hello all, new contributor/developer here. I have just submitted a bug > for > > python-openstackclient project : > > https://storyboard.openstack.org/#!/story/2007440 > > > > I am not sure how to provide --force option to openstackclient .. > > > > I am comparing quota.py files in nova-client and in openstackclient.but > > can't seem to understand how to provide --force option. Any suggestions? > > at leas when it comes to some nova action we intentionally do not want to > support --force > for thing like migration > specifcally for live migration we removed --force in microversion 2.68 > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id62 > so since osc never supported --force we dont want it to be enabled. > > for "openstack quota set" i dont think we have specifically disucced if we > want to support --force > before. > we deprecated the proxing of quotas via nova in micro-version 2.36 > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#microversion > > 2.39 > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id36 > 2.50 > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id46 > 2.57 > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id53 > also modifed quotas in different ways. > > non of these microversion remove the force parameter like we did with live > migration so it hink it would > be valid to add support for --force. > > however nova and keystone are currently working on removing support for > nova based quotas using the unified > limits proposal > > > https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/unified-limits-nova.html > > https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html > > so i dont know if we have disucsed how that will impact the openstack > clint and if we will deprecate > "openstack quota set" and intoduce a "openstack limit set" to interact > with the new api or if we will > use a microverion but maintain the same cli. > > i dont know if the unified limits api will support a force paramater. i > hope its not nessisary to have a force paramter > with the new design so while there is no conflict i am awere of wehre > exposing --force might encurage usage that the > nova team does not want opertors to do, im not sure it would be consitent > with the future direction. > > so tentative +0.5 from me in terms of closing the gap for older openstack > releases > but i think it would be good to have the unified limit folks also way in > on if it should > be added and how that will interact with the current planned changes. > > > > Best Regards > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Mar 19 15:50:03 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 15:50:03 +0000 Subject: [python-openstackclient] --force bug In-Reply-To: References: <5884eb023abd91f422e46204eacee408460e4193.camel@redhat.com> Message-ID: <26e5fdf2d97a46a950721fcf2b3ed0f9214ab2c9.camel@redhat.com> On Thu, 2020-03-19 at 16:45 +0100, Belmiro Moreira wrote: > Hi Sean, > applying quotas using "--force" is used by operators when it's required to > reduce a quota that's been used by a user. > Ex: user has 10 instances running in the project but now the operator wants > to set the quota 5 without asking the user to delete instances. > > I see this as important functionality for operators. > It's supported by nova client but not by the openstack client. Reducing the > feature gap between them it will easy operations. ya that makes sense to me. > > cheers, > Belmiro > > > On Thu, Mar 19, 2020 at 4:20 PM Sean Mooney wrote: > > > On Thu, 2020-03-19 at 16:19 +0530, jayaditya gupta wrote: > > > Hello all, new contributor/developer here. I have just submitted a bug > > > > for > > > python-openstackclient project : > > > https://storyboard.openstack.org/#!/story/2007440 > > > > > > I am not sure how to provide --force option to openstackclient .. > > > > > > I am comparing quota.py files in nova-client and in openstackclient.but > > > can't seem to understand how to provide --force option. Any suggestions? > > > > at leas when it comes to some nova action we intentionally do not want to > > support --force > > for thing like migration > > specifcally for live migration we removed --force in microversion 2.68 > > > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id62 > > so since osc never supported --force we dont want it to be enabled. > > > > for "openstack quota set" i dont think we have specifically disucced if we > > want to support --force > > before. > > we deprecated the proxing of quotas via nova in micro-version 2.36 > > > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#microversion > > > > 2.39 > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id36 > > 2.50 > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id46 > > 2.57 > > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id53 > > also modifed quotas in different ways. > > > > non of these microversion remove the force parameter like we did with live > > migration so it hink it would > > be valid to add support for --force. > > > > however nova and keystone are currently working on removing support for > > nova based quotas using the unified > > limits proposal > > > > > > https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/unified-limits-nova.html > > > > https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html > > > > so i dont know if we have disucsed how that will impact the openstack > > clint and if we will deprecate > > "openstack quota set" and intoduce a "openstack limit set" to interact > > with the new api or if we will > > use a microverion but maintain the same cli. > > > > i dont know if the unified limits api will support a force paramater. i > > hope its not nessisary to have a force paramter > > with the new design so while there is no conflict i am awere of wehre > > exposing --force might encurage usage that the > > nova team does not want opertors to do, im not sure it would be consitent > > with the future direction. > > > > so tentative +0.5 from me in terms of closing the gap for older openstack > > releases > > but i think it would be good to have the unified limit folks also way in > > on if it should > > be added and how that will interact with the current planned changes. > > > > > > Best Regards > > > > > > From iurygregory at gmail.com Thu Mar 19 15:50:34 2020 From: iurygregory at gmail.com (Iury Gregory) Date: Thu, 19 Mar 2020 16:50:34 +0100 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> Message-ID: Hi Rodolfo, Thanks for raising this topic. I like the idea and I can work in the ironic part. Em qui., 19 de mar. de 2020 às 16:47, Rodolfo Alonso escreveu: > Hello all: > > With this mail I would like to propose the goal to move away from > oslo.rootwrap and migrate to > oslo.privsep. The last one offers a superior security model, faster and > more secure. During the last > cycles and since privsep was released, the Community has encouraged the > usage of privsep and the > deprecation of any existing code still using rootwrap. > > For any developer willing to collaborate, there are plenty of code > examples, as I’ll provide later, > implementing and using privsep for new methods and migrations. > > If this goal is approved, I'll open a Story ( > https://storyboard.openstack.org/) and any developer > will be able to add a task for each patch or set of them related. This > would be the tracker for this > common effort. > > > PROJECTS TO MIGRATE. > -------------------- > Projects that are still using rootwrap: > http://codesearch.openstack.org/?q=rootwrap&i=nope&files=.*.py&repos= > neutron > os-brick > designate > cinder > ironic-inspector > neutron-vpnaas > nova > solum > glance_store > ironic > zun > magnum > manila > networking-bagpipe > sahara > ceilometer > cinderlib > freezer > ironic-lib > monasca-agent > tacker > tripleo-common > > > USAGE DOCUMENTATION ABOUT PRIVSEP. > ---------------------------------- > How to create a privsep context, assign privileges and use it as a > decorator: > https://docs.openstack.org/oslo.privsep/latest/user/index.html > > > HOW TO MIGRATE FROM ROOTWRAP TO PRIVSEP. > ---------------------------------------- > From the same link provided previously, in the section “Converting from > rootwrap to privsep”: > > https://docs.openstack.org/oslo.privsep/latest/user/index.html#converting-from-rootwrap-to-privsep > > oslo.privsep provides a class, PrivContext, that can be used to create a > decorator function. The > instance created is a context of execution and has defined a list of > capabilities, matching the > Linux capabilities. The privsep context decorator should contain the > minimum needed capabilities to > execute the decorated function. > > For example: > > default = priv_context.PrivContext( > __name__, > cfg_section='privsep', > pypath=__name__ + '.default', > capabilities=[caps.CAP_SYS_ADMIN, > caps.CAP_NET_ADMIN, > caps.CAP_DAC_OVERRIDE, > caps.CAP_DAC_READ_SEARCH, > caps.CAP_SYS_PTRACE], > ) > > > The function “entrypoint” of this instance can be used as a decorator for > another function: > > @privileged.default.entrypoint > def delete_interface(ifname, namespace, **kwargs): > _run_iproute_link("del", ifname, namespace, **kwargs) > > > As commented in the given link, a straight 1:1 filter:function replacement > generally results in > functions that are still too broad for good security. It is better to > replace each chmod rootwrap > call with a narrow privsep function that will limit it to specific files. > > > MIGRATION EXAMPLES. > ------------------- > Nova: > > https://review.opendev.org/#/q/project:openstack/nova+branch:master+topic:my-own-personal-alternative-universe > Neutron: > > https://review.opendev.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bug/1492714 > os-vif > : > https://review.opendev.org/#/c/287725/ > > > Thank you and regards. > > > > -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Thu Mar 19 15:51:25 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 19 Mar 2020 10:51:25 -0500 Subject: [nova] bug triage In-Reply-To: <1583748087.12170.20@est.tech> References: <1583748087.12170.20@est.tech> Message-ID: <170f37dfe5a.c763a0b2349289.2259758601392224983@ghanshyammann.com> ---- On Mon, 09 Mar 2020 05:01:27 -0500 Balázs_Gibizer_ wrote ---- > Hi, > > We surpassed the somewhat magical line of having more than 100 > untriaged nova bugs [1]. For me it seems that how we, as a team, are > handling the incoming bugs is not sustainable. I'm guilty as well of > not doing bug triage in a last two months. So I'm personally trying to > change now and dedicate a weekly times lot to look at the bug list. But > I also want to open a discussion about bug triage in genera. > > How can we handle the incoming bugs? > > I see that neutron does a weekly rotation of bug deputy and for them it > works nicely. What do you think? Do we want to try that? Do we have > enough volunteers to create a nice rotation period? yeah, neutron is doing really good in term of rotation where nova might not have that many volunteers. Anyways I created etherpad to have weekly bug triage info or any bug needs more discussion/opinion. -https://etherpad.openstack.org/p/nova-bug-triage-ussuri I will try to do triage weekly basis and others can also do the same. Once we have more volunteer than we can think about rotation also. Basic idea is to triage if possible or initial investigation and then ask expert for their advice/feedback or further investigation. -gmann > > Cheers, > gibi > > > [1] https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New > > > > From flux.adam at gmail.com Thu Mar 19 15:55:39 2020 From: flux.adam at gmail.com (Adam Harwell) Date: Fri, 20 Mar 2020 00:55:39 +0900 Subject: [octavia] dropping support for stable/queens Message-ID: I'm officially proposing dropping support for the queens stable branch of Octavia. Given a quorum of cores, we will move forward with marking it EOL and all that entails. Thanks, --Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Mar 19 16:04:10 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 16:04:10 +0000 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> Message-ID: <8a7e12c0d63e7144f2ce3424e6d703d1a3a40d9c.camel@redhat.com> On Thu, 2020-03-19 at 16:50 +0100, Iury Gregory wrote: > Hi Rodolfo, > > Thanks for raising this topic. I like the idea and I can work in the ironic > part. > > Em qui., 19 de mar. de 2020 às 16:47, Rodolfo Alonso > escreveu: > > > Hello all: > > > > With this mail I would like to propose the goal to move away from > > oslo.rootwrap and migrate to > > oslo.privsep. The last one offers a superior security model, faster and > > more secure. During the last > > cycles and since privsep was released, the Community has encouraged the > > usage of privsep and the > > deprecation of any existing code still using rootwrap. > > > > For any developer willing to collaborate, there are plenty of code > > examples, as I’ll provide later, > > implementing and using privsep for new methods and migrations. > > > > If this goal is approved, I'll open a Story ( > > https://storyboard.openstack.org/) and any developer > > will be able to add a task for each patch or set of them related. This > > would be the tracker for this > > common effort. > > > > > > PROJECTS TO MIGRATE. > > -------------------- > > Projects that are still using rootwrap: > > http://codesearch.openstack.org/?q=rootwrap&i=nope&files=.*.py&repos= > > neutron > > os-brick > > designate > > cinder > > ironic-inspector > > neutron-vpnaas > > nova nova does not use rootwarp anymore. > > solum > > glance_store > > ironic > > zun > > magnum > > manila > > networking-bagpipe > > sahara > > ceilometer > > cinderlib > > freezer > > ironic-lib > > monasca-agent > > tacker > > tripleo-common > > > > > > USAGE DOCUMENTATION ABOUT PRIVSEP. > > ---------------------------------- > > How to create a privsep context, assign privileges and use it as a > > decorator: > > https://docs.openstack.org/oslo.privsep/latest/user/index.html > > > > > > HOW TO MIGRATE FROM ROOTWRAP TO PRIVSEP. > > ---------------------------------------- > > From the same link provided previously, in the section “Converting from > > rootwrap to privsep”: > > > > https://docs.openstack.org/oslo.privsep/latest/user/index.html#converting-from-rootwrap-to-privsep > > > > oslo.privsep provides a class, PrivContext, that can be used to create a > > decorator function. The > > instance created is a context of execution and has defined a list of > > capabilities, matching the > > Linux capabilities. The privsep context decorator should contain the > > minimum needed capabilities to > > execute the decorated function. > > > > For example: > > > > default = priv_context.PrivContext( > > __name__, > > cfg_section='privsep', > > pypath=__name__ + '.default', > > capabilities=[caps.CAP_SYS_ADMIN, > > caps.CAP_NET_ADMIN, > > caps.CAP_DAC_OVERRIDE, > > caps.CAP_DAC_READ_SEARCH, > > caps.CAP_SYS_PTRACE], > > ) > > > > > > The function “entrypoint” of this instance can be used as a decorator for > > another function: > > > > @privileged.default.entrypoint > > def delete_interface(ifname, namespace, **kwargs): > > _run_iproute_link("del", ifname, namespace, **kwargs) > > > > > > As commented in the given link, a straight 1:1 filter:function replacement > > generally results in > > functions that are still too broad for good security. It is better to > > replace each chmod rootwrap > > call with a narrow privsep function that will limit it to specific files. > > > > > > MIGRATION EXAMPLES. > > ------------------- > > Nova: > > > > https://review.opendev.org/#/q/project:openstack/nova+branch:master+topic:my-own-personal-alternative-universe > > Neutron: > > > > https://review.opendev.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bug/1492714 > > os-vif > > : > > https://review.opendev.org/#/c/287725/ > > > > > > Thank you and regards. > > > > > > > > > > From ignaziocassano at gmail.com Thu Mar 19 16:10:10 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 19 Mar 2020 17:10:10 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Hello Jakub, I will try again but if there is a bug on queens I do not think it will be corrected because is going out of support. Thanks Ignazio Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar ha scritto: > On 13/03/2020 08:24, Ignazio Cassano wrote: > > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node switched > > on openvswitch works fine . The problem is this migration create the qbr > on > > the mode switched to openvswitch. > > But when I switch another compute node to openvswitch and I try to live > > migrate the same vm (openvswitch to qopenswitch) it does not work because > > the qbr presence. > > I verified on nova logs. > > Ignazio > > Hi Ignazio, > > I think the first step - migrating from hybrid_iptables to ovs should > not create the qbr on the target node. It sounds like a bug - IIRC the > libvirt domxml should not have the qbr when migrating. > > > > > > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha > scritto: > > > >> On 12/03/2020 11:38, Ignazio Cassano wrote: > >>> Hello All, I am facing some problems migrating from iptables_hybrid > >>> frirewall to openvswitch firewall on centos 7 queens, > >>> I am doing this because I want enable security groups logs which > require > >>> openvswitch firewall. > >>> I would like to migrate without restarting my instances. > >>> I startded moving all instances from compute node 1. > >>> Then I configured openvswitch firewall on compute node 1, > >>> Instances migrated from compute node 2 to compute node 1 without > >> problems. > >>> Once the compute node 2 was empty, I migrated it to openvswitch. > >>> But now instances does not migrate from node 1 to node 2 because it > >>> requires the presence of qbr bridge on node 2 > >>> > >>> This happened because migrating instances from node2 with > iptables_hybrid > >>> to compute node 1 with openvswitch, does not put the tap under br-int > as > >>> requested by openvswich firewall, but qbr is still present on compute > >> node > >>> 1. > >>> Once I enabled openvswitch on compute node 2, migration from compute > >> node 1 > >>> fails because it exprects qbr on compute node 2 . > >>> So I think I should moving on the fly tap interfaces from qbr to br-int > >> on > >>> compute node 1 before migrating to compute node 2 but it is a huge work > >> on > >>> a lot of instances. > >>> > >>> Any workaround, please ? > >>> > >>> Ignazio > >>> > >> > >> I may be a little outdated here but to the best of my knowledge there > >> are two ways how to migrate from iptables to openvswitch. > >> > >> 1) If you don't mind the intermediate linux bridge and you care about > >> logs, you can just change the config file on compute node to start using > >> openvswitch firewall and restart the ovs agent. That should trigger a > >> mechanism that deletes iptables rules and starts using openflow rules. > >> It will leave the intermediate bridge there but except the extra hop in > >> networking stack, it doesn't mind. > >> > >> 2) With multiple-port binding feature, what you described above should > >> be working. I know Miguel spent some time working on that so perhaps he > >> has more information about which release it should be functional at, I > >> think it was Queens. Not sure if any Nova work was required to make it > >> work. > >> > >> Hope that helps. > >> Kuba > >> > >> > >> > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 19 16:27:28 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 19 Mar 2020 12:27:28 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> Message-ID: Erik, If i want to adopt following setting then where i should add them in Queens openstack, neutron-server or all my compute nodes? which setting will go where? heartbeat_timeout_threshold = 0 rpc_conn_pool_size = 300 rpc_thread_pool_size = 2048 rpc_response_timeout = 3600 rpc_poll_timeout = 60 ## Rpc all executor_thread_pool_size = 64 rpc_response_timeout = 3600 On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: > > We are hitting something awfully similar. > > We have basically been hitting a few pretty serious bugs with RabbitMQ. > > The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. > > e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 > > I wrote two quick scripts to audit these issues. > http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). > http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. > > The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. > > Best Regards, Erik Olof Gunnar Andersson > > -----Original Message----- > From: Satish Patel > Sent: Wednesday, March 11, 2020 5:14 PM > To: Grant Morley > Cc: openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. > > This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ > > On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: > > > > Hi all, > > > > We are currently experiencing some fairly major issues with our > > OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We > > are seeing a lot of time out messages in responses to replies and > > because of this instance creation or anything to do with instances and > > networking is broken. > > > > We are running OpenStack Queens. > > > > We have already tuned Rabbit for Neutron by doing the following on neutron: > > > > heartbeat_timeout_threshold = 0 > > rpc_conn_pool_size = 300 > > rpc_thread_pool_size = 2048 > > rpc_response_timeout = 3600 > > rpc_poll_timeout = 60 > > > > ## Rpc all > > executor_thread_pool_size = 64 > > rpc_response_timeout = 3600 > > > > What we are seeing in the error logs for neutron for all services > > (l3-agent, dhcp, linux-bridge etc ) are these timeouts: > > > > https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n > > 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK > > 9aOA$ > > > > We have manually tried to get everything in sync by forcing fail-over > > of the networking which seems to get routers in sync. > > > > We are also seeing that there are a lot of "unacknowledged" messages > > in RabbitMQ for 'q-plugin' in the neutron queues. > > > > Some times restarting of the services on neutron gets these back > > acknowledged again, however the timeouts come back. > > > > The RabbitMQ servers themselves are not loaded at all. All memory, > > file descriptors and errlang processes have plenty of resources available. > > > > We are also seeing a lot of rpc issues: > > > > Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds > > before next attempt. If the server is not down, consider increasing > > the rpc_response_timeout option as Neutron server(s) may be overloaded > > and unable to respond quickly enough.: MessagingTimeout: Timed out > > waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 > > 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc > > [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC > > method release_dhcp_port. Waiting for 3347 seconds before next attempt. > > If the server is not down, consider increasing the > > rpc_response_timeout option as Neutron server(s) may be overloaded and > > unable to respond quickly enough.: MessagingTimeout: Timed out waiting > > for a reply to message ID 7937465f15634fbfa443fe1758a12a9c > > > > Does anyone know if there is anymore tuning to be done at all? > > Upgrading for us at the moment to a newer version isn't really an > > option unfortunately. > > > > Because of our setup, we also have roughly 800 routers enabled and I > > know that will be putting a load on the system. However these problems > > have only started to happen roughly 1 week ago and have steadily got worse. > > > > If anyone has any use cases for this or any more recommendations that > > would be great. > > > > Many thanks, > > > > > From grant at civo.com Thu Mar 19 16:35:07 2020 From: grant at civo.com (Grant Morley) Date: Thu, 19 Mar 2020 16:35:07 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> Message-ID: <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> Hi Satish, You will need to add those to the "neutron.conf" file on your network nodes. If you are running OS-A I would do it on your "neutron-server" nodes and add the following to your agents containers: executor_thread_pool_size = 64 rpc_response_timeout = 3600 Regards, On 19/03/2020 16:27, Satish Patel wrote: > Erik, > > If i want to adopt following setting then where i should add them in > Queens openstack, neutron-server or all my compute nodes? which > setting will go where? > > heartbeat_timeout_threshold = 0 > rpc_conn_pool_size = 300 > rpc_thread_pool_size = 2048 > rpc_response_timeout = 3600 > rpc_poll_timeout = 60 > > ## Rpc all > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson > wrote: >> We are hitting something awfully similar. >> >> We have basically been hitting a few pretty serious bugs with RabbitMQ. >> >> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >> >> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >> >> I wrote two quick scripts to audit these issues. >> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >> >> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >> >> Best Regards, Erik Olof Gunnar Andersson >> >> -----Original Message----- >> From: Satish Patel >> Sent: Wednesday, March 11, 2020 5:14 PM >> To: Grant Morley >> Cc: openstack-discuss at lists.openstack.org >> Subject: Re: Neutron RabbitMQ issues >> >> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >> >> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >> >> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>> Hi all, >>> >>> We are currently experiencing some fairly major issues with our >>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>> are seeing a lot of time out messages in responses to replies and >>> because of this instance creation or anything to do with instances and >>> networking is broken. >>> >>> We are running OpenStack Queens. >>> >>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>> >>> heartbeat_timeout_threshold = 0 >>> rpc_conn_pool_size = 300 >>> rpc_thread_pool_size = 2048 >>> rpc_response_timeout = 3600 >>> rpc_poll_timeout = 60 >>> >>> ## Rpc all >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> What we are seeing in the error logs for neutron for all services >>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>> >>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>> 9aOA$ >>> >>> We have manually tried to get everything in sync by forcing fail-over >>> of the networking which seems to get routers in sync. >>> >>> We are also seeing that there are a lot of "unacknowledged" messages >>> in RabbitMQ for 'q-plugin' in the neutron queues. >>> >>> Some times restarting of the services on neutron gets these back >>> acknowledged again, however the timeouts come back. >>> >>> The RabbitMQ servers themselves are not loaded at all. All memory, >>> file descriptors and errlang processes have plenty of resources available. >>> >>> We are also seeing a lot of rpc issues: >>> >>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>> before next attempt. If the server is not down, consider increasing >>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>> If the server is not down, consider increasing the >>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>> >>> Does anyone know if there is anymore tuning to be done at all? >>> Upgrading for us at the moment to a newer version isn't really an >>> option unfortunately. >>> >>> Because of our setup, we also have roughly 800 routers enabled and I >>> know that will be putting a load on the system. However these problems >>> have only started to happen roughly 1 week ago and have steadily got worse. >>> >>> If anyone has any use cases for this or any more recommendations that >>> would be great. >>> >>> Many thanks, >>> >>> -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From grant at civo.com Thu Mar 19 16:37:03 2020 From: grant at civo.com (Grant Morley) Date: Thu, 19 Mar 2020 16:37:03 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> Message-ID: <8a186e5e-45a7-a021-d68d-ed062c54f2b4@civo.com> To add to this as well, I would recommend you run the latest version of the neutron code "17.1.17" for OS-A Queens as well. We found that upgrading to that version of the code as well as making those changes really helped. Grant On 19/03/2020 16:35, Grant Morley wrote: > > Hi Satish, > > You will need to add those to the "neutron.conf" file on your network > nodes. If you are running OS-A I would do it on your "neutron-server" > nodes and add the following to your agents containers: > > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > Regards, > > On 19/03/2020 16:27, Satish Patel wrote: >> Erik, >> >> If i want to adopt following setting then where i should add them in >> Queens openstack, neutron-server or all my compute nodes? which >> setting will go where? >> >> heartbeat_timeout_threshold = 0 >> rpc_conn_pool_size = 300 >> rpc_thread_pool_size = 2048 >> rpc_response_timeout = 3600 >> rpc_poll_timeout = 60 >> >> ## Rpc all >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 >> >> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson >> wrote: >>> We are hitting something awfully similar. >>> >>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>> >>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>> >>> e.g.https://github.com/rabbitmq/rabbitmq-server/issues/641 >>> >>> I wrote two quick scripts to audit these issues. >>> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >>> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>> >>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>> >>> Best Regards, Erik Olof Gunnar Andersson >>> >>> -----Original Message----- >>> From: Satish Patel >>> Sent: Wednesday, March 11, 2020 5:14 PM >>> To: Grant Morley >>> Cc:openstack-discuss at lists.openstack.org >>> Subject: Re: Neutron RabbitMQ issues >>> >>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>> >>> This is my favorite video, not sure you have seen this before or not but anyway posting here -https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>> >>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>> Hi all, >>>> >>>> We are currently experiencing some fairly major issues with our >>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>> are seeing a lot of time out messages in responses to replies and >>>> because of this instance creation or anything to do with instances and >>>> networking is broken. >>>> >>>> We are running OpenStack Queens. >>>> >>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>> >>>> heartbeat_timeout_threshold = 0 >>>> rpc_conn_pool_size = 300 >>>> rpc_thread_pool_size = 2048 >>>> rpc_response_timeout = 3600 >>>> rpc_poll_timeout = 60 >>>> >>>> ## Rpc all >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> What we are seeing in the error logs for neutron for all services >>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>> >>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>> 9aOA$ >>>> >>>> We have manually tried to get everything in sync by forcing fail-over >>>> of the networking which seems to get routers in sync. >>>> >>>> We are also seeing that there are a lot of "unacknowledged" messages >>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>> >>>> Some times restarting of the services on neutron gets these back >>>> acknowledged again, however the timeouts come back. >>>> >>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>> file descriptors and errlang processes have plenty of resources available. >>>> >>>> We are also seeing a lot of rpc issues: >>>> >>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>> before next attempt. If the server is not down, consider increasing >>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>> If the server is not down, consider increasing the >>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>> >>>> Does anyone know if there is anymore tuning to be done at all? >>>> Upgrading for us at the moment to a newer version isn't really an >>>> option unfortunately. >>>> >>>> Because of our setup, we also have roughly 800 routers enabled and I >>>> know that will be putting a load on the system. However these problems >>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>> >>>> If anyone has any use cases for this or any more recommendations that >>>> would be great. >>>> >>>> Many thanks, >>>> >>>> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 19 16:53:55 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 19 Mar 2020 12:53:55 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> Message-ID: I am running openstack-ansible (Queens / Stein both) so this is what i am going to do, am i doing correctly? neutron-server (container) I have 3 neutron node. > > heartbeat_timeout_threshold = 0 > > rpc_conn_pool_size = 300 > > rpc_thread_pool_size = 2048 > > rpc_response_timeout = 3600 > > rpc_poll_timeout = 60 330 compute nodes (agent neutron.conf) going to add following: >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 How about nova? should i be doing that on nova as well to reduce load on rabbitMQ? On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: > Hi Satish, > > You will need to add those to the "neutron.conf" file on your network > nodes. If you are running OS-A I would do it on your "neutron-server" nodes > and add the following to your agents containers: > > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > Regards, > On 19/03/2020 16:27, Satish Patel wrote: > > Erik, > > If i want to adopt following setting then where i should add them in > Queens openstack, neutron-server or all my compute nodes? which > setting will go where? > > heartbeat_timeout_threshold = 0 > rpc_conn_pool_size = 300 > rpc_thread_pool_size = 2048 > rpc_response_timeout = 3600 > rpc_poll_timeout = 60 > > ## Rpc all > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: > > We are hitting something awfully similar. > > We have basically been hitting a few pretty serious bugs with RabbitMQ. > > The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. > > e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 > > I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. > > The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. > > Best Regards, Erik Olof Gunnar Andersson > > -----Original Message----- > From: Satish Patel > Sent: Wednesday, March 11, 2020 5:14 PM > To: Grant Morley > Cc: openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. > > This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ > > On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: > > Hi all, > > We are currently experiencing some fairly major issues with our > OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We > are seeing a lot of time out messages in responses to replies and > because of this instance creation or anything to do with instances and > networking is broken. > > We are running OpenStack Queens. > > We have already tuned Rabbit for Neutron by doing the following on neutron: > > heartbeat_timeout_threshold = 0 > rpc_conn_pool_size = 300 > rpc_thread_pool_size = 2048 > rpc_response_timeout = 3600 > rpc_poll_timeout = 60 > > ## Rpc all > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > What we are seeing in the error logs for neutron for all services > (l3-agent, dhcp, linux-bridge etc ) are these timeouts: > https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n > 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK > 9aOA$ > > We have manually tried to get everything in sync by forcing fail-over > of the networking which seems to get routers in sync. > > We are also seeing that there are a lot of "unacknowledged" messages > in RabbitMQ for 'q-plugin' in the neutron queues. > > Some times restarting of the services on neutron gets these back > acknowledged again, however the timeouts come back. > > The RabbitMQ servers themselves are not loaded at all. All memory, > file descriptors and errlang processes have plenty of resources available. > > We are also seeing a lot of rpc issues: > > Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds > before next attempt. If the server is not down, consider increasing > the rpc_response_timeout option as Neutron server(s) may be overloaded > and unable to respond quickly enough.: MessagingTimeout: Timed out > waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 > 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc > [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC > method release_dhcp_port. Waiting for 3347 seconds before next attempt. > If the server is not down, consider increasing the > rpc_response_timeout option as Neutron server(s) may be overloaded and > unable to respond quickly enough.: MessagingTimeout: Timed out waiting > for a reply to message ID 7937465f15634fbfa443fe1758a12a9c > > Does anyone know if there is anymore tuning to be done at all? > Upgrading for us at the moment to a newer version isn't really an > option unfortunately. > > Because of our setup, we also have roughly 800 routers enabled and I > know that will be putting a load on the system. However these problems > have only started to happen roughly 1 week ago and have steadily got worse. > > If anyone has any use cases for this or any more recommendations that > would be great. > > Many thanks, > > > > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grant at civo.com Thu Mar 19 17:02:29 2020 From: grant at civo.com (Grant Morley) Date: Thu, 19 Mar 2020 17:02:29 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> Message-ID: <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> Correct, you need to add: > > heartbeat_timeout_threshold = 0 > > rpc_conn_pool_size = 300 > > rpc_thread_pool_size = 2048 > > rpc_response_timeout = 3600 > > rpc_poll_timeout = 60 To your Neutron nodes And you can add: >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 To your compute nodes (neutron.conf) However I found just adding the changes to the neturon servers really helped. I would recommend just starting with your neutron nodes first to see if that helps. If you find your compute nodes are still having issues then change the settings on those after. Regards, On 19/03/2020 16:53, Satish Patel wrote: > I am running openstack-ansible (Queens / Stein both) so this is what i > am going to do, am i doing correctly? > > neutron-server (container) I have 3 neutron node. > > > heartbeat_timeout_threshold = 0 > > > rpc_conn_pool_size = 300 > > > rpc_thread_pool_size = 2048 > > > rpc_response_timeout = 3600 > > > rpc_poll_timeout = 60 > > 330 compute nodes (agent neutron.conf) going to add following: > >> executor_thread_pool_size = 64 > >> rpc_response_timeout = 3600 > > > > How about nova? should i be doing that on nova as well to reduce load > on rabbitMQ? > > > On Thu, Mar 19, 2020 at 12:35 PM Grant Morley > wrote: > > Hi Satish, > > You will need to add those to the "neutron.conf" file on your > network nodes. If you are running OS-A I would do it on your > "neutron-server" nodes and add the following to your agents > containers: > > executor_thread_pool_size = 64 > rpc_response_timeout = 3600 > > Regards, > > On 19/03/2020 16:27, Satish Patel wrote: >> Erik, >> >> If i want to adopt following setting then where i should add them in >> Queens openstack, neutron-server or all my compute nodes? which >> setting will go where? >> >> heartbeat_timeout_threshold = 0 >> rpc_conn_pool_size = 300 >> rpc_thread_pool_size = 2048 >> rpc_response_timeout = 3600 >> rpc_poll_timeout = 60 >> >> ## Rpc all >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 >> >> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson >> wrote: >>> We are hitting something awfully similar. >>> >>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>> >>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>> >>> e.g.https://github.com/rabbitmq/rabbitmq-server/issues/641 >>> >>> I wrote two quick scripts to audit these issues. >>> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >>> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>> >>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>> >>> Best Regards, Erik Olof Gunnar Andersson >>> >>> -----Original Message----- >>> From: Satish Patel >>> Sent: Wednesday, March 11, 2020 5:14 PM >>> To: Grant Morley >>> Cc:openstack-discuss at lists.openstack.org >>> Subject: Re: Neutron RabbitMQ issues >>> >>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>> >>> This is my favorite video, not sure you have seen this before or not but anyway posting here -https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>> >>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>> Hi all, >>>> >>>> We are currently experiencing some fairly major issues with our >>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>> are seeing a lot of time out messages in responses to replies and >>>> because of this instance creation or anything to do with instances and >>>> networking is broken. >>>> >>>> We are running OpenStack Queens. >>>> >>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>> >>>> heartbeat_timeout_threshold = 0 >>>> rpc_conn_pool_size = 300 >>>> rpc_thread_pool_size = 2048 >>>> rpc_response_timeout = 3600 >>>> rpc_poll_timeout = 60 >>>> >>>> ## Rpc all >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> What we are seeing in the error logs for neutron for all services >>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>> >>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>> 9aOA$ >>>> >>>> We have manually tried to get everything in sync by forcing fail-over >>>> of the networking which seems to get routers in sync. >>>> >>>> We are also seeing that there are a lot of "unacknowledged" messages >>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>> >>>> Some times restarting of the services on neutron gets these back >>>> acknowledged again, however the timeouts come back. >>>> >>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>> file descriptors and errlang processes have plenty of resources available. >>>> >>>> We are also seeing a lot of rpc issues: >>>> >>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>> before next attempt. If the server is not down, consider increasing >>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>> If the server is not down, consider increasing the >>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>> >>>> Does anyone know if there is anymore tuning to be done at all? >>>> Upgrading for us at the moment to a newer version isn't really an >>>> option unfortunately. >>>> >>>> Because of our setup, we also have roughly 800 routers enabled and I >>>> know that will be putting a load on the system. However these problems >>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>> >>>> If anyone has any use cases for this or any more recommendations that >>>> would be great. >>>> >>>> Many thanks, >>>> >>>> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > > -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 19 17:13:04 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 19 Mar 2020 13:13:04 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> Message-ID: how about rpc_worker ? currently i have rpc_worker=1 On Thu, Mar 19, 2020 at 1:02 PM Grant Morley wrote: > Correct, you need to add: > > > > heartbeat_timeout_threshold = 0 > > > rpc_conn_pool_size = 300 > > > rpc_thread_pool_size = 2048 > > > rpc_response_timeout = 3600 > > > rpc_poll_timeout = 60 > > To your Neutron nodes > > And you can add: > > > >> executor_thread_pool_size = 64 > >> rpc_response_timeout = 3600 > > To your compute nodes (neutron.conf) However I found just adding the > changes to the neturon servers really helped. > > I would recommend just starting with your neutron nodes first to see if > that helps. If you find your compute nodes are still having issues then > change the settings on those after. > > Regards, > On 19/03/2020 16:53, Satish Patel wrote: > > I am running openstack-ansible (Queens / Stein both) so this is what i am > going to do, am i doing correctly? > > neutron-server (container) I have 3 neutron node. > > > heartbeat_timeout_threshold = 0 > > > rpc_conn_pool_size = 300 > > > rpc_thread_pool_size = 2048 > > > rpc_response_timeout = 3600 > > > rpc_poll_timeout = 60 > > 330 compute nodes (agent neutron.conf) going to add following: > >> executor_thread_pool_size = 64 > >> rpc_response_timeout = 3600 > > > > How about nova? should i be doing that on nova as well to reduce load on > rabbitMQ? > > > On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: > >> Hi Satish, >> >> You will need to add those to the "neutron.conf" file on your network >> nodes. If you are running OS-A I would do it on your "neutron-server" nodes >> and add the following to your agents containers: >> >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 >> >> Regards, >> On 19/03/2020 16:27, Satish Patel wrote: >> >> Erik, >> >> If i want to adopt following setting then where i should add them in >> Queens openstack, neutron-server or all my compute nodes? which >> setting will go where? >> >> heartbeat_timeout_threshold = 0 >> rpc_conn_pool_size = 300 >> rpc_thread_pool_size = 2048 >> rpc_response_timeout = 3600 >> rpc_poll_timeout = 60 >> >> ## Rpc all >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 >> >> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: >> >> We are hitting something awfully similar. >> >> We have basically been hitting a few pretty serious bugs with RabbitMQ. >> >> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >> >> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >> >> I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >> >> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >> >> Best Regards, Erik Olof Gunnar Andersson >> >> -----Original Message----- >> From: Satish Patel >> Sent: Wednesday, March 11, 2020 5:14 PM >> To: Grant Morley >> Cc: openstack-discuss at lists.openstack.org >> Subject: Re: Neutron RabbitMQ issues >> >> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >> >> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >> >> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >> >> Hi all, >> >> We are currently experiencing some fairly major issues with our >> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >> are seeing a lot of time out messages in responses to replies and >> because of this instance creation or anything to do with instances and >> networking is broken. >> >> We are running OpenStack Queens. >> >> We have already tuned Rabbit for Neutron by doing the following on neutron: >> >> heartbeat_timeout_threshold = 0 >> rpc_conn_pool_size = 300 >> rpc_thread_pool_size = 2048 >> rpc_response_timeout = 3600 >> rpc_poll_timeout = 60 >> >> ## Rpc all >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 >> >> What we are seeing in the error logs for neutron for all services >> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >> 9aOA$ >> >> We have manually tried to get everything in sync by forcing fail-over >> of the networking which seems to get routers in sync. >> >> We are also seeing that there are a lot of "unacknowledged" messages >> in RabbitMQ for 'q-plugin' in the neutron queues. >> >> Some times restarting of the services on neutron gets these back >> acknowledged again, however the timeouts come back. >> >> The RabbitMQ servers themselves are not loaded at all. All memory, >> file descriptors and errlang processes have plenty of resources available. >> >> We are also seeing a lot of rpc issues: >> >> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >> before next attempt. If the server is not down, consider increasing >> the rpc_response_timeout option as Neutron server(s) may be overloaded >> and unable to respond quickly enough.: MessagingTimeout: Timed out >> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >> If the server is not down, consider increasing the >> rpc_response_timeout option as Neutron server(s) may be overloaded and >> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >> >> Does anyone know if there is anymore tuning to be done at all? >> Upgrading for us at the moment to a newer version isn't really an >> option unfortunately. >> >> Because of our setup, we also have roughly 800 routers enabled and I >> know that will be putting a load on the system. However these problems >> have only started to happen roughly 1 week ago and have steadily got worse. >> >> If anyone has any use cases for this or any more recommendations that >> would be great. >> >> Many thanks, >> >> >> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an account! >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Thu Mar 19 17:23:50 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Thu, 19 Mar 2020 18:23:50 +0100 Subject: [nova] feasibility to keep the two-company rule Message-ID: <1584638630.12170.41@est.tech> Hi, Nova has an unwritten rule that requires to have at least two companies involved in any new feature development (or even bugfix?). In the current Nova core diversity situation this rule puts extra burden to the remaining non Red Hat cores and I guess it also makes any Red Hat driven feature development harder. In parallel to working on increasing the size of the core team I suggest to reconsider dropping this rule. Some discussion happened already on the today's meeting[1]. Cheers, gibi [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 From grant at civo.com Thu Mar 19 17:26:04 2020 From: grant at civo.com (Grant Morley) Date: Thu, 19 Mar 2020 17:26:04 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> Message-ID: <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> We left ours on the default value of 1 and that still seems to be fine. Grant On 19/03/2020 17:13, Satish Patel wrote: > how about rpc_worker ? > currently i have rpc_worker=1 > > On Thu, Mar 19, 2020 at 1:02 PM Grant Morley > wrote: > > Correct, you need to add: > > > > heartbeat_timeout_threshold = 0 > > > rpc_conn_pool_size = 300 > > > rpc_thread_pool_size = 2048 > > > rpc_response_timeout = 3600 > > > rpc_poll_timeout = 60 > > To your Neutron nodes > > And you can add: > > > >> executor_thread_pool_size = 64 > >> rpc_response_timeout = 3600 > > To your compute nodes (neutron.conf) However I found just adding > the changes to the neturon servers really helped. > > I would recommend just starting with your neutron nodes first to > see if that helps. If you find your compute nodes are still having > issues then change the settings on those after. > > Regards, > > On 19/03/2020 16:53, Satish Patel wrote: >> I am running openstack-ansible (Queens / Stein both) so this is >> what i am going to do, am i doing correctly? >> >> neutron-server (container) I have 3 neutron node. >> > > heartbeat_timeout_threshold = 0 >> > > rpc_conn_pool_size = 300 >> > > rpc_thread_pool_size = 2048 >> > > rpc_response_timeout = 3600 >> > > rpc_poll_timeout = 60 >> >> 330 compute nodes (agent neutron.conf) going to add following: >> >> executor_thread_pool_size = 64 >> >> rpc_response_timeout = 3600 >> >> >> >> How about nova? should i be doing that on nova as well to reduce >> load on rabbitMQ? >> >> >> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley > > wrote: >> >> Hi Satish, >> >> You will need to add those to the "neutron.conf" file on your >> network nodes. If you are running OS-A I would do it on your >> "neutron-server" nodes and add the following to your agents >> containers: >> >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 >> >> Regards, >> >> On 19/03/2020 16:27, Satish Patel wrote: >>> Erik, >>> >>> If i want to adopt following setting then where i should add them in >>> Queens openstack, neutron-server or all my compute nodes? which >>> setting will go where? >>> >>> heartbeat_timeout_threshold = 0 >>> rpc_conn_pool_size = 300 >>> rpc_thread_pool_size = 2048 >>> rpc_response_timeout = 3600 >>> rpc_poll_timeout = 60 >>> >>> ## Rpc all >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson >>> wrote: >>>> We are hitting something awfully similar. >>>> >>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>> >>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>> >>>> e.g.https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>> >>>> I wrote two quick scripts to audit these issues. >>>> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >>>> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>> >>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>> >>>> Best Regards, Erik Olof Gunnar Andersson >>>> >>>> -----Original Message----- >>>> From: Satish Patel >>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>> To: Grant Morley >>>> Cc:openstack-discuss at lists.openstack.org >>>> Subject: Re: Neutron RabbitMQ issues >>>> >>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>> >>>> This is my favorite video, not sure you have seen this before or not but anyway posting here -https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>> >>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>> Hi all, >>>>> >>>>> We are currently experiencing some fairly major issues with our >>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>> are seeing a lot of time out messages in responses to replies and >>>>> because of this instance creation or anything to do with instances and >>>>> networking is broken. >>>>> >>>>> We are running OpenStack Queens. >>>>> >>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>> >>>>> heartbeat_timeout_threshold = 0 >>>>> rpc_conn_pool_size = 300 >>>>> rpc_thread_pool_size = 2048 >>>>> rpc_response_timeout = 3600 >>>>> rpc_poll_timeout = 60 >>>>> >>>>> ## Rpc all >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> What we are seeing in the error logs for neutron for all services >>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>> >>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>> 9aOA$ >>>>> >>>>> We have manually tried to get everything in sync by forcing fail-over >>>>> of the networking which seems to get routers in sync. >>>>> >>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>> >>>>> Some times restarting of the services on neutron gets these back >>>>> acknowledged again, however the timeouts come back. >>>>> >>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>> file descriptors and errlang processes have plenty of resources available. >>>>> >>>>> We are also seeing a lot of rpc issues: >>>>> >>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>> before next attempt. If the server is not down, consider increasing >>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>> If the server is not down, consider increasing the >>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>> >>>>> Does anyone know if there is anymore tuning to be done at all? >>>>> Upgrading for us at the moment to a newer version isn't really an >>>>> option unfortunately. >>>>> >>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>> know that will be putting a load on the system. However these problems >>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>> >>>>> If anyone has any use cases for this or any more recommendations that >>>>> would be great. >>>>> >>>>> Many thanks, >>>>> >>>>> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an account! >> >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > > -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 19 17:32:20 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 19 Mar 2020 13:32:20 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: Great, thanks! Did you guys tune your nova component for rabbitMQ? On Thu, Mar 19, 2020 at 1:26 PM Grant Morley wrote: > We left ours on the default value of 1 and that still seems to be fine. > > Grant > On 19/03/2020 17:13, Satish Patel wrote: > > how about rpc_worker ? > > currently i have rpc_worker=1 > > On Thu, Mar 19, 2020 at 1:02 PM Grant Morley wrote: > >> Correct, you need to add: >> >> > > heartbeat_timeout_threshold = 0 >> > > rpc_conn_pool_size = 300 >> > > rpc_thread_pool_size = 2048 >> > > rpc_response_timeout = 3600 >> > > rpc_poll_timeout = 60 >> >> To your Neutron nodes >> >> And you can add: >> >> >> >> executor_thread_pool_size = 64 >> >> rpc_response_timeout = 3600 >> >> To your compute nodes (neutron.conf) However I found just adding the >> changes to the neturon servers really helped. >> >> I would recommend just starting with your neutron nodes first to see if >> that helps. If you find your compute nodes are still having issues then >> change the settings on those after. >> >> Regards, >> On 19/03/2020 16:53, Satish Patel wrote: >> >> I am running openstack-ansible (Queens / Stein both) so this is what i am >> going to do, am i doing correctly? >> >> neutron-server (container) I have 3 neutron node. >> > > heartbeat_timeout_threshold = 0 >> > > rpc_conn_pool_size = 300 >> > > rpc_thread_pool_size = 2048 >> > > rpc_response_timeout = 3600 >> > > rpc_poll_timeout = 60 >> >> 330 compute nodes (agent neutron.conf) going to add following: >> >> executor_thread_pool_size = 64 >> >> rpc_response_timeout = 3600 >> >> >> >> How about nova? should i be doing that on nova as well to reduce load on >> rabbitMQ? >> >> >> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: >> >>> Hi Satish, >>> >>> You will need to add those to the "neutron.conf" file on your network >>> nodes. If you are running OS-A I would do it on your "neutron-server" nodes >>> and add the following to your agents containers: >>> >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> Regards, >>> On 19/03/2020 16:27, Satish Patel wrote: >>> >>> Erik, >>> >>> If i want to adopt following setting then where i should add them in >>> Queens openstack, neutron-server or all my compute nodes? which >>> setting will go where? >>> >>> heartbeat_timeout_threshold = 0 >>> rpc_conn_pool_size = 300 >>> rpc_thread_pool_size = 2048 >>> rpc_response_timeout = 3600 >>> rpc_poll_timeout = 60 >>> >>> ## Rpc all >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: >>> >>> We are hitting something awfully similar. >>> >>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>> >>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>> >>> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >>> >>> I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>> >>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>> >>> Best Regards, Erik Olof Gunnar Andersson >>> >>> -----Original Message----- >>> From: Satish Patel >>> Sent: Wednesday, March 11, 2020 5:14 PM >>> To: Grant Morley >>> Cc: openstack-discuss at lists.openstack.org >>> Subject: Re: Neutron RabbitMQ issues >>> >>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>> >>> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>> >>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>> >>> Hi all, >>> >>> We are currently experiencing some fairly major issues with our >>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>> are seeing a lot of time out messages in responses to replies and >>> because of this instance creation or anything to do with instances and >>> networking is broken. >>> >>> We are running OpenStack Queens. >>> >>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>> >>> heartbeat_timeout_threshold = 0 >>> rpc_conn_pool_size = 300 >>> rpc_thread_pool_size = 2048 >>> rpc_response_timeout = 3600 >>> rpc_poll_timeout = 60 >>> >>> ## Rpc all >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> What we are seeing in the error logs for neutron for all services >>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>> 9aOA$ >>> >>> We have manually tried to get everything in sync by forcing fail-over >>> of the networking which seems to get routers in sync. >>> >>> We are also seeing that there are a lot of "unacknowledged" messages >>> in RabbitMQ for 'q-plugin' in the neutron queues. >>> >>> Some times restarting of the services on neutron gets these back >>> acknowledged again, however the timeouts come back. >>> >>> The RabbitMQ servers themselves are not loaded at all. All memory, >>> file descriptors and errlang processes have plenty of resources available. >>> >>> We are also seeing a lot of rpc issues: >>> >>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>> before next attempt. If the server is not down, consider increasing >>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>> If the server is not down, consider increasing the >>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>> >>> Does anyone know if there is anymore tuning to be done at all? >>> Upgrading for us at the moment to a newer version isn't really an >>> option unfortunately. >>> >>> Because of our setup, we also have roughly 800 routers enabled and I >>> know that will be putting a load on the system. However these problems >>> have only started to happen roughly 1 week ago and have steadily got worse. >>> >>> If anyone has any use cases for this or any more recommendations that >>> would be great. >>> >>> Many thanks, >>> >>> >>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an account! >>> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an account! >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emccormick at cirrusseven.com Thu Mar 19 18:36:39 2020 From: emccormick at cirrusseven.com (Erik McCormick) Date: Thu, 19 Mar 2020 14:36:39 -0400 Subject: [octavia] dropping support for stable/queens In-Reply-To: References: Message-ID: On Thu, Mar 19, 2020, 11:57 AM Adam Harwell wrote: > I'm officially proposing dropping support for the queens stable branch of > Octavia. > Given a quorum of cores, we will move forward with marking it EOL and all > that entails. > Adam, Why can't it go into EM Instead of EOL? Thanks Erik > Thanks, > --Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Thu Mar 19 18:53:22 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 19 Mar 2020 14:53:22 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: I have question related following setting, why are you disabling heartbeat timeout? heartbeat_timeout_threshold = 0 On Thu, Mar 19, 2020 at 1:32 PM Satish Patel wrote: > Great, thanks! Did you guys tune your nova component for rabbitMQ? > > On Thu, Mar 19, 2020 at 1:26 PM Grant Morley wrote: > >> We left ours on the default value of 1 and that still seems to be fine. >> >> Grant >> On 19/03/2020 17:13, Satish Patel wrote: >> >> how about rpc_worker ? >> >> currently i have rpc_worker=1 >> >> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley wrote: >> >>> Correct, you need to add: >>> >>> > > heartbeat_timeout_threshold = 0 >>> > > rpc_conn_pool_size = 300 >>> > > rpc_thread_pool_size = 2048 >>> > > rpc_response_timeout = 3600 >>> > > rpc_poll_timeout = 60 >>> >>> To your Neutron nodes >>> >>> And you can add: >>> >>> >>> >> executor_thread_pool_size = 64 >>> >> rpc_response_timeout = 3600 >>> >>> To your compute nodes (neutron.conf) However I found just adding the >>> changes to the neturon servers really helped. >>> >>> I would recommend just starting with your neutron nodes first to see if >>> that helps. If you find your compute nodes are still having issues then >>> change the settings on those after. >>> >>> Regards, >>> On 19/03/2020 16:53, Satish Patel wrote: >>> >>> I am running openstack-ansible (Queens / Stein both) so this is what i >>> am going to do, am i doing correctly? >>> >>> neutron-server (container) I have 3 neutron node. >>> > > heartbeat_timeout_threshold = 0 >>> > > rpc_conn_pool_size = 300 >>> > > rpc_thread_pool_size = 2048 >>> > > rpc_response_timeout = 3600 >>> > > rpc_poll_timeout = 60 >>> >>> 330 compute nodes (agent neutron.conf) going to add following: >>> >> executor_thread_pool_size = 64 >>> >> rpc_response_timeout = 3600 >>> >>> >>> >>> How about nova? should i be doing that on nova as well to reduce load on >>> rabbitMQ? >>> >>> >>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: >>> >>>> Hi Satish, >>>> >>>> You will need to add those to the "neutron.conf" file on your network >>>> nodes. If you are running OS-A I would do it on your "neutron-server" nodes >>>> and add the following to your agents containers: >>>> >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> Regards, >>>> On 19/03/2020 16:27, Satish Patel wrote: >>>> >>>> Erik, >>>> >>>> If i want to adopt following setting then where i should add them in >>>> Queens openstack, neutron-server or all my compute nodes? which >>>> setting will go where? >>>> >>>> heartbeat_timeout_threshold = 0 >>>> rpc_conn_pool_size = 300 >>>> rpc_thread_pool_size = 2048 >>>> rpc_response_timeout = 3600 >>>> rpc_poll_timeout = 60 >>>> >>>> ## Rpc all >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: >>>> >>>> We are hitting something awfully similar. >>>> >>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>> >>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>> >>>> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>> >>>> I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>> >>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>> >>>> Best Regards, Erik Olof Gunnar Andersson >>>> >>>> -----Original Message----- >>>> From: Satish Patel >>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>> To: Grant Morley >>>> Cc: openstack-discuss at lists.openstack.org >>>> Subject: Re: Neutron RabbitMQ issues >>>> >>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>> >>>> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>> >>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>> >>>> Hi all, >>>> >>>> We are currently experiencing some fairly major issues with our >>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>> are seeing a lot of time out messages in responses to replies and >>>> because of this instance creation or anything to do with instances and >>>> networking is broken. >>>> >>>> We are running OpenStack Queens. >>>> >>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>> >>>> heartbeat_timeout_threshold = 0 >>>> rpc_conn_pool_size = 300 >>>> rpc_thread_pool_size = 2048 >>>> rpc_response_timeout = 3600 >>>> rpc_poll_timeout = 60 >>>> >>>> ## Rpc all >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> What we are seeing in the error logs for neutron for all services >>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>> 9aOA$ >>>> >>>> We have manually tried to get everything in sync by forcing fail-over >>>> of the networking which seems to get routers in sync. >>>> >>>> We are also seeing that there are a lot of "unacknowledged" messages >>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>> >>>> Some times restarting of the services on neutron gets these back >>>> acknowledged again, however the timeouts come back. >>>> >>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>> file descriptors and errlang processes have plenty of resources available. >>>> >>>> We are also seeing a lot of rpc issues: >>>> >>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>> before next attempt. If the server is not down, consider increasing >>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>> If the server is not down, consider increasing the >>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>> >>>> Does anyone know if there is anymore tuning to be done at all? >>>> Upgrading for us at the moment to a newer version isn't really an >>>> option unfortunately. >>>> >>>> Because of our setup, we also have roughly 800 routers enabled and I >>>> know that will be putting a load on the system. However these problems >>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>> >>>> If anyone has any use cases for this or any more recommendations that >>>> would be great. >>>> >>>> Many thanks, >>>> >>>> >>>> >>>> -- >>>> >>>> Grant Morley >>>> Cloud Lead, Civo Ltd >>>> www.civo.com | Signup for an account! >>>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an account! >>> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an account! >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mordred at inaugust.com Thu Mar 19 19:06:51 2020 From: mordred at inaugust.com (Monty Taylor) Date: Thu, 19 Mar 2020 14:06:51 -0500 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <1584638630.12170.41@est.tech> References: <1584638630.12170.41@est.tech> Message-ID: <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> > On Mar 19, 2020, at 12:23 PM, Balázs Gibizer wrote: > > Hi, > > Nova has an unwritten rule that requires to have at least two companies involved in any new feature development (or even bugfix?). In the current Nova core diversity situation this rule puts extra burden to the remaining non Red Hat cores and I guess it also makes any Red Hat driven feature development harder. In parallel to working on increasing the size of the core team I suggest to reconsider dropping this rule. > > Some discussion happened already on the today's meeting[1] I think that’s a great idea. FWIW I’ve never liked that rule, because it assume that developers from a company are putting employer requirements over project requirements when acting in their capacity as a core reviewer - which is contrary to our general expectation of how a core reviewer behaves. I think it’s a great idea to get rid of this policy - and then if anyone is behaving in a manner that abuses the trust of the rest of the core reviewers, such as slamming through a new feature that other people obviously have misgivings about … that can be dealt with the same way any other breach of trust would happen. > Cheers, > gibi > > > [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 > > > > > From satish.txt at gmail.com Thu Mar 19 20:46:35 2020 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 19 Mar 2020 16:46:35 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: One more question where i should add these option in neutron.conf file. I have following to section, should i be adding all those option inside oslo_messaging_rabbit or DEFAULT? also what is the difference between executor_thread_pool_size vs rpc_thread_pool_size ? or they are both samething? [DEFAULT] ... executor_thread_pool_size = 64 rpc_response_timeout = 60 ... ... [oslo_messaging_rabbit] ssl = True rpc_conn_pool_size = 30 On Thu, Mar 19, 2020 at 2:53 PM Satish Patel wrote: > I have question related following setting, why are you disabling > heartbeat timeout? > > heartbeat_timeout_threshold = 0 > > On Thu, Mar 19, 2020 at 1:32 PM Satish Patel wrote: > >> Great, thanks! Did you guys tune your nova component for rabbitMQ? >> >> On Thu, Mar 19, 2020 at 1:26 PM Grant Morley wrote: >> >>> We left ours on the default value of 1 and that still seems to be fine. >>> >>> Grant >>> On 19/03/2020 17:13, Satish Patel wrote: >>> >>> how about rpc_worker ? >>> >>> currently i have rpc_worker=1 >>> >>> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley wrote: >>> >>>> Correct, you need to add: >>>> >>>> > > heartbeat_timeout_threshold = 0 >>>> > > rpc_conn_pool_size = 300 >>>> > > rpc_thread_pool_size = 2048 >>>> > > rpc_response_timeout = 3600 >>>> > > rpc_poll_timeout = 60 >>>> >>>> To your Neutron nodes >>>> >>>> And you can add: >>>> >>>> >>>> >> executor_thread_pool_size = 64 >>>> >> rpc_response_timeout = 3600 >>>> >>>> To your compute nodes (neutron.conf) However I found just adding the >>>> changes to the neturon servers really helped. >>>> >>>> I would recommend just starting with your neutron nodes first to see if >>>> that helps. If you find your compute nodes are still having issues then >>>> change the settings on those after. >>>> >>>> Regards, >>>> On 19/03/2020 16:53, Satish Patel wrote: >>>> >>>> I am running openstack-ansible (Queens / Stein both) so this is what i >>>> am going to do, am i doing correctly? >>>> >>>> neutron-server (container) I have 3 neutron node. >>>> > > heartbeat_timeout_threshold = 0 >>>> > > rpc_conn_pool_size = 300 >>>> > > rpc_thread_pool_size = 2048 >>>> > > rpc_response_timeout = 3600 >>>> > > rpc_poll_timeout = 60 >>>> >>>> 330 compute nodes (agent neutron.conf) going to add following: >>>> >> executor_thread_pool_size = 64 >>>> >> rpc_response_timeout = 3600 >>>> >>>> >>>> >>>> How about nova? should i be doing that on nova as well to reduce load >>>> on rabbitMQ? >>>> >>>> >>>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: >>>> >>>>> Hi Satish, >>>>> >>>>> You will need to add those to the "neutron.conf" file on your network >>>>> nodes. If you are running OS-A I would do it on your "neutron-server" nodes >>>>> and add the following to your agents containers: >>>>> >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> Regards, >>>>> On 19/03/2020 16:27, Satish Patel wrote: >>>>> >>>>> Erik, >>>>> >>>>> If i want to adopt following setting then where i should add them in >>>>> Queens openstack, neutron-server or all my compute nodes? which >>>>> setting will go where? >>>>> >>>>> heartbeat_timeout_threshold = 0 >>>>> rpc_conn_pool_size = 300 >>>>> rpc_thread_pool_size = 2048 >>>>> rpc_response_timeout = 3600 >>>>> rpc_poll_timeout = 60 >>>>> >>>>> ## Rpc all >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: >>>>> >>>>> We are hitting something awfully similar. >>>>> >>>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>>> >>>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>>> >>>>> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>>> >>>>> I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>>> >>>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>>> >>>>> Best Regards, Erik Olof Gunnar Andersson >>>>> >>>>> -----Original Message----- >>>>> From: Satish Patel >>>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>>> To: Grant Morley >>>>> Cc: openstack-discuss at lists.openstack.org >>>>> Subject: Re: Neutron RabbitMQ issues >>>>> >>>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>>> >>>>> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>>> >>>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>> >>>>> Hi all, >>>>> >>>>> We are currently experiencing some fairly major issues with our >>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>> are seeing a lot of time out messages in responses to replies and >>>>> because of this instance creation or anything to do with instances and >>>>> networking is broken. >>>>> >>>>> We are running OpenStack Queens. >>>>> >>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>> >>>>> heartbeat_timeout_threshold = 0 >>>>> rpc_conn_pool_size = 300 >>>>> rpc_thread_pool_size = 2048 >>>>> rpc_response_timeout = 3600 >>>>> rpc_poll_timeout = 60 >>>>> >>>>> ## Rpc all >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> What we are seeing in the error logs for neutron for all services >>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>> 9aOA$ >>>>> >>>>> We have manually tried to get everything in sync by forcing fail-over >>>>> of the networking which seems to get routers in sync. >>>>> >>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>> >>>>> Some times restarting of the services on neutron gets these back >>>>> acknowledged again, however the timeouts come back. >>>>> >>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>> file descriptors and errlang processes have plenty of resources available. >>>>> >>>>> We are also seeing a lot of rpc issues: >>>>> >>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>> before next attempt. If the server is not down, consider increasing >>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>> If the server is not down, consider increasing the >>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>> >>>>> Does anyone know if there is anymore tuning to be done at all? >>>>> Upgrading for us at the moment to a newer version isn't really an >>>>> option unfortunately. >>>>> >>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>> know that will be putting a load on the system. However these problems >>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>> >>>>> If anyone has any use cases for this or any more recommendations that >>>>> would be great. >>>>> >>>>> Many thanks, >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Grant Morley >>>>> Cloud Lead, Civo Ltd >>>>> www.civo.com | Signup for an account! >>>>> >>>> -- >>>> >>>> Grant Morley >>>> Cloud Lead, Civo Ltd >>>> www.civo.com | Signup for an account! >>>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an account! >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From melwittt at gmail.com Thu Mar 19 20:48:45 2020 From: melwittt at gmail.com (melanie witt) Date: Thu, 19 Mar 2020 13:48:45 -0700 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <1584638630.12170.41@est.tech> References: <1584638630.12170.41@est.tech> Message-ID: On 3/19/20 10:23, Balázs Gibizer wrote: > Hi, > > Nova has an unwritten rule that requires to have at least two companies > involved in any new feature development (or even bugfix?). In the > current Nova core diversity situation this rule puts extra burden to the > remaining non Red Hat cores and I guess it also makes any Red Hat driven > feature development harder. In parallel to working on increasing the > size of the core team I suggest to reconsider dropping this rule. I have historically always appreciated the two-company approval rule because it ensures each patch has been reviewed with at least two different and diverse company perspectives. However I recognize that the situation has changed over time and we may not have the luxury any longer to ensure diverse company approvals. I do not wish to overburden the remaining non Red Hat cores with review requests for approvals. So, given the evolved landscape, I am understanding and open to adapt our process to drop the two-company rule if there is such consensus in the rest of the team. Cheers, -melanie > Some discussion happened already on the today's meeting[1]. > > Cheers, > gibi > > > [1] > http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 > > > > > From sean.mcginnis at gmail.com Thu Mar 19 22:07:41 2020 From: sean.mcginnis at gmail.com (Sean McGinnis) Date: Thu, 19 Mar 2020 17:07:41 -0500 Subject: [Release-job-failures] Tag of openstack/glance_store for ref refs/tags/rocky-em failed In-Reply-To: References: Message-ID: <12a81cd1-0b5f-eb20-942b-4c3000d1cf59@gmail.com> On 3/19/20 4:56 PM, zuul at openstack.org wrote: > Build failed. > > - publish-openstack-releasenotes-python3 https://zuul.opendev.org/t/openstack/build/66b1173250724747bc9b7630277a5b1a : POST_FAILURE in 4m 24s > > _______________________________________________ > Release-job-failures mailing list > Release-job-failures at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures This failure appears to be from a temporary glitch that we've seen time to time in the gate: "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" This was on publishing release notes. Since no new release notes have been added as part of this (just tagging rocky-em) it is safe to ignore this failure. Sean From hongbin034 at gmail.com Thu Mar 19 22:21:32 2020 From: hongbin034 at gmail.com (Hongbin Lu) Date: Thu, 19 Mar 2020 18:21:32 -0400 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> Message-ID: On Thu, Mar 19, 2020 at 11:52 AM Rodolfo Alonso wrote: > Hello all: > > With this mail I would like to propose the goal to move away from > oslo.rootwrap and migrate to > oslo.privsep. The last one offers a superior security model, faster and > more secure. During the last > cycles and since privsep was released, the Community has encouraged the > usage of privsep and the > deprecation of any existing code still using rootwrap. > > For any developer willing to collaborate, there are plenty of code > examples, as I’ll provide later, > implementing and using privsep for new methods and migrations. > > If this goal is approved, I'll open a Story ( > https://storyboard.openstack.org/) and any developer > will be able to add a task for each patch or set of them related. This > would be the tracker for this > common effort. > > > PROJECTS TO MIGRATE. > -------------------- > Projects that are still using rootwrap: > http://codesearch.openstack.org/?q=rootwrap&i=nope&files=.*.py&repos= > neutron > os-brick > designate > cinder > ironic-inspector > neutron-vpnaas > nova > solum > glance_store > ironic > zun > Zun adopted privsep as early as two years ago: https://review.opendev.org/#/c/511430/ . > magnum > manila > networking-bagpipe > sahara > ceilometer > cinderlib > freezer > ironic-lib > monasca-agent > tacker > tripleo-common > > > USAGE DOCUMENTATION ABOUT PRIVSEP. > ---------------------------------- > How to create a privsep context, assign privileges and use it as a > decorator: > https://docs.openstack.org/oslo.privsep/latest/user/index.html > > > HOW TO MIGRATE FROM ROOTWRAP TO PRIVSEP. > ---------------------------------------- > From the same link provided previously, in the section “Converting from > rootwrap to privsep”: > > https://docs.openstack.org/oslo.privsep/latest/user/index.html#converting-from-rootwrap-to-privsep > > oslo.privsep provides a class, PrivContext, that can be used to create a > decorator function. The > instance created is a context of execution and has defined a list of > capabilities, matching the > Linux capabilities. The privsep context decorator should contain the > minimum needed capabilities to > execute the decorated function. > > For example: > > default = priv_context.PrivContext( > __name__, > cfg_section='privsep', > pypath=__name__ + '.default', > capabilities=[caps.CAP_SYS_ADMIN, > caps.CAP_NET_ADMIN, > caps.CAP_DAC_OVERRIDE, > caps.CAP_DAC_READ_SEARCH, > caps.CAP_SYS_PTRACE], > ) > > > The function “entrypoint” of this instance can be used as a decorator for > another function: > > @privileged.default.entrypoint > def delete_interface(ifname, namespace, **kwargs): > _run_iproute_link("del", ifname, namespace, **kwargs) > > > As commented in the given link, a straight 1:1 filter:function replacement > generally results in > functions that are still too broad for good security. It is better to > replace each chmod rootwrap > call with a narrow privsep function that will limit it to specific files. > > > MIGRATION EXAMPLES. > ------------------- > Nova: > > https://review.opendev.org/#/q/project:openstack/nova+branch:master+topic:my-own-personal-alternative-universe > Neutron: > > https://review.opendev.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bug/1492714 > os-vif > : > https://review.opendev.org/#/c/287725/ > > > Thank you and regards. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Mar 19 22:22:23 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 22:22:23 +0000 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: <8a7e12c0d63e7144f2ce3424e6d703d1a3a40d9c.camel@redhat.com> References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> <8a7e12c0d63e7144f2ce3424e6d703d1a3a40d9c.camel@redhat.com> Message-ID: <3cdc43f3417209d7c7c9de95b27d31978cd0b3c5.camel@redhat.com> On Thu, 2020-03-19 at 16:04 +0000, Sean Mooney wrote: > On Thu, 2020-03-19 at 16:50 +0100, Iury Gregory wrote: > > Hi Rodolfo, > > > > Thanks for raising this topic. I like the idea and I can work in the ironic > > part. > > > > Em qui., 19 de mar. de 2020 às 16:47, Rodolfo Alonso > > escreveu: > > > > > Hello all: > > > > > > With this mail I would like to propose the goal to move away from > > > oslo.rootwrap and migrate to > > > oslo.privsep. The last one offers a superior security model, faster and > > > more secure. During the last > > > cycles and since privsep was released, the Community has encouraged the > > > usage of privsep and the > > > deprecation of any existing code still using rootwrap. i had a rather long rant about why privsep is only more secure if you use it correctly i would say that today most project that use privsep are not as secure as you think they are or as secure as you would like them to be. i would say that the privsep guide also is not entirely correct as it advocate groping privileged function in a singel module and gives an example of a context that has two many capabilities. we have the unfortunate situation that in nova and neuton a singel privsep context is used for all privadge function meaning it has the some of all posible capablites that any code path can require. that is an anti pattern. also we group privaldage funcition in a /privsep/... module which encurages function reuse which encurages widing the contract to make them more resuable which again is an anti patteren. this lead to function like nova.privsep.path.writefile(path, mode, data). which litally allows you to write to any file on the system with any mod and put any data. that breaks all the rules. first it has a wide contract by takeing both the path and mode. second its not got _privaladged in the fucntion name so we have to have a hacking check to prevent you doing "from nova.privsep import fs; ... fs.writefile(...)" without realising its privaldaged. thrid its grouped in privsep module with encurage it to be used form may part of the code base require int the wide contract, and finally becasue nova only has one privsep context capabilities=[capabilities.CAP_CHOWN, capabilities.CAP_DAC_OVERRIDE, capabilities.CAP_DAC_READ_SEARCH, capabilities.CAP_FOWNER, capabilities.CAP_NET_ADMIN, capabilities.CAP_SYS_ADMIN], instead of just the file capablies it also has cap_net_admin and cap_sys_admin so it can mess with sysfs and so other thing that would normaly be prevented if it only had the capablites required for the specalised fucntion it callse writefile. for exampel create a vm conosle log.... this has come up in the past on the mainlist and at at least two ptgs http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004358.html so in parallel to this project that already use privsep might need to have a second goal of use privsep properly. p.s. yes this is the less ranty version fo this mail :) i do think this is a good goal. > > > > > > For any developer willing to collaborate, there are plenty of code > > > examples, as I’ll provide later, > > > implementing and using privsep for new methods and migrations. > > > > > > If this goal is approved, I'll open a Story ( > > > https://storyboard.openstack.org/) and any developer > > > will be able to add a task for each patch or set of them related. This > > > would be the tracker for this > > > common effort. > > > > > > > > > PROJECTS TO MIGRATE. > > > -------------------- > > > Projects that are still using rootwrap: > > > http://codesearch.openstack.org/?q=rootwrap&i=nope&files=.*.py&repos= > > > neutron > > > os-brick > > > designate > > > cinder > > > ironic-inspector > > > neutron-vpnaas > > > nova > > nova does not use rootwarp anymore. so it came up on irc that nova still has one use of rootwrap which is for os-bricks. os-brick itself actully does nto use rootwap directly as of newton https://github.com/openstack/os-brick/commit/dbf77fba1061cb4e93b3db5f8117d6ccc689f702#diff-0d141267b46cdfd7a9dfe6100d79fe33 but instead rely on the calling agent (nova or cinder) to pass in a root_helper string with is passed to oslo_concurrency.processutils.execute in effect this is only used to spawn the privsep deamon process. out side of test code os-brick only usage of root_helper is when set via nova or cinder and all prviladge ecalation happens by calling a single function in os_brick.rootwap which does not use rootwap unless root_helper is set to use rootwap. so execute in there rootwrap module https://github.com/openstack/os-brick/blob/9649f17228203186b523e400080a300f28b7e6ff/os_brick/privileged/rootwrap.py#L163 only elevets to root via calling execute_root which gains privladges via privsep https://github.com/openstack/os-brick/blob/9649f17228203186b523e400080a300f28b7e6ff/os_brick/privileged/rootwrap.py#L190-L194 so while os-brick shoudl definetly be looked at to make the usage of privsep actully more secure so that it does not simple pass eveythin to a single function that forad args to process utils, they dont actully need to do anyting to drop rootwrap. instead os_brick need to work on the second potential goal of use privsep correctly. i personally think the use of rootwap in nova for os-bircks provide almost no security benifit and would suggest we should remove it. once os-brick passes the command to execute it will be dispatch to the privesep process via the unix socket which will spawn a shell using oslo_concurrancy.process_utils.execute with CAP_SYS_ADMIN privaldges and no root wrap filtering will be applied at that point. > > > solum > > > glance_store > > > ironic > > > zun > > > magnum > > > manila > > > networking-bagpipe > > > sahara > > > ceilometer > > > cinderlib > > > freezer > > > ironic-lib > > > monasca-agent > > > tacker > > > tripleo-common > > > > > > > > > USAGE DOCUMENTATION ABOUT PRIVSEP. > > > ---------------------------------- > > > How to create a privsep context, assign privileges and use it as a > > > decorator: > > > https://docs.openstack.org/oslo.privsep/latest/user/index.html > > > > > > > > > HOW TO MIGRATE FROM ROOTWRAP TO PRIVSEP. > > > ---------------------------------------- > > > From the same link provided previously, in the section “Converting from > > > rootwrap to privsep”: > > > > > > https://docs.openstack.org/oslo.privsep/latest/user/index.html#converting-from-rootwrap-to-privsep > > > > > > oslo.privsep provides a class, PrivContext, that can be used to create a > > > decorator function. The > > > instance created is a context of execution and has defined a list of > > > capabilities, matching the > > > Linux capabilities. The privsep context decorator should contain the > > > minimum needed capabilities to > > > execute the decorated function. > > > > > > For example: > > > > > > default = priv_context.PrivContext( > > > __name__, > > > cfg_section='privsep', > > > pypath=__name__ + '.default', > > > capabilities=[caps.CAP_SYS_ADMIN, > > > caps.CAP_NET_ADMIN, > > > caps.CAP_DAC_OVERRIDE, > > > caps.CAP_DAC_READ_SEARCH, > > > caps.CAP_SYS_PTRACE], > > > ) actully i didnt comment on this before but ^ is a terable example. we should use several smllaer context and not one big one with multple capablities. that one is form neutron and should really be 4 context one for cap_sys_admin one for cap_net_admin one for file access ( caps.CAP_DAC_OVERRIDE, caps.CAP_DAC_READ_SEARCH,) and one for ptrace so if people are converting project please done creat large context like the one above crate small ones that only have the caps a specific function reuqires. > > > > > > > > > The function “entrypoint” of this instance can be used as a decorator for > > > another function: > > > > > > @privileged.default.entrypoint > > > def delete_interface(ifname, namespace, **kwargs): > > > _run_iproute_link("del", ifname, namespace, **kwargs) > > > > > > > > > As commented in the given link, a straight 1:1 filter:function replacement > > > generally results in > > > functions that are still too broad for good security. It is better to > > > replace each chmod rootwrap > > > call with a narrow privsep function that will limit it to specific files. yep and that also means that you shoudl avoid groupign them in a /privspec/ folder and instead put the spealised funtion int he module that uses it to avoid the tempation to make overly broad itnerfaces. the function should also be sufixed with _privileged so that when its called its imediatly obvious this function has elevated privldatees e.g. delete_interface_privileged not delete_interface > > > > > > > > > MIGRATION EXAMPLES. > > > ------------------- > > > Nova: > > > > > > https://review.opendev.org/#/q/project:openstack/nova+branch:master+topic:my-own-personal-alternative-universe > > > Neutron: > > > > > > https://review.opendev.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bug/1492714 > > > os-vif > > > : > > > https://review.opendev.org/#/c/287725/ > > > as i alluded too above you can look at nova os-vif and neuton for inpseration but may dont follow everythin they do blindly. os-vif is proaly closest to what i woudl think is correct usage but we have not been fully consitent with idea that all privileged function should have _privileged in the name. we do however use multiple privsep contexts, one per plugin, we only need CAP_NET_ADMIN so we dont have multiple context per plugin for different capablities. os-vif also does not use rootwap to launch the privesep deamon which is what nova is doing with os-bricks. os-vif does that iself and you can alter it via config opitn in the nova.conf which we do in our fucntional tests by setting the helper_command flag in the os_vif_privileged section dynamically https://github.com/openstack/os-vif/blob/d5b61d10655b3b7a9d32378b5f7bcdb06479e15d/os_vif/tests/functional/base.py#L117-L122 but you could change this in the nova.conf if you wanted to inject the use of rootwap or any other pviladge escalation mechanisum to futer restict the privsep deamon. this makes use of the fact that every time you creat a privsep context you can specify a config secition for it to read privsep parmater form such as https://github.com/openstack/os-vif/blob/master/os_vif/tests/functional/privsep.py#L18 whe we defien the os_vif_privileged we we can set any of the privsep libs standard config itmes such as helper_command https://github.com/openstack/oslo.privsep/blob/e896ed39c4b2d8492bb91283d99c6446bacc657c/oslo_privsep/priv_context.py#L59-L67 this is how nova and os-bicks and other project should configure and spawan privsep deamons instead of using rootwap to do that. os-vif isnt perfect either but it is easy to fool your self an think privsep will do more fo you then it is if you blindly follow the example form the privsep usage docs without thinking through each step and what shoudl be the responsibility of libs or clients and where your configuration shoudl reside. e.g. nova/cinder should not be respocnisble for securign os-brick > > > > > > Thank you and regards. > > > > > > > > > > > > > > > > > > From cboylan at sapwetik.org Thu Mar 19 22:43:58 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 19 Mar 2020 15:43:58 -0700 Subject: [nova][gate] status of some gate bugs In-Reply-To: <20200319082548.GL29109@sync> References: <6119b4f1-a235-b1b5-93e5-eb5d23e2e0fa@gmail.com> <20200319082548.GL29109@sync> Message-ID: On Thu, Mar 19, 2020, at 1:25 AM, Arnaud Morin wrote: > > Hey Melanie, all, > > About OVH case (company I work for). > We are digging into the issue. > > First thing, we do not limit anymore the IOPS. I dont remember when we > removed this limit, but this is not new. > > However, the hypervisor are quite old now, and our policy on this old > servers was to use some swap. > And we think that the host may slow down when overcommitting on RAM > (swapping on disk). > > Anyway, we also know that we can have better latency when upgrading > QEMU. We are currently in the middle of testing a new QEMU version, I > will push to upgrade your hypervisors first, so we will see if the > latency on QEMU side can help the gate. > > Finally, we plan to change the hardware and stop doing overcommit on RAM > (and swapping on disk). However, I have no ETA about that, but for sure, > this will improve the IOPS. You all likely know far more about this than I do, but our use case is likely ideal for kernel same page merging because we boot a relatively small number of identical images that rotate relatively slowly (24 hours). Turning that on, if not already, could potentially reduce memory pressure. > > I'll keep you in touch. > > Cheers, > > -- > Arnaud Morin > From openstack at nemebean.com Thu Mar 19 23:04:37 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 19 Mar 2020 18:04:37 -0500 Subject: [openstack.org] SEO Improvements In-Reply-To: <20200314142029.lf55ksds2avnzdbx@yuggoth.org> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> <20200314142029.lf55ksds2avnzdbx@yuggoth.org> Message-ID: <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> On 3/14/20 9:20 AM, Jeremy Stanley wrote: > On 2020-03-14 14:05:32 +0000 (+0000), Sean Mooney wrote: >> On Sat, 2020-03-14 at 09:36 -0400, Donny Davis wrote: > [...] >>> It would be really great if when you click the current release >>> is X button at the top of the page it would reload the same doc >>> and just replace the release instead of directing you back to >>> the home page of the doc release. > [...] >> yes althought that is not a SEO thing i think we need to change >> how that baner is create but i basically always ignore it because >> it does not do what i want so removing it entirly or more helpflly >> making it link to the latest verion of the current doc would both >> be greate improvemnts. > > The solution to the technical problem is straightforward (even a > simple HTML form element would to the trick), the bigger challenge > is logistical: Content moves around between releases, it's not just > renamed but often gets split or combined too, and lots of times > folks don't think to include redirects from old filenames when they > make major edits like that. All too often when I'm looking something > up in the published HTML copies of our documentation I'll land on > the wrong version, edit the URL to a later release, and get a 404 > for that file. Someone (really multiple someones) will need to do a > thorough audit to work out what redirects we missed between previous > releases as well as coming up with ways to better remind folks to > add redirects and maybe leave other breadcrumb trails when making > future changes. > I just ran into this and I think there's a middle ground that would still help a lot. Right now if you click on the banner to go to the latest release it redirects you back to the main docs.openstack.org page. Could it at least redirect you to the root of the project you were already looking at? I realize that might break on project renames, but that seems like a massively smaller problem to solve than trying to keep track of internal structural changes in each project's docs. From openstack at fried.cc Thu Mar 19 23:10:01 2020 From: openstack at fried.cc (Eric Fried) Date: Thu, 19 Mar 2020 18:10:01 -0500 Subject: [openstack.org] SEO Improvements In-Reply-To: <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> <20200314142029.lf55ksds2avnzdbx@yuggoth.org> <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> Message-ID: <7272a8ff-798b-5741-0422-6383cefd725b@fried.cc> > Could it at least redirect you to the root of the project you were > already looking at? I realize that might break on project renames, but > that seems like a massively smaller problem to solve than trying to keep > track of internal structural changes in each project's docs. Slightly heavier-weight, but potentially much more useful, can't the link do some js for you (either on page load or when you click it) to probe, in order: - The page you were looking for, s/$release/latest/ - failing that, the root of the project, as Ben suggests - failing that, the current behavior efried . From fungi at yuggoth.org Thu Mar 19 23:22:27 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 19 Mar 2020 23:22:27 +0000 Subject: [openstack.org] SEO Improvements In-Reply-To: <7272a8ff-798b-5741-0422-6383cefd725b@fried.cc> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> <20200314142029.lf55ksds2avnzdbx@yuggoth.org> <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> <7272a8ff-798b-5741-0422-6383cefd725b@fried.cc> Message-ID: <20200319232227.4pr5iowae36vsv7t@yuggoth.org> On 2020-03-19 18:10:01 -0500 (-0500), Eric Fried wrote: > > Could it at least redirect you to the root of the project you were > > already looking at? I realize that might break on project renames, but > > that seems like a massively smaller problem to solve than trying to keep > > track of internal structural changes in each project's docs. > > Slightly heavier-weight, but potentially much more useful, can't the link do > some js for you (either on page load or when you click it) to probe, in > order: > > - The page you were looking for, s/$release/latest/ > - failing that, the root of the project, as Ben suggests > - failing that, the current behavior I'm pretty sure Apache .htaccess rules could manage that without any client-side scripting. I know it has ways to define conditional redirect behavior depending on whether specific files exist or not. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From openstack at nemebean.com Thu Mar 19 23:24:39 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 19 Mar 2020 18:24:39 -0500 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: <3cdc43f3417209d7c7c9de95b27d31978cd0b3c5.camel@redhat.com> References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> <8a7e12c0d63e7144f2ce3424e6d703d1a3a40d9c.camel@redhat.com> <3cdc43f3417209d7c7c9de95b27d31978cd0b3c5.camel@redhat.com> Message-ID: On 3/19/20 5:22 PM, Sean Mooney wrote: > On Thu, 2020-03-19 at 16:04 +0000, Sean Mooney wrote: >> On Thu, 2020-03-19 at 16:50 +0100, Iury Gregory wrote: >>> Hi Rodolfo, >>> >>> Thanks for raising this topic. I like the idea and I can work in the ironic >>> part. >>> >>> Em qui., 19 de mar. de 2020 às 16:47, Rodolfo Alonso >>> escreveu: >>> >>>> Hello all: >>>> >>>> With this mail I would like to propose the goal to move away from >>>> oslo.rootwrap and migrate to >>>> oslo.privsep. The last one offers a superior security model, faster and >>>> more secure. During the last >>>> cycles and since privsep was released, the Community has encouraged the >>>> usage of privsep and the >>>> deprecation of any existing code still using rootwrap. > i had a rather long rant about why privsep is only more secure if you use it correctly > i would say that today most project that use privsep are not as secure as you think they > are or as secure as you would like them to be. > > i would say that the privsep guide also is not entirely correct as it advocate > groping privileged function in a singel module and gives an example of a context that > has two many capabilities. > > we have the unfortunate situation that in nova and neuton a singel privsep context is used > for all privadge function meaning it has the some of all posible capablites that any code > path can require. that is an anti pattern. also we group privaldage funcition > in a /privsep/... module which encurages function reuse which encurages widing the > contract to make them more resuable which again is an anti patteren. Wasn't there a reason for them to all be in one module though? Like a hacking check to verify that privileged functions were used safely? You're right about widening the contract for reuse being bad, but it seems to me that could happen wherever the code lives. A lot of the problems with the Nova implementation are that it is (or at least was) a straight port of the rootwrap calls, so in some ways it just exposed the insecurity of the rootwrap design too. > > this lead to function like nova.privsep.path.writefile(path, mode, data). > which litally allows you to write to any file on the system with any mod and put any data. > that breaks all the rules. first it has a wide contract by takeing both the path and mode. > second its not got _privaladged in the fucntion name so we have to have a hacking check to prevent > you doing "from nova.privsep import fs; ... fs.writefile(...)" without realising its privaldaged. Ah, this is what I was thinking of. I guess you either use a naming convention for all privileged functions or put them in a single/few modules that you can run hacking checks on. I've meant to see if we could pull the Nova hacking check into privsep so other projects could use it. Either approach has its benefits and drawbacks. We could probably mention both in the privsep user docs and just say that projects should be consistent in which they use. > thrid its grouped in privsep module with encurage it to be used form may part of the code base require > int the wide contract, and finally becasue nova only has one privsep context > capabilities=[capabilities.CAP_CHOWN, > capabilities.CAP_DAC_OVERRIDE, > capabilities.CAP_DAC_READ_SEARCH, > capabilities.CAP_FOWNER, > capabilities.CAP_NET_ADMIN, > capabilities.CAP_SYS_ADMIN], > > instead of just the file capablies it also has cap_net_admin and cap_sys_admin so it can mess > with sysfs and so other thing that would normaly be prevented if it only had the capablites required > for the specalised fucntion it callse writefile. for exampel create a vm conosle log.... > > this has come up in the past on the mainlist and at at least two ptgs > http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004358.html Yeah, I think it has been acknowledged that the Nova use of privsep is not ideal. I do want to note that the use of general-purpose privileged functions like writefile is specifically discouraged in multiple places in the privsep docs. That was a lesson learned from the initial Nova migration. I believe privsep already supports multiple contexts, so maybe the docs need to be expanded to include that as a best practice. From smooney at redhat.com Thu Mar 19 23:26:52 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 23:26:52 +0000 Subject: [openstack.org] SEO Improvements In-Reply-To: <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> <20200314142029.lf55ksds2avnzdbx@yuggoth.org> <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> Message-ID: <4d51df8000e9c5ff2b9c53d76403615c15620d16.camel@redhat.com> On Thu, 2020-03-19 at 18:04 -0500, Ben Nemec wrote: > > On 3/14/20 9:20 AM, Jeremy Stanley wrote: > > On 2020-03-14 14:05:32 +0000 (+0000), Sean Mooney wrote: > > > On Sat, 2020-03-14 at 09:36 -0400, Donny Davis wrote: > > > > [...] > > > > It would be really great if when you click the current release > > > > is X button at the top of the page it would reload the same doc > > > > and just replace the release instead of directing you back to > > > > the home page of the doc release. > > > > [...] > > > yes althought that is not a SEO thing i think we need to change > > > how that baner is create but i basically always ignore it because > > > it does not do what i want so removing it entirly or more helpflly > > > making it link to the latest verion of the current doc would both > > > be greate improvemnts. > > > > The solution to the technical problem is straightforward (even a > > simple HTML form element would to the trick), the bigger challenge > > is logistical: Content moves around between releases, it's not just > > renamed but often gets split or combined too, and lots of times > > folks don't think to include redirects from old filenames when they > > make major edits like that. All too often when I'm looking something > > up in the published HTML copies of our documentation I'll land on > > the wrong version, edit the URL to a later release, and get a 404 > > for that file. Someone (really multiple someones) will need to do a > > thorough audit to work out what redirects we missed between previous > > releases as well as coming up with ways to better remind folks to > > add redirects and maybe leave other breadcrumb trails when making > > future changes. > > > > I just ran into this and I think there's a middle ground that would > still help a lot. Right now if you click on the banner to go to the > latest release it redirects you back to the main docs.openstack.org > page. Could it at least redirect you to the root of the project you were > already looking at? I realize that might break on project renames, but > that seems like a massively smaller problem to solve than trying to keep > track of internal structural changes in each project's docs. that would be an improvment. medium term regardign doc resturcing i was wondering if there was a way to add a metadta tag fo some form to a page like a uuid that would live with that doument enve if it was renamed that we could link too instead? e.g. in each doc add a .. unique_id : and then create a directory at the top that would have a fill per uuid which just contains a redirect to the relevent file. so the baner would link to ${base_domain}//latest/reirects/${uuid} which would be a file genereated by sphinx that redirect to the file corresponding to ${uuid} > From smooney at redhat.com Thu Mar 19 23:55:04 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 19 Mar 2020 23:55:04 +0000 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> <8a7e12c0d63e7144f2ce3424e6d703d1a3a40d9c.camel@redhat.com> <3cdc43f3417209d7c7c9de95b27d31978cd0b3c5.camel@redhat.com> Message-ID: <14f7be40df5af528ce852dfc9235672b28bd7dbf.camel@redhat.com> On Thu, 2020-03-19 at 18:24 -0500, Ben Nemec wrote: > > On 3/19/20 5:22 PM, Sean Mooney wrote: > > On Thu, 2020-03-19 at 16:04 +0000, Sean Mooney wrote: > > > On Thu, 2020-03-19 at 16:50 +0100, Iury Gregory wrote: > > > > Hi Rodolfo, > > > > > > > > Thanks for raising this topic. I like the idea and I can work in the ironic > > > > part. > > > > > > > > Em qui., 19 de mar. de 2020 às 16:47, Rodolfo Alonso > > > > escreveu: > > > > > > > > > Hello all: > > > > > > > > > > With this mail I would like to propose the goal to move away from > > > > > oslo.rootwrap and migrate to > > > > > oslo.privsep. The last one offers a superior security model, faster and > > > > > more secure. During the last > > > > > cycles and since privsep was released, the Community has encouraged the > > > > > usage of privsep and the > > > > > deprecation of any existing code still using rootwrap. > > > > i had a rather long rant about why privsep is only more secure if you use it correctly > > i would say that today most project that use privsep are not as secure as you think they > > are or as secure as you would like them to be. > > > > i would say that the privsep guide also is not entirely correct as it advocate > > groping privileged function in a singel module and gives an example of a context that > > has two many capabilities. > > > > we have the unfortunate situation that in nova and neuton a singel privsep context is used > > for all privadge function meaning it has the some of all posible capablites that any code > > path can require. that is an anti pattern. also we group privaldage funcition > > in a /privsep/... module which encurages function reuse which encurages widing the > > contract to make them more resuable which again is an anti patteren. > > Wasn't there a reason for them to all be in one module though? Like a > hacking check to verify that privileged functions were used safely? > > You're right about widening the contract for reuse being bad, but it > seems to me that could happen wherever the code lives. A lot of the > problems with the Nova implementation are that it is (or at least was) a > straight port of the rootwrap calls, so in some ways it just exposed the > insecurity of the rootwrap design too. > > > > > this lead to function like nova.privsep.path.writefile(path, mode, data). > > which litally allows you to write to any file on the system with any mod and put any data. > > that breaks all the rules. first it has a wide contract by takeing both the path and mode. > > second its not got _privaladged in the fucntion name so we have to have a hacking check to prevent > > you doing "from nova.privsep import fs; ... fs.writefile(...)" without realising its privaldaged. > > Ah, this is what I was thinking of. actully i think what you were thinking of was the prefix https://github.com/openstack/nova/blob/master/nova/privsep/__init__.py#L22 which is used in the entrypoint decorator https://github.com/openstack/oslo.privsep/blob/e896ed39c4b2d8492bb91283d99c6446bacc657c/oslo_privsep/priv_context.py#L217 to ensure that the decorate can not be used out side of the module or sub module where the context is defiend. so in the nova case we do sys_admin_pctxt = priv_context.PrivContext( 'nova', cfg_section='nova_sys_admin', pypath=__name__ + '.sys_admin_pctxt', capabilities=[capabilities.CAP_CHOWN, capabilities.CAP_DAC_OVERRIDE, capabilities.CAP_DAC_READ_SEARCH, capabilities.CAP_FOWNER, capabilities.CAP_NET_ADMIN, capabilities.CAP_SYS_ADMIN], ) and teh first parmater 'nova' ensure that any function that use the decorator must be part of nova e.g. this will not work --- ~/my_project/my_module.py--- from nova import privsep @privsep.sys_admin_pctxt.entrypoint def my_func(): print("hi") in nova we technically could use 'nova.privsep' to prevent any other moduel in nova form containg privladged funcitons based on that context. in the os-vif ovs plugin we do vif_plug = priv_context.PrivContext( "vif_plug_ovs", cfg_section="vif_plug_ovs_privileged", pypath=__name__ + ".vif_plug", capabilities=[c.CAP_NET_ADMIN], ) so in the linxu bridge plugin you could not incorrectly use the ovs context because they are not in the same submodule. > I guess you either use a naming > convention for all privileged functions or put them in a single/few > modules that you can run hacking checks on. I've meant to see if we > could pull the Nova hacking check into privsep so other projects could > use it. > > Either approach has its benefits and drawbacks. We could probably > mention both in the privsep user docs and just say that projects should > be consistent in which they use. line lenght/typing is the main drawback for the suffix/nameing convention but yes both approchs work i obviously have a bias to the one i prefer but since i have not gone through and change all of os-vif to follow the suffix approch i dont think its worth needless code change just to conform to it. > > > thrid its grouped in privsep module with encurage it to be used form may part of the code base require > > int the wide contract, and finally becasue nova only has one privsep context > > capabilities=[capabilities.CAP_CHOWN, > > capabilities.CAP_DAC_OVERRIDE, > > capabilities.CAP_DAC_READ_SEARCH, > > capabilities.CAP_FOWNER, > > capabilities.CAP_NET_ADMIN, > > capabilities.CAP_SYS_ADMIN], > > > > instead of just the file capablies it also has cap_net_admin and cap_sys_admin so it can mess > > with sysfs and so other thing that would normaly be prevented if it only had the capablites required > > for the specalised fucntion it callse writefile. for exampel create a vm conosle log.... > > > > this has come up in the past on the mainlist and at at least two ptgs > > http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004358.html > > Yeah, I think it has been acknowledged that the Nova use of privsep is > not ideal. I do want to note that the use of general-purpose privileged > functions like writefile is specifically discouraged in multiple places > in the privsep docs. That was a lesson learned from the initial Nova > migration. > > I believe privsep already supports multiple contexts, so maybe the docs > need to be expanded to include that as a best practice. ya you can. os-vif is always using two contexts. we have 1 context per plugin an one for the root of the libary. strictly speakign you dont need to do that but we wanted to use the in tree plugins as a template for creating your own and in general each plug should have its own security context but normally you would only have muliple contexts if you they had differnet capabliies. since each context mean an addtional unix socket and demon you don twant to have hundres of them but nova should proably have 3 looking at the set of capablites ie uses. to use multiple context after they are defiend you just need to use the correct decorator so its really easy to work with. > From jimmy at openstack.org Fri Mar 20 00:06:03 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Thu, 19 Mar 2020 19:06:03 -0500 Subject: [openstack.org] SEO Improvements In-Reply-To: <20200319232227.4pr5iowae36vsv7t@yuggoth.org> References: <5E6BECCE.2070404@openstack.org> <5E6BF120.9040103@openstack.org> <20200313213341.ivhagnfygtfce5vm@yuggoth.org> <5E6BFFE4.8090905@openstack.org> <4ee649f4f3faea617c362090e284ac404412b311.camel@redhat.com> <20200314142029.lf55ksds2avnzdbx@yuggoth.org> <6904af1e-f20f-d132-331d-c331c90d4f07@nemebean.com> <7272a8ff-798b-5741-0422-6383cefd725b@fried.cc> <20200319232227.4pr5iowae36vsv7t@yuggoth.org> Message-ID: <5E7408EB.3080209@openstack.org> > Jeremy Stanley > March 19, 2020 at 6:22 PM > > I'm pretty sure Apache .htaccess rules could manage that without any > client-side scripting. I know it has ways to define conditional > redirect behavior depending on whether specific files exist or not. Yep. I think you could do this with RegEx and maybe enhance it with some javascript. Adding Sebastian and JP to this thread to hopefully catch up :) > Eric Fried > March 19, 2020 at 6:10 PM > > Slightly heavier-weight, but potentially much more useful, can't the > link do some js for you (either on page load or when you click it) to > probe, in order: > > - The page you were looking for, s/$release/latest/ > - failing that, the root of the project, as Ben suggests > - failing that, the current behavior > > efried > . > > Ben Nemec > March 19, 2020 at 6:04 PM > > > > > I just ran into this and I think there's a middle ground that would > still help a lot. Right now if you click on the banner to go to the > latest release it redirects you back to the main docs.openstack.org > page. Could it at least redirect you to the root of the project you > were already looking at? I realize that might break on project > renames, but that seems like a massively smaller problem to solve than > trying to keep track of internal structural changes in each project's > docs. > > Jeremy Stanley > March 14, 2020 at 9:20 AM > On 2020-03-14 14:05:32 +0000 (+0000), Sean Mooney wrote: >> On Sat, 2020-03-14 at 09:36 -0400, Donny Davis wrote: > [...] >>> It would be really great if when you click the current release >>> is X button at the top of the page it would reload the same doc >>> and just replace the release instead of directing you back to >>> the home page of the doc release. > [...] >> yes althought that is not a SEO thing i think we need to change >> how that baner is create but i basically always ignore it because >> it does not do what i want so removing it entirly or more helpflly >> making it link to the latest verion of the current doc would both >> be greate improvemnts. > > The solution to the technical problem is straightforward (even a > simple HTML form element would to the trick), the bigger challenge > is logistical: Content moves around between releases, it's not just > renamed but often gets split or combined too, and lots of times > folks don't think to include redirects from old filenames when they > make major edits like that. All too often when I'm looking something > up in the published HTML copies of our documentation I'll land on > the wrong version, edit the URL to a later release, and get a 404 > for that file. Someone (really multiple someones) will need to do a > thorough audit to work out what redirects we missed between previous > releases as well as coming up with ways to better remind folks to > add redirects and maybe leave other breadcrumb trails when making > future changes. > Sean Mooney > March 14, 2020 at 9:05 AM > On Sat, 2020-03-14 at 09:36 -0400, Donny Davis wrote: >> On Fri, Mar 13, 2020 at 5:53 PM Jimmy McArthur wrote: >> >>> Jeremy Stanley >>> March 13, 2020 at 4:33 PM >>> On 2020-03-13 20:59:30 +0000 (+0000), Sean Mooney wrote: >>> [...] >>> >>> Yep, this trips me up fairly often as well. The pattern of serving >>> multiple versions of documentation is a fairly widespread one, far >>> beyond just OpenStack circles, so maybe we should look at some other >>> examples and see if we can reverse engineer how they manage to >>> direct search results to their latest versions. For example, why do >>> searches for Python module names return results under >>> https://docs.python.org/3/ before they return results for >>> https://docs.python.org/3.5/ ? I briefly skimmed the page sources >>> for some examples but nothing jumped out at me, nor did the site's >>> robots.txt provide any insight. Perhaps SEO specialists know what >>> trick is at play there? >>> >>> I will add it to my list of questions. Normally you'd do it with >>> redirects to the latest, but that doesn't help if you're trying to keep >>> archived documentation. >>> >>> Sean Mooney >>> March 13, 2020 at 3:59 PM >>> >>> On Fri, 2020-03-13 at 15:46 -0500, Jimmy McArthur wrote: >>> >>> Sorry - I accidentally left Zuul keywords and examples in there. Fixed >>> below: >>> >>> >>> Jimmy McArthur >>> March 13, 2020 at 3:27 PM >>> Hi all - >>> >>> We've contracted a professional SEO firm to help improve search >>> placement for all OSF projects. I'd like to crowd source this on each >>> of the project mailing lists, so the community is able to be >>> involved. Could you all help out in providing the below: >>> >>> “Wish List” of keywords: 8-12 big terms you think fit your domain (if >>> you only have 4-5 to share, fewer is okay) >>> - open infrastructure >>> - ? >>> >>> At least 3 competitors >>> - AWS >>> - ? >>> >>> Any other input on positioning, offerings, etc. that you think will >>> help best filter for relevance >>> - ? >>> >>> honestly the only think i would like to see is fixing the search result so that the 'latest' version of all our doc >>> are at the top of the list instead of pike. >>> the docs for our older release always come up first and its hard fo fine the direct link to the 'latest' version >>> which >>> tracks master. >>> >>> granted i have it save in my broswer history but when users are looking for docs on things it would be nice if they >>> got >>> the more recent docs. >>> >>> Cheers, >>> Jimmy >>> >>> >>> Jimmy McArthur >>> March 13, 2020 at 3:46 PM >>> Sorry - I accidentally left Zuul keywords and examples in there. Fixed >>> below: >>> >>> >>> Jimmy McArthur >>> March 13, 2020 at 3:27 PM >>> Hi all - >>> >>> We've contracted a professional SEO firm to help improve search placement >>> for all OSF projects. I'd like to crowd source this on each of the project >>> mailing lists, so the community is able to be involved. Could you all help >>> out in providing the below: >>> >>> “Wish List” of keywords: 8-12 big terms you think fit your domain (if you >>> only have 4-5 to share, fewer is okay) >>> - open source ci >>> - ? >>> >>> At least 3 competitors >>> - Jenkins >>> - ? >>> >>> Any other input on positioning, offerings, etc. that you think will help >>> best filter for relevance >>> - ? >>> >>> Cheers, >>> Jimmy >>> >>> >>> >> It would be really great if when you click the current release is X button >> at the top of the page it would reload the same doc and just replace the >> release instead of directing you back to the home page of the doc release. > yes althought that is not a SEO thing i think we need to change how that baner is create but > i basically always ignore it because it does not do what i want so removing it entirly or more > helpflly making it link to the latest verion of the current doc would both be greate improvemnts. >> [image: image.png] > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnsomor at gmail.com Fri Mar 20 01:42:48 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Thu, 19 Mar 2020 18:42:48 -0700 Subject: [octavia] dropping support for stable/queens In-Reply-To: References: Message-ID: Hi Erik, The Queens branch entered Extended Maintenance (EM) on August 25th, 2019 (later extended to October 25th, 2019 to accommodate release team resources[1]) and was tagged EM on November 6th, 2019. This proposal is to exit Extended Maintenance and mark the Queens release as End-of-Life(EOL). This is following the guidelines for stable branch releases[2]. Currently the Octavia team is maintaining five branches and supporting queens is becoming more difficult as the test jobs are still based on the Ubuntu Xenial release (released April 21. 2016). The python 2.7 end-of-life combined with the older OS version has made it hard to keep reliable test jobs[3]. Backporting patches is also becoming more difficult as the Octavia code base has rapidly evolved in the two years since it was released. If you or others are willing to help us with fixing the queens jobs and backporting patches, that would be great and we may be able to leave the queens branch in Extended Maintenance. If not, Adam is proposing we move it into EOL. Personally, I agree with Adam that it is time to mark the Queens release as EOL. Since you can run newer releases of Octavia on older clouds (as many do), the branch cannot release under EM, and patches on that branch are few. Michael [1] https://github.com/openstack/releases/commit/be797b61677b1eafc0162bd664f09db54287ac81#diff-8fa0a5ef8165646609d2f3d50646c597 [2] https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-life [3] https://zuul.openstack.org/builds?project=openstack%2Foctavia&branch=stable%2Fqueens&pipeline=check&job_name=octavia-v2-dsvm-scenario On Thu, Mar 19, 2020 at 11:40 AM Erik McCormick wrote: > > > On Thu, Mar 19, 2020, 11:57 AM Adam Harwell wrote: >> >> I'm officially proposing dropping support for the queens stable branch of Octavia. >> Given a quorum of cores, we will move forward with marking it EOL and all that entails. > > > Adam, > > Why can't it go into EM Instead of EOL? > > Thanks > Erik > >> >> Thanks, >> --Adam From grant at civo.com Fri Mar 20 08:55:55 2020 From: grant at civo.com (Grant Morley) Date: Fri, 20 Mar 2020 08:55:55 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: Hi, There was a bug in Queens that meant there was an issue with the heartbeat timeouts. Setting it to 0 gets around that bug. I believe that was fixed in Rocky and above, so your Stein installation should be fine. Setting the value to 0 For us meant we stopped getting errors in the logs for: "Too many heartbeats missed, trying to force connect to RabbitMQ" Regards, On 19/03/2020 18:53, Satish Patel wrote: > I have question related following setting, why are you disabling > heartbeat timeout? > > heartbeat_timeout_threshold = 0 > > On Thu, Mar 19, 2020 at 1:32 PM Satish Patel > wrote: > > Great, thanks!  Did you guys tune your nova component for rabbitMQ? > > On Thu, Mar 19, 2020 at 1:26 PM Grant Morley > wrote: > > We left ours on the default value of 1 and that still seems to > be fine. > > Grant > > On 19/03/2020 17:13, Satish Patel wrote: >> how about rpc_worker ? >> currently i have rpc_worker=1 >> >> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley > > wrote: >> >> Correct, you need to add: >> >> > > heartbeat_timeout_threshold = 0 >> > > rpc_conn_pool_size = 300 >> > > rpc_thread_pool_size = 2048 >> > > rpc_response_timeout = 3600 >> > > rpc_poll_timeout = 60 >> >> To your Neutron nodes >> >> And you can add: >> >> >> >> executor_thread_pool_size = 64 >> >> rpc_response_timeout = 3600 >> >> To your compute nodes (neutron.conf) However I found just >> adding the changes to the neturon servers really helped. >> >> I would recommend just starting with your neutron nodes >> first to see if that helps. If you find your compute >> nodes are still having issues then change the settings on >> those after. >> >> Regards, >> >> On 19/03/2020 16:53, Satish Patel wrote: >>> I am running openstack-ansible (Queens / Stein both) so >>> this is what i am going to do, am i doing correctly? >>> >>> neutron-server (container) I have 3 neutron node. >>> > > heartbeat_timeout_threshold = 0 >>> > > rpc_conn_pool_size = 300 >>> > > rpc_thread_pool_size = 2048 >>> > > rpc_response_timeout = 3600 >>> > > rpc_poll_timeout = 60 >>> >>> 330 compute nodes (agent neutron.conf) going to add >>> following: >>> >> executor_thread_pool_size = 64 >>> >> rpc_response_timeout = 3600 >>> >>> >>> >>> How about nova? should i be doing that on nova as well >>> to reduce load on rabbitMQ? >>> >>> >>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley >>> > wrote: >>> >>> Hi Satish, >>> >>> You will need to add those to the "neutron.conf" >>> file on your network nodes. If you are running OS-A >>> I would do it on your "neutron-server" nodes and add >>> the following to your agents containers: >>> >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> Regards, >>> >>> On 19/03/2020 16:27, Satish Patel wrote: >>>> Erik, >>>> >>>> If i want to adopt following setting then where i should add them in >>>> Queens openstack, neutron-server or all my compute nodes? which >>>> setting will go where? >>>> >>>> heartbeat_timeout_threshold = 0 >>>> rpc_conn_pool_size = 300 >>>> rpc_thread_pool_size = 2048 >>>> rpc_response_timeout = 3600 >>>> rpc_poll_timeout = 60 >>>> >>>> ## Rpc all >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson >>>> wrote: >>>>> We are hitting something awfully similar. >>>>> >>>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>>> >>>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>>> >>>>> e.g.https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>>> >>>>> I wrote two quick scripts to audit these issues. >>>>> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >>>>> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>>> >>>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>>> >>>>> Best Regards, Erik Olof Gunnar Andersson >>>>> >>>>> -----Original Message----- >>>>> From: Satish Patel >>>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>>> To: Grant Morley >>>>> Cc:openstack-discuss at lists.openstack.org >>>>> Subject: Re: Neutron RabbitMQ issues >>>>> >>>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>>> >>>>> This is my favorite video, not sure you have seen this before or not but anyway posting here -https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>>> >>>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>>> Hi all, >>>>>> >>>>>> We are currently experiencing some fairly major issues with our >>>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>>> are seeing a lot of time out messages in responses to replies and >>>>>> because of this instance creation or anything to do with instances and >>>>>> networking is broken. >>>>>> >>>>>> We are running OpenStack Queens. >>>>>> >>>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>>> >>>>>> heartbeat_timeout_threshold = 0 >>>>>> rpc_conn_pool_size = 300 >>>>>> rpc_thread_pool_size = 2048 >>>>>> rpc_response_timeout = 3600 >>>>>> rpc_poll_timeout = 60 >>>>>> >>>>>> ## Rpc all >>>>>> executor_thread_pool_size = 64 >>>>>> rpc_response_timeout = 3600 >>>>>> >>>>>> What we are seeing in the error logs for neutron for all services >>>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>>> >>>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>>> 9aOA$ >>>>>> >>>>>> We have manually tried to get everything in sync by forcing fail-over >>>>>> of the networking which seems to get routers in sync. >>>>>> >>>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>>> >>>>>> Some times restarting of the services on neutron gets these back >>>>>> acknowledged again, however the timeouts come back. >>>>>> >>>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>>> file descriptors and errlang processes have plenty of resources available. >>>>>> >>>>>> We are also seeing a lot of rpc issues: >>>>>> >>>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>>> before next attempt. If the server is not down, consider increasing >>>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>>> If the server is not down, consider increasing the >>>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>>> >>>>>> Does anyone know if there is anymore tuning to be done at all? >>>>>> Upgrading for us at the moment to a newer version isn't really an >>>>>> option unfortunately. >>>>>> >>>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>>> know that will be putting a load on the system. However these problems >>>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>>> >>>>>> If anyone has any use cases for this or any more recommendations that >>>>>> would be great. >>>>>> >>>>>> Many thanks, >>>>>> >>>>>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an >>> account! >>> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an >> account! >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > > -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From grant at civo.com Fri Mar 20 08:56:49 2020 From: grant at civo.com (Grant Morley) Date: Fri, 20 Mar 2020 08:56:49 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: <0a157dd9-1e3e-e406-4e5c-5b745162470a@civo.com> Hi, We personally did not tune Nova for RabbitMQ,  we found that it was doing a good enough job for us. Grant On 19/03/2020 17:32, Satish Patel wrote: > Great, thanks!  Did you guys tune your nova component for rabbitMQ? > > On Thu, Mar 19, 2020 at 1:26 PM Grant Morley > wrote: > > We left ours on the default value of 1 and that still seems to be > fine. > > Grant > > On 19/03/2020 17:13, Satish Patel wrote: >> how about rpc_worker ? >> currently i have rpc_worker=1 >> >> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley > > wrote: >> >> Correct, you need to add: >> >> > > heartbeat_timeout_threshold = 0 >> > > rpc_conn_pool_size = 300 >> > > rpc_thread_pool_size = 2048 >> > > rpc_response_timeout = 3600 >> > > rpc_poll_timeout = 60 >> >> To your Neutron nodes >> >> And you can add: >> >> >> >> executor_thread_pool_size = 64 >> >> rpc_response_timeout = 3600 >> >> To your compute nodes (neutron.conf) However I found just >> adding the changes to the neturon servers really helped. >> >> I would recommend just starting with your neutron nodes first >> to see if that helps. If you find your compute nodes are >> still having issues then change the settings on those after. >> >> Regards, >> >> On 19/03/2020 16:53, Satish Patel wrote: >>> I am running openstack-ansible (Queens / Stein both) so this >>> is what i am going to do, am i doing correctly? >>> >>> neutron-server (container) I have 3 neutron node. >>> > > heartbeat_timeout_threshold = 0 >>> > > rpc_conn_pool_size = 300 >>> > > rpc_thread_pool_size = 2048 >>> > > rpc_response_timeout = 3600 >>> > > rpc_poll_timeout = 60 >>> >>> 330 compute nodes (agent neutron.conf) going to add following: >>> >> executor_thread_pool_size = 64 >>> >> rpc_response_timeout = 3600 >>> >>> >>> >>> How about nova? should i be doing that on nova as well to >>> reduce load on rabbitMQ? >>> >>> >>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley >>> > wrote: >>> >>> Hi Satish, >>> >>> You will need to add those to the "neutron.conf" file on >>> your network nodes. If you are running OS-A I would do >>> it on your "neutron-server" nodes and add the following >>> to your agents containers: >>> >>> executor_thread_pool_size = 64 >>> rpc_response_timeout = 3600 >>> >>> Regards, >>> >>> On 19/03/2020 16:27, Satish Patel wrote: >>>> Erik, >>>> >>>> If i want to adopt following setting then where i should add them in >>>> Queens openstack, neutron-server or all my compute nodes? which >>>> setting will go where? >>>> >>>> heartbeat_timeout_threshold = 0 >>>> rpc_conn_pool_size = 300 >>>> rpc_thread_pool_size = 2048 >>>> rpc_response_timeout = 3600 >>>> rpc_poll_timeout = 60 >>>> >>>> ## Rpc all >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson >>>> wrote: >>>>> We are hitting something awfully similar. >>>>> >>>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>>> >>>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>>> >>>>> e.g.https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>>> >>>>> I wrote two quick scripts to audit these issues. >>>>> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >>>>> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>>> >>>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>>> >>>>> Best Regards, Erik Olof Gunnar Andersson >>>>> >>>>> -----Original Message----- >>>>> From: Satish Patel >>>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>>> To: Grant Morley >>>>> Cc:openstack-discuss at lists.openstack.org >>>>> Subject: Re: Neutron RabbitMQ issues >>>>> >>>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>>> >>>>> This is my favorite video, not sure you have seen this before or not but anyway posting here -https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>>> >>>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>>> Hi all, >>>>>> >>>>>> We are currently experiencing some fairly major issues with our >>>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>>> are seeing a lot of time out messages in responses to replies and >>>>>> because of this instance creation or anything to do with instances and >>>>>> networking is broken. >>>>>> >>>>>> We are running OpenStack Queens. >>>>>> >>>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>>> >>>>>> heartbeat_timeout_threshold = 0 >>>>>> rpc_conn_pool_size = 300 >>>>>> rpc_thread_pool_size = 2048 >>>>>> rpc_response_timeout = 3600 >>>>>> rpc_poll_timeout = 60 >>>>>> >>>>>> ## Rpc all >>>>>> executor_thread_pool_size = 64 >>>>>> rpc_response_timeout = 3600 >>>>>> >>>>>> What we are seeing in the error logs for neutron for all services >>>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>>> >>>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>>> 9aOA$ >>>>>> >>>>>> We have manually tried to get everything in sync by forcing fail-over >>>>>> of the networking which seems to get routers in sync. >>>>>> >>>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>>> >>>>>> Some times restarting of the services on neutron gets these back >>>>>> acknowledged again, however the timeouts come back. >>>>>> >>>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>>> file descriptors and errlang processes have plenty of resources available. >>>>>> >>>>>> We are also seeing a lot of rpc issues: >>>>>> >>>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>>> before next attempt. If the server is not down, consider increasing >>>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>>> If the server is not down, consider increasing the >>>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>>> >>>>>> Does anyone know if there is anymore tuning to be done at all? >>>>>> Upgrading for us at the moment to a newer version isn't really an >>>>>> option unfortunately. >>>>>> >>>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>>> know that will be putting a load on the system. However these problems >>>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>>> >>>>>> If anyone has any use cases for this or any more recommendations that >>>>>> would be great. >>>>>> >>>>>> Many thanks, >>>>>> >>>>>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an >>> account! >>> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an account! >> >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > > -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyarwood at redhat.com Fri Mar 20 10:33:42 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Fri, 20 Mar 2020 10:33:42 +0000 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <1584638630.12170.41@est.tech> References: <1584638630.12170.41@est.tech> Message-ID: <20200320103342.xrjzlnv46ba5d2mw@lyarwood.usersys.redhat.com> On 19-03-20 18:23:50, Balázs Gibizer wrote: > Hi, > > Nova has an unwritten rule that requires to have at least two companies > involved in any new feature development (or even bugfix?). In the current > Nova core diversity situation this rule puts extra burden to the remaining > non Red Hat cores and I guess it also makes any Red Hat driven feature > development harder. In parallel to working on increasing the size of the > core team I suggest to reconsider dropping this rule. > > Some discussion happened already on the today's meeting[1]. > > Cheers, > gibi > > [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 Thanks again for raising this gibi! +1 from me, while this topic has been discussed at length many times now I agree that given the current core diversity situation we need to look at this again. I would personally be happy to see the rule dropped for the time being while in parallel we also try to increase the pool of cores from outside of RH. Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From skaplons at redhat.com Fri Mar 20 11:11:07 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 20 Mar 2020 12:11:07 +0100 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> References: <1584638630.12170.41@est.tech> <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> Message-ID: <20200320111107.bx7ubpmnzda6phep@skaplons-mac> Hi, On Thu, Mar 19, 2020 at 02:06:51PM -0500, Monty Taylor wrote: > > > > On Mar 19, 2020, at 12:23 PM, Balázs Gibizer wrote: > > > > Hi, > > > > Nova has an unwritten rule that requires to have at least two companies involved in any new feature development (or even bugfix?). In the current Nova core diversity situation this rule puts extra burden to the remaining non Red Hat cores and I guess it also makes any Red Hat driven feature development harder. In parallel to working on increasing the size of the core team I suggest to reconsider dropping this rule. > > > > Some discussion happened already on the today's meeting[1] > > I think that’s a great idea. FWIW I’ve never liked that rule, because it assume that developers from a company are putting employer requirements over project requirements when acting in their capacity as a core reviewer - which is contrary to our general expectation of how a core reviewer behaves. We are doing it exactly like that in Neutron and IMHO it works pretty good so far :) > > I think it’s a great idea to get rid of this policy - and then if anyone is behaving in a manner that abuses the trust of the rest of the core reviewers, such as slamming through a new feature that other people obviously have misgivings about … that can be dealt with the same way any other breach of trust would happen. > > > Cheers, > > gibi > > > > > > [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 > > > > > > > > > > > > -- Slawek Kaplonski Senior software engineer Red Hat From ignaziocassano at gmail.com Fri Mar 20 11:21:56 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 20 Mar 2020 12:21:56 +0100 Subject: [queens][cinder][unity] iscsi driver Message-ID: Hello All, I would like to suggest to modify the unity iscsi cinder driver documentation. After 1 month of intensive test, we discovered the mutipath.conf needs the following option in multipath.conf kvm nodes: skip_kpartx yes Without the above option instances with more than 2 volumes attached could fail live migration because the os_brick code fails in "multipath -f" command reporting map in use error. Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From aj at suse.com Fri Mar 20 13:43:19 2020 From: aj at suse.com (Andreas Jaeger) Date: Fri, 20 Mar 2020 14:43:19 +0100 Subject: [all] Re: review.opendev.org Downtime March 20, 2020 at 1400UTC In-Reply-To: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> References: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> Message-ID: <2e3e4b3a-9a48-545e-eb85-be6b8613d090@suse.com> A friendly reminder: In a few minutes review.opendev.org will be completely down as explained below, Andreas On 3/17/20 10:08 PM, Clark Boylan wrote: > Hello all, > > The OpenDev team is planning to take a short downtime March 20, 2020 from 14:00UTC to 15:00UTC. This outage will enable us to rename a few projects and redeploy Gerrit using containers. The container based deployment is an important first step in our longer term plan for upgrading Gerrit to a modern version. > > We recognize that this will be a disruption during an already disrupted week for many. We feel this is worthwhile as it fits into project release schedules and sets us up well for the future. Thank you for your patience. > > Feel free to ask us any questions you may have on this thread or in #opendev on Freenode. > > Clark > > -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From stephenfin at redhat.com Fri Mar 20 14:04:57 2020 From: stephenfin at redhat.com (Stephen Finucane) Date: Fri, 20 Mar 2020 14:04:57 +0000 Subject: [oslo][nova][vmware] Replacement for suds library Message-ID: The suds-jurko library used by oslo.vmware is emitting the following warnings in nova tests. /nova/.tox/py36/lib/python3.6/site-packages/suds/resolver.py:89: DeprecationWarning: invalid escape sequence \% self.splitp = re.compile('({.+})*[^\%s]+' % ps[0]) /nova/.tox/py36/lib/python3.6/site-packages/suds/wsdl.py:619: DeprecationWarning: invalid escape sequence \s body.parts = re.split('[\s,]', parts) These warnings are going to be errors in Python 3.10 [1]. We have over 18 months before we need to worry about this [2], but I'd like to see some movement on this well before then. It seems the suds-jurko fork is dead [3] and movement on yet another fork, suds-community, is slow [4]. How difficult would it be to switch over to something that does seem maintained like zeep [5] and, assuming it's doable, is anyone from VMWare/SAP able to do this work? Stephen [1] https://docs.python.org/3.9/whatsnew/3.9.html#you-should-check-for-deprecationwarning-in-your-code [2] https://www.python.org/dev/peps/pep-0596/ [3] https://bitbucket.org/jurko/suds/src/default/ [4] https://github.com/suds-community/suds/pull/32 [5] https://python-zeep.readthedocs.io/en/master/ From smooney at redhat.com Fri Mar 20 14:14:57 2020 From: smooney at redhat.com (Sean Mooney) Date: Fri, 20 Mar 2020 14:14:57 +0000 Subject: [all] Re: review.opendev.org Downtime March 20, 2020 at 1400UTC In-Reply-To: <2e3e4b3a-9a48-545e-eb85-be6b8613d090@suse.com> References: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> <2e3e4b3a-9a48-545e-eb85-be6b8613d090@suse.com> Message-ID: <97152a82a566ee7be2e5cd8386c65d79093a00af.camel@redhat.com> On Fri, 2020-03-20 at 14:43 +0100, Andreas Jaeger wrote: > A friendly reminder: In a few minutes review.opendev.org will be > completely down as explained below, on a related note i noticed i was getting different contenent from https://review.openstack.org/p/openstack/nova and ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git last week. these were my remotes in my nove repo gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (fetch) gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (push) origin https://review.openstack.org/p/openstack/nova (fetch) origin https://review.openstack.org/p/openstack/nova (push) the gerrit remote when i did a fetch was up to date with https://opendev.org/openstack/nova and with https://github.com/openstack/nova but https://review.openstack.org/p/openstack/nova was behind both of them by a few hours maybe a day i dont rememeber but i just remember tinking it was odd. is this somethign ye were aware of that could happen? it was if the redirect from review.openstack.org was pointing an out of sync gerrit backend. i know i should use the updated urls but this is an old copy of nova form before the switch so just said i would mention this incase anyone else hits this issue. > > Andreas > > On 3/17/20 10:08 PM, Clark Boylan wrote: > > Hello all, > > > > The OpenDev team is planning to take a short downtime March 20, 2020 from 14:00UTC to 15:00UTC. This outage will > > enable us to rename a few projects and redeploy Gerrit using containers. The container based deployment is an > > important first step in our longer term plan for upgrading Gerrit to a modern version. > > > > We recognize that this will be a disruption during an already disrupted week for many. We feel this is worthwhile as > > it fits into project release schedules and sets us up well for the future. Thank you for your patience. > > > > Feel free to ask us any questions you may have on this thread or in #opendev on Freenode. > > > > Clark > > > > > > From rosmaita.fossdev at gmail.com Fri Mar 20 14:15:22 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Fri, 20 Mar 2020 10:15:22 -0400 Subject: [cinder][qa] propose Liang Fang for devstack-plugin-open-cas-core Message-ID: Liang Fang is driving the effort to create the devstack-plugin-open-cas and use it to test the cinder/nova volume-local-cache feature with tempest. Since he's our SME on open-cas, I propose that he be made a member of devstack-plugin-open-cas-core. (Currently, the only members of that group are the members of cinder-core and devstack-core.) In the absence of objections, I'll make the change before the next Cinder team meeting (Wednesday, 25 March at 1400 UTC in its new location of #openstack-meeting-alt). cheers, brian From sean.mcginnis at gmx.com Fri Mar 20 14:22:31 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Fri, 20 Mar 2020 09:22:31 -0500 Subject: [release] Release countdown for week R-7, March 23-27 Message-ID: <0d20ad3c-5571-6c76-a00e-8f13b043ca85@gmx.com> Development Focus ----------------- We are entering the last weeks of the Ussuri development cycle. From now until the final release, we'll send a countdown email like this every week. It's probably a good time for teams to take stock of their library and client work that needs to be completed yet. The non-client library freeze is coming up, followed closely by the client lib freeze. Please plan accordingly so avoid any last minute rushes to get key functionality in. General Information ------------------- Next week is the Extra-ATC freeze, in preparation for elections. All contributions to OpenStack are valuable, but some are not expressed as Gerrit code changes. Please list active contributors to your project team who do not have a code contribution this cycle, and therefore won't automatically be considered an Active Technical Contributor and allowed to vote. This is done by adding extra-atcs to https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml before the Extra-ATC freeze on March 26. A quick reminder of the upcoming freeze dates. Those vary depending on deliverable type: * General libraries (except client libraries) need to have their last feature release before Non-client library freeze (April 2). Their stable branches are cut early. * Client libraries (think python-*client libraries) need to have their last feature release before Client library freeze (April 9) * Deliverables following a cycle-with-rc model (that would be most services) observe a Feature freeze on that same date, April 9. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. After feature freeze, cycle-with-rc deliverables need to produce a first release candidate (and a stable branch) before RC1 deadline (April 23) * Deliverables following cycle-with-intermediary model can release as necessary, but in all cases before Final RC deadline (May 7) Finally, now is also a good time to start planning what highlights you want for your deliverables in the cycle highlights. The deadline to submit an initial version for those is set to Feature freeze (April 9). Background on cycle-highlights: http://lists.openstack.org/pipermail/openstack-dev/2017-December/125613.html Project Team Guide, Cycle-Highlights: https://docs.openstack.org/project-team-guide/release-management.html#cycle-highlights knelson [at] openstack.org/diablo_rojo on IRC is available if you need help selecting or writing your highlights Note that cycle highlights are per-team, not per-deliverable. Upcoming Deadlines & Dates -------------------------- Extra-ATC freeze: March 26 (R-7 week) Non-client library freeze: April 2 (R-6 week) Client library freeze: April 9 (R-5 week) Ussuri-3 milestone: April 9 (R-5 week) Cycle Highlights Due: April 9 (R-5 week) -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri Mar 20 14:28:17 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 20 Mar 2020 14:28:17 +0000 Subject: [all] Re: review.opendev.org Downtime March 20, 2020 at 1400UTC In-Reply-To: <97152a82a566ee7be2e5cd8386c65d79093a00af.camel@redhat.com> References: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> <2e3e4b3a-9a48-545e-eb85-be6b8613d090@suse.com> <97152a82a566ee7be2e5cd8386c65d79093a00af.camel@redhat.com> Message-ID: <20200320142816.h2pd4n3fv6nefz4v@yuggoth.org> On 2020-03-20 14:14:57 +0000 (+0000), Sean Mooney wrote: > on a related note i noticed i was getting different contenent from > https://review.openstack.org/p/openstack/nova and > ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git > last week. > > these were my remotes in my nove repo > gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (fetch) > gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (push) > origin https://review.openstack.org/p/openstack/nova (fetch) > origin https://review.openstack.org/p/openstack/nova (push) I definitely don't recommend cloning from review.o.o, better to rely on https://opendev.org/openstack/nova in your origin remotes. > the gerrit remote when i did a fetch was up to date with > https://opendev.org/openstack/nova and with > https://github.com/openstack/nova but > https://review.openstack.org/p/openstack/nova was behind both of > them by a few hours maybe a day i dont rememeber but i just > remember tinking it was odd. That https://review.openstack.org/p/openstack/nova URL is "just another replica" like opendev.org and github.com (the /p is directed to a local copy Gerrit is replicating to a local copy on the server's filesystem). > is this somethign ye were aware of that could happen? it was if > the redirect from review.openstack.org was pointing an out of sync > gerrit backend. [...] Gerrit can't guarantee consistency for its replication tasks, so it can certainly happen. I think we had plans to remove that /p replica anyway (it's going to potentially conflict with some URLs for newer Gerrit versions), but generally if you notice a discrepancy between ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git and https://opendev.org/openstack/nova do let us know, we've been trying to get on top of a race condition in our Gitea updates where Gerrit will silently fail to replicate refs to some of the servers while they're in the middle of restarting. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From skaplons at redhat.com Fri Mar 20 14:37:49 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 20 Mar 2020 15:37:49 +0100 Subject: [neutron] How to fix break of connectivity in case of L3 HA after reboot of backup node Message-ID: <20200320143749.qmi3ucfh43vao3wp@skaplons-mac> Hi, We have bug [1] to solve. Basically, when node which is backup node for some router, connectivity to external gateway may be broken for some time. It's like that because when host is up and L3 agent is configuring qrouter namespace, it flush all IPv6 addresses from the qg- interface. And due to that some MLDv2 packets are sent from this interface which may cause that ToR switch will learn qg port's mac address on wrong (backup) node. This don't happens every time and for every router because it is a race between L3 agent and OVS agent. When L3 agent creates qg interface in br-int, it sets tag 4095 for it and traffic sent with such vlan tag is always dropped in br-int. So if L3 agent will flush IPv6 addresses before OVS agent wires the port and sets correct tag for it, then all is fine. But if OVS agent is first, then MLDv2 packets are sent to the wire and there is this connectivity break. There are proposed 2 ways of fixing this: - [2] which propsoes to add some kind of "communication" between L3 agent and OVS agent and tell OVS agent that tag can be changed only after IPv6 config is finished by L3 agent. Downside of this solution is that it works for OVS agent only, Linuxbridge agent may still hit the same issue. But plus is that after initial configuration of the router, everything else regarding to failover is handled by keepalived only - in same way like it is now. - [3] which sets qg NIC to be DOWN always on backup nodes. So when keepalived failover active router to new node, L3 agent needs to come and switch interfaces to be UP before it will work. The plus of this solution is that it works for all OVS and Linuxbridge L2 agents (and probably for others too) but downside is that failover process is a bit longer and there may be potentially another race condition between L3 agent and keepalived. Keepalived tries to sent gARP packets after switch node to be active, first attempt will always fail as interface is still DOWN. But keepalived will retry those gARPs after some time and this should be fine if L3 agent will already bring interface to be UP. Both patches are waiting for pretty long time in gerrit and I want to bring more visibility for both of them. Please check them and maybe You will have some opinions about which solution would be better and which we should go with. [1] https://bugs.launchpad.net/neutron/+bug/1859832 [2] https://review.opendev.org/#/c/702856/ [3] https://review.opendev.org/#/c/707406/ -- Slawek Kaplonski Senior software engineer Red Hat From smooney at redhat.com Fri Mar 20 14:48:27 2020 From: smooney at redhat.com (Sean Mooney) Date: Fri, 20 Mar 2020 14:48:27 +0000 Subject: [all] Re: review.opendev.org Downtime March 20, 2020 at 1400UTC In-Reply-To: <20200320142816.h2pd4n3fv6nefz4v@yuggoth.org> References: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> <2e3e4b3a-9a48-545e-eb85-be6b8613d090@suse.com> <97152a82a566ee7be2e5cd8386c65d79093a00af.camel@redhat.com> <20200320142816.h2pd4n3fv6nefz4v@yuggoth.org> Message-ID: On Fri, 2020-03-20 at 14:28 +0000, Jeremy Stanley wrote: > On 2020-03-20 14:14:57 +0000 (+0000), Sean Mooney wrote: > > on a related note i noticed i was getting different contenent from > > https://review.openstack.org/p/openstack/nova and > > ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git > > last week. > > > > these were my remotes in my nove repo > > gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (fetch) > > gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (push) > > origin https://review.openstack.org/p/openstack/nova (fetch) > > origin https://review.openstack.org/p/openstack/nova (push) > > I definitely don't recommend cloning from review.o.o, better to rely > on https://opendev.org/openstack/nova in your origin remotes. ya i normally set it up against either github or https://opendev.org/openstack/nova the review.o.o is much larger and take up way more space on disk so i normally avoid it but i think i origianly create this repo usign gertty for reviews. this was on my local laptop and i normally only use that repo for doing quick fixes. > > > the gerrit remote when i did a fetch was up to date with > > https://opendev.org/openstack/nova and with > > https://github.com/openstack/nova but > > https://review.openstack.org/p/openstack/nova was behind both of > > them by a few hours maybe a day i dont rememeber but i just > > remember tinking it was odd. > > That https://review.openstack.org/p/openstack/nova URL is "just > another replica" like opendev.org and github.com (the /p is directed > to a local copy Gerrit is replicating to a local copy on the > server's filesystem). > > > is this somethign ye were aware of that could happen? it was if > > the redirect from review.openstack.org was pointing an out of sync > > gerrit backend. > > [...] > > Gerrit can't guarantee consistency for its replication tasks, so it > can certainly happen. I think we had plans to remove that /p replica > anyway (it's going to potentially conflict with some URLs for newer > Gerrit versions), but generally if you notice a discrepancy between > ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git > and https://opendev.org/openstack/nova do let us know, we've been > trying to get on top of a race condition in our Gitea updates where > Gerrit will silently fail to replicate refs to some of the servers > while they're in the middle of restarting. ya if i notice an issue between those too ill let you know but in generall they seam to mostly be in sync. From Arkady.Kanevsky at dell.com Fri Mar 20 14:57:44 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Fri, 20 Mar 2020 14:57:44 +0000 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <20200320103342.xrjzlnv46ba5d2mw@lyarwood.usersys.redhat.com> References: <1584638630.12170.41@est.tech> <20200320103342.xrjzlnv46ba5d2mw@lyarwood.usersys.redhat.com> Message-ID: <4e415cf66ea347a5be5de08bb847dd83@AUSX13MPS308.AMER.DELL.COM> Absolutely. Very good rule to follow. -----Original Message----- From: Lee Yarwood Sent: Friday, March 20, 2020 5:34 AM To: Balázs Gibizer Cc: OpenStack Discuss Subject: Re: [nova] feasibility to keep the two-company rule On 19-03-20 18:23:50, Balázs Gibizer wrote: > Hi, > > Nova has an unwritten rule that requires to have at least two > companies involved in any new feature development (or even bugfix?). > In the current Nova core diversity situation this rule puts extra > burden to the remaining non Red Hat cores and I guess it also makes > any Red Hat driven feature development harder. In parallel to working > on increasing the size of the core team I suggest to reconsider dropping this rule. > > Some discussion happened already on the today's meeting[1]. > > Cheers, > gibi > > [1] > http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.0 > 0.log.html#l-90 Thanks again for raising this gibi! +1 from me, while this topic has been discussed at length many times now I agree that given the current core diversity situation we need to look at this again. I would personally be happy to see the rule dropped for the time being while in parallel we also try to increase the pool of cores from outside of RH. Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 From jungleboyj at gmail.com Fri Mar 20 15:20:42 2020 From: jungleboyj at gmail.com (Jay S. Bryant) Date: Fri, 20 Mar 2020 10:20:42 -0500 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: References: <1584638630.12170.41@est.tech> Message-ID: Melanie, Good topic here: On 3/19/2020 3:48 PM, melanie witt wrote: > On 3/19/20 10:23, Balázs Gibizer wrote: >> Hi, >> >> Nova has an unwritten rule that requires to have at least two >> companies involved in any new feature development (or even bugfix?). >> In the current Nova core diversity situation this rule puts extra >> burden to the remaining non Red Hat cores and I guess it also makes >> any Red Hat driven feature development harder. In parallel to working >> on increasing the size of the core team I suggest to reconsider >> dropping this rule. > Cinder has also followed this rule in the past but I think that the enforcement has slipped as our pool of cores has gotten smaller.  I have not seen it cause any issues. > I have historically always appreciated the two-company approval rule > because it ensures each patch has been reviewed with at least two > different and diverse company perspectives. > I think, historically this was also intended to prevent one company from unfairly imposing its will upon the community.  Now that development has slowed and wills are not so strong I think this is less of a concern. > However I recognize that the situation has changed over time and we > may not have the luxury any longer to ensure diverse company > approvals. I do not wish to overburden the remaining non Red Hat cores > with review requests for approvals. > > So, given the evolved landscape, I am understanding and open to adapt > our process to drop the two-company rule if there is such consensus in > the rest of the team. > Support this if there is consensus from the team. > Cheers, > -melanie > >> Some discussion happened already on the today's meeting[1]. >> >> Cheers, >> gibi >> >> >> [1] >> http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 >> >> >> >> >> > > From pramchan at yahoo.com Fri Mar 20 15:37:39 2020 From: pramchan at yahoo.com (prakash RAMCHANDRAN) Date: Fri, 20 Mar 2020 15:37:39 +0000 (UTC) Subject: [Tempest][Restack]Call for help on 2020:06 Certificaion References: <371984283.1451826.1584718659038.ref@mail.yahoo.com> Message-ID: <371984283.1451826.1584718659038@mail.yahoo.com> Hi all, Appreciate all your inputs  in mail and one-on-one  conversations I had with few of you. Like to set proper expectations before we deal with extending our goals. First we must deliver on continuity of Refstack, while we evaluate the value it adds to end users through survey.   I. Every Release - There is a new criterion for Certification - Do we have one for 2020.06? - Who has volunteered or would like to volunteer for this effort? II.  Can we have previous WG Chair or anyone can brief us (by call or by email ) as what the process was and how it was managed. |||. Who were last release OSF Reviewers? Can they volunteer for 2020:06 release definition  for Refstack  - https://opendev.org/openstack/refstack/ VI. Given the circumstances (CoViD-19)  will it be possible to deliver Refstack in June /July 2020? (Need some estimates from TC or others who managed this in past years) V. Is Zun - ?,  Magnum -?  Are both OCI and/or  CNI compliant? - Can they be tested using Tempest and add those two or any other container projects to refstack core? For Survey  (suggest any specifics Questions to  ask- may be few  questions to  - Customer / Vendors/ new feedback  Q1-Value of Certification , Q2. How Often customers ask for it in RFP or sales cycle?, Q3. What additional Tests, process, frequency etc will add value for them?Q4. Do you think we should continue the efforts of Refstack?) Let me know through email and have added this to TC meeting for Thursday April 2.https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting ThanksPrakash Sent from Yahoo Mail on Android -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungleboyj at gmail.com Fri Mar 20 15:38:26 2020 From: jungleboyj at gmail.com (Jay S. Bryant) Date: Fri, 20 Mar 2020 10:38:26 -0500 Subject: [cinder][qa] propose Liang Fang for devstack-plugin-open-cas-core In-Reply-To: References: Message-ID: <81820719-4445-a62c-98f6-5b8d4399c337@gmail.com> +1 Great idea Brian. Thanks! Jay On 3/20/2020 9:15 AM, Brian Rosmaita wrote: > Liang Fang is driving the effort to create the > devstack-plugin-open-cas and use it to test the cinder/nova > volume-local-cache feature with tempest.  Since he's our SME on > open-cas, I propose that he be made a member of > devstack-plugin-open-cas-core.  (Currently, the only members of that > group are the members of cinder-core and devstack-core.) > > In the absence of objections, I'll make the change before the next > Cinder team meeting (Wednesday, 25 March at 1400 UTC in its new > location of #openstack-meeting-alt). > > > cheers, > brian > From whayutin at redhat.com Fri Mar 20 15:51:19 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Fri, 20 Mar 2020 09:51:19 -0600 Subject: [tripleo] launchpad bug policy Message-ID: Greetings, Just a quick refresher on TripleO's bug policy. =================================== Please self-triage the bug: - Set the bug to Triaged - Set an Importance (note: "Critical" is something that prevents any TripleO deployment, use it carefully) - Assign the bug to yourself if you plan to work on it - Set a milestone where you think the bugs can be or needs to be fixed. If any doubt, check with the PTL. - Set one or multiple tags related to the bug. See available tags on https://bugs.launchpad.net/tripleo (right column). If you can't update these informations, it's because you're not member of TripleO in Launchpad group. Please ping the PTL and you'll be added. Please double-check if you provided the minimum of needed information in the description of this report. Nova provides an excellent example of Bug Template. It can be found at: https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate =================================== The following bugs were opened in the last 365 days incorrectly. We are at 60 bugs :( http://dashboard-ci.tripleo.org/d/cockpit/cockpit?panelId=357&fullscreen&orgId=1 If you are on this list, please help me out and self triage. If you have found something critical and want to ensure it's on a watched list, set the "alert" tag. Thanks all!! rabi https://bugs.launchpad.net/tripleo/+bug/1830837 bogdando https://bugs.launchpad.net/tripleo/+bug/1838972 jfrancoa https://bugs.launchpad.net/tripleo/+bug/1846206 bogdando https://bugs.launchpad.net/tripleo/+bug/1868075 adamjr https://bugs.launchpad.net/tripleo/+bug/1831841 anta-nok https://bugs.launchpad.net/tripleo/+bug/1832931 rabi https://bugs.launchpad.net/tripleo/+bug/1834054 waleedm https://bugs.launchpad.net/tripleo/+bug/1836011 priyotoshtsp https://bugs.launchpad.net/tripleo/+bug/1836350 donny-g https://bugs.launchpad.net/tripleo/+bug/1837760 mschuppert https://bugs.launchpad.net/tripleo/+bug/1838272 alee-3 https://bugs.launchpad.net/tripleo/+bug/1838804 gongysh https://bugs.launchpad.net/tripleo/+bug/1839126 asrmarco13 https://bugs.launchpad.net/tripleo/+bug/1839298 pkopec https://bugs.launchpad.net/tripleo/+bug/1840600 jfreud https://bugs.launchpad.net/tripleo/+bug/1840714 sai438 https://bugs.launchpad.net/tripleo/+bug/1841395 sclarke1 https://bugs.launchpad.net/tripleo/+bug/1842988 anta-nok https://bugs.launchpad.net/tripleo/+bug/1844438 cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1845268 bgill https://bugs.launchpad.net/tripleo/+bug/1847289 versus-vv https://bugs.launchpad.net/tripleo/+bug/1848546 sandeepyadav93 https://bugs.launchpad.net/tripleo/+bug/1849668 zodiac https://bugs.launchpad.net/tripleo/+bug/1850080 mschuppert https://bugs.launchpad.net/tripleo/+bug/1850610 alee-3 https://bugs.launchpad.net/tripleo/+bug/1850647 arequipeno https://bugs.launchpad.net/tripleo/+bug/1851503 damani42 https://bugs.launchpad.net/tripleo/+bug/1851618 adamjr https://bugs.launchpad.net/tripleo/+bug/1852509 david-hill-ubisoft https://bugs.launchpad.net/tripleo/+bug/1852762 adamjr https://bugs.launchpad.net/tripleo/+bug/1852893 ksambor https://bugs.launchpad.net/tripleo/+bug/1853000 pandasoubhagya https://bugs.launchpad.net/tripleo/+bug/1853562 kajinamit https://bugs.launchpad.net/tripleo/+bug/1854117 jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856095 mschuppert https://bugs.launchpad.net/tripleo/+bug/1856322 jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856411 shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1856680 hkominos https://bugs.launchpad.net/tripleo/+bug/1856734 hontu https://bugs.launchpad.net/tripleo/+bug/1856921 xarlos https://bugs.launchpad.net/tripleo/+bug/1857003 versus-vv https://bugs.launchpad.net/tripleo/+bug/1857153 bshephar https://bugs.launchpad.net/tripleo/+bug/1858010 shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1859608 cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1860486 denis-karpov https://bugs.launchpad.net/tripleo/+bug/1860528 bshephar https://bugs.launchpad.net/tripleo/+bug/1860609 suzumushi https://bugs.launchpad.net/tripleo/+bug/1861315 ekultails https://bugs.launchpad.net/tripleo/+bug/1862179 mschuppert https://bugs.launchpad.net/tripleo/+bug/1862321 slavonic https://bugs.launchpad.net/tripleo/+bug/1863122 ffernand https://bugs.launchpad.net/tripleo/+bug/1863243 cgoncalves https://bugs.launchpad.net/tripleo/+bug/1863595 larsks https://bugs.launchpad.net/tripleo/+bug/1864232 jbemmel https://bugs.launchpad.net/tripleo/+bug/1864924 kajinamit https://bugs.launchpad.net/tripleo/+bug/1865477 cgoncalves https://bugs.launchpad.net/tripleo/+bug/1867144 prabiegx https://bugs.launchpad.net/tripleo/+bug/1867333 valleedelisle https://bugs.launchpad.net/tripleo/+bug/1867362 amoralej https://bugs.launchpad.net/tripleo/+bug/1867608 -------------- next part -------------- An HTML attachment was scrubbed... URL: From nate.johnston at redhat.com Fri Mar 20 15:57:57 2020 From: nate.johnston at redhat.com (Nate Johnston) Date: Fri, 20 Mar 2020 11:57:57 -0400 Subject: [neutron] How to fix break of connectivity in case of L3 HA after reboot of backup node In-Reply-To: <20200320143749.qmi3ucfh43vao3wp@skaplons-mac> References: <20200320143749.qmi3ucfh43vao3wp@skaplons-mac> Message-ID: <20200320155757.l7pv7r64oua4eu5o@firewall> On Fri, Mar 20, 2020 at 03:37:49PM +0100, Slawek Kaplonski wrote: > Hi, > > We have bug [1] to solve. Basically, when node which is backup node for some > router, connectivity to external gateway may be broken for some time. It's like > that because when host is up and L3 agent is configuring qrouter namespace, it > flush all IPv6 addresses from the qg- interface. And due to that some MLDv2 > packets are sent from this interface which may cause that ToR switch will learn > qg port's mac address on wrong (backup) node. > > This don't happens every time and for every router because it is a race between > L3 agent and OVS agent. When L3 agent creates qg interface in br-int, it sets > tag 4095 for it and traffic sent with such vlan tag is always dropped in br-int. > So if L3 agent will flush IPv6 addresses before OVS agent wires the port and > sets correct tag for it, then all is fine. But if OVS agent is first, then MLDv2 > packets are sent to the wire and there is this connectivity break. > > There are proposed 2 ways of fixing this: > - [2] which propsoes to add some kind of "communication" between L3 agent and > OVS agent and tell OVS agent that tag can be changed only after IPv6 config > is finished by L3 agent. > Downside of this solution is that it works for OVS agent only, Linuxbridge > agent may still hit the same issue. But plus is that after initial > configuration of the router, everything else regarding to failover is handled > by keepalived only - in same way like it is now. > - [3] which sets qg NIC to be DOWN always on backup nodes. So when keepalived > failover active router to new node, L3 agent needs to come and switch > interfaces to be UP before it will work. > The plus of this solution is that it works for all OVS and > Linuxbridge L2 agents (and probably for others too) but downside is that > failover process is a bit longer and there may be potentially another race > condition between L3 agent and keepalived. Keepalived tries to sent gARP > packets after switch node to be active, first attempt will always fail as > interface is still DOWN. But keepalived will retry those gARPs after some > time and this should be fine if L3 agent will already bring interface to be > UP. Personally I find [2] more appealing. I think that if we find many linuxbridge users hitting this issue then we can replicate the solution for linuxbridge at that time, but until then let's not worry about it - the majority of users use OVS. And the gARP timegap for solution #3 to me seems like a possbility for problems or downtime. Nate > Both patches are waiting for pretty long time in gerrit and I want to bring more > visibility for both of them. Please check them and maybe You will have some > opinions about which solution would be better and which we should go with. > > [1] https://bugs.launchpad.net/neutron/+bug/1859832 > [2] https://review.opendev.org/#/c/702856/ > [3] https://review.opendev.org/#/c/707406/ > > -- > Slawek Kaplonski > Senior software engineer > Red Hat > > From eandersson at blizzard.com Fri Mar 20 17:51:54 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Fri, 20 Mar 2020 17:51:54 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: Best Regards, Erik Olof Gunnar Andersson From: Satish Patel Sent: Friday, March 20, 2020 9:23 AM To: Grant Morley Cc: Erik Olof Gunnar Andersson ; openstack-discuss at lists.openstack.org Subject: Re: Neutron RabbitMQ issues Oh you are right here, i have following stuff in my neutron.conf on server # Notifications [oslo_messaging_notifications] driver = messagingv2 topics = notifications transport_url = rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a043f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22:5671//neutron?ssl=1 # Messaging [oslo_messaging_rabbit] rpc_conn_pool_size = 30 ssl = True Following change i am going to made let me know if anything missing. [DEFAULT] executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" rpc_response_timeout = 3600 [oslo_messaging_notifications] topics = notifications driver = noop # Messaging [oslo_messaging_rabbit] rpc_conn_pool_size = 300 heartbeat_timeout_threshold = 0 ssl = True Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. https://bugs.launchpad.net/neutron/+bug/1853071 Best Regards, Erik Andersson Should i be adding this to all my compute nodes also ? On Fri, Mar 20, 2020 at 11:40 AM Grant Morley > wrote: If you tune rabbit then for: heartbeat_timeout_threshold = 0 That should help with the error message you are getting. That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. Are all of the ready messages for "notifications.info" for the various services ( Nova, Neutron, Keystone etc ) If that is the case you can disable those messages in your config files for those services. Look for: # Notifications [oslo_messaging_notifications] notification_topics = notifications driver = noop Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. Hope that -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Fri Mar 20 18:18:33 2020 From: satish.txt at gmail.com (Satish Patel) Date: Fri, 20 Mar 2020 14:18:33 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: Erik, That is good finding i have check on following file /openstack/venvs/neutron-19.0.0.0rc3.dev6/lib/python2.7/site-packages/neutron/objects/agent.py Do you think i should add following option and restart neutron-server? is this for all compute nodes agent or just for server? new_facade = True On Fri, Mar 20, 2020 at 1:51 PM Erik Olof Gunnar Andersson wrote: > > > > > > Best Regards, Erik Olof Gunnar Andersson > > > > From: Satish Patel > Sent: Friday, March 20, 2020 9:23 AM > To: Grant Morley > Cc: Erik Olof Gunnar Andersson ; openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > > > Oh you are right here, i have following stuff in my neutron.conf on server > > > > # Notifications > [oslo_messaging_notifications] > driver = messagingv2 > topics = notifications > transport_url = rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a043f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22:5671//neutron?ssl=1 > > # Messaging > [oslo_messaging_rabbit] > rpc_conn_pool_size = 30 > ssl = True > > > > > > Following change i am going to made let me know if anything missing. > > > > [DEFAULT] > > executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" > > rpc_response_timeout = 3600 > > > > > > [oslo_messaging_notifications] > topics = notifications > driver = noop > > > > # Messaging > [oslo_messaging_rabbit] > rpc_conn_pool_size = 300 > > heartbeat_timeout_threshold = 0 > > ssl = True > > > > Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. > > https://bugs.launchpad.net/neutron/+bug/1853071 > > > > Best Regards, Erik Andersson > > > > Should i be adding this to all my compute nodes also ? > > > > On Fri, Mar 20, 2020 at 11:40 AM Grant Morley wrote: > > If you tune rabbit then for: > > heartbeat_timeout_threshold = 0 > > That should help with the error message you are getting. > > That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. > > Are all of the ready messages for "notifications.info" for the various services ( Nova, Neutron, Keystone etc ) > > If that is the case you can disable those messages in your config files for those services. Look for: > > # Notifications > [oslo_messaging_notifications] > notification_topics = notifications > driver = noop > > Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. > > We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. > > Hope that > > From haleyb.dev at gmail.com Fri Mar 20 18:40:09 2020 From: haleyb.dev at gmail.com (Brian Haley) Date: Fri, 20 Mar 2020 14:40:09 -0400 Subject: [neutron] How to fix break of connectivity in case of L3 HA after reboot of backup node In-Reply-To: <20200320155757.l7pv7r64oua4eu5o@firewall> References: <20200320143749.qmi3ucfh43vao3wp@skaplons-mac> <20200320155757.l7pv7r64oua4eu5o@firewall> Message-ID: <41947c59-cdf7-f6ae-6873-f5e854805201@gmail.com> On 3/20/20 11:57 AM, Nate Johnston wrote: > On Fri, Mar 20, 2020 at 03:37:49PM +0100, Slawek Kaplonski wrote: >> Hi, >> >> We have bug [1] to solve. Basically, when node which is backup node for some >> router, connectivity to external gateway may be broken for some time. It's like >> that because when host is up and L3 agent is configuring qrouter namespace, it >> flush all IPv6 addresses from the qg- interface. And due to that some MLDv2 >> packets are sent from this interface which may cause that ToR switch will learn >> qg port's mac address on wrong (backup) node. >> >> This don't happens every time and for every router because it is a race between >> L3 agent and OVS agent. When L3 agent creates qg interface in br-int, it sets >> tag 4095 for it and traffic sent with such vlan tag is always dropped in br-int. >> So if L3 agent will flush IPv6 addresses before OVS agent wires the port and >> sets correct tag for it, then all is fine. But if OVS agent is first, then MLDv2 >> packets are sent to the wire and there is this connectivity break. >> >> There are proposed 2 ways of fixing this: >> - [2] which propsoes to add some kind of "communication" between L3 agent and >> OVS agent and tell OVS agent that tag can be changed only after IPv6 config >> is finished by L3 agent. >> Downside of this solution is that it works for OVS agent only, Linuxbridge >> agent may still hit the same issue. But plus is that after initial >> configuration of the router, everything else regarding to failover is handled >> by keepalived only - in same way like it is now. >> - [3] which sets qg NIC to be DOWN always on backup nodes. So when keepalived >> failover active router to new node, L3 agent needs to come and switch >> interfaces to be UP before it will work. >> The plus of this solution is that it works for all OVS and >> Linuxbridge L2 agents (and probably for others too) but downside is that >> failover process is a bit longer and there may be potentially another race >> condition between L3 agent and keepalived. Keepalived tries to sent gARP >> packets after switch node to be active, first attempt will always fail as >> interface is still DOWN. But keepalived will retry those gARPs after some >> time and this should be fine if L3 agent will already bring interface to be >> UP. > > Personally I find [2] more appealing. I think that if we find many linuxbridge > users hitting this issue then we can replicate the solution for linuxbridge at > that time, but until then let's not worry about it - the majority of users use > OVS. And the gARP timegap for solution #3 to me seems like a possbility for > problems or downtime. I would agree, it seemed easier to understand to me as well. -Brian >> Both patches are waiting for pretty long time in gerrit and I want to bring more >> visibility for both of them. Please check them and maybe You will have some >> opinions about which solution would be better and which we should go with. >> >> [1] https://bugs.launchpad.net/neutron/+bug/1859832 >> [2] https://review.opendev.org/#/c/702856/ >> [3] https://review.opendev.org/#/c/707406/ >> >> -- >> Slawek Kaplonski >> Senior software engineer >> Red Hat >> >> > > From ralonsoh at redhat.com Fri Mar 20 19:13:04 2020 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Fri, 20 Mar 2020 19:13:04 +0000 Subject: [neutron] How to fix break of connectivity in case of L3 HA after reboot of backup node In-Reply-To: <41947c59-cdf7-f6ae-6873-f5e854805201@gmail.com> References: <20200320143749.qmi3ucfh43vao3wp@skaplons-mac> <20200320155757.l7pv7r64oua4eu5o@firewall> <41947c59-cdf7-f6ae-6873-f5e854805201@gmail.com> Message-ID: Hello: As commented by Nate and Brian, and myself in [2] and [3], I prefer [2]. I understand this is a fix only for OVS, but: - It limits the solution to the external GW port plugging process, where the problem appears. - The second solution, as you commented, can introduce a race condition between the L3 agent and keepalived process, and a possible delay in the HA switch process. Regards. On Fri, 2020-03-20 at 14:40 -0400, Brian Haley wrote: > On 3/20/20 11:57 AM, Nate Johnston wrote: > > On Fri, Mar 20, 2020 at 03:37:49PM +0100, Slawek Kaplonski wrote: > > > Hi, > > > > > > We have bug [1] to solve. Basically, when node which is backup node for some > > > router, connectivity to external gateway may be broken for some time. It's like > > > that because when host is up and L3 agent is configuring qrouter namespace, it > > > flush all IPv6 addresses from the qg- interface. And due to that some MLDv2 > > > packets are sent from this interface which may cause that ToR switch will learn > > > qg port's mac address on wrong (backup) node. > > > > > > This don't happens every time and for every router because it is a race between > > > L3 agent and OVS agent. When L3 agent creates qg interface in br-int, it sets > > > tag 4095 for it and traffic sent with such vlan tag is always dropped in br-int. > > > So if L3 agent will flush IPv6 addresses before OVS agent wires the port and > > > sets correct tag for it, then all is fine. But if OVS agent is first, then MLDv2 > > > packets are sent to the wire and there is this connectivity break. > > > > > > There are proposed 2 ways of fixing this: > > > - [2] which propsoes to add some kind of "communication" between L3 agent and > > > OVS agent and tell OVS agent that tag can be changed only after IPv6 config > > > is finished by L3 agent. > > > Downside of this solution is that it works for OVS agent only, Linuxbridge > > > agent may still hit the same issue. But plus is that after initial > > > configuration of the router, everything else regarding to failover is handled > > > by keepalived only - in same way like it is now. > > > - [3] which sets qg NIC to be DOWN always on backup nodes. So when keepalived > > > failover active router to new node, L3 agent needs to come and switch > > > interfaces to be UP before it will work. > > > The plus of this solution is that it works for all OVS and > > > Linuxbridge L2 agents (and probably for others too) but downside is that > > > failover process is a bit longer and there may be potentially another race > > > condition between L3 agent and keepalived. Keepalived tries to sent gARP > > > packets after switch node to be active, first attempt will always fail as > > > interface is still DOWN. But keepalived will retry those gARPs after some > > > time and this should be fine if L3 agent will already bring interface to be > > > UP. > > > > Personally I find [2] more appealing. I think that if we find many linuxbridge > > users hitting this issue then we can replicate the solution for linuxbridge at > > that time, but until then let's not worry about it - the majority of users use > > OVS. And the gARP timegap for solution #3 to me seems like a possbility for > > problems or downtime. > > I would agree, it seemed easier to understand to me as well. > > -Brian > > > > Both patches are waiting for pretty long time in gerrit and I want to bring more > > > visibility for both of them. Please check them and maybe You will have some > > > opinions about which solution would be better and which we should go with. > > > > > > [1] https://bugs.launchpad.net/neutron/+bug/1859832 > > > [2] https://review.opendev.org/#/c/702856/ > > > [3] https://review.opendev.org/#/c/707406/ > > > > > > -- > > > Slawek Kaplonski > > > Senior software engineer > > > Red Hat > > > > > > From eandersson at blizzard.com Fri Mar 20 19:32:19 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Fri, 20 Mar 2020 19:32:19 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: This should just be for the server afaik. I haven't tried it out myself, but we for sure have the same issue. We just scaled out the number of workers as a workaround. In fact we even added neutron-servers on VMs to handle the issue. Best Regards, Erik Olof Gunnar Andersson -----Original Message----- From: Satish Patel Sent: Friday, March 20, 2020 11:19 AM To: Erik Olof Gunnar Andersson Cc: Grant Morley ; openstack-discuss at lists.openstack.org Subject: Re: Neutron RabbitMQ issues Erik, That is good finding i have check on following file /openstack/venvs/neutron-19.0.0.0rc3.dev6/lib/python2.7/site-packages/neutron/objects/agent.py Do you think i should add following option and restart neutron-server? is this for all compute nodes agent or just for server? new_facade = True On Fri, Mar 20, 2020 at 1:51 PM Erik Olof Gunnar Andersson wrote: > > > > > > Best Regards, Erik Olof Gunnar Andersson > > > > From: Satish Patel > Sent: Friday, March 20, 2020 9:23 AM > To: Grant Morley > Cc: Erik Olof Gunnar Andersson ; > openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > > > Oh you are right here, i have following stuff in my neutron.conf on > server > > > > # Notifications > [oslo_messaging_notifications] > driver = messagingv2 > topics = notifications > transport_url = > rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a043 > f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22:56 > 71//neutron?ssl=1 > > # Messaging > [oslo_messaging_rabbit] > rpc_conn_pool_size = 30 > ssl = True > > > > > > Following change i am going to made let me know if anything missing. > > > > [DEFAULT] > > executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" > > rpc_response_timeout = 3600 > > > > > > [oslo_messaging_notifications] > topics = notifications > driver = noop > > > > # Messaging > [oslo_messaging_rabbit] > rpc_conn_pool_size = 300 > > heartbeat_timeout_threshold = 0 > > ssl = True > > > > Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/18 > 53071__;Kw!!Ci6f514n9QsL8ck!zYBaONMbYxgOLiv4UJY51DLOI4H-qHjCOACdH6inbj > e694WzyxY-Eqpyl5QtywV6BQ$ > > > > Best Regards, Erik Andersson > > > > Should i be adding this to all my compute nodes also ? > > > > On Fri, Mar 20, 2020 at 11:40 AM Grant Morley wrote: > > If you tune rabbit then for: > > heartbeat_timeout_threshold = 0 > > That should help with the error message you are getting. > > That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. > > Are all of the ready messages for "notifications.info" for the various > services ( Nova, Neutron, Keystone etc ) > > If that is the case you can disable those messages in your config files for those services. Look for: > > # Notifications > [oslo_messaging_notifications] > notification_topics = notifications > driver = noop > > Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. > > We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. > > Hope that > > From Arkady.Kanevsky at dell.com Fri Mar 20 19:40:35 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Fri, 20 Mar 2020 19:40:35 +0000 Subject: [tripleo] launchpad bug policy In-Reply-To: References: Message-ID: Wes, Do we have a plan to move to storyboard? From: Wesley Hayutin Sent: Friday, March 20, 2020 10:51 AM To: OpenStack Discuss Subject: [tripleo] launchpad bug policy [EXTERNAL EMAIL] Greetings, Just a quick refresher on TripleO's bug policy. =================================== Please self-triage the bug: - Set the bug to Triaged - Set an Importance (note: "Critical" is something that prevents any TripleO deployment, use it carefully) - Assign the bug to yourself if you plan to work on it - Set a milestone where you think the bugs can be or needs to be fixed. If any doubt, check with the PTL. - Set one or multiple tags related to the bug. See available tags on https://bugs.launchpad.net/tripleo (right column). If you can't update these informations, it's because you're not member of TripleO in Launchpad group. Please ping the PTL and you'll be added. Please double-check if you provided the minimum of needed information in the description of this report. Nova provides an excellent example of Bug Template. It can be found at: https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate =================================== The following bugs were opened in the last 365 days incorrectly. We are at 60 bugs :( http://dashboard-ci.tripleo.org/d/cockpit/cockpit?panelId=357&fullscreen&orgId=1 If you are on this list, please help me out and self triage. If you have found something critical and want to ensure it's on a watched list, set the "alert" tag. Thanks all!! rabi https://bugs.launchpad.net/tripleo/+bug/1830837 bogdando https://bugs.launchpad.net/tripleo/+bug/1838972 jfrancoa https://bugs.launchpad.net/tripleo/+bug/1846206 bogdando https://bugs.launchpad.net/tripleo/+bug/1868075 adamjr https://bugs.launchpad.net/tripleo/+bug/1831841 anta-nok https://bugs.launchpad.net/tripleo/+bug/1832931 rabi https://bugs.launchpad.net/tripleo/+bug/1834054 waleedm https://bugs.launchpad.net/tripleo/+bug/1836011 priyotoshtsp https://bugs.launchpad.net/tripleo/+bug/1836350 donny-g https://bugs.launchpad.net/tripleo/+bug/1837760 mschuppert https://bugs.launchpad.net/tripleo/+bug/1838272 alee-3 https://bugs.launchpad.net/tripleo/+bug/1838804 gongysh https://bugs.launchpad.net/tripleo/+bug/1839126 asrmarco13 https://bugs.launchpad.net/tripleo/+bug/1839298 pkopec https://bugs.launchpad.net/tripleo/+bug/1840600 jfreud https://bugs.launchpad.net/tripleo/+bug/1840714 sai438 https://bugs.launchpad.net/tripleo/+bug/1841395 sclarke1 https://bugs.launchpad.net/tripleo/+bug/1842988 anta-nok https://bugs.launchpad.net/tripleo/+bug/1844438 cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1845268 bgill https://bugs.launchpad.net/tripleo/+bug/1847289 versus-vv https://bugs.launchpad.net/tripleo/+bug/1848546 sandeepyadav93 https://bugs.launchpad.net/tripleo/+bug/1849668 zodiac https://bugs.launchpad.net/tripleo/+bug/1850080 mschuppert https://bugs.launchpad.net/tripleo/+bug/1850610 alee-3 https://bugs.launchpad.net/tripleo/+bug/1850647 arequipeno https://bugs.launchpad.net/tripleo/+bug/1851503 damani42 https://bugs.launchpad.net/tripleo/+bug/1851618 adamjr https://bugs.launchpad.net/tripleo/+bug/1852509 david-hill-ubisoft https://bugs.launchpad.net/tripleo/+bug/1852762 adamjr https://bugs.launchpad.net/tripleo/+bug/1852893 ksambor https://bugs.launchpad.net/tripleo/+bug/1853000 pandasoubhagya https://bugs.launchpad.net/tripleo/+bug/1853562 kajinamit https://bugs.launchpad.net/tripleo/+bug/1854117 jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856095 mschuppert https://bugs.launchpad.net/tripleo/+bug/1856322 jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856411 shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1856680 hkominos https://bugs.launchpad.net/tripleo/+bug/1856734 hontu https://bugs.launchpad.net/tripleo/+bug/1856921 xarlos https://bugs.launchpad.net/tripleo/+bug/1857003 versus-vv https://bugs.launchpad.net/tripleo/+bug/1857153 bshephar https://bugs.launchpad.net/tripleo/+bug/1858010 shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1859608 cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1860486 denis-karpov https://bugs.launchpad.net/tripleo/+bug/1860528 bshephar https://bugs.launchpad.net/tripleo/+bug/1860609 suzumushi https://bugs.launchpad.net/tripleo/+bug/1861315 ekultails https://bugs.launchpad.net/tripleo/+bug/1862179 mschuppert https://bugs.launchpad.net/tripleo/+bug/1862321 slavonic https://bugs.launchpad.net/tripleo/+bug/1863122 ffernand https://bugs.launchpad.net/tripleo/+bug/1863243 cgoncalves https://bugs.launchpad.net/tripleo/+bug/1863595 larsks https://bugs.launchpad.net/tripleo/+bug/1864232 jbemmel https://bugs.launchpad.net/tripleo/+bug/1864924 kajinamit https://bugs.launchpad.net/tripleo/+bug/1865477 cgoncalves https://bugs.launchpad.net/tripleo/+bug/1867144 prabiegx https://bugs.launchpad.net/tripleo/+bug/1867333 valleedelisle https://bugs.launchpad.net/tripleo/+bug/1867362 amoralej https://bugs.launchpad.net/tripleo/+bug/1867608 -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Fri Mar 20 19:46:41 2020 From: satish.txt at gmail.com (Satish Patel) Date: Fri, 20 Mar 2020 15:46:41 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: Do you think i should try to add following option by hand "new_facade = True" on server and restart neutron-server services? I am not seeing any extra dependency for that option so looks very simple to add. When you said scaled out number of workers means you added multiple neutron-servers on bunch of VM to spread load? (I have 3x controller nodes running on physical servers) On Fri, Mar 20, 2020 at 3:32 PM Erik Olof Gunnar Andersson wrote: > > This should just be for the server afaik. I haven't tried it out myself, but we for sure have the same issue. We just scaled out the number of workers as a workaround. In fact we even added neutron-servers on VMs to handle the issue. > > Best Regards, Erik Olof Gunnar Andersson > > -----Original Message----- > From: Satish Patel > Sent: Friday, March 20, 2020 11:19 AM > To: Erik Olof Gunnar Andersson > Cc: Grant Morley ; openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > Erik, > > That is good finding i have check on following file > > /openstack/venvs/neutron-19.0.0.0rc3.dev6/lib/python2.7/site-packages/neutron/objects/agent.py > > Do you think i should add following option and restart neutron-server? > is this for all compute nodes agent or just for server? > > new_facade = True > > On Fri, Mar 20, 2020 at 1:51 PM Erik Olof Gunnar Andersson wrote: > > > > > > > > > > > > Best Regards, Erik Olof Gunnar Andersson > > > > > > > > From: Satish Patel > > Sent: Friday, March 20, 2020 9:23 AM > > To: Grant Morley > > Cc: Erik Olof Gunnar Andersson ; > > openstack-discuss at lists.openstack.org > > Subject: Re: Neutron RabbitMQ issues > > > > > > > > Oh you are right here, i have following stuff in my neutron.conf on > > server > > > > > > > > # Notifications > > [oslo_messaging_notifications] > > driver = messagingv2 > > topics = notifications > > transport_url = > > rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a043 > > f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22:56 > > 71//neutron?ssl=1 > > > > # Messaging > > [oslo_messaging_rabbit] > > rpc_conn_pool_size = 30 > > ssl = True > > > > > > > > > > > > Following change i am going to made let me know if anything missing. > > > > > > > > [DEFAULT] > > > > executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" > > > > rpc_response_timeout = 3600 > > > > > > > > > > > > [oslo_messaging_notifications] > > topics = notifications > > driver = noop > > > > > > > > # Messaging > > [oslo_messaging_rabbit] > > rpc_conn_pool_size = 300 > > > > heartbeat_timeout_threshold = 0 > > > > ssl = True > > > > > > > > Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/18 > > 53071__;Kw!!Ci6f514n9QsL8ck!zYBaONMbYxgOLiv4UJY51DLOI4H-qHjCOACdH6inbj > > e694WzyxY-Eqpyl5QtywV6BQ$ > > > > > > > > Best Regards, Erik Andersson > > > > > > > > Should i be adding this to all my compute nodes also ? > > > > > > > > On Fri, Mar 20, 2020 at 11:40 AM Grant Morley wrote: > > > > If you tune rabbit then for: > > > > heartbeat_timeout_threshold = 0 > > > > That should help with the error message you are getting. > > > > That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. > > > > Are all of the ready messages for "notifications.info" for the various > > services ( Nova, Neutron, Keystone etc ) > > > > If that is the case you can disable those messages in your config files for those services. Look for: > > > > # Notifications > > [oslo_messaging_notifications] > > notification_topics = notifications > > driver = noop > > > > Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. > > > > We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. > > > > Hope that > > > > From ekultails at gmail.com Fri Mar 20 19:49:21 2020 From: ekultails at gmail.com (Luke Short) Date: Fri, 20 Mar 2020 15:49:21 -0400 Subject: [tripleo] launchpad bug policy In-Reply-To: References: Message-ID: Greetings Arkady, For bug tracking, I believe TripleO will continue to use Launchpad. My team has been using Storyboard for tracking progress on big-picture items with smaller tasks assigned (for example, removing Mistral and Zaqar from the Undercloud) with great success. Those two platforms have different focuses and use-cases. Both of which can add value to any contributor in the OpenStack community. Sincerely, Luke Short On Fri, Mar 20, 2020 at 3:42 PM wrote: > Wes, > > Do we have a plan to move to storyboard? > > > > *From:* Wesley Hayutin > *Sent:* Friday, March 20, 2020 10:51 AM > *To:* OpenStack Discuss > *Subject:* [tripleo] launchpad bug policy > > > > [EXTERNAL EMAIL] > > Greetings, > > > > Just a quick refresher on TripleO's bug policy. > > > > =================================== > > Please self-triage the bug: > - Set the bug to Triaged > - Set an Importance (note: "Critical" is something that prevents any > TripleO deployment, use it carefully) > - Assign the bug to yourself if you plan to work on it > - Set a milestone where you think the bugs can be or needs to be fixed. If > any doubt, check with the PTL. > - Set one or multiple tags related to the bug. See available tags on > https://bugs.launchpad.net/tripleo (right column). > > If you can't update these informations, it's because you're not member of > TripleO in Launchpad group. Please ping the PTL and you'll be added. > > Please double-check if you provided the minimum of needed information in > the description of this report. > Nova provides an excellent example of Bug Template. It can be found at: > https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate > > =================================== > > > > The following bugs were opened in the last 365 days incorrectly. We are > at 60 bugs :( > > > http://dashboard-ci.tripleo.org/d/cockpit/cockpit?panelId=357&fullscreen&orgId=1 > > > > If you are on this list, please help me out and self triage. If you have > found something critical and want to ensure it's on a watched list, set the > "alert" tag. > > > > Thanks all!! > > > > rabi https://bugs.launchpad.net/tripleo/+bug/1830837 > bogdando https://bugs.launchpad.net/tripleo/+bug/1838972 > jfrancoa https://bugs.launchpad.net/tripleo/+bug/1846206 > bogdando https://bugs.launchpad.net/tripleo/+bug/1868075 > adamjr https://bugs.launchpad.net/tripleo/+bug/1831841 > anta-nok https://bugs.launchpad.net/tripleo/+bug/1832931 > rabi https://bugs.launchpad.net/tripleo/+bug/1834054 > waleedm https://bugs.launchpad.net/tripleo/+bug/1836011 > priyotoshtsp https://bugs.launchpad.net/tripleo/+bug/1836350 > donny-g https://bugs.launchpad.net/tripleo/+bug/1837760 > mschuppert https://bugs.launchpad.net/tripleo/+bug/1838272 > alee-3 https://bugs.launchpad.net/tripleo/+bug/1838804 > gongysh https://bugs.launchpad.net/tripleo/+bug/1839126 > asrmarco13 https://bugs.launchpad.net/tripleo/+bug/1839298 > pkopec https://bugs.launchpad.net/tripleo/+bug/1840600 > jfreud https://bugs.launchpad.net/tripleo/+bug/1840714 > sai438 https://bugs.launchpad.net/tripleo/+bug/1841395 > sclarke1 https://bugs.launchpad.net/tripleo/+bug/1842988 > anta-nok https://bugs.launchpad.net/tripleo/+bug/1844438 > cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1845268 > bgill https://bugs.launchpad.net/tripleo/+bug/1847289 > versus-vv https://bugs.launchpad.net/tripleo/+bug/1848546 > sandeepyadav93 https://bugs.launchpad.net/tripleo/+bug/1849668 > zodiac https://bugs.launchpad.net/tripleo/+bug/1850080 > mschuppert https://bugs.launchpad.net/tripleo/+bug/1850610 > alee-3 https://bugs.launchpad.net/tripleo/+bug/1850647 > arequipeno https://bugs.launchpad.net/tripleo/+bug/1851503 > damani42 https://bugs.launchpad.net/tripleo/+bug/1851618 > adamjr https://bugs.launchpad.net/tripleo/+bug/1852509 > david-hill-ubisoft https://bugs.launchpad.net/tripleo/+bug/1852762 > adamjr https://bugs.launchpad.net/tripleo/+bug/1852893 > ksambor https://bugs.launchpad.net/tripleo/+bug/1853000 > pandasoubhagya https://bugs.launchpad.net/tripleo/+bug/1853562 > kajinamit https://bugs.launchpad.net/tripleo/+bug/1854117 > jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856095 > mschuppert https://bugs.launchpad.net/tripleo/+bug/1856322 > jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856411 > shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1856680 > hkominos https://bugs.launchpad.net/tripleo/+bug/1856734 > hontu https://bugs.launchpad.net/tripleo/+bug/1856921 > xarlos https://bugs.launchpad.net/tripleo/+bug/1857003 > versus-vv https://bugs.launchpad.net/tripleo/+bug/1857153 > bshephar https://bugs.launchpad.net/tripleo/+bug/1858010 > shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1859608 > cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1860486 > denis-karpov https://bugs.launchpad.net/tripleo/+bug/1860528 > bshephar https://bugs.launchpad.net/tripleo/+bug/1860609 > suzumushi https://bugs.launchpad.net/tripleo/+bug/1861315 > ekultails https://bugs.launchpad.net/tripleo/+bug/1862179 > mschuppert https://bugs.launchpad.net/tripleo/+bug/1862321 > slavonic https://bugs.launchpad.net/tripleo/+bug/1863122 > ffernand https://bugs.launchpad.net/tripleo/+bug/1863243 > cgoncalves https://bugs.launchpad.net/tripleo/+bug/1863595 > larsks https://bugs.launchpad.net/tripleo/+bug/1864232 > jbemmel https://bugs.launchpad.net/tripleo/+bug/1864924 > kajinamit https://bugs.launchpad.net/tripleo/+bug/1865477 > cgoncalves https://bugs.launchpad.net/tripleo/+bug/1867144 > prabiegx https://bugs.launchpad.net/tripleo/+bug/1867333 > valleedelisle https://bugs.launchpad.net/tripleo/+bug/1867362 > amoralej https://bugs.launchpad.net/tripleo/+bug/1867608 > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eandersson at blizzard.com Fri Mar 20 19:52:51 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Fri, 20 Mar 2020 19:52:51 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: When I say scale up I mean that yea. Plus of course bumping the number of workers (rpc_workers) to an appropriate value. There are risks with modifying that. Might be worth asking in the neutron channel on irc, at least if this is your production deployment. If possible maybe test it in a lab or staging deployment first. Best Regards, Erik Olof Gunnar Andersson -----Original Message----- From: Satish Patel Sent: Friday, March 20, 2020 12:47 PM To: Erik Olof Gunnar Andersson Cc: Grant Morley ; openstack-discuss at lists.openstack.org Subject: Re: Neutron RabbitMQ issues Do you think i should try to add following option by hand "new_facade = True" on server and restart neutron-server services? I am not seeing any extra dependency for that option so looks very simple to add. When you said scaled out number of workers means you added multiple neutron-servers on bunch of VM to spread load? (I have 3x controller nodes running on physical servers) On Fri, Mar 20, 2020 at 3:32 PM Erik Olof Gunnar Andersson wrote: > > This should just be for the server afaik. I haven't tried it out myself, but we for sure have the same issue. We just scaled out the number of workers as a workaround. In fact we even added neutron-servers on VMs to handle the issue. > > Best Regards, Erik Olof Gunnar Andersson > > -----Original Message----- > From: Satish Patel > Sent: Friday, March 20, 2020 11:19 AM > To: Erik Olof Gunnar Andersson > Cc: Grant Morley ; > openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > Erik, > > That is good finding i have check on following file > > /openstack/venvs/neutron-19.0.0.0rc3.dev6/lib/python2.7/site-packages/ > neutron/objects/agent.py > > Do you think i should add following option and restart neutron-server? > is this for all compute nodes agent or just for server? > > new_facade = True > > On Fri, Mar 20, 2020 at 1:51 PM Erik Olof Gunnar Andersson wrote: > > > > > > > > > > > > Best Regards, Erik Olof Gunnar Andersson > > > > > > > > From: Satish Patel > > Sent: Friday, March 20, 2020 9:23 AM > > To: Grant Morley > > Cc: Erik Olof Gunnar Andersson ; > > openstack-discuss at lists.openstack.org > > Subject: Re: Neutron RabbitMQ issues > > > > > > > > Oh you are right here, i have following stuff in my neutron.conf on > > server > > > > > > > > # Notifications > > [oslo_messaging_notifications] > > driver = messagingv2 > > topics = notifications > > transport_url = > > rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a0 > > 43 > > f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22: > > 56 > > 71//neutron?ssl=1 > > > > # Messaging > > [oslo_messaging_rabbit] > > rpc_conn_pool_size = 30 > > ssl = True > > > > > > > > > > > > Following change i am going to made let me know if anything missing. > > > > > > > > [DEFAULT] > > > > executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" > > > > rpc_response_timeout = 3600 > > > > > > > > > > > > [oslo_messaging_notifications] > > topics = notifications > > driver = noop > > > > > > > > # Messaging > > [oslo_messaging_rabbit] > > rpc_conn_pool_size = 300 > > > > heartbeat_timeout_threshold = 0 > > > > ssl = True > > > > > > > > Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/ > > 18 > > 53071__;Kw!!Ci6f514n9QsL8ck!zYBaONMbYxgOLiv4UJY51DLOI4H-qHjCOACdH6in > > bj > > e694WzyxY-Eqpyl5QtywV6BQ$ > > > > > > > > Best Regards, Erik Andersson > > > > > > > > Should i be adding this to all my compute nodes also ? > > > > > > > > On Fri, Mar 20, 2020 at 11:40 AM Grant Morley wrote: > > > > If you tune rabbit then for: > > > > heartbeat_timeout_threshold = 0 > > > > That should help with the error message you are getting. > > > > That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. > > > > Are all of the ready messages for "notifications.info" for the > > various services ( Nova, Neutron, Keystone etc ) > > > > If that is the case you can disable those messages in your config files for those services. Look for: > > > > # Notifications > > [oslo_messaging_notifications] > > notification_topics = notifications > > driver = noop > > > > Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. > > > > We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. > > > > Hope that > > > > From satish.txt at gmail.com Fri Mar 20 20:00:32 2020 From: satish.txt at gmail.com (Satish Patel) Date: Fri, 20 Mar 2020 16:00:32 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: Sure, I will do that. Thanks On Fri, Mar 20, 2020 at 3:52 PM Erik Olof Gunnar Andersson wrote: > > When I say scale up I mean that yea. Plus of course bumping the number of workers (rpc_workers) to an appropriate value. > > There are risks with modifying that. Might be worth asking in the neutron channel on irc, at least if this is your production deployment. If possible maybe test it in a lab or staging deployment first. > > Best Regards, Erik Olof Gunnar Andersson > > -----Original Message----- > From: Satish Patel > Sent: Friday, March 20, 2020 12:47 PM > To: Erik Olof Gunnar Andersson > Cc: Grant Morley ; openstack-discuss at lists.openstack.org > Subject: Re: Neutron RabbitMQ issues > > Do you think i should try to add following option by hand "new_facade = True" on server and restart neutron-server services? I am not seeing any extra dependency for that option so looks very simple to add. > > When you said scaled out number of workers means you added multiple neutron-servers on bunch of VM to spread load? (I have 3x controller nodes running on physical servers) > > > > On Fri, Mar 20, 2020 at 3:32 PM Erik Olof Gunnar Andersson wrote: > > > > This should just be for the server afaik. I haven't tried it out myself, but we for sure have the same issue. We just scaled out the number of workers as a workaround. In fact we even added neutron-servers on VMs to handle the issue. > > > > Best Regards, Erik Olof Gunnar Andersson > > > > -----Original Message----- > > From: Satish Patel > > Sent: Friday, March 20, 2020 11:19 AM > > To: Erik Olof Gunnar Andersson > > Cc: Grant Morley ; > > openstack-discuss at lists.openstack.org > > Subject: Re: Neutron RabbitMQ issues > > > > Erik, > > > > That is good finding i have check on following file > > > > /openstack/venvs/neutron-19.0.0.0rc3.dev6/lib/python2.7/site-packages/ > > neutron/objects/agent.py > > > > Do you think i should add following option and restart neutron-server? > > is this for all compute nodes agent or just for server? > > > > new_facade = True > > > > On Fri, Mar 20, 2020 at 1:51 PM Erik Olof Gunnar Andersson wrote: > > > > > > > > > > > > > > > > > > Best Regards, Erik Olof Gunnar Andersson > > > > > > > > > > > > From: Satish Patel > > > Sent: Friday, March 20, 2020 9:23 AM > > > To: Grant Morley > > > Cc: Erik Olof Gunnar Andersson ; > > > openstack-discuss at lists.openstack.org > > > Subject: Re: Neutron RabbitMQ issues > > > > > > > > > > > > Oh you are right here, i have following stuff in my neutron.conf on > > > server > > > > > > > > > > > > # Notifications > > > [oslo_messaging_notifications] > > > driver = messagingv2 > > > topics = notifications > > > transport_url = > > > rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a0 > > > 43 > > > f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22: > > > 56 > > > 71//neutron?ssl=1 > > > > > > # Messaging > > > [oslo_messaging_rabbit] > > > rpc_conn_pool_size = 30 > > > ssl = True > > > > > > > > > > > > > > > > > > Following change i am going to made let me know if anything missing. > > > > > > > > > > > > [DEFAULT] > > > > > > executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" > > > > > > rpc_response_timeout = 3600 > > > > > > > > > > > > > > > > > > [oslo_messaging_notifications] > > > topics = notifications > > > driver = noop > > > > > > > > > > > > # Messaging > > > [oslo_messaging_rabbit] > > > rpc_conn_pool_size = 300 > > > > > > heartbeat_timeout_threshold = 0 > > > > > > ssl = True > > > > > > > > > > > > Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. > > > > > > https://urldefense.com/v3/__https://bugs.launchpad.net/neutron/*bug/ > > > 18 > > > 53071__;Kw!!Ci6f514n9QsL8ck!zYBaONMbYxgOLiv4UJY51DLOI4H-qHjCOACdH6in > > > bj > > > e694WzyxY-Eqpyl5QtywV6BQ$ > > > > > > > > > > > > Best Regards, Erik Andersson > > > > > > > > > > > > Should i be adding this to all my compute nodes also ? > > > > > > > > > > > > On Fri, Mar 20, 2020 at 11:40 AM Grant Morley wrote: > > > > > > If you tune rabbit then for: > > > > > > heartbeat_timeout_threshold = 0 > > > > > > That should help with the error message you are getting. > > > > > > That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. > > > > > > Are all of the ready messages for "notifications.info" for the > > > various services ( Nova, Neutron, Keystone etc ) > > > > > > If that is the case you can disable those messages in your config files for those services. Look for: > > > > > > # Notifications > > > [oslo_messaging_notifications] > > > notification_topics = notifications > > > driver = noop > > > > > > Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. > > > > > > We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. > > > > > > Hope that > > > > > > From whayutin at redhat.com Fri Mar 20 20:09:55 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Fri, 20 Mar 2020 14:09:55 -0600 Subject: [tripleo] launchpad bug policy In-Reply-To: References: Message-ID: On Fri, Mar 20, 2020 at 1:49 PM Luke Short wrote: > Greetings Arkady, > > For bug tracking, I believe TripleO will continue to use Launchpad. My > team has been using Storyboard for tracking progress on big-picture items > with smaller tasks assigned (for example, removing Mistral and Zaqar from > the Undercloud) with great success. Those two platforms have different > focuses and use-cases. Both of which can add value to any contributor in > the OpenStack community. > > Sincerely, > Luke Short > Yes, building on what Luke said we'll continue to use launchpad. I know there was a lot of planning around storyboard at the last PTG, and we'll continue to monitor and evaluate its use in the future. Thanks > > On Fri, Mar 20, 2020 at 3:42 PM wrote: > >> Wes, >> >> Do we have a plan to move to storyboard? >> >> >> >> *From:* Wesley Hayutin >> *Sent:* Friday, March 20, 2020 10:51 AM >> *To:* OpenStack Discuss >> *Subject:* [tripleo] launchpad bug policy >> >> >> >> [EXTERNAL EMAIL] >> >> Greetings, >> >> >> >> Just a quick refresher on TripleO's bug policy. >> >> >> >> =================================== >> >> Please self-triage the bug: >> - Set the bug to Triaged >> - Set an Importance (note: "Critical" is something that prevents any >> TripleO deployment, use it carefully) >> - Assign the bug to yourself if you plan to work on it >> - Set a milestone where you think the bugs can be or needs to be fixed. >> If any doubt, check with the PTL. >> - Set one or multiple tags related to the bug. See available tags on >> https://bugs.launchpad.net/tripleo (right column). >> >> If you can't update these informations, it's because you're not member of >> TripleO in Launchpad group. Please ping the PTL and you'll be added. >> >> Please double-check if you provided the minimum of needed information in >> the description of this report. >> Nova provides an excellent example of Bug Template. It can be found at: >> https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate >> >> =================================== >> >> >> >> The following bugs were opened in the last 365 days incorrectly. We are >> at 60 bugs :( >> >> >> http://dashboard-ci.tripleo.org/d/cockpit/cockpit?panelId=357&fullscreen&orgId=1 >> >> >> >> If you are on this list, please help me out and self triage. If you have >> found something critical and want to ensure it's on a watched list, set the >> "alert" tag. >> >> >> >> Thanks all!! >> >> >> >> rabi https://bugs.launchpad.net/tripleo/+bug/1830837 >> bogdando https://bugs.launchpad.net/tripleo/+bug/1838972 >> jfrancoa https://bugs.launchpad.net/tripleo/+bug/1846206 >> bogdando https://bugs.launchpad.net/tripleo/+bug/1868075 >> adamjr https://bugs.launchpad.net/tripleo/+bug/1831841 >> anta-nok https://bugs.launchpad.net/tripleo/+bug/1832931 >> rabi https://bugs.launchpad.net/tripleo/+bug/1834054 >> waleedm https://bugs.launchpad.net/tripleo/+bug/1836011 >> priyotoshtsp https://bugs.launchpad.net/tripleo/+bug/1836350 >> donny-g https://bugs.launchpad.net/tripleo/+bug/1837760 >> mschuppert https://bugs.launchpad.net/tripleo/+bug/1838272 >> alee-3 https://bugs.launchpad.net/tripleo/+bug/1838804 >> gongysh https://bugs.launchpad.net/tripleo/+bug/1839126 >> asrmarco13 https://bugs.launchpad.net/tripleo/+bug/1839298 >> pkopec https://bugs.launchpad.net/tripleo/+bug/1840600 >> jfreud https://bugs.launchpad.net/tripleo/+bug/1840714 >> sai438 https://bugs.launchpad.net/tripleo/+bug/1841395 >> sclarke1 https://bugs.launchpad.net/tripleo/+bug/1842988 >> anta-nok https://bugs.launchpad.net/tripleo/+bug/1844438 >> cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1845268 >> bgill https://bugs.launchpad.net/tripleo/+bug/1847289 >> versus-vv https://bugs.launchpad.net/tripleo/+bug/1848546 >> sandeepyadav93 https://bugs.launchpad.net/tripleo/+bug/1849668 >> zodiac https://bugs.launchpad.net/tripleo/+bug/1850080 >> mschuppert https://bugs.launchpad.net/tripleo/+bug/1850610 >> alee-3 https://bugs.launchpad.net/tripleo/+bug/1850647 >> arequipeno https://bugs.launchpad.net/tripleo/+bug/1851503 >> damani42 https://bugs.launchpad.net/tripleo/+bug/1851618 >> adamjr https://bugs.launchpad.net/tripleo/+bug/1852509 >> david-hill-ubisoft https://bugs.launchpad.net/tripleo/+bug/1852762 >> adamjr https://bugs.launchpad.net/tripleo/+bug/1852893 >> ksambor https://bugs.launchpad.net/tripleo/+bug/1853000 >> pandasoubhagya https://bugs.launchpad.net/tripleo/+bug/1853562 >> kajinamit https://bugs.launchpad.net/tripleo/+bug/1854117 >> jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856095 >> mschuppert https://bugs.launchpad.net/tripleo/+bug/1856322 >> jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856411 >> shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1856680 >> hkominos https://bugs.launchpad.net/tripleo/+bug/1856734 >> hontu https://bugs.launchpad.net/tripleo/+bug/1856921 >> xarlos https://bugs.launchpad.net/tripleo/+bug/1857003 >> versus-vv https://bugs.launchpad.net/tripleo/+bug/1857153 >> bshephar https://bugs.launchpad.net/tripleo/+bug/1858010 >> shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1859608 >> cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1860486 >> denis-karpov https://bugs.launchpad.net/tripleo/+bug/1860528 >> bshephar https://bugs.launchpad.net/tripleo/+bug/1860609 >> suzumushi https://bugs.launchpad.net/tripleo/+bug/1861315 >> ekultails https://bugs.launchpad.net/tripleo/+bug/1862179 >> mschuppert https://bugs.launchpad.net/tripleo/+bug/1862321 >> slavonic https://bugs.launchpad.net/tripleo/+bug/1863122 >> ffernand https://bugs.launchpad.net/tripleo/+bug/1863243 >> cgoncalves https://bugs.launchpad.net/tripleo/+bug/1863595 >> larsks https://bugs.launchpad.net/tripleo/+bug/1864232 >> jbemmel https://bugs.launchpad.net/tripleo/+bug/1864924 >> kajinamit https://bugs.launchpad.net/tripleo/+bug/1865477 >> cgoncalves https://bugs.launchpad.net/tripleo/+bug/1867144 >> prabiegx https://bugs.launchpad.net/tripleo/+bug/1867333 >> valleedelisle https://bugs.launchpad.net/tripleo/+bug/1867362 >> amoralej https://bugs.launchpad.net/tripleo/+bug/1867608 >> >> >> >> >> >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ssbarnea at redhat.com Fri Mar 20 20:51:58 2020 From: ssbarnea at redhat.com (Sorin Sbarnea) Date: Fri, 20 Mar 2020 20:51:58 +0000 Subject: [tripleo] launchpad bug policy In-Reply-To: References: Message-ID: Happy to hear that tripleo will continue to “monitor” storyboard development because from what I see it would take another decade until SB would become practical to use for both issue tracking and kanban or scrum planning. On Fri, 20 Mar 2020 at 20:14, Wesley Hayutin wrote: > > > On Fri, Mar 20, 2020 at 1:49 PM Luke Short wrote: > >> Greetings Arkady, >> >> For bug tracking, I believe TripleO will continue to use Launchpad. My >> team has been using Storyboard for tracking progress on big-picture items >> with smaller tasks assigned (for example, removing Mistral and Zaqar from >> the Undercloud) with great success. Those two platforms have different >> focuses and use-cases. Both of which can add value to any contributor in >> the OpenStack community. >> >> Sincerely, >> Luke Short >> > > Yes, building on what Luke said we'll continue to use launchpad. I know > there was a lot of planning around storyboard at the last PTG, and we'll > continue to monitor and evaluate its use in the future. > > Thanks > > >> >> On Fri, Mar 20, 2020 at 3:42 PM wrote: >> >>> Wes, >>> >>> Do we have a plan to move to storyboard? >>> >>> >>> >>> *From:* Wesley Hayutin >>> *Sent:* Friday, March 20, 2020 10:51 AM >>> *To:* OpenStack Discuss >>> *Subject:* [tripleo] launchpad bug policy >>> >>> >>> >>> [EXTERNAL EMAIL] >>> >>> Greetings, >>> >>> >>> >>> Just a quick refresher on TripleO's bug policy. >>> >>> >>> >>> =================================== >>> >>> Please self-triage the bug: >>> - Set the bug to Triaged >>> - Set an Importance (note: "Critical" is something that prevents any >>> TripleO deployment, use it carefully) >>> - Assign the bug to yourself if you plan to work on it >>> - Set a milestone where you think the bugs can be or needs to be fixed. >>> If any doubt, check with the PTL. >>> - Set one or multiple tags related to the bug. See available tags on >>> https://bugs.launchpad.net/tripleo (right column). >>> >>> If you can't update these informations, it's because you're not member >>> of TripleO in Launchpad group. Please ping the PTL and you'll be added. >>> >>> Please double-check if you provided the minimum of needed information in >>> the description of this report. >>> Nova provides an excellent example of Bug Template. It can be found at: >>> https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate >>> >>> =================================== >>> >>> >>> >>> The following bugs were opened in the last 365 days incorrectly. We are >>> at 60 bugs :( >>> >>> >>> http://dashboard-ci.tripleo.org/d/cockpit/cockpit?panelId=357&fullscreen&orgId=1 >>> >>> >>> >>> If you are on this list, please help me out and self triage. If you >>> have found something critical and want to ensure it's on a watched list, >>> set the "alert" tag. >>> >>> >>> >>> Thanks all!! >>> >>> >>> >>> rabi https://bugs.launchpad.net/tripleo/+bug/1830837 >>> bogdando https://bugs.launchpad.net/tripleo/+bug/1838972 >>> jfrancoa https://bugs.launchpad.net/tripleo/+bug/1846206 >>> bogdando https://bugs.launchpad.net/tripleo/+bug/1868075 >>> adamjr https://bugs.launchpad.net/tripleo/+bug/1831841 >>> anta-nok https://bugs.launchpad.net/tripleo/+bug/1832931 >>> rabi https://bugs.launchpad.net/tripleo/+bug/1834054 >>> waleedm https://bugs.launchpad.net/tripleo/+bug/1836011 >>> priyotoshtsp https://bugs.launchpad.net/tripleo/+bug/1836350 >>> donny-g https://bugs.launchpad.net/tripleo/+bug/1837760 >>> mschuppert https://bugs.launchpad.net/tripleo/+bug/1838272 >>> alee-3 https://bugs.launchpad.net/tripleo/+bug/1838804 >>> gongysh https://bugs.launchpad.net/tripleo/+bug/1839126 >>> asrmarco13 https://bugs.launchpad.net/tripleo/+bug/1839298 >>> pkopec https://bugs.launchpad.net/tripleo/+bug/1840600 >>> jfreud https://bugs.launchpad.net/tripleo/+bug/1840714 >>> sai438 https://bugs.launchpad.net/tripleo/+bug/1841395 >>> sclarke1 https://bugs.launchpad.net/tripleo/+bug/1842988 >>> anta-nok https://bugs.launchpad.net/tripleo/+bug/1844438 >>> cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1845268 >>> bgill https://bugs.launchpad.net/tripleo/+bug/1847289 >>> versus-vv https://bugs.launchpad.net/tripleo/+bug/1848546 >>> sandeepyadav93 https://bugs.launchpad.net/tripleo/+bug/1849668 >>> zodiac https://bugs.launchpad.net/tripleo/+bug/1850080 >>> mschuppert https://bugs.launchpad.net/tripleo/+bug/1850610 >>> alee-3 https://bugs.launchpad.net/tripleo/+bug/1850647 >>> arequipeno https://bugs.launchpad.net/tripleo/+bug/1851503 >>> damani42 https://bugs.launchpad.net/tripleo/+bug/1851618 >>> adamjr https://bugs.launchpad.net/tripleo/+bug/1852509 >>> david-hill-ubisoft https://bugs.launchpad.net/tripleo/+bug/1852762 >>> adamjr https://bugs.launchpad.net/tripleo/+bug/1852893 >>> ksambor https://bugs.launchpad.net/tripleo/+bug/1853000 >>> pandasoubhagya https://bugs.launchpad.net/tripleo/+bug/1853562 >>> kajinamit https://bugs.launchpad.net/tripleo/+bug/1854117 >>> jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856095 >>> mschuppert https://bugs.launchpad.net/tripleo/+bug/1856322 >>> jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856411 >>> shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1856680 >>> hkominos https://bugs.launchpad.net/tripleo/+bug/1856734 >>> hontu https://bugs.launchpad.net/tripleo/+bug/1856921 >>> xarlos https://bugs.launchpad.net/tripleo/+bug/1857003 >>> versus-vv https://bugs.launchpad.net/tripleo/+bug/1857153 >>> bshephar https://bugs.launchpad.net/tripleo/+bug/1858010 >>> shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1859608 >>> cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1860486 >>> denis-karpov https://bugs.launchpad.net/tripleo/+bug/1860528 >>> bshephar https://bugs.launchpad.net/tripleo/+bug/1860609 >>> suzumushi https://bugs.launchpad.net/tripleo/+bug/1861315 >>> ekultails https://bugs.launchpad.net/tripleo/+bug/1862179 >>> mschuppert https://bugs.launchpad.net/tripleo/+bug/1862321 >>> slavonic https://bugs.launchpad.net/tripleo/+bug/1863122 >>> ffernand https://bugs.launchpad.net/tripleo/+bug/1863243 >>> cgoncalves https://bugs.launchpad.net/tripleo/+bug/1863595 >>> larsks https://bugs.launchpad.net/tripleo/+bug/1864232 >>> jbemmel https://bugs.launchpad.net/tripleo/+bug/1864924 >>> kajinamit https://bugs.launchpad.net/tripleo/+bug/1865477 >>> cgoncalves https://bugs.launchpad.net/tripleo/+bug/1867144 >>> prabiegx https://bugs.launchpad.net/tripleo/+bug/1867333 >>> valleedelisle https://bugs.launchpad.net/tripleo/+bug/1867362 >>> amoralej https://bugs.launchpad.net/tripleo/+bug/1867608 >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> -- -- /sorin -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri Mar 20 21:10:45 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 20 Mar 2020 21:10:45 +0000 Subject: [tripleo] launchpad bug policy In-Reply-To: References: Message-ID: <20200320211045.cdwaz5ebracpvidc@yuggoth.org> On 2020-03-20 20:51:58 +0000 (+0000), Sorin Sbarnea wrote: > Happy to hear that tripleo will continue to “monitor” storyboard > development because from what I see it would take another decade > until SB would become practical to use for both issue tracking and > kanban or scrum planning. [...] That's a rather harsh criticism, and not constructive. Saying it's not ready for your uses (and better, exactly why) would have been preferable to predicting the trajectory of a project to which you're not contributing. We all work on software with inertial challenges, but let's not make pessimism a habit here. Also, collaboration is preferable to monitoring any day of the week. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From i at liuyulong.me Fri Mar 20 21:30:31 2020 From: i at liuyulong.me (=?utf-8?B?TElVIFl1bG9uZw==?=) Date: Sat, 21 Mar 2020 05:30:31 +0800 Subject: [neutron] How to fix break of connectivity in case of L3 HA after reboot of backup node In-Reply-To: References: <20200320143749.qmi3ucfh43vao3wp@skaplons-mac> <20200320155757.l7pv7r64oua4eu5o@firewall> <41947c59-cdf7-f6ae-6873-f5e854805201@gmail.com> Message-ID: > Hello: >  > As commented by Nate and Brian, and myself in [2] and [3], I prefer [2]. I understand this is a fix > only for OVS, but: > - It limits the solution to the external GW port plugging process, where the problem appears. > - The second solution, as you commented, can introduce a race condition between the L3 agent and > keepalived process, and a possible delay in the HA switch process. >  > Regards. We have run that code [3] for a long time, and no state change delay was seen. So I may wonder is there any test results show a delay?  On Fri, 2020-03-20 at 14:40 -0400, Brian Haley wrote: > On 3/20/20 11:57 AM, Nate Johnston wrote: > > On Fri, Mar 20, 2020 at 03:37:49PM +0100, Slawek Kaplonski wrote: > > > Hi, > > > > > > We have bug [1] to solve. Basically, when node which is backup node for some > > > router, connectivity to external gateway may be broken for some time. It's like > > > that because when host is up and L3 agent is configuring qrouter namespace, it > > > flush all IPv6 addresses from the qg- interface. And due to that some MLDv2 > > > packets are sent from this interface which may cause that ToR switch will learn > > > qg port's mac address on wrong (backup) node. > > > > > > This don't happens every time and for every router because it is a race between > > > L3 agent and OVS agent. When L3 agent creates qg interface in br-int, it sets > > > tag 4095 for it and traffic sent with such vlan tag is always dropped in br-int. > > > So if L3 agent will flush IPv6 addresses before OVS agent wires the port and > > > sets correct tag for it, then all is fine. But if OVS agent is first, then MLDv2 > > > packets are sent to the wire and there is this connectivity break. > > > > > > There are proposed 2 ways of fixing this: > > >   - [2] which propsoes to add some kind of "communication" between L3 agent and > > >     OVS agent and tell OVS agent that tag can be changed only after IPv6 config > > >     is finished by L3 agent. What if ovs-agent has finished the port processing, and then L3 agent just set the port to "INTERNAL_STATUS_ACTIVE = "active"". I don't think the port will be processed again. So it will 4095 forever? Is that a race condition? > > >     Downside of this solution is that it works for OVS agent only, Linuxbridge > > >     agent may still hit the same issue. But plus is that after initial > > >     configuration of the router, everything else regarding to failover is handled > > >     by keepalived only - in same way like it is now. HA router failover is one case, add HA router a schedule instance (neutron l3-agent-router-add) to a new L3-agent is facing the same root cause of IPv6 related packets. > > >   - [3] which sets qg NIC to be DOWN always on backup nodes. So when keepalived > > >     failover active router to new node, L3 agent needs to come and switch > > >     interfaces to be UP before it will work. > > >     The plus of this solution is that it works for all OVS and > > >     Linuxbridge L2 agents (and probably for others too) but downside is that > > >     failover process is a bit longer and there may be potentially another race > > >     condition between L3 agent and keepalived. Keepalived tries to sent gARP > > >     packets after switch node to be active, first attempt will always fail as > > >     interface is still DOWN. But keepalived will retry those gARPs after some > > >     time and this should be fine if L3 agent will already bring interface to be > > >     UP. This is what the patch https://review.opendev.org/#/c/712474/ is doing now. The keepalived will try to send 5 times garp (default value of vrrp_garp_master_repeat) after transition to MASTER. And there is a delay (vrrp_garp_interval) between gratuitous ARP messages sent on an interface (https://www.keepalived.org/manpage.html). The default value is zero that means if one get failed, try next time immediately. In some extreme situations the keepalived may get failed to send the garp packets out due to the device or underlay dataplane is not ready. Actually with the help of the fix [3] and related testing, we just found out the potential lacks of the Keepalived config options. So it should be a good change to tune it. So about the race condition I may say it was not seen locally. If there are any test results, that would be very useful for distinguishing problems. > > > > Personally I find [2] more appealing.  I think that if we find many linuxbridge > > users hitting this issue then we can replicate the solution for linuxbridge at > > that time, but until then let's not worry about it - the majority of users use > > OVS.  And the gARP timegap for solution #3 to me seems like a possbility for > > problems or downtime. Linux bridge driver meets the same issue. The driver is still alive, and for stable branches the fix is also worth to do. It is very simple to simulate the issue, just link up a veth pair device, you will dump the packets on the interface. > > I would agree, it seemed easier to understand to me as well. > > -Brian For a running cloud you need to restart ovs-agent and l3-agent to achive the fix [2], and mostly the centralized network node may have tons of ports which will take significant time to "re-added" for the ovs-agent. And absolutely, restart time for L3-agent is also needed. And my opinions at the very beginning, the fix [2] is trying to expand the HA logical from L3 to L2, and introduce protential fail point in ovs-agent for HA routers. That could have some side-effect like unexpected code aggression on ovs-agent. Someday a guy may say: "I just changed the ovs-agent code, but why HA router does not work?" > > > > Both patches are waiting for pretty long time in gerrit and I want to bring more > > > visibility for both of them. Please check them and maybe You will have some > > > opinions about which solution would be better and which we should go with. > > > > > > [1] https://bugs.launchpad.net/neutron/+bug/1859832 > > > [2] https://review.opendev.org/#/c/702856/ > > > [3] https://review.opendev.org/#/c/707406/ > > > > > > -- > > > Slawek Kaplonski > > > Senior software engineer > > > Red Hat > > > > > > Seems we are repeating the discuss here, why not back to gerrit? since all the code links are pasted here. Regards, LIU Yulong -------------- next part -------------- An HTML attachment was scrubbed... URL: From johfulto at redhat.com Fri Mar 20 22:07:41 2020 From: johfulto at redhat.com (John Fulton) Date: Fri, 20 Mar 2020 18:07:41 -0400 Subject: [tripleo][operators] Removal of mistral from the TripleO Undercloud In-Reply-To: References: Message-ID: On Thu, Mar 19, 2020 at 5:37 AM Saravanan KR wrote: > > On Thu, Mar 19, 2020 at 1:02 AM John Fulton wrote: > > > > On Sat, Mar 14, 2020 at 8:06 AM Rabi Mishra wrote: > > > > > > On Sat, Mar 14, 2020 at 2:10 AM John Fulton wrote: > > >> > > >> On Fri, Mar 13, 2020 at 3:27 PM Kevin Carter wrote: > > >> > > > >> > Hello stackers, > > >> > > > >> > In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4]. > > >> > > >> As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud. > > >> > > >> > > > >> > So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases? > > > > > > > > > Yeah, as John mentioned, tripleo.derive_params.v1.derive_parameters workflow is surely being used for NFV (DPDK/SR-IOV) and HCI use cases and can't be deprecated or dropped. Though we've a generic interface in tripleoclient to run any workflow in plan-environment, I have not seen it being used for anything other than the mentioned workflow. > > > > > > In the scope of 'mistral-ansible' work, we seem to have two options. > > > > > > 1. Convert the workflow to ansible playbook 'as-is' i.e calculate and merge the derived parameters in plan-environment and as you've mentioned, change tripleoclient code to call any playbook in plan-environment.yaml and the parameters/vars. > > > > Nice idea. I hadn't thought of that. > > > > If there's a "hello world" example of this (which results in a THT > > param in the deployment plan being set to "hello world"), then I could > > try writing an ansible module to derive the HCI parameters and set > > them in place of the "hello world". > > > I am fine with the approach, but the only concern is, we have plans to > remove Heat in the coming cycles. One of inputs for the Mistral derive > parameters is fetched from the heat stack. If we are going to retain > it, then it has to be re-worked during the Heat removal. Mistral to > ansible could be the first step towards it. Hey Saravanan, That works for me. I'm glad we were able to come up with a way to do this. Kevin put some patches together today that will help a lot on this. 1. tht: https://review.opendev.org/#/c/714217/ 2. tripleo-ansible: https://review.opendev.org/#/c/714232/ 3. trilpeoclient: https://review.opendev.org/#/c/714198/ If I put these on my undercloud, then I think I can run: 'openstack overcloud deploy ... -p plan-environment-derived-params.yaml' as usual and then the updated tripleoclient and tht patch should trigger the new tripleo-ansible playbook in place of the Mistral workbook. I think I can then just update that tripleo-ansible patch to have it include a new derive_params_hci role and add a new derive_params_hci module where I'll stick code from the original Python prototype I did for it. I'll probably just shell to `openstack baremetal introspection data save ID` from ansible to get the Ironic data. I'll give it a try next week and update this thread. Even if Heat is not in the flow, at least the Ansible role and module can be reused. Note that it uses the new tripleo_plan_parameters_update module that Rabi wrote so that should make it easier to deal with the deployment plan itself (https://review.opendev.org/712604). John > Regards, > Saravanan KR > > > John > > > > > 2. Move the functionality further down the component chain in TripleO to have the required ansible host/group_vars set for them to be used by config-download playbooks/ansible/puppet. > > > > > > I guess option 1 would be easier within the timelines. I've done some preliminary work to move some of the functionality in relevant mistral actions to utils modules[1], so that they can be called from ansible modules without depending on mistral/mistral-lib and use those in a playbook that kinda replicate the tasks in the mistral workflow. > > > > > > Having said that, it would be good to know what the DFG:NFV folks think, as they are the original authors/maintainers of that workflow. > > > > > > > > > > > >> The Mistral based workflow took advantage of the deployment plan which > > >> was stored in Swift on the undercloud. My understanding is that too is > > >> going away. > > > > > > > > > I'm not sure that would be in the scope of 'mstral-to-ansible' work. Dropping swift would probably be a bit more complex, as we use it to store templates, plan-environment, plan backups (possibly missing few more) etc and would require significant design rework (may be possible when we get rid of heat in undercloud). In spite of heat using the templates from swift and merging environments on the client side, we've had already bumped heat's REST API json body size limit (max_json_body_size) on the undercloud to 4MB[2] from the default 1MB and sending all required templates as part of API request would not be a good idea from undercloud scalability pov. > > > > > > [1] https://review.opendev.org/#/c/709546/ > > > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml#L109 > > > > > > -- > > > Regards, > > > Rabi Mishra > > > > > > From Arkady.Kanevsky at dell.com Fri Mar 20 22:30:49 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Fri, 20 Mar 2020 22:30:49 +0000 Subject: [tripleo] launchpad bug policy In-Reply-To: References: Message-ID: <493e37c90c1a4bb0ad01209f2f84cc0a@AUSX13MPS308.AMER.DELL.COM> Thanks for clarification. Would be good to have uniformity across all OpenStack projects. From: Wesley Hayutin Sent: Friday, March 20, 2020 3:10 PM To: Luke Short Cc: Kanevsky, Arkady; OpenStack Discuss Subject: Re: [tripleo] launchpad bug policy [EXTERNAL EMAIL] On Fri, Mar 20, 2020 at 1:49 PM Luke Short > wrote: Greetings Arkady, For bug tracking, I believe TripleO will continue to use Launchpad. My team has been using Storyboard for tracking progress on big-picture items with smaller tasks assigned (for example, removing Mistral and Zaqar from the Undercloud) with great success. Those two platforms have different focuses and use-cases. Both of which can add value to any contributor in the OpenStack community. Sincerely, Luke Short Yes, building on what Luke said we'll continue to use launchpad. I know there was a lot of planning around storyboard at the last PTG, and we'll continue to monitor and evaluate its use in the future. Thanks On Fri, Mar 20, 2020 at 3:42 PM > wrote: Wes, Do we have a plan to move to storyboard? From: Wesley Hayutin > Sent: Friday, March 20, 2020 10:51 AM To: OpenStack Discuss Subject: [tripleo] launchpad bug policy [EXTERNAL EMAIL] Greetings, Just a quick refresher on TripleO's bug policy. =================================== Please self-triage the bug: - Set the bug to Triaged - Set an Importance (note: "Critical" is something that prevents any TripleO deployment, use it carefully) - Assign the bug to yourself if you plan to work on it - Set a milestone where you think the bugs can be or needs to be fixed. If any doubt, check with the PTL. - Set one or multiple tags related to the bug. See available tags on https://bugs.launchpad.net/tripleo (right column). If you can't update these informations, it's because you're not member of TripleO in Launchpad group. Please ping the PTL and you'll be added. Please double-check if you provided the minimum of needed information in the description of this report. Nova provides an excellent example of Bug Template. It can be found at: https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate =================================== The following bugs were opened in the last 365 days incorrectly. We are at 60 bugs :( http://dashboard-ci.tripleo.org/d/cockpit/cockpit?panelId=357&fullscreen&orgId=1 If you are on this list, please help me out and self triage. If you have found something critical and want to ensure it's on a watched list, set the "alert" tag. Thanks all!! rabi https://bugs.launchpad.net/tripleo/+bug/1830837 bogdando https://bugs.launchpad.net/tripleo/+bug/1838972 jfrancoa https://bugs.launchpad.net/tripleo/+bug/1846206 bogdando https://bugs.launchpad.net/tripleo/+bug/1868075 adamjr https://bugs.launchpad.net/tripleo/+bug/1831841 anta-nok https://bugs.launchpad.net/tripleo/+bug/1832931 rabi https://bugs.launchpad.net/tripleo/+bug/1834054 waleedm https://bugs.launchpad.net/tripleo/+bug/1836011 priyotoshtsp https://bugs.launchpad.net/tripleo/+bug/1836350 donny-g https://bugs.launchpad.net/tripleo/+bug/1837760 mschuppert https://bugs.launchpad.net/tripleo/+bug/1838272 alee-3 https://bugs.launchpad.net/tripleo/+bug/1838804 gongysh https://bugs.launchpad.net/tripleo/+bug/1839126 asrmarco13 https://bugs.launchpad.net/tripleo/+bug/1839298 pkopec https://bugs.launchpad.net/tripleo/+bug/1840600 jfreud https://bugs.launchpad.net/tripleo/+bug/1840714 sai438 https://bugs.launchpad.net/tripleo/+bug/1841395 sclarke1 https://bugs.launchpad.net/tripleo/+bug/1842988 anta-nok https://bugs.launchpad.net/tripleo/+bug/1844438 cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1845268 bgill https://bugs.launchpad.net/tripleo/+bug/1847289 versus-vv https://bugs.launchpad.net/tripleo/+bug/1848546 sandeepyadav93 https://bugs.launchpad.net/tripleo/+bug/1849668 zodiac https://bugs.launchpad.net/tripleo/+bug/1850080 mschuppert https://bugs.launchpad.net/tripleo/+bug/1850610 alee-3 https://bugs.launchpad.net/tripleo/+bug/1850647 arequipeno https://bugs.launchpad.net/tripleo/+bug/1851503 damani42 https://bugs.launchpad.net/tripleo/+bug/1851618 adamjr https://bugs.launchpad.net/tripleo/+bug/1852509 david-hill-ubisoft https://bugs.launchpad.net/tripleo/+bug/1852762 adamjr https://bugs.launchpad.net/tripleo/+bug/1852893 ksambor https://bugs.launchpad.net/tripleo/+bug/1853000 pandasoubhagya https://bugs.launchpad.net/tripleo/+bug/1853562 kajinamit https://bugs.launchpad.net/tripleo/+bug/1854117 jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856095 mschuppert https://bugs.launchpad.net/tripleo/+bug/1856322 jimbagwell https://bugs.launchpad.net/tripleo/+bug/1856411 shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1856680 hkominos https://bugs.launchpad.net/tripleo/+bug/1856734 hontu https://bugs.launchpad.net/tripleo/+bug/1856921 xarlos https://bugs.launchpad.net/tripleo/+bug/1857003 versus-vv https://bugs.launchpad.net/tripleo/+bug/1857153 bshephar https://bugs.launchpad.net/tripleo/+bug/1858010 shyam.biradar https://bugs.launchpad.net/tripleo/+bug/1859608 cagri-ersen https://bugs.launchpad.net/tripleo/+bug/1860486 denis-karpov https://bugs.launchpad.net/tripleo/+bug/1860528 bshephar https://bugs.launchpad.net/tripleo/+bug/1860609 suzumushi https://bugs.launchpad.net/tripleo/+bug/1861315 ekultails https://bugs.launchpad.net/tripleo/+bug/1862179 mschuppert https://bugs.launchpad.net/tripleo/+bug/1862321 slavonic https://bugs.launchpad.net/tripleo/+bug/1863122 ffernand https://bugs.launchpad.net/tripleo/+bug/1863243 cgoncalves https://bugs.launchpad.net/tripleo/+bug/1863595 larsks https://bugs.launchpad.net/tripleo/+bug/1864232 jbemmel https://bugs.launchpad.net/tripleo/+bug/1864924 kajinamit https://bugs.launchpad.net/tripleo/+bug/1865477 cgoncalves https://bugs.launchpad.net/tripleo/+bug/1867144 prabiegx https://bugs.launchpad.net/tripleo/+bug/1867333 valleedelisle https://bugs.launchpad.net/tripleo/+bug/1867362 amoralej https://bugs.launchpad.net/tripleo/+bug/1867608 -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri Mar 20 22:46:49 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 20 Mar 2020 22:46:49 +0000 Subject: [tripleo] launchpad bug policy In-Reply-To: <493e37c90c1a4bb0ad01209f2f84cc0a@AUSX13MPS308.AMER.DELL.COM> References: <493e37c90c1a4bb0ad01209f2f84cc0a@AUSX13MPS308.AMER.DELL.COM> Message-ID: <20200320224648.rrjox4weklwmtrpn@yuggoth.org> On 2020-03-20 22:30:49 +0000 (+0000), Arkady.Kanevsky at dell.com wrote: [...] > Would be good to have uniformity across all OpenStack projects. [...] For the time being I think we've collectively given up on achieving that degree of uniformity for task and defect tracking throughout all of OpenStack. Different teams have sufficiently varied workflows and cadences that they need particular tools to be efficient, and I'm fine with embracing that reality rather than trying to force something for the sake of consistency. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From tpb at dyncloud.net Sat Mar 21 01:29:22 2020 From: tpb at dyncloud.net (Tom Barron) Date: Fri, 20 Mar 2020 21:29:22 -0400 Subject: [tripleo] launchpad bug policy In-Reply-To: <20200320224648.rrjox4weklwmtrpn@yuggoth.org> References: <493e37c90c1a4bb0ad01209f2f84cc0a@AUSX13MPS308.AMER.DELL.COM> <20200320224648.rrjox4weklwmtrpn@yuggoth.org> Message-ID: <20200321012922.4vrq2fz32pxymhaa@barron.net> On 20/03/20 22:46 +0000, Jeremy Stanley wrote: >On 2020-03-20 22:30:49 +0000 (+0000), Arkady.Kanevsky at dell.com wrote: >[...] >> Would be good to have uniformity across all OpenStack projects. >[...] > >For the time being I think we've collectively given up on achieving >that degree of uniformity for task and defect tracking throughout >all of OpenStack. Different teams have sufficiently varied workflows >and cadences that they need particular tools to be efficient, and >I'm fine with embracing that reality rather than trying to force >something for the sake of consistency. >-- >Jeremy Stanley +1, there are better hobgoblins to chase atm :D From saphi070 at gmail.com Sat Mar 21 11:37:52 2020 From: saphi070 at gmail.com (Sa Pham) Date: Sat, 21 Mar 2020 18:37:52 +0700 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Hello Ignazio, Does your openstack environment using self-service network ? I have tried openvswitch firewall native with openstack queens version using provider network. But It's not working good. On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano wrote: > Hello Jakub, > I will try again but if there is a bug on queens I do not think it will be > corrected because is going out of support. > Thanks > Ignazio > > Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < > jlibosva at redhat.com> ha scritto: > >> On 13/03/2020 08:24, Ignazio Cassano wrote: >> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >> switched >> > on openvswitch works fine . The problem is this migration create the >> qbr on >> > the mode switched to openvswitch. >> > But when I switch another compute node to openvswitch and I try to live >> > migrate the same vm (openvswitch to qopenswitch) it does not work >> because >> > the qbr presence. >> > I verified on nova logs. >> > Ignazio >> >> Hi Ignazio, >> >> I think the first step - migrating from hybrid_iptables to ovs should >> not create the qbr on the target node. It sounds like a bug - IIRC the >> libvirt domxml should not have the qbr when migrating. >> >> >> > >> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha >> scritto: >> > >> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >> >>> Hello All, I am facing some problems migrating from iptables_hybrid >> >>> frirewall to openvswitch firewall on centos 7 queens, >> >>> I am doing this because I want enable security groups logs which >> require >> >>> openvswitch firewall. >> >>> I would like to migrate without restarting my instances. >> >>> I startded moving all instances from compute node 1. >> >>> Then I configured openvswitch firewall on compute node 1, >> >>> Instances migrated from compute node 2 to compute node 1 without >> >> problems. >> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >> >>> But now instances does not migrate from node 1 to node 2 because it >> >>> requires the presence of qbr bridge on node 2 >> >>> >> >>> This happened because migrating instances from node2 with >> iptables_hybrid >> >>> to compute node 1 with openvswitch, does not put the tap under br-int >> as >> >>> requested by openvswich firewall, but qbr is still present on compute >> >> node >> >>> 1. >> >>> Once I enabled openvswitch on compute node 2, migration from compute >> >> node 1 >> >>> fails because it exprects qbr on compute node 2 . >> >>> So I think I should moving on the fly tap interfaces from qbr to >> br-int >> >> on >> >>> compute node 1 before migrating to compute node 2 but it is a huge >> work >> >> on >> >>> a lot of instances. >> >>> >> >>> Any workaround, please ? >> >>> >> >>> Ignazio >> >>> >> >> >> >> I may be a little outdated here but to the best of my knowledge there >> >> are two ways how to migrate from iptables to openvswitch. >> >> >> >> 1) If you don't mind the intermediate linux bridge and you care about >> >> logs, you can just change the config file on compute node to start >> using >> >> openvswitch firewall and restart the ovs agent. That should trigger a >> >> mechanism that deletes iptables rules and starts using openflow rules. >> >> It will leave the intermediate bridge there but except the extra hop in >> >> networking stack, it doesn't mind. >> >> >> >> 2) With multiple-port binding feature, what you described above should >> >> be working. I know Miguel spent some time working on that so perhaps he >> >> has more information about which release it should be functional at, I >> >> think it was Queens. Not sure if any Nova work was required to make it >> >> work. >> >> >> >> Hope that helps. >> >> Kuba >> >> >> >> >> >> >> >> >> > >> >> -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 12:32:48 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 13:32:48 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Hello Sa, I am using self service and provider networks.It works fine in both cases. The problem is the migration from iptables hybrid to openvswitch without rebooting instanes. Do you mean security groups do not work on provider networks ? Ignazio Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: > Hello Ignazio, > > Does your openstack environment using self-service network ? > > I have tried openvswitch firewall native with openstack queens version > using provider network. But It's not working good. > > > > On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano > wrote: > >> Hello Jakub, >> I will try again but if there is a bug on queens I do not think it will >> be corrected because is going out of support. >> Thanks >> Ignazio >> >> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >> jlibosva at redhat.com> ha scritto: >> >>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>> switched >>> > on openvswitch works fine . The problem is this migration create the >>> qbr on >>> > the mode switched to openvswitch. >>> > But when I switch another compute node to openvswitch and I try to live >>> > migrate the same vm (openvswitch to qopenswitch) it does not work >>> because >>> > the qbr presence. >>> > I verified on nova logs. >>> > Ignazio >>> >>> Hi Ignazio, >>> >>> I think the first step - migrating from hybrid_iptables to ovs should >>> not create the qbr on the target node. It sounds like a bug - IIRC the >>> libvirt domxml should not have the qbr when migrating. >>> >>> >>> > >>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha >>> scritto: >>> > >>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>> >>> Hello All, I am facing some problems migrating from iptables_hybrid >>> >>> frirewall to openvswitch firewall on centos 7 queens, >>> >>> I am doing this because I want enable security groups logs which >>> require >>> >>> openvswitch firewall. >>> >>> I would like to migrate without restarting my instances. >>> >>> I startded moving all instances from compute node 1. >>> >>> Then I configured openvswitch firewall on compute node 1, >>> >>> Instances migrated from compute node 2 to compute node 1 without >>> >> problems. >>> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>> >>> But now instances does not migrate from node 1 to node 2 because it >>> >>> requires the presence of qbr bridge on node 2 >>> >>> >>> >>> This happened because migrating instances from node2 with >>> iptables_hybrid >>> >>> to compute node 1 with openvswitch, does not put the tap under >>> br-int as >>> >>> requested by openvswich firewall, but qbr is still present on >>> compute >>> >> node >>> >>> 1. >>> >>> Once I enabled openvswitch on compute node 2, migration from compute >>> >> node 1 >>> >>> fails because it exprects qbr on compute node 2 . >>> >>> So I think I should moving on the fly tap interfaces from qbr to >>> br-int >>> >> on >>> >>> compute node 1 before migrating to compute node 2 but it is a huge >>> work >>> >> on >>> >>> a lot of instances. >>> >>> >>> >>> Any workaround, please ? >>> >>> >>> >>> Ignazio >>> >>> >>> >> >>> >> I may be a little outdated here but to the best of my knowledge there >>> >> are two ways how to migrate from iptables to openvswitch. >>> >> >>> >> 1) If you don't mind the intermediate linux bridge and you care about >>> >> logs, you can just change the config file on compute node to start >>> using >>> >> openvswitch firewall and restart the ovs agent. That should trigger a >>> >> mechanism that deletes iptables rules and starts using openflow rules. >>> >> It will leave the intermediate bridge there but except the extra hop >>> in >>> >> networking stack, it doesn't mind. >>> >> >>> >> 2) With multiple-port binding feature, what you described above should >>> >> be working. I know Miguel spent some time working on that so perhaps >>> he >>> >> has more information about which release it should be functional at, I >>> >> think it was Queens. Not sure if any Nova work was required to make it >>> >> work. >>> >> >>> >> Hope that helps. >>> >> Kuba >>> >> >>> >> >>> >> >>> >> >>> > >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jlibosva at redhat.com Thu Mar 19 12:54:12 2020 From: jlibosva at redhat.com (Jakub Libosvar) Date: Thu, 19 Mar 2020 13:54:12 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: Message-ID: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> On 13/03/2020 08:24, Ignazio Cassano wrote: > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node switched > on openvswitch works fine . The problem is this migration create the qbr on > the mode switched to openvswitch. > But when I switch another compute node to openvswitch and I try to live > migrate the same vm (openvswitch to qopenswitch) it does not work because > the qbr presence. > I verified on nova logs. > Ignazio Hi Ignazio, I think the first step - migrating from hybrid_iptables to ovs should not create the qbr on the target node. It sounds like a bug - IIRC the libvirt domxml should not have the qbr when migrating. > > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha scritto: > >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>> Hello All, I am facing some problems migrating from iptables_hybrid >>> frirewall to openvswitch firewall on centos 7 queens, >>> I am doing this because I want enable security groups logs which require >>> openvswitch firewall. >>> I would like to migrate without restarting my instances. >>> I startded moving all instances from compute node 1. >>> Then I configured openvswitch firewall on compute node 1, >>> Instances migrated from compute node 2 to compute node 1 without >> problems. >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>> But now instances does not migrate from node 1 to node 2 because it >>> requires the presence of qbr bridge on node 2 >>> >>> This happened because migrating instances from node2 with iptables_hybrid >>> to compute node 1 with openvswitch, does not put the tap under br-int as >>> requested by openvswich firewall, but qbr is still present on compute >> node >>> 1. >>> Once I enabled openvswitch on compute node 2, migration from compute >> node 1 >>> fails because it exprects qbr on compute node 2 . >>> So I think I should moving on the fly tap interfaces from qbr to br-int >> on >>> compute node 1 before migrating to compute node 2 but it is a huge work >> on >>> a lot of instances. >>> >>> Any workaround, please ? >>> >>> Ignazio >>> >> >> I may be a little outdated here but to the best of my knowledge there >> are two ways how to migrate from iptables to openvswitch. >> >> 1) If you don't mind the intermediate linux bridge and you care about >> logs, you can just change the config file on compute node to start using >> openvswitch firewall and restart the ovs agent. That should trigger a >> mechanism that deletes iptables rules and starts using openflow rules. >> It will leave the intermediate bridge there but except the extra hop in >> networking stack, it doesn't mind. >> >> 2) With multiple-port binding feature, what you described above should >> be working. I know Miguel spent some time working on that so perhaps he >> has more information about which release it should be functional at, I >> think it was Queens. Not sure if any Nova work was required to make it >> work. >> >> Hope that helps. >> Kuba >> >> >> >> > From kbaegis at gmail.com Thu Mar 19 17:58:28 2020 From: kbaegis at gmail.com (Stephen Nemeth) Date: Thu, 19 Mar 2020 11:58:28 -0600 Subject: Diskimage-builder complex disk setup In-Reply-To: References: Message-ID: <628e153a-16de-4bdc-96d4-0bdc2d5dc3b2@Spark> Hi all, We’re attempting to create user images to deploy with metal^3/ironic. I’m making almost no headway starting from any of the examples provided for the DIB_BLOCK_DEVICE_CONFIG setup. Is there a repository with practical values located anywhere? I’m finding the documentation insufficient to do anything more complicated than a single partition setup. Here’s what I’ve gotten so far. Help greatly appreciated: ``` [   {     "local_loop": {       "name": "image0",       "size": "20GiB"     }   },   {     "partitioning": {       "base": "image0",       "label": "gpt",       "partitions": [         {           "name": "ESP",           "type": "EF00",           "size": "16MiB"         },         {           "name": "boot",           "type": "EF02",           "size": "6GiB"         },         {           "name": "lvm",           "type": 8,           "size": "100%"         }       ]     }   },   {     "lvm": {       "name": "lvm2",       "pvs": [         {           "name": "pv",           "options": [             "--force"           ],           "device": "lvm",           "base": "image0"         }       ],       "vgs": [         {           "name": "vg",           "base": [             "pv"           ],           "options": [             "--force"           ]         }       ],       "lvs": [         {           "name": "root",           "base": "vg",           "extents": "100%FREE",           "options": [             "--zero=n"           ]         }       ]     }   },   {     "mkfs": {         {           "name": "mkfs_root",           "base": "root",           "type": "btrfs",           "label": "root",           "opts": "-f",           "mount": { "mount_point": "/" }         },         {           "name": "mkfs_efi",           "base": "ESP",           "type": "vfat",           "label": "efi",           "opts": "-f",           "mount": { "mount_point": "/boot/efi/" }         },         {           "name": "mkfs_boot",           "base": "boot",           "type": "btrfs",           "label": "boot",           "opts": "-f",           "mount": { "mount_point": "/boot/" }         }     }   } ] ``` -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Fri Mar 20 14:38:04 2020 From: satish.txt at gmail.com (Satish Patel) Date: Fri, 20 Mar 2020 10:38:04 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: Grant, But i am seeing lots of following logs on my compute nodes running stein release. 2020-03-20 10:34:46.132 53425 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: ConnectionForced: Too many heartbeats missed. Find attached screenshot of one of rabbitMQ node, i have lots of messages in "Ready: 15339" is this looks normal to you? On Fri, Mar 20, 2020 at 4:55 AM Grant Morley wrote: > Hi, > > There was a bug in Queens that meant there was an issue with the heartbeat > timeouts. Setting it to 0 gets around that bug. I believe that was fixed in > Rocky and above, so your Stein installation should be fine. > > Setting the value to 0 For us meant we stopped getting errors in the logs > for: > > "Too many heartbeats missed, trying to force connect to RabbitMQ" > > Regards, > On 19/03/2020 18:53, Satish Patel wrote: > > I have question related following setting, why are you disabling > heartbeat timeout? > > heartbeat_timeout_threshold = 0 > > On Thu, Mar 19, 2020 at 1:32 PM Satish Patel wrote: > >> Great, thanks! Did you guys tune your nova component for rabbitMQ? >> >> On Thu, Mar 19, 2020 at 1:26 PM Grant Morley wrote: >> >>> We left ours on the default value of 1 and that still seems to be fine. >>> >>> Grant >>> On 19/03/2020 17:13, Satish Patel wrote: >>> >>> how about rpc_worker ? >>> >>> currently i have rpc_worker=1 >>> >>> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley wrote: >>> >>>> Correct, you need to add: >>>> >>>> > > heartbeat_timeout_threshold = 0 >>>> > > rpc_conn_pool_size = 300 >>>> > > rpc_thread_pool_size = 2048 >>>> > > rpc_response_timeout = 3600 >>>> > > rpc_poll_timeout = 60 >>>> >>>> To your Neutron nodes >>>> >>>> And you can add: >>>> >>>> >>>> >> executor_thread_pool_size = 64 >>>> >> rpc_response_timeout = 3600 >>>> >>>> To your compute nodes (neutron.conf) However I found just adding the >>>> changes to the neturon servers really helped. >>>> >>>> I would recommend just starting with your neutron nodes first to see if >>>> that helps. If you find your compute nodes are still having issues then >>>> change the settings on those after. >>>> >>>> Regards, >>>> On 19/03/2020 16:53, Satish Patel wrote: >>>> >>>> I am running openstack-ansible (Queens / Stein both) so this is what i >>>> am going to do, am i doing correctly? >>>> >>>> neutron-server (container) I have 3 neutron node. >>>> > > heartbeat_timeout_threshold = 0 >>>> > > rpc_conn_pool_size = 300 >>>> > > rpc_thread_pool_size = 2048 >>>> > > rpc_response_timeout = 3600 >>>> > > rpc_poll_timeout = 60 >>>> >>>> 330 compute nodes (agent neutron.conf) going to add following: >>>> >> executor_thread_pool_size = 64 >>>> >> rpc_response_timeout = 3600 >>>> >>>> >>>> >>>> How about nova? should i be doing that on nova as well to reduce load >>>> on rabbitMQ? >>>> >>>> >>>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: >>>> >>>>> Hi Satish, >>>>> >>>>> You will need to add those to the "neutron.conf" file on your network >>>>> nodes. If you are running OS-A I would do it on your "neutron-server" nodes >>>>> and add the following to your agents containers: >>>>> >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> Regards, >>>>> On 19/03/2020 16:27, Satish Patel wrote: >>>>> >>>>> Erik, >>>>> >>>>> If i want to adopt following setting then where i should add them in >>>>> Queens openstack, neutron-server or all my compute nodes? which >>>>> setting will go where? >>>>> >>>>> heartbeat_timeout_threshold = 0 >>>>> rpc_conn_pool_size = 300 >>>>> rpc_thread_pool_size = 2048 >>>>> rpc_response_timeout = 3600 >>>>> rpc_poll_timeout = 60 >>>>> >>>>> ## Rpc all >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: >>>>> >>>>> We are hitting something awfully similar. >>>>> >>>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>>> >>>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>>> >>>>> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>>> >>>>> I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>>> >>>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>>> >>>>> Best Regards, Erik Olof Gunnar Andersson >>>>> >>>>> -----Original Message----- >>>>> From: Satish Patel >>>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>>> To: Grant Morley >>>>> Cc: openstack-discuss at lists.openstack.org >>>>> Subject: Re: Neutron RabbitMQ issues >>>>> >>>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>>> >>>>> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>>> >>>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>> >>>>> Hi all, >>>>> >>>>> We are currently experiencing some fairly major issues with our >>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>> are seeing a lot of time out messages in responses to replies and >>>>> because of this instance creation or anything to do with instances and >>>>> networking is broken. >>>>> >>>>> We are running OpenStack Queens. >>>>> >>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>> >>>>> heartbeat_timeout_threshold = 0 >>>>> rpc_conn_pool_size = 300 >>>>> rpc_thread_pool_size = 2048 >>>>> rpc_response_timeout = 3600 >>>>> rpc_poll_timeout = 60 >>>>> >>>>> ## Rpc all >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> What we are seeing in the error logs for neutron for all services >>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>> 9aOA$ >>>>> >>>>> We have manually tried to get everything in sync by forcing fail-over >>>>> of the networking which seems to get routers in sync. >>>>> >>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>> >>>>> Some times restarting of the services on neutron gets these back >>>>> acknowledged again, however the timeouts come back. >>>>> >>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>> file descriptors and errlang processes have plenty of resources available. >>>>> >>>>> We are also seeing a lot of rpc issues: >>>>> >>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>> before next attempt. If the server is not down, consider increasing >>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>> If the server is not down, consider increasing the >>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>> >>>>> Does anyone know if there is anymore tuning to be done at all? >>>>> Upgrading for us at the moment to a newer version isn't really an >>>>> option unfortunately. >>>>> >>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>> know that will be putting a load on the system. However these problems >>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>> >>>>> If anyone has any use cases for this or any more recommendations that >>>>> would be great. >>>>> >>>>> Many thanks, >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Grant Morley >>>>> Cloud Lead, Civo Ltd >>>>> www.civo.com | Signup for an account! >>>>> >>>> -- >>>> >>>> Grant Morley >>>> Cloud Lead, Civo Ltd >>>> www.civo.com | Signup for an account! >>>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an account! >>> >> -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2020-03-19 at 6.04.48 PM.png Type: image/png Size: 737789 bytes Desc: not available URL: From grant at civo.com Fri Mar 20 15:39:59 2020 From: grant at civo.com (Grant Morley) Date: Fri, 20 Mar 2020 15:39:59 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> Message-ID: <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> If you tune rabbit then for: heartbeat_timeout_threshold = 0 That should help with the error message you are getting. That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. Are all of the ready messages for "notifications.info" for the various services ( Nova, Neutron, Keystone etc ) If that is the case you can disable those messages in your config files for those services. Look for: # Notifications [oslo_messaging_notifications] notification_topics = notifications driver = noop Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. Hope that helps. Grant On 20/03/2020 14:38, Satish Patel wrote: > Grant, > > But i am seeing lots of following logs on my compute nodes running > stein release. > > 2020-03-20 10:34:46.132 53425 WARNING > oslo.messaging._drivers.impl_rabbit [-] Unexpected error during > heartbeart thread processing, retrying...: ConnectionForced: Too many > heartbeats missed. > > Find attached screenshot of one of rabbitMQ node, i have lots of > messages in "Ready: 15339"  is this looks normal to you? > > > > > On Fri, Mar 20, 2020 at 4:55 AM Grant Morley > wrote: > > Hi, > > There was a bug in Queens that meant there was an issue with the > heartbeat timeouts. Setting it to 0 gets around that bug. I > believe that was fixed in Rocky and above, so your Stein > installation should be fine. > > Setting the value to 0 For us meant we stopped getting errors in > the logs for: > > "Too many heartbeats missed, trying to force connect to RabbitMQ" > > Regards, > > On 19/03/2020 18:53, Satish Patel wrote: >> I have question related following setting, why are you disabling >> heartbeat timeout? >> >> heartbeat_timeout_threshold = 0 >> >> On Thu, Mar 19, 2020 at 1:32 PM Satish Patel >> > wrote: >> >> Great, thanks!  Did you guys tune your nova component for >> rabbitMQ? >> >> On Thu, Mar 19, 2020 at 1:26 PM Grant Morley > > wrote: >> >> We left ours on the default value of 1 and that still >> seems to be fine. >> >> Grant >> >> On 19/03/2020 17:13, Satish Patel wrote: >>> how about rpc_worker ? >>> currently i have rpc_worker=1 >>> >>> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley >>> > wrote: >>> >>> Correct, you need to add: >>> >>> > > heartbeat_timeout_threshold = 0 >>> > > rpc_conn_pool_size = 300 >>> > > rpc_thread_pool_size = 2048 >>> > > rpc_response_timeout = 3600 >>> > > rpc_poll_timeout = 60 >>> >>> To your Neutron nodes >>> >>> And you can add: >>> >>> >>> >> executor_thread_pool_size = 64 >>> >> rpc_response_timeout = 3600 >>> >>> To your compute nodes (neutron.conf) However I found >>> just adding the changes to the neturon servers >>> really helped. >>> >>> I would recommend just starting with your neutron >>> nodes first to see if that helps. If you find your >>> compute nodes are still having issues then change >>> the settings on those after. >>> >>> Regards, >>> >>> On 19/03/2020 16:53, Satish Patel wrote: >>>> I am running openstack-ansible (Queens / Stein >>>> both) so this is what i am going to do, am i doing >>>> correctly? >>>> >>>> neutron-server (container) I have 3 neutron node. >>>> > > heartbeat_timeout_threshold = 0 >>>> > > rpc_conn_pool_size = 300 >>>> > > rpc_thread_pool_size = 2048 >>>> > > rpc_response_timeout = 3600 >>>> > > rpc_poll_timeout = 60 >>>> >>>> 330 compute nodes (agent neutron.conf) going to add >>>> following: >>>> >> executor_thread_pool_size = 64 >>>> >> rpc_response_timeout = 3600 >>>> >>>> >>>> >>>> How about nova? should i be doing that on nova as >>>> well to reduce load on rabbitMQ? >>>> >>>> >>>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley >>>> > wrote: >>>> >>>> Hi Satish, >>>> >>>> You will need to add those to the >>>> "neutron.conf" file on your network nodes. If >>>> you are running OS-A I would do it on your >>>> "neutron-server" nodes and add the following to >>>> your agents containers: >>>> >>>> executor_thread_pool_size = 64 >>>> rpc_response_timeout = 3600 >>>> >>>> Regards, >>>> >>>> On 19/03/2020 16:27, Satish Patel wrote: >>>>> Erik, >>>>> >>>>> If i want to adopt following setting then where i should add them in >>>>> Queens openstack, neutron-server or all my compute nodes? which >>>>> setting will go where? >>>>> >>>>> heartbeat_timeout_threshold = 0 >>>>> rpc_conn_pool_size = 300 >>>>> rpc_thread_pool_size = 2048 >>>>> rpc_response_timeout = 3600 >>>>> rpc_poll_timeout = 60 >>>>> >>>>> ## Rpc all >>>>> executor_thread_pool_size = 64 >>>>> rpc_response_timeout = 3600 >>>>> >>>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson >>>>> wrote: >>>>>> We are hitting something awfully similar. >>>>>> >>>>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>>>> >>>>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>>>> >>>>>> e.g.https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>>>> >>>>>> I wrote two quick scripts to audit these issues. >>>>>> http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). >>>>>> http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>>>> >>>>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>>>> >>>>>> Best Regards, Erik Olof Gunnar Andersson >>>>>> >>>>>> -----Original Message----- >>>>>> From: Satish Patel >>>>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>>>> To: Grant Morley >>>>>> Cc:openstack-discuss at lists.openstack.org >>>>>> Subject: Re: Neutron RabbitMQ issues >>>>>> >>>>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>>>> >>>>>> This is my favorite video, not sure you have seen this before or not but anyway posting here -https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>>>> >>>>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> We are currently experiencing some fairly major issues with our >>>>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>>>> are seeing a lot of time out messages in responses to replies and >>>>>>> because of this instance creation or anything to do with instances and >>>>>>> networking is broken. >>>>>>> >>>>>>> We are running OpenStack Queens. >>>>>>> >>>>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>>>> >>>>>>> heartbeat_timeout_threshold = 0 >>>>>>> rpc_conn_pool_size = 300 >>>>>>> rpc_thread_pool_size = 2048 >>>>>>> rpc_response_timeout = 3600 >>>>>>> rpc_poll_timeout = 60 >>>>>>> >>>>>>> ## Rpc all >>>>>>> executor_thread_pool_size = 64 >>>>>>> rpc_response_timeout = 3600 >>>>>>> >>>>>>> What we are seeing in the error logs for neutron for all services >>>>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>>>> >>>>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>>>> 9aOA$ >>>>>>> >>>>>>> We have manually tried to get everything in sync by forcing fail-over >>>>>>> of the networking which seems to get routers in sync. >>>>>>> >>>>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>>>> >>>>>>> Some times restarting of the services on neutron gets these back >>>>>>> acknowledged again, however the timeouts come back. >>>>>>> >>>>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>>>> file descriptors and errlang processes have plenty of resources available. >>>>>>> >>>>>>> We are also seeing a lot of rpc issues: >>>>>>> >>>>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>>>> before next attempt. If the server is not down, consider increasing >>>>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>>>> If the server is not down, consider increasing the >>>>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>>>> >>>>>>> Does anyone know if there is anymore tuning to be done at all? >>>>>>> Upgrading for us at the moment to a newer version isn't really an >>>>>>> option unfortunately. >>>>>>> >>>>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>>>> know that will be putting a load on the system. However these problems >>>>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>>>> >>>>>>> If anyone has any use cases for this or any more recommendations that >>>>>>> would be great. >>>>>>> >>>>>>> Many thanks, >>>>>>> >>>>>>> >>>> -- >>>> >>>> Grant Morley >>>> Cloud Lead, Civo Ltd >>>> www.civo.com | Signup >>>> for an account! >>>> >>> -- >>> >>> Grant Morley >>> Cloud Lead, Civo Ltd >>> www.civo.com | Signup for an >>> account! >>> >> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an >> account! >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > > -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Fri Mar 20 16:22:54 2020 From: satish.txt at gmail.com (Satish Patel) Date: Fri, 20 Mar 2020 12:22:54 -0400 Subject: Neutron RabbitMQ issues In-Reply-To: <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: Oh you are right here, i have following stuff in my neutron.conf on server # Notifications [oslo_messaging_notifications] driver = messagingv2 topics = notifications transport_url = rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671, neutron:5be2a043f9a93adbd at 172.28.15.248:5671, neutron:5be2a043f9a93adbd at 172.28.15.22:5671//neutron?ssl=1 # Messaging [oslo_messaging_rabbit] rpc_conn_pool_size = 30 ssl = True Following change i am going to made let me know if anything missing. [DEFAULT] executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" rpc_response_timeout = 3600 [oslo_messaging_notifications] topics = notifications driver = noop # Messaging [oslo_messaging_rabbit] rpc_conn_pool_size = 300 heartbeat_timeout_threshold = 0 ssl = True Should i be adding this to all my compute nodes also ? On Fri, Mar 20, 2020 at 11:40 AM Grant Morley wrote: > If you tune rabbit then for: > > heartbeat_timeout_threshold = 0 > > That should help with the error message you are getting. > > That is a lot of messages queued. We had the same because we were not > using ceilometer but had the "notifications" still turned on for it for > services. > > Are all of the ready messages for "notifications.info" for the various > services ( Nova, Neutron, Keystone etc ) > > If that is the case you can disable those messages in your config files > for those services. Look for: > > # Notifications > [oslo_messaging_notifications] > notification_topics = notifications > driver = noop > > Make sure the driver option is set to "noop" by default it will be set too > "messagingv2" and then restart the service and that should stop sending > messages to the queue. You can then purge the "notifications.info" queue > for Nova or Neutron etc.. > > We only had the "message ready" when we had the setting for ceilometer set > but was not using it. Also only purge a queue if it is for that reason. Do > not purge the queue if it is for any other reason than that as it can cause > issues. > > Hope that helps. > > Grant > On 20/03/2020 14:38, Satish Patel wrote: > > Grant, > > But i am seeing lots of following logs on my compute nodes running stein > release. > > 2020-03-20 10:34:46.132 53425 WARNING oslo.messaging._drivers.impl_rabbit > [-] Unexpected error during heartbeart thread processing, retrying...: > ConnectionForced: Too many heartbeats missed. > > Find attached screenshot of one of rabbitMQ node, i have lots of messages > in "Ready: 15339" is this looks normal to you? > > > > > On Fri, Mar 20, 2020 at 4:55 AM Grant Morley wrote: > >> Hi, >> >> There was a bug in Queens that meant there was an issue with the >> heartbeat timeouts. Setting it to 0 gets around that bug. I believe that >> was fixed in Rocky and above, so your Stein installation should be fine. >> >> Setting the value to 0 For us meant we stopped getting errors in the logs >> for: >> >> "Too many heartbeats missed, trying to force connect to RabbitMQ" >> >> Regards, >> On 19/03/2020 18:53, Satish Patel wrote: >> >> I have question related following setting, why are you disabling >> heartbeat timeout? >> >> heartbeat_timeout_threshold = 0 >> >> On Thu, Mar 19, 2020 at 1:32 PM Satish Patel >> wrote: >> >>> Great, thanks! Did you guys tune your nova component for rabbitMQ? >>> >>> On Thu, Mar 19, 2020 at 1:26 PM Grant Morley wrote: >>> >>>> We left ours on the default value of 1 and that still seems to be fine. >>>> >>>> Grant >>>> On 19/03/2020 17:13, Satish Patel wrote: >>>> >>>> how about rpc_worker ? >>>> >>>> currently i have rpc_worker=1 >>>> >>>> On Thu, Mar 19, 2020 at 1:02 PM Grant Morley wrote: >>>> >>>>> Correct, you need to add: >>>>> >>>>> > > heartbeat_timeout_threshold = 0 >>>>> > > rpc_conn_pool_size = 300 >>>>> > > rpc_thread_pool_size = 2048 >>>>> > > rpc_response_timeout = 3600 >>>>> > > rpc_poll_timeout = 60 >>>>> >>>>> To your Neutron nodes >>>>> >>>>> And you can add: >>>>> >>>>> >>>>> >> executor_thread_pool_size = 64 >>>>> >> rpc_response_timeout = 3600 >>>>> >>>>> To your compute nodes (neutron.conf) However I found just adding the >>>>> changes to the neturon servers really helped. >>>>> >>>>> I would recommend just starting with your neutron nodes first to see >>>>> if that helps. If you find your compute nodes are still having issues then >>>>> change the settings on those after. >>>>> >>>>> Regards, >>>>> On 19/03/2020 16:53, Satish Patel wrote: >>>>> >>>>> I am running openstack-ansible (Queens / Stein both) so this is what i >>>>> am going to do, am i doing correctly? >>>>> >>>>> neutron-server (container) I have 3 neutron node. >>>>> > > heartbeat_timeout_threshold = 0 >>>>> > > rpc_conn_pool_size = 300 >>>>> > > rpc_thread_pool_size = 2048 >>>>> > > rpc_response_timeout = 3600 >>>>> > > rpc_poll_timeout = 60 >>>>> >>>>> 330 compute nodes (agent neutron.conf) going to add following: >>>>> >> executor_thread_pool_size = 64 >>>>> >> rpc_response_timeout = 3600 >>>>> >>>>> >>>>> >>>>> How about nova? should i be doing that on nova as well to reduce load >>>>> on rabbitMQ? >>>>> >>>>> >>>>> On Thu, Mar 19, 2020 at 12:35 PM Grant Morley wrote: >>>>> >>>>>> Hi Satish, >>>>>> >>>>>> You will need to add those to the "neutron.conf" file on your network >>>>>> nodes. If you are running OS-A I would do it on your "neutron-server" nodes >>>>>> and add the following to your agents containers: >>>>>> >>>>>> executor_thread_pool_size = 64 >>>>>> rpc_response_timeout = 3600 >>>>>> >>>>>> Regards, >>>>>> On 19/03/2020 16:27, Satish Patel wrote: >>>>>> >>>>>> Erik, >>>>>> >>>>>> If i want to adopt following setting then where i should add them in >>>>>> Queens openstack, neutron-server or all my compute nodes? which >>>>>> setting will go where? >>>>>> >>>>>> heartbeat_timeout_threshold = 0 >>>>>> rpc_conn_pool_size = 300 >>>>>> rpc_thread_pool_size = 2048 >>>>>> rpc_response_timeout = 3600 >>>>>> rpc_poll_timeout = 60 >>>>>> >>>>>> ## Rpc all >>>>>> executor_thread_pool_size = 64 >>>>>> rpc_response_timeout = 3600 >>>>>> >>>>>> On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: >>>>>> >>>>>> We are hitting something awfully similar. >>>>>> >>>>>> We have basically been hitting a few pretty serious bugs with RabbitMQ. >>>>>> >>>>>> The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. >>>>>> >>>>>> e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 >>>>>> >>>>>> I wrote two quick scripts to audit these issues.http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment).http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. >>>>>> >>>>>> The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. >>>>>> >>>>>> Best Regards, Erik Olof Gunnar Andersson >>>>>> >>>>>> -----Original Message----- >>>>>> From: Satish Patel >>>>>> Sent: Wednesday, March 11, 2020 5:14 PM >>>>>> To: Grant Morley >>>>>> Cc: openstack-discuss at lists.openstack.org >>>>>> Subject: Re: Neutron RabbitMQ issues >>>>>> >>>>>> I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. >>>>>> >>>>>> This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ >>>>>> >>>>>> On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> We are currently experiencing some fairly major issues with our >>>>>> OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We >>>>>> are seeing a lot of time out messages in responses to replies and >>>>>> because of this instance creation or anything to do with instances and >>>>>> networking is broken. >>>>>> >>>>>> We are running OpenStack Queens. >>>>>> >>>>>> We have already tuned Rabbit for Neutron by doing the following on neutron: >>>>>> >>>>>> heartbeat_timeout_threshold = 0 >>>>>> rpc_conn_pool_size = 300 >>>>>> rpc_thread_pool_size = 2048 >>>>>> rpc_response_timeout = 3600 >>>>>> rpc_poll_timeout = 60 >>>>>> >>>>>> ## Rpc all >>>>>> executor_thread_pool_size = 64 >>>>>> rpc_response_timeout = 3600 >>>>>> >>>>>> What we are seeing in the error logs for neutron for all services >>>>>> (l3-agent, dhcp, linux-bridge etc ) are these timeouts: >>>>>> https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n >>>>>> 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK >>>>>> 9aOA$ >>>>>> >>>>>> We have manually tried to get everything in sync by forcing fail-over >>>>>> of the networking which seems to get routers in sync. >>>>>> >>>>>> We are also seeing that there are a lot of "unacknowledged" messages >>>>>> in RabbitMQ for 'q-plugin' in the neutron queues. >>>>>> >>>>>> Some times restarting of the services on neutron gets these back >>>>>> acknowledged again, however the timeouts come back. >>>>>> >>>>>> The RabbitMQ servers themselves are not loaded at all. All memory, >>>>>> file descriptors and errlang processes have plenty of resources available. >>>>>> >>>>>> We are also seeing a lot of rpc issues: >>>>>> >>>>>> Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds >>>>>> before next attempt. If the server is not down, consider increasing >>>>>> the rpc_response_timeout option as Neutron server(s) may be overloaded >>>>>> and unable to respond quickly enough.: MessagingTimeout: Timed out >>>>>> waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 >>>>>> 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc >>>>>> [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC >>>>>> method release_dhcp_port. Waiting for 3347 seconds before next attempt. >>>>>> If the server is not down, consider increasing the >>>>>> rpc_response_timeout option as Neutron server(s) may be overloaded and >>>>>> unable to respond quickly enough.: MessagingTimeout: Timed out waiting >>>>>> for a reply to message ID 7937465f15634fbfa443fe1758a12a9c >>>>>> >>>>>> Does anyone know if there is anymore tuning to be done at all? >>>>>> Upgrading for us at the moment to a newer version isn't really an >>>>>> option unfortunately. >>>>>> >>>>>> Because of our setup, we also have roughly 800 routers enabled and I >>>>>> know that will be putting a load on the system. However these problems >>>>>> have only started to happen roughly 1 week ago and have steadily got worse. >>>>>> >>>>>> If anyone has any use cases for this or any more recommendations that >>>>>> would be great. >>>>>> >>>>>> Many thanks, >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Grant Morley >>>>>> Cloud Lead, Civo Ltd >>>>>> www.civo.com | Signup for an account! >>>>>> >>>>> -- >>>>> >>>>> Grant Morley >>>>> Cloud Lead, Civo Ltd >>>>> www.civo.com | Signup for an account! >>>>> >>>> -- >>>> >>>> Grant Morley >>>> Cloud Lead, Civo Ltd >>>> www.civo.com | Signup for an account! >>>> >>> -- >> >> Grant Morley >> Cloud Lead, Civo Ltd >> www.civo.com | Signup for an account! >> > -- > > Grant Morley > Cloud Lead, Civo Ltd > www.civo.com | Signup for an account! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eandersson at blizzard.com Fri Mar 20 17:47:33 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Fri, 20 Mar 2020 17:47:33 +0000 Subject: Neutron RabbitMQ issues In-Reply-To: References: <825e802d-5a6f-4e96-dcf5-9b10332ebf3e@civo.com> <4ba08cde-ce96-b8e4-b8cc-a7ded9edd48f@civo.com> <14cc654d-05f8-f5d2-83c6-4e7e24c4ee7b@civo.com> <5cdd4a27-eba8-cdd3-5926-76e534a3286f@civo.com> <704ebc80-6341-ed10-06db-c3a2db80398a@civo.com> Message-ID: Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this. https://bugs.launchpad.net/neutron/+bug/1853071 Best Regards, Erik Olof Gunnar Andersson From: Satish Patel Sent: Friday, March 20, 2020 9:23 AM To: Grant Morley Cc: Erik Olof Gunnar Andersson ; openstack-discuss at lists.openstack.org Subject: Re: Neutron RabbitMQ issues Oh you are right here, i have following stuff in my neutron.conf on server # Notifications [oslo_messaging_notifications] driver = messagingv2 topics = notifications transport_url = rabbit://neutron:5be2a043f9a93adbd at 172.28.15.192:5671,neutron:5be2a043f9a93adbd at 172.28.15.248:5671,neutron:5be2a043f9a93adbd at 172.28.15.22:5671//neutron?ssl=1 # Messaging [oslo_messaging_rabbit] rpc_conn_pool_size = 30 ssl = True Following change i am going to made let me know if anything missing. [DEFAULT] executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size" rpc_response_timeout = 3600 [oslo_messaging_notifications] topics = notifications driver = noop # Messaging [oslo_messaging_rabbit] rpc_conn_pool_size = 300 heartbeat_timeout_threshold = 0 ssl = True Should i be adding this to all my compute nodes also ? On Fri, Mar 20, 2020 at 11:40 AM Grant Morley > wrote: If you tune rabbit then for: heartbeat_timeout_threshold = 0 That should help with the error message you are getting. That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services. Are all of the ready messages for "notifications.info" for the various services ( Nova, Neutron, Keystone etc ) If that is the case you can disable those messages in your config files for those services. Look for: # Notifications [oslo_messaging_notifications] notification_topics = notifications driver = noop Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "notifications.info" queue for Nova or Neutron etc.. We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues. Hope that helps. Grant On 20/03/2020 14:38, Satish Patel wrote: Grant, But i am seeing lots of following logs on my compute nodes running stein release. 2020-03-20 10:34:46.132 53425 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: ConnectionForced: Too many heartbeats missed. Find attached screenshot of one of rabbitMQ node, i have lots of messages in "Ready: 15339" is this looks normal to you? On Fri, Mar 20, 2020 at 4:55 AM Grant Morley > wrote: Hi, There was a bug in Queens that meant there was an issue with the heartbeat timeouts. Setting it to 0 gets around that bug. I believe that was fixed in Rocky and above, so your Stein installation should be fine. Setting the value to 0 For us meant we stopped getting errors in the logs for: "Too many heartbeats missed, trying to force connect to RabbitMQ" Regards, On 19/03/2020 18:53, Satish Patel wrote: I have question related following setting, why are you disabling heartbeat timeout? heartbeat_timeout_threshold = 0 On Thu, Mar 19, 2020 at 1:32 PM Satish Patel > wrote: Great, thanks! Did you guys tune your nova component for rabbitMQ? On Thu, Mar 19, 2020 at 1:26 PM Grant Morley > wrote: We left ours on the default value of 1 and that still seems to be fine. Grant On 19/03/2020 17:13, Satish Patel wrote: how about rpc_worker ? currently i have rpc_worker=1 On Thu, Mar 19, 2020 at 1:02 PM Grant Morley > wrote: Correct, you need to add: > > heartbeat_timeout_threshold = 0 > > rpc_conn_pool_size = 300 > > rpc_thread_pool_size = 2048 > > rpc_response_timeout = 3600 > > rpc_poll_timeout = 60 To your Neutron nodes And you can add: >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 To your compute nodes (neutron.conf) However I found just adding the changes to the neturon servers really helped. I would recommend just starting with your neutron nodes first to see if that helps. If you find your compute nodes are still having issues then change the settings on those after. Regards, On 19/03/2020 16:53, Satish Patel wrote: I am running openstack-ansible (Queens / Stein both) so this is what i am going to do, am i doing correctly? neutron-server (container) I have 3 neutron node. > > heartbeat_timeout_threshold = 0 > > rpc_conn_pool_size = 300 > > rpc_thread_pool_size = 2048 > > rpc_response_timeout = 3600 > > rpc_poll_timeout = 60 330 compute nodes (agent neutron.conf) going to add following: >> executor_thread_pool_size = 64 >> rpc_response_timeout = 3600 How about nova? should i be doing that on nova as well to reduce load on rabbitMQ? On Thu, Mar 19, 2020 at 12:35 PM Grant Morley > wrote: Hi Satish, You will need to add those to the "neutron.conf" file on your network nodes. If you are running OS-A I would do it on your "neutron-server" nodes and add the following to your agents containers: executor_thread_pool_size = 64 rpc_response_timeout = 3600 Regards, On 19/03/2020 16:27, Satish Patel wrote: Erik, If i want to adopt following setting then where i should add them in Queens openstack, neutron-server or all my compute nodes? which setting will go where? heartbeat_timeout_threshold = 0 rpc_conn_pool_size = 300 rpc_thread_pool_size = 2048 rpc_response_timeout = 3600 rpc_poll_timeout = 60 ## Rpc all executor_thread_pool_size = 64 rpc_response_timeout = 3600 On Wed, Mar 11, 2020 at 9:05 PM Erik Olof Gunnar Andersson wrote: We are hitting something awfully similar. We have basically been hitting a few pretty serious bugs with RabbitMQ. The main one is when a RabbitMQ server crashes, or gets split brain it does not always recover, or even when just one node is restarted. We sometimes end up with orphaned consumers that keep consuming messages, but goes to /dev/null pretty much. Another issue is that sometimes bindings stop working. They are visually there, but simply does not route traffic to the intended queues. e.g. https://github.com/rabbitmq/rabbitmq-server/issues/641 I wrote two quick scripts to audit these issues. http://paste.openstack.org/show/790569/ - Check if you have orphaned consumers (may need pagination if you have a large deployment). http://paste.openstack.org/show/790570/ - Check if the bindings are bad for a specific queue. The main issue seems to be the number of queues + connections causing the recovery after restarting a node to cause bindings and/or queues to get into an "orphaned" state. Best Regards, Erik Olof Gunnar Andersson -----Original Message----- From: Satish Patel Sent: Wednesday, March 11, 2020 5:14 PM To: Grant Morley Cc: openstack-discuss at lists.openstack.org Subject: Re: Neutron RabbitMQ issues I am also dealing with some short of rabbitmq performance issue but its not as worst you your issue. This is my favorite video, not sure you have seen this before or not but anyway posting here - https://urldefense.com/v3/__https://www.youtube.com/watch?v=bpmgxrPOrZw__;!!Ci6f514n9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvarUEZv4Mw$ On Wed, Mar 11, 2020 at 10:24 AM Grant Morley wrote: Hi all, We are currently experiencing some fairly major issues with our OpenStack cluster. It all appears to be with Neutron and RabbitMQ. We are seeing a lot of time out messages in responses to replies and because of this instance creation or anything to do with instances and networking is broken. We are running OpenStack Queens. We have already tuned Rabbit for Neutron by doing the following on neutron: heartbeat_timeout_threshold = 0 rpc_conn_pool_size = 300 rpc_thread_pool_size = 2048 rpc_response_timeout = 3600 rpc_poll_timeout = 60 ## Rpc all executor_thread_pool_size = 64 rpc_response_timeout = 3600 What we are seeing in the error logs for neutron for all services (l3-agent, dhcp, linux-bridge etc ) are these timeouts: https://urldefense.com/v3/__https://pastebin.com/Fjh23A5a__;!!Ci6f514n 9QsL8ck!1rOR_L7ya6zmMgZ0owpfO7NvhsPOzbgyUplonob2awcg8hd80yCAT_ynvapLQK 9aOA$ We have manually tried to get everything in sync by forcing fail-over of the networking which seems to get routers in sync. We are also seeing that there are a lot of "unacknowledged" messages in RabbitMQ for 'q-plugin' in the neutron queues. Some times restarting of the services on neutron gets these back acknowledged again, however the timeouts come back. The RabbitMQ servers themselves are not loaded at all. All memory, file descriptors and errlang processes have plenty of resources available. We are also seeing a lot of rpc issues: Timeout in RPC method release_dhcp_port. Waiting for 1523 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: MessagingTimeout: Timed out waiting for a reply to message ID 965fa44ab4f6462fa378a1cf7259aad4 2020-03-10 19:02:33.548 16242 ERROR neutron.common.rpc [req-a858afbb-5083-4e21-a309-6ee53582c4d9 - - - - -] Timeout in RPC method release_dhcp_port. Waiting for 3347 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: MessagingTimeout: Timed out waiting for a reply to message ID 7937465f15634fbfa443fe1758a12a9c Does anyone know if there is anymore tuning to be done at all? Upgrading for us at the moment to a newer version isn't really an option unfortunately. Because of our setup, we also have roughly 800 routers enabled and I know that will be putting a load on the system. However these problems have only started to happen roughly 1 week ago and have steadily got worse. If anyone has any use cases for this or any more recommendations that would be great. Many thanks, -- [https://www.civo.com/images/email-logo.jpg] Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -- [https://www.civo.com/images/email-logo.jpg] Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -- [https://www.civo.com/images/email-logo.jpg] Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -- [https://www.civo.com/images/email-logo.jpg] Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -- [https://www.civo.com/images/email-logo.jpg] Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From saphi070 at gmail.com Sat Mar 21 13:22:34 2020 From: saphi070 at gmail.com (Sa Pham) Date: Sat, 21 Mar 2020 20:22:34 +0700 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: With VM uses provider network directly, When I hard reboot that VM, I cannot reach that VM again. Can you test in your environment? On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano wrote: > Hello Sa, I am using self service and provider networks.It works fine in > both cases. The problem is the migration from iptables hybrid to > openvswitch without rebooting instanes. > Do you mean security groups do not work on provider networks ? > Ignazio > > > Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: > >> Hello Ignazio, >> >> Does your openstack environment using self-service network ? >> >> I have tried openvswitch firewall native with openstack queens version >> using provider network. But It's not working good. >> >> >> >> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >> ignaziocassano at gmail.com> wrote: >> >>> Hello Jakub, >>> I will try again but if there is a bug on queens I do not think it will >>> be corrected because is going out of support. >>> Thanks >>> Ignazio >>> >>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>> jlibosva at redhat.com> ha scritto: >>> >>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>>> switched >>>> > on openvswitch works fine . The problem is this migration create the >>>> qbr on >>>> > the mode switched to openvswitch. >>>> > But when I switch another compute node to openvswitch and I try to >>>> live >>>> > migrate the same vm (openvswitch to qopenswitch) it does not work >>>> because >>>> > the qbr presence. >>>> > I verified on nova logs. >>>> > Ignazio >>>> >>>> Hi Ignazio, >>>> >>>> I think the first step - migrating from hybrid_iptables to ovs should >>>> not create the qbr on the target node. It sounds like a bug - IIRC the >>>> libvirt domxml should not have the qbr when migrating. >>>> >>>> >>>> > >>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha >>>> scritto: >>>> > >>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>> >>> Hello All, I am facing some problems migrating from iptables_hybrid >>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>> >>> I am doing this because I want enable security groups logs which >>>> require >>>> >>> openvswitch firewall. >>>> >>> I would like to migrate without restarting my instances. >>>> >>> I startded moving all instances from compute node 1. >>>> >>> Then I configured openvswitch firewall on compute node 1, >>>> >>> Instances migrated from compute node 2 to compute node 1 without >>>> >> problems. >>>> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>>> >>> But now instances does not migrate from node 1 to node 2 because it >>>> >>> requires the presence of qbr bridge on node 2 >>>> >>> >>>> >>> This happened because migrating instances from node2 with >>>> iptables_hybrid >>>> >>> to compute node 1 with openvswitch, does not put the tap under >>>> br-int as >>>> >>> requested by openvswich firewall, but qbr is still present on >>>> compute >>>> >> node >>>> >>> 1. >>>> >>> Once I enabled openvswitch on compute node 2, migration from compute >>>> >> node 1 >>>> >>> fails because it exprects qbr on compute node 2 . >>>> >>> So I think I should moving on the fly tap interfaces from qbr to >>>> br-int >>>> >> on >>>> >>> compute node 1 before migrating to compute node 2 but it is a huge >>>> work >>>> >> on >>>> >>> a lot of instances. >>>> >>> >>>> >>> Any workaround, please ? >>>> >>> >>>> >>> Ignazio >>>> >>> >>>> >> >>>> >> I may be a little outdated here but to the best of my knowledge there >>>> >> are two ways how to migrate from iptables to openvswitch. >>>> >> >>>> >> 1) If you don't mind the intermediate linux bridge and you care about >>>> >> logs, you can just change the config file on compute node to start >>>> using >>>> >> openvswitch firewall and restart the ovs agent. That should trigger a >>>> >> mechanism that deletes iptables rules and starts using openflow >>>> rules. >>>> >> It will leave the intermediate bridge there but except the extra hop >>>> in >>>> >> networking stack, it doesn't mind. >>>> >> >>>> >> 2) With multiple-port binding feature, what you described above >>>> should >>>> >> be working. I know Miguel spent some time working on that so perhaps >>>> he >>>> >> has more information about which release it should be functional at, >>>> I >>>> >> think it was Queens. Not sure if any Nova work was required to make >>>> it >>>> >> work. >>>> >> >>>> >> Hope that helps. >>>> >> Kuba >>>> >> >>>> >> >>>> >> >>>> >> >>>> > >>>> >>>> >> >> -- >> Sa Pham Dang >> Skype: great_bn >> Phone/Telegram: 0986.849.582 >> >> >> -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 14:41:26 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 15:41:26 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Sure, Sa. I have tested it 2 minutes ago. It works . I also changed security groups rules to allow/deny ssh access . It works also after hard reboot Ignazio Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham ha scritto: > With VM uses provider network directly, When I hard reboot that VM, I > cannot reach that VM again. Can you test in your environment? > > On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano > wrote: > >> Hello Sa, I am using self service and provider networks.It works fine in >> both cases. The problem is the migration from iptables hybrid to >> openvswitch without rebooting instanes. >> Do you mean security groups do not work on provider networks ? >> Ignazio >> >> >> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >> >>> Hello Ignazio, >>> >>> Does your openstack environment using self-service network ? >>> >>> I have tried openvswitch firewall native with openstack queens version >>> using provider network. But It's not working good. >>> >>> >>> >>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>> ignaziocassano at gmail.com> wrote: >>> >>>> Hello Jakub, >>>> I will try again but if there is a bug on queens I do not think it will >>>> be corrected because is going out of support. >>>> Thanks >>>> Ignazio >>>> >>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>> jlibosva at redhat.com> ha scritto: >>>> >>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>>>> switched >>>>> > on openvswitch works fine . The problem is this migration create the >>>>> qbr on >>>>> > the mode switched to openvswitch. >>>>> > But when I switch another compute node to openvswitch and I try to >>>>> live >>>>> > migrate the same vm (openvswitch to qopenswitch) it does not work >>>>> because >>>>> > the qbr presence. >>>>> > I verified on nova logs. >>>>> > Ignazio >>>>> >>>>> Hi Ignazio, >>>>> >>>>> I think the first step - migrating from hybrid_iptables to ovs should >>>>> not create the qbr on the target node. It sounds like a bug - IIRC the >>>>> libvirt domxml should not have the qbr when migrating. >>>>> >>>>> >>>>> > >>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha >>>>> scritto: >>>>> > >>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>> >>> Hello All, I am facing some problems migrating from iptables_hybrid >>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>> >>> I am doing this because I want enable security groups logs which >>>>> require >>>>> >>> openvswitch firewall. >>>>> >>> I would like to migrate without restarting my instances. >>>>> >>> I startded moving all instances from compute node 1. >>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>> >>> Instances migrated from compute node 2 to compute node 1 without >>>>> >> problems. >>>>> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>>>> >>> But now instances does not migrate from node 1 to node 2 because it >>>>> >>> requires the presence of qbr bridge on node 2 >>>>> >>> >>>>> >>> This happened because migrating instances from node2 with >>>>> iptables_hybrid >>>>> >>> to compute node 1 with openvswitch, does not put the tap under >>>>> br-int as >>>>> >>> requested by openvswich firewall, but qbr is still present on >>>>> compute >>>>> >> node >>>>> >>> 1. >>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>> compute >>>>> >> node 1 >>>>> >>> fails because it exprects qbr on compute node 2 . >>>>> >>> So I think I should moving on the fly tap interfaces from qbr to >>>>> br-int >>>>> >> on >>>>> >>> compute node 1 before migrating to compute node 2 but it is a huge >>>>> work >>>>> >> on >>>>> >>> a lot of instances. >>>>> >>> >>>>> >>> Any workaround, please ? >>>>> >>> >>>>> >>> Ignazio >>>>> >>> >>>>> >> >>>>> >> I may be a little outdated here but to the best of my knowledge >>>>> there >>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>> >> >>>>> >> 1) If you don't mind the intermediate linux bridge and you care >>>>> about >>>>> >> logs, you can just change the config file on compute node to start >>>>> using >>>>> >> openvswitch firewall and restart the ovs agent. That should trigger >>>>> a >>>>> >> mechanism that deletes iptables rules and starts using openflow >>>>> rules. >>>>> >> It will leave the intermediate bridge there but except the extra >>>>> hop in >>>>> >> networking stack, it doesn't mind. >>>>> >> >>>>> >> 2) With multiple-port binding feature, what you described above >>>>> should >>>>> >> be working. I know Miguel spent some time working on that so >>>>> perhaps he >>>>> >> has more information about which release it should be functional >>>>> at, I >>>>> >> think it was Queens. Not sure if any Nova work was required to make >>>>> it >>>>> >> work. >>>>> >> >>>>> >> Hope that helps. >>>>> >> Kuba >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> > >>>>> >>>>> >>> >>> -- >>> Sa Pham Dang >>> Skype: great_bn >>> Phone/Telegram: 0986.849.582 >>> >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 14:48:16 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 15:48:16 +0100 Subject: Diskimage-builder complex disk setup In-Reply-To: <628e153a-16de-4bdc-96d4-0bdc2d5dc3b2@Spark> References: <628e153a-16de-4bdc-96d4-0bdc2d5dc3b2@Spark> Message-ID: Hello, this is my example: export DIB_BLOCK_DEVICE_CONFIG=$(cat mylvm_centos8_nmsf.yaml) this is my mylvm_centos8_nmsf.yaml: - local_loop: name: image0 size: 30G - partitioning: base: image0 label: mbr partitions: - name: root flags: [ boot,primary ] size: 100% - lvm: name: lvm base: [root] pvs: - name: pv base: root vgs: - name: vg_rootcentos8 base: ["pv"] lvs: - name: lv_root base: vg_rootcentos8 size: 5000M - name: lv_var base: vg_rootcentos8 size: 5000M - name: lv_usr base: vg_rootcentos8 size: 5000M - name: lv_swap base: vg_rootcentos8 size: 2000M - name: lv_tmp base: vg_rootcentos8 size: 1500M - name: lv_appserv base: vg_rootcentos8 size: 8000M - mkfs: name: mkfs_root base: lv_root label: "img-rootfs" type: "ext4" mount: mount_point: / fstab: options: "noacl,errors=remount-ro" fsck-passno: 1 - mkfs: name: mkfs_var base: lv_var type: "ext4" mount: mount_point: /var fstab: options: "noacl" fsck-passno: 2 - mkfs: name: mkfs_usr base: lv_usr type: "ext4" mount: mount_point: /usr fstab: options: "noacl" fsck-passno: 2 - mkfs: name: mkfs_swap base: lv_swap type: "ext3" mount: mount_point: swap fstab: name: swap options: "swap" - mkfs: name: mkfs_tmp base: lv_tmp type: "ext4" mount: mount_point: /tmp fstab: options: "noacl" fsck-passno: 2 - mkfs: name: mkfs_appserv base: lv_appserv type: "ext4" mount: mount_point: /appserv fstab: options: "noacl" fsck-passno: 2 After creating the image I modified it as follows: guestfish -a /tmp/centos8_nmsf_ansible_heat.qcow2 -i mkswap-L mkfs_swap /dev/vg_rootcentos8/lv_swap Il giorno sab 21 mar 2020 alle ore 14:24 Stephen Nemeth ha scritto: > Hi all, > > We’re attempting to create user images to deploy with metal^3/ironic. > > I’m making almost no headway starting from any of the examples provided > for the DIB_BLOCK_DEVICE_CONFIG setup. > > Is there a repository with practical values located anywhere? I’m finding > the documentation insufficient to do anything more complicated than a > single partition setup. > > Here’s what I’ve gotten so far. Help greatly appreciated: > > ``` > [ > { > "local_loop": { > "name": "image0", > "size": "20GiB" > } > }, > { > "partitioning": { > "base": "image0", > "label": "gpt", > "partitions": [ > { > "name": "ESP", > "type": "EF00", > "size": "16MiB" > }, > { > "name": "boot", > "type": "EF02", > "size": "6GiB" > }, > { > "name": "lvm", > "type": 8, > "size": "100%" > } > ] > } > }, > { > "lvm": { > "name": "lvm2", > "pvs": [ > { > "name": "pv", > "options": [ > "--force" > ], > "device": "lvm", > "base": "image0" > } > ], > "vgs": [ > { > "name": "vg", > "base": [ > "pv" > ], > "options": [ > "--force" > ] > } > ], > "lvs": [ > { > "name": "root", > "base": "vg", > "extents": "100%FREE", > "options": [ > "--zero=n" > ] > } > ] > } > }, > { > "mkfs": { > { > "name": "mkfs_root", > "base": "root", > "type": "btrfs", > "label": "root", > "opts": "-f", > "mount": { "mount_point": "/" } > }, > { > "name": "mkfs_efi", > "base": "ESP", > "type": "vfat", > "label": "efi", > "opts": "-f", > "mount": { "mount_point": "/boot/efi/" } > }, > { > "name": "mkfs_boot", > "base": "boot", > "type": "btrfs", > "label": "boot", > "opts": "-f", > "mount": { "mount_point": "/boot/" } > } > } > } > ] > ``` > -------------- next part -------------- An HTML attachment was scrubbed... URL: From saphi070 at gmail.com Sat Mar 21 14:48:54 2020 From: saphi070 at gmail.com (Sa Pham) Date: Sat, 21 Mar 2020 21:48:54 +0700 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: One problem which I got few days ago. I have existing openstack with iptables_hybrid. I changed the firewall driver to openvswitch then restart neutron-openvswitch-agent. I couldn't reach that VM any more. I tried to reboot or hard reboot that VM but It didn't work. On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano wrote: > Sure, Sa. > I have tested it 2 minutes ago. > It works . > I also changed security groups rules to allow/deny ssh access . It works > also after hard reboot > Ignazio > > Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham ha > scritto: > >> With VM uses provider network directly, When I hard reboot that VM, I >> cannot reach that VM again. Can you test in your environment? >> >> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano >> wrote: >> >>> Hello Sa, I am using self service and provider networks.It works fine in >>> both cases. The problem is the migration from iptables hybrid to >>> openvswitch without rebooting instanes. >>> Do you mean security groups do not work on provider networks ? >>> Ignazio >>> >>> >>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>> >>>> Hello Ignazio, >>>> >>>> Does your openstack environment using self-service network ? >>>> >>>> I have tried openvswitch firewall native with openstack queens version >>>> using provider network. But It's not working good. >>>> >>>> >>>> >>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>> ignaziocassano at gmail.com> wrote: >>>> >>>>> Hello Jakub, >>>>> I will try again but if there is a bug on queens I do not think it >>>>> will be corrected because is going out of support. >>>>> Thanks >>>>> Ignazio >>>>> >>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>> jlibosva at redhat.com> ha scritto: >>>>> >>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>>>>> switched >>>>>> > on openvswitch works fine . The problem is this migration create >>>>>> the qbr on >>>>>> > the mode switched to openvswitch. >>>>>> > But when I switch another compute node to openvswitch and I try to >>>>>> live >>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not work >>>>>> because >>>>>> > the qbr presence. >>>>>> > I verified on nova logs. >>>>>> > Ignazio >>>>>> >>>>>> Hi Ignazio, >>>>>> >>>>>> I think the first step - migrating from hybrid_iptables to ovs should >>>>>> not create the qbr on the target node. It sounds like a bug - IIRC the >>>>>> libvirt domxml should not have the qbr when migrating. >>>>>> >>>>>> >>>>>> > >>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha >>>>>> scritto: >>>>>> > >>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>> >>> Hello All, I am facing some problems migrating from >>>>>> iptables_hybrid >>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>> >>> I am doing this because I want enable security groups logs which >>>>>> require >>>>>> >>> openvswitch firewall. >>>>>> >>> I would like to migrate without restarting my instances. >>>>>> >>> I startded moving all instances from compute node 1. >>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>> >>> Instances migrated from compute node 2 to compute node 1 without >>>>>> >> problems. >>>>>> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>>>>> >>> But now instances does not migrate from node 1 to node 2 because >>>>>> it >>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>> >>> >>>>>> >>> This happened because migrating instances from node2 with >>>>>> iptables_hybrid >>>>>> >>> to compute node 1 with openvswitch, does not put the tap under >>>>>> br-int as >>>>>> >>> requested by openvswich firewall, but qbr is still present on >>>>>> compute >>>>>> >> node >>>>>> >>> 1. >>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>> compute >>>>>> >> node 1 >>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>> >>> So I think I should moving on the fly tap interfaces from qbr to >>>>>> br-int >>>>>> >> on >>>>>> >>> compute node 1 before migrating to compute node 2 but it is a >>>>>> huge work >>>>>> >> on >>>>>> >>> a lot of instances. >>>>>> >>> >>>>>> >>> Any workaround, please ? >>>>>> >>> >>>>>> >>> Ignazio >>>>>> >>> >>>>>> >> >>>>>> >> I may be a little outdated here but to the best of my knowledge >>>>>> there >>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>> >> >>>>>> >> 1) If you don't mind the intermediate linux bridge and you care >>>>>> about >>>>>> >> logs, you can just change the config file on compute node to start >>>>>> using >>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>> trigger a >>>>>> >> mechanism that deletes iptables rules and starts using openflow >>>>>> rules. >>>>>> >> It will leave the intermediate bridge there but except the extra >>>>>> hop in >>>>>> >> networking stack, it doesn't mind. >>>>>> >> >>>>>> >> 2) With multiple-port binding feature, what you described above >>>>>> should >>>>>> >> be working. I know Miguel spent some time working on that so >>>>>> perhaps he >>>>>> >> has more information about which release it should be functional >>>>>> at, I >>>>>> >> think it was Queens. Not sure if any Nova work was required to >>>>>> make it >>>>>> >> work. >>>>>> >> >>>>>> >> Hope that helps. >>>>>> >> Kuba >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> > >>>>>> >>>>>> >>>> >>>> -- >>>> Sa Pham Dang >>>> Skype: great_bn >>>> Phone/Telegram: 0986.849.582 >>>> >>>> >>>> >> >> -- >> Sa Pham Dang >> Skype: great_bn >> Phone/Telegram: 0986.849.582 >> >> >> -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 14:53:34 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 15:53:34 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Sa, have you modified only the compute node side ? I've modified also the controller node (neutron node) side ad reported in documentation for enabling security groups logs. https://docs.openstack.org/neutron/queens/admin/config-logging.html Ignazio Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham ha scritto: > One problem which I got few days ago. > > I have existing openstack with iptables_hybrid. I changed the firewall > driver to openvswitch then restart neutron-openvswitch-agent. > I couldn't reach that VM any more. I tried to reboot or hard reboot that > VM but It didn't work. > > > > On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano > wrote: > >> Sure, Sa. >> I have tested it 2 minutes ago. >> It works . >> I also changed security groups rules to allow/deny ssh access . It works >> also after hard reboot >> Ignazio >> >> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham ha >> scritto: >> >>> With VM uses provider network directly, When I hard reboot that VM, I >>> cannot reach that VM again. Can you test in your environment? >>> >>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>> ignaziocassano at gmail.com> wrote: >>> >>>> Hello Sa, I am using self service and provider networks.It works fine >>>> in both cases. The problem is the migration from iptables hybrid to >>>> openvswitch without rebooting instanes. >>>> Do you mean security groups do not work on provider networks ? >>>> Ignazio >>>> >>>> >>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>> >>>>> Hello Ignazio, >>>>> >>>>> Does your openstack environment using self-service network ? >>>>> >>>>> I have tried openvswitch firewall native with openstack queens version >>>>> using provider network. But It's not working good. >>>>> >>>>> >>>>> >>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>> ignaziocassano at gmail.com> wrote: >>>>> >>>>>> Hello Jakub, >>>>>> I will try again but if there is a bug on queens I do not think it >>>>>> will be corrected because is going out of support. >>>>>> Thanks >>>>>> Ignazio >>>>>> >>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>> jlibosva at redhat.com> ha scritto: >>>>>> >>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>>>>>> switched >>>>>>> > on openvswitch works fine . The problem is this migration create >>>>>>> the qbr on >>>>>>> > the mode switched to openvswitch. >>>>>>> > But when I switch another compute node to openvswitch and I try to >>>>>>> live >>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not work >>>>>>> because >>>>>>> > the qbr presence. >>>>>>> > I verified on nova logs. >>>>>>> > Ignazio >>>>>>> >>>>>>> Hi Ignazio, >>>>>>> >>>>>>> I think the first step - migrating from hybrid_iptables to ovs should >>>>>>> not create the qbr on the target node. It sounds like a bug - IIRC >>>>>>> the >>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>> >>>>>>> >>>>>>> > >>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar ha >>>>>>> scritto: >>>>>>> > >>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>> iptables_hybrid >>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>> >>> I am doing this because I want enable security groups logs which >>>>>>> require >>>>>>> >>> openvswitch firewall. >>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>> >>> Instances migrated from compute node 2 to compute node 1 without >>>>>>> >> problems. >>>>>>> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>>>>>> >>> But now instances does not migrate from node 1 to node 2 because >>>>>>> it >>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>> >>> >>>>>>> >>> This happened because migrating instances from node2 with >>>>>>> iptables_hybrid >>>>>>> >>> to compute node 1 with openvswitch, does not put the tap under >>>>>>> br-int as >>>>>>> >>> requested by openvswich firewall, but qbr is still present on >>>>>>> compute >>>>>>> >> node >>>>>>> >>> 1. >>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>> compute >>>>>>> >> node 1 >>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>> >>> So I think I should moving on the fly tap interfaces from qbr to >>>>>>> br-int >>>>>>> >> on >>>>>>> >>> compute node 1 before migrating to compute node 2 but it is a >>>>>>> huge work >>>>>>> >> on >>>>>>> >>> a lot of instances. >>>>>>> >>> >>>>>>> >>> Any workaround, please ? >>>>>>> >>> >>>>>>> >>> Ignazio >>>>>>> >>> >>>>>>> >> >>>>>>> >> I may be a little outdated here but to the best of my knowledge >>>>>>> there >>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>> >> >>>>>>> >> 1) If you don't mind the intermediate linux bridge and you care >>>>>>> about >>>>>>> >> logs, you can just change the config file on compute node to >>>>>>> start using >>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>> trigger a >>>>>>> >> mechanism that deletes iptables rules and starts using openflow >>>>>>> rules. >>>>>>> >> It will leave the intermediate bridge there but except the extra >>>>>>> hop in >>>>>>> >> networking stack, it doesn't mind. >>>>>>> >> >>>>>>> >> 2) With multiple-port binding feature, what you described above >>>>>>> should >>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>> perhaps he >>>>>>> >> has more information about which release it should be functional >>>>>>> at, I >>>>>>> >> think it was Queens. Not sure if any Nova work was required to >>>>>>> make it >>>>>>> >> work. >>>>>>> >> >>>>>>> >> Hope that helps. >>>>>>> >> Kuba >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> > >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Sa Pham Dang >>>>> Skype: great_bn >>>>> Phone/Telegram: 0986.849.582 >>>>> >>>>> >>>>> >>> >>> -- >>> Sa Pham Dang >>> Skype: great_bn >>> Phone/Telegram: 0986.849.582 >>> >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From saphi070 at gmail.com Sat Mar 21 14:56:49 2020 From: saphi070 at gmail.com (Sa Pham) Date: Sat, 21 Mar 2020 21:56:49 +0700 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: I just use Openvswitch for firewall driver. I did not use log plugin. You said you conffigured sec group rules to allow and deny. As I know, Security group cannot add deny rule. On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano wrote: > Sa, have you modified only the compute node side ? > I've modified also the controller node (neutron node) side ad reported in > documentation for enabling security groups logs. > > https://docs.openstack.org/neutron/queens/admin/config-logging.html > > Ignazio > > > > Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham ha > scritto: > >> One problem which I got few days ago. >> >> I have existing openstack with iptables_hybrid. I changed the firewall >> driver to openvswitch then restart neutron-openvswitch-agent. >> I couldn't reach that VM any more. I tried to reboot or hard reboot that >> VM but It didn't work. >> >> >> >> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano >> wrote: >> >>> Sure, Sa. >>> I have tested it 2 minutes ago. >>> It works . >>> I also changed security groups rules to allow/deny ssh access . It works >>> also after hard reboot >>> Ignazio >>> >>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>> ha scritto: >>> >>>> With VM uses provider network directly, When I hard reboot that VM, I >>>> cannot reach that VM again. Can you test in your environment? >>>> >>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>> ignaziocassano at gmail.com> wrote: >>>> >>>>> Hello Sa, I am using self service and provider networks.It works fine >>>>> in both cases. The problem is the migration from iptables hybrid to >>>>> openvswitch without rebooting instanes. >>>>> Do you mean security groups do not work on provider networks ? >>>>> Ignazio >>>>> >>>>> >>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>> >>>>>> Hello Ignazio, >>>>>> >>>>>> Does your openstack environment using self-service network ? >>>>>> >>>>>> I have tried openvswitch firewall native with openstack queens >>>>>> version using provider network. But It's not working good. >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>> ignaziocassano at gmail.com> wrote: >>>>>> >>>>>>> Hello Jakub, >>>>>>> I will try again but if there is a bug on queens I do not think it >>>>>>> will be corrected because is going out of support. >>>>>>> Thanks >>>>>>> Ignazio >>>>>>> >>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>> >>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>>>>>>> switched >>>>>>>> > on openvswitch works fine . The problem is this migration create >>>>>>>> the qbr on >>>>>>>> > the mode switched to openvswitch. >>>>>>>> > But when I switch another compute node to openvswitch and I try >>>>>>>> to live >>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not work >>>>>>>> because >>>>>>>> > the qbr presence. >>>>>>>> > I verified on nova logs. >>>>>>>> > Ignazio >>>>>>>> >>>>>>>> Hi Ignazio, >>>>>>>> >>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>> should >>>>>>>> not create the qbr on the target node. It sounds like a bug - IIRC >>>>>>>> the >>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>> ha scritto: >>>>>>>> > >>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>> iptables_hybrid >>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>> which require >>>>>>>> >>> openvswitch firewall. >>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 without >>>>>>>> >> problems. >>>>>>>> >>> Once the compute node 2 was empty, I migrated it to openvswitch. >>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>> because it >>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>> >>> >>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>> iptables_hybrid >>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap under >>>>>>>> br-int as >>>>>>>> >>> requested by openvswich firewall, but qbr is still present on >>>>>>>> compute >>>>>>>> >> node >>>>>>>> >>> 1. >>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>>> compute >>>>>>>> >> node 1 >>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>> >>> So I think I should moving on the fly tap interfaces from qbr >>>>>>>> to br-int >>>>>>>> >> on >>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is a >>>>>>>> huge work >>>>>>>> >> on >>>>>>>> >>> a lot of instances. >>>>>>>> >>> >>>>>>>> >>> Any workaround, please ? >>>>>>>> >>> >>>>>>>> >>> Ignazio >>>>>>>> >>> >>>>>>>> >> >>>>>>>> >> I may be a little outdated here but to the best of my knowledge >>>>>>>> there >>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>> >> >>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you care >>>>>>>> about >>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>> start using >>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>> trigger a >>>>>>>> >> mechanism that deletes iptables rules and starts using openflow >>>>>>>> rules. >>>>>>>> >> It will leave the intermediate bridge there but except the extra >>>>>>>> hop in >>>>>>>> >> networking stack, it doesn't mind. >>>>>>>> >> >>>>>>>> >> 2) With multiple-port binding feature, what you described above >>>>>>>> should >>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>> perhaps he >>>>>>>> >> has more information about which release it should be functional >>>>>>>> at, I >>>>>>>> >> think it was Queens. Not sure if any Nova work was required to >>>>>>>> make it >>>>>>>> >> work. >>>>>>>> >> >>>>>>>> >> Hope that helps. >>>>>>>> >> Kuba >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> Sa Pham Dang >>>>>> Skype: great_bn >>>>>> Phone/Telegram: 0986.849.582 >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> Sa Pham Dang >>>> Skype: great_bn >>>> Phone/Telegram: 0986.849.582 >>>> >>>> >>>> >> >> -- >> Sa Pham Dang >> Skype: great_bn >> Phone/Telegram: 0986.849.582 >> >> >> -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 15:02:41 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 16:02:41 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Sorry, I mean I added ssh access and then I removed it Openviswitch is a requirement for security group logs. So , if you read at the documentation, it suggests to modify iptables_hybrid on neutron node as well. 1 month ago I addes a compute node with openvswitch on an openstack with iptables_hybrid on neutron node: it did not worked until I modified the neutron node. I do not know why Il giorno sab 21 mar 2020 alle ore 15:57 Sa Pham ha scritto: > I just use Openvswitch for firewall driver. I did not use log plugin. > > You said you conffigured sec group rules to allow and deny. As I know, > Security group cannot add deny rule. > > On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano > wrote: > >> Sa, have you modified only the compute node side ? >> I've modified also the controller node (neutron node) side ad reported in >> documentation for enabling security groups logs. >> >> https://docs.openstack.org/neutron/queens/admin/config-logging.html >> >> Ignazio >> >> >> >> Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham ha >> scritto: >> >>> One problem which I got few days ago. >>> >>> I have existing openstack with iptables_hybrid. I changed the firewall >>> driver to openvswitch then restart neutron-openvswitch-agent. >>> I couldn't reach that VM any more. I tried to reboot or hard reboot >>> that VM but It didn't work. >>> >>> >>> >>> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano < >>> ignaziocassano at gmail.com> wrote: >>> >>>> Sure, Sa. >>>> I have tested it 2 minutes ago. >>>> It works . >>>> I also changed security groups rules to allow/deny ssh access . It >>>> works also after hard reboot >>>> Ignazio >>>> >>>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>>> ha scritto: >>>> >>>>> With VM uses provider network directly, When I hard reboot that VM, I >>>>> cannot reach that VM again. Can you test in your environment? >>>>> >>>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>>> ignaziocassano at gmail.com> wrote: >>>>> >>>>>> Hello Sa, I am using self service and provider networks.It works fine >>>>>> in both cases. The problem is the migration from iptables hybrid to >>>>>> openvswitch without rebooting instanes. >>>>>> Do you mean security groups do not work on provider networks ? >>>>>> Ignazio >>>>>> >>>>>> >>>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>>> >>>>>>> Hello Ignazio, >>>>>>> >>>>>>> Does your openstack environment using self-service network ? >>>>>>> >>>>>>> I have tried openvswitch firewall native with openstack queens >>>>>>> version using provider network. But It's not working good. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>> >>>>>>>> Hello Jakub, >>>>>>>> I will try again but if there is a bug on queens I do not think it >>>>>>>> will be corrected because is going out of support. >>>>>>>> Thanks >>>>>>>> Ignazio >>>>>>>> >>>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>>> >>>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a node >>>>>>>>> switched >>>>>>>>> > on openvswitch works fine . The problem is this migration create >>>>>>>>> the qbr on >>>>>>>>> > the mode switched to openvswitch. >>>>>>>>> > But when I switch another compute node to openvswitch and I try >>>>>>>>> to live >>>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not >>>>>>>>> work because >>>>>>>>> > the qbr presence. >>>>>>>>> > I verified on nova logs. >>>>>>>>> > Ignazio >>>>>>>>> >>>>>>>>> Hi Ignazio, >>>>>>>>> >>>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>>> should >>>>>>>>> not create the qbr on the target node. It sounds like a bug - IIRC >>>>>>>>> the >>>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>>> >>>>>>>>> >>>>>>>>> > >>>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>>> ha scritto: >>>>>>>>> > >>>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>>> iptables_hybrid >>>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>>> which require >>>>>>>>> >>> openvswitch firewall. >>>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 >>>>>>>>> without >>>>>>>>> >> problems. >>>>>>>>> >>> Once the compute node 2 was empty, I migrated it to >>>>>>>>> openvswitch. >>>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>>> because it >>>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>>> >>> >>>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>>> iptables_hybrid >>>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap under >>>>>>>>> br-int as >>>>>>>>> >>> requested by openvswich firewall, but qbr is still present on >>>>>>>>> compute >>>>>>>>> >> node >>>>>>>>> >>> 1. >>>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>>>> compute >>>>>>>>> >> node 1 >>>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>>> >>> So I think I should moving on the fly tap interfaces from qbr >>>>>>>>> to br-int >>>>>>>>> >> on >>>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is a >>>>>>>>> huge work >>>>>>>>> >> on >>>>>>>>> >>> a lot of instances. >>>>>>>>> >>> >>>>>>>>> >>> Any workaround, please ? >>>>>>>>> >>> >>>>>>>>> >>> Ignazio >>>>>>>>> >>> >>>>>>>>> >> >>>>>>>>> >> I may be a little outdated here but to the best of my knowledge >>>>>>>>> there >>>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>>> >> >>>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you care >>>>>>>>> about >>>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>>> start using >>>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>>> trigger a >>>>>>>>> >> mechanism that deletes iptables rules and starts using openflow >>>>>>>>> rules. >>>>>>>>> >> It will leave the intermediate bridge there but except the >>>>>>>>> extra hop in >>>>>>>>> >> networking stack, it doesn't mind. >>>>>>>>> >> >>>>>>>>> >> 2) With multiple-port binding feature, what you described above >>>>>>>>> should >>>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>>> perhaps he >>>>>>>>> >> has more information about which release it should be >>>>>>>>> functional at, I >>>>>>>>> >> think it was Queens. Not sure if any Nova work was required to >>>>>>>>> make it >>>>>>>>> >> work. >>>>>>>>> >> >>>>>>>>> >> Hope that helps. >>>>>>>>> >> Kuba >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> > >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sa Pham Dang >>>>>>> Skype: great_bn >>>>>>> Phone/Telegram: 0986.849.582 >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Sa Pham Dang >>>>> Skype: great_bn >>>>> Phone/Telegram: 0986.849.582 >>>>> >>>>> >>>>> >>> >>> -- >>> Sa Pham Dang >>> Skype: great_bn >>> Phone/Telegram: 0986.849.582 >>> >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From saphi070 at gmail.com Sat Mar 21 15:35:33 2020 From: saphi070 at gmail.com (Sa Pham) Date: Sat, 21 Mar 2020 22:35:33 +0700 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Which configuration did you use? Or You configured log plugin in neutron node? On Sat, Mar 21, 2020 at 10:02 PM Ignazio Cassano wrote: > Sorry, I mean I added ssh access and then I removed it > Openviswitch is a requirement for security group logs. > So , if you read at the documentation, it suggests to modify > iptables_hybrid on neutron node as well. > > 1 month ago I addes a compute node with openvswitch on an openstack with > iptables_hybrid on neutron node: it did not worked until I modified the > neutron node. I do not know why > > > > Il giorno sab 21 mar 2020 alle ore 15:57 Sa Pham ha > scritto: > >> I just use Openvswitch for firewall driver. I did not use log plugin. >> >> You said you conffigured sec group rules to allow and deny. As I know, >> Security group cannot add deny rule. >> >> On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano >> wrote: >> >>> Sa, have you modified only the compute node side ? >>> I've modified also the controller node (neutron node) side ad reported >>> in documentation for enabling security groups logs. >>> >>> https://docs.openstack.org/neutron/queens/admin/config-logging.html >>> >>> Ignazio >>> >>> >>> >>> Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham >>> ha scritto: >>> >>>> One problem which I got few days ago. >>>> >>>> I have existing openstack with iptables_hybrid. I changed the firewall >>>> driver to openvswitch then restart neutron-openvswitch-agent. >>>> I couldn't reach that VM any more. I tried to reboot or hard reboot >>>> that VM but It didn't work. >>>> >>>> >>>> >>>> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano < >>>> ignaziocassano at gmail.com> wrote: >>>> >>>>> Sure, Sa. >>>>> I have tested it 2 minutes ago. >>>>> It works . >>>>> I also changed security groups rules to allow/deny ssh access . It >>>>> works also after hard reboot >>>>> Ignazio >>>>> >>>>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>>>> ha scritto: >>>>> >>>>>> With VM uses provider network directly, When I hard reboot that VM, I >>>>>> cannot reach that VM again. Can you test in your environment? >>>>>> >>>>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>>>> ignaziocassano at gmail.com> wrote: >>>>>> >>>>>>> Hello Sa, I am using self service and provider networks.It works >>>>>>> fine in both cases. The problem is the migration from iptables hybrid to >>>>>>> openvswitch without rebooting instanes. >>>>>>> Do you mean security groups do not work on provider networks ? >>>>>>> Ignazio >>>>>>> >>>>>>> >>>>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>>>> >>>>>>>> Hello Ignazio, >>>>>>>> >>>>>>>> Does your openstack environment using self-service network ? >>>>>>>> >>>>>>>> I have tried openvswitch firewall native with openstack queens >>>>>>>> version using provider network. But It's not working good. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hello Jakub, >>>>>>>>> I will try again but if there is a bug on queens I do not think it >>>>>>>>> will be corrected because is going out of support. >>>>>>>>> Thanks >>>>>>>>> Ignazio >>>>>>>>> >>>>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>>>> >>>>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a >>>>>>>>>> node switched >>>>>>>>>> > on openvswitch works fine . The problem is this migration >>>>>>>>>> create the qbr on >>>>>>>>>> > the mode switched to openvswitch. >>>>>>>>>> > But when I switch another compute node to openvswitch and I try >>>>>>>>>> to live >>>>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not >>>>>>>>>> work because >>>>>>>>>> > the qbr presence. >>>>>>>>>> > I verified on nova logs. >>>>>>>>>> > Ignazio >>>>>>>>>> >>>>>>>>>> Hi Ignazio, >>>>>>>>>> >>>>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>>>> should >>>>>>>>>> not create the qbr on the target node. It sounds like a bug - >>>>>>>>>> IIRC the >>>>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> > >>>>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>>>> ha scritto: >>>>>>>>>> > >>>>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>>>> iptables_hybrid >>>>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>>>> which require >>>>>>>>>> >>> openvswitch firewall. >>>>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 >>>>>>>>>> without >>>>>>>>>> >> problems. >>>>>>>>>> >>> Once the compute node 2 was empty, I migrated it to >>>>>>>>>> openvswitch. >>>>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>>>> because it >>>>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>>>> >>> >>>>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>>>> iptables_hybrid >>>>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap >>>>>>>>>> under br-int as >>>>>>>>>> >>> requested by openvswich firewall, but qbr is still present >>>>>>>>>> on compute >>>>>>>>>> >> node >>>>>>>>>> >>> 1. >>>>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>>>>> compute >>>>>>>>>> >> node 1 >>>>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>>>> >>> So I think I should moving on the fly tap interfaces from qbr >>>>>>>>>> to br-int >>>>>>>>>> >> on >>>>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is a >>>>>>>>>> huge work >>>>>>>>>> >> on >>>>>>>>>> >>> a lot of instances. >>>>>>>>>> >>> >>>>>>>>>> >>> Any workaround, please ? >>>>>>>>>> >>> >>>>>>>>>> >>> Ignazio >>>>>>>>>> >>> >>>>>>>>>> >> >>>>>>>>>> >> I may be a little outdated here but to the best of my >>>>>>>>>> knowledge there >>>>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>>>> >> >>>>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you >>>>>>>>>> care about >>>>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>>>> start using >>>>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>>>> trigger a >>>>>>>>>> >> mechanism that deletes iptables rules and starts using >>>>>>>>>> openflow rules. >>>>>>>>>> >> It will leave the intermediate bridge there but except the >>>>>>>>>> extra hop in >>>>>>>>>> >> networking stack, it doesn't mind. >>>>>>>>>> >> >>>>>>>>>> >> 2) With multiple-port binding feature, what you described >>>>>>>>>> above should >>>>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>>>> perhaps he >>>>>>>>>> >> has more information about which release it should be >>>>>>>>>> functional at, I >>>>>>>>>> >> think it was Queens. Not sure if any Nova work was required to >>>>>>>>>> make it >>>>>>>>>> >> work. >>>>>>>>>> >> >>>>>>>>>> >> Hope that helps. >>>>>>>>>> >> Kuba >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Sa Pham Dang >>>>>>>> Skype: great_bn >>>>>>>> Phone/Telegram: 0986.849.582 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> Sa Pham Dang >>>>>> Skype: great_bn >>>>>> Phone/Telegram: 0986.849.582 >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> Sa Pham Dang >>>> Skype: great_bn >>>> Phone/Telegram: 0986.849.582 >>>> >>>> >>>> >> >> -- >> Sa Pham Dang >> Skype: great_bn >> Phone/Telegram: 0986.849.582 >> >> >> -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 15:59:29 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 16:59:29 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: I followed exactly the link I sent you. Il Sab 21 Mar 2020, 16:35 Sa Pham ha scritto: > Which configuration did you use? Or You configured log plugin in neutron > node? > > On Sat, Mar 21, 2020 at 10:02 PM Ignazio Cassano > wrote: > >> Sorry, I mean I added ssh access and then I removed it >> Openviswitch is a requirement for security group logs. >> So , if you read at the documentation, it suggests to modify >> iptables_hybrid on neutron node as well. >> >> 1 month ago I addes a compute node with openvswitch on an openstack with >> iptables_hybrid on neutron node: it did not worked until I modified the >> neutron node. I do not know why >> >> >> >> Il giorno sab 21 mar 2020 alle ore 15:57 Sa Pham ha >> scritto: >> >>> I just use Openvswitch for firewall driver. I did not use log plugin. >>> >>> You said you conffigured sec group rules to allow and deny. As I know, >>> Security group cannot add deny rule. >>> >>> On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano < >>> ignaziocassano at gmail.com> wrote: >>> >>>> Sa, have you modified only the compute node side ? >>>> I've modified also the controller node (neutron node) side ad reported >>>> in documentation for enabling security groups logs. >>>> >>>> https://docs.openstack.org/neutron/queens/admin/config-logging.html >>>> >>>> Ignazio >>>> >>>> >>>> >>>> Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham >>>> ha scritto: >>>> >>>>> One problem which I got few days ago. >>>>> >>>>> I have existing openstack with iptables_hybrid. I changed the firewall >>>>> driver to openvswitch then restart neutron-openvswitch-agent. >>>>> I couldn't reach that VM any more. I tried to reboot or hard reboot >>>>> that VM but It didn't work. >>>>> >>>>> >>>>> >>>>> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano < >>>>> ignaziocassano at gmail.com> wrote: >>>>> >>>>>> Sure, Sa. >>>>>> I have tested it 2 minutes ago. >>>>>> It works . >>>>>> I also changed security groups rules to allow/deny ssh access . It >>>>>> works also after hard reboot >>>>>> Ignazio >>>>>> >>>>>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>>>>> ha scritto: >>>>>> >>>>>>> With VM uses provider network directly, When I hard reboot that VM, >>>>>>> I cannot reach that VM again. Can you test in your environment? >>>>>>> >>>>>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>> >>>>>>>> Hello Sa, I am using self service and provider networks.It works >>>>>>>> fine in both cases. The problem is the migration from iptables hybrid to >>>>>>>> openvswitch without rebooting instanes. >>>>>>>> Do you mean security groups do not work on provider networks ? >>>>>>>> Ignazio >>>>>>>> >>>>>>>> >>>>>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>>>>> >>>>>>>>> Hello Ignazio, >>>>>>>>> >>>>>>>>> Does your openstack environment using self-service network ? >>>>>>>>> >>>>>>>>> I have tried openvswitch firewall native with openstack queens >>>>>>>>> version using provider network. But It's not working good. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hello Jakub, >>>>>>>>>> I will try again but if there is a bug on queens I do not think >>>>>>>>>> it will be corrected because is going out of support. >>>>>>>>>> Thanks >>>>>>>>>> Ignazio >>>>>>>>>> >>>>>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>>>>> >>>>>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a >>>>>>>>>>> node switched >>>>>>>>>>> > on openvswitch works fine . The problem is this migration >>>>>>>>>>> create the qbr on >>>>>>>>>>> > the mode switched to openvswitch. >>>>>>>>>>> > But when I switch another compute node to openvswitch and I >>>>>>>>>>> try to live >>>>>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not >>>>>>>>>>> work because >>>>>>>>>>> > the qbr presence. >>>>>>>>>>> > I verified on nova logs. >>>>>>>>>>> > Ignazio >>>>>>>>>>> >>>>>>>>>>> Hi Ignazio, >>>>>>>>>>> >>>>>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>>>>> should >>>>>>>>>>> not create the qbr on the target node. It sounds like a bug - >>>>>>>>>>> IIRC the >>>>>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> > >>>>>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>>>>> ha scritto: >>>>>>>>>>> > >>>>>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>>>>> iptables_hybrid >>>>>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>>>>> which require >>>>>>>>>>> >>> openvswitch firewall. >>>>>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 >>>>>>>>>>> without >>>>>>>>>>> >> problems. >>>>>>>>>>> >>> Once the compute node 2 was empty, I migrated it to >>>>>>>>>>> openvswitch. >>>>>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>>>>> because it >>>>>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>>>>> >>> >>>>>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>>>>> iptables_hybrid >>>>>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap >>>>>>>>>>> under br-int as >>>>>>>>>>> >>> requested by openvswich firewall, but qbr is still present >>>>>>>>>>> on compute >>>>>>>>>>> >> node >>>>>>>>>>> >>> 1. >>>>>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>>>>>> compute >>>>>>>>>>> >> node 1 >>>>>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>>>>> >>> So I think I should moving on the fly tap interfaces from >>>>>>>>>>> qbr to br-int >>>>>>>>>>> >> on >>>>>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is >>>>>>>>>>> a huge work >>>>>>>>>>> >> on >>>>>>>>>>> >>> a lot of instances. >>>>>>>>>>> >>> >>>>>>>>>>> >>> Any workaround, please ? >>>>>>>>>>> >>> >>>>>>>>>>> >>> Ignazio >>>>>>>>>>> >>> >>>>>>>>>>> >> >>>>>>>>>>> >> I may be a little outdated here but to the best of my >>>>>>>>>>> knowledge there >>>>>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>>>>> >> >>>>>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you >>>>>>>>>>> care about >>>>>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>>>>> start using >>>>>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>>>>> trigger a >>>>>>>>>>> >> mechanism that deletes iptables rules and starts using >>>>>>>>>>> openflow rules. >>>>>>>>>>> >> It will leave the intermediate bridge there but except the >>>>>>>>>>> extra hop in >>>>>>>>>>> >> networking stack, it doesn't mind. >>>>>>>>>>> >> >>>>>>>>>>> >> 2) With multiple-port binding feature, what you described >>>>>>>>>>> above should >>>>>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>>>>> perhaps he >>>>>>>>>>> >> has more information about which release it should be >>>>>>>>>>> functional at, I >>>>>>>>>>> >> think it was Queens. Not sure if any Nova work was required >>>>>>>>>>> to make it >>>>>>>>>>> >> work. >>>>>>>>>>> >> >>>>>>>>>>> >> Hope that helps. >>>>>>>>>>> >> Kuba >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Sa Pham Dang >>>>>>>>> Skype: great_bn >>>>>>>>> Phone/Telegram: 0986.849.582 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sa Pham Dang >>>>>>> Skype: great_bn >>>>>>> Phone/Telegram: 0986.849.582 >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Sa Pham Dang >>>>> Skype: great_bn >>>>> Phone/Telegram: 0986.849.582 >>>>> >>>>> >>>>> >>> >>> -- >>> Sa Pham Dang >>> Skype: great_bn >>> Phone/Telegram: 0986.849.582 >>> >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Mar 21 16:07:05 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 21 Mar 2020 17:07:05 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: I am sorry Sam, on that link there is not all configurations you need. On neutron node you must enable in /etc/neutron/plugins/ml2/openvswitch_agent.ini under [securitygroup] : firewall_driver = openvswitch Restart the openvswitch agent on neutron node Ignazio Il giorno sab 21 mar 2020 alle ore 16:35 Sa Pham ha scritto: > Which configuration did you use? Or You configured log plugin in neutron > node? > > On Sat, Mar 21, 2020 at 10:02 PM Ignazio Cassano > wrote: > >> Sorry, I mean I added ssh access and then I removed it >> Openviswitch is a requirement for security group logs. >> So , if you read at the documentation, it suggests to modify >> iptables_hybrid on neutron node as well. >> >> 1 month ago I addes a compute node with openvswitch on an openstack with >> iptables_hybrid on neutron node: it did not worked until I modified the >> neutron node. I do not know why >> >> >> >> Il giorno sab 21 mar 2020 alle ore 15:57 Sa Pham ha >> scritto: >> >>> I just use Openvswitch for firewall driver. I did not use log plugin. >>> >>> You said you conffigured sec group rules to allow and deny. As I know, >>> Security group cannot add deny rule. >>> >>> On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano < >>> ignaziocassano at gmail.com> wrote: >>> >>>> Sa, have you modified only the compute node side ? >>>> I've modified also the controller node (neutron node) side ad reported >>>> in documentation for enabling security groups logs. >>>> >>>> https://docs.openstack.org/neutron/queens/admin/config-logging.html >>>> >>>> Ignazio >>>> >>>> >>>> >>>> Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham >>>> ha scritto: >>>> >>>>> One problem which I got few days ago. >>>>> >>>>> I have existing openstack with iptables_hybrid. I changed the firewall >>>>> driver to openvswitch then restart neutron-openvswitch-agent. >>>>> I couldn't reach that VM any more. I tried to reboot or hard reboot >>>>> that VM but It didn't work. >>>>> >>>>> >>>>> >>>>> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano < >>>>> ignaziocassano at gmail.com> wrote: >>>>> >>>>>> Sure, Sa. >>>>>> I have tested it 2 minutes ago. >>>>>> It works . >>>>>> I also changed security groups rules to allow/deny ssh access . It >>>>>> works also after hard reboot >>>>>> Ignazio >>>>>> >>>>>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>>>>> ha scritto: >>>>>> >>>>>>> With VM uses provider network directly, When I hard reboot that VM, >>>>>>> I cannot reach that VM again. Can you test in your environment? >>>>>>> >>>>>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>> >>>>>>>> Hello Sa, I am using self service and provider networks.It works >>>>>>>> fine in both cases. The problem is the migration from iptables hybrid to >>>>>>>> openvswitch without rebooting instanes. >>>>>>>> Do you mean security groups do not work on provider networks ? >>>>>>>> Ignazio >>>>>>>> >>>>>>>> >>>>>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>>>>> >>>>>>>>> Hello Ignazio, >>>>>>>>> >>>>>>>>> Does your openstack environment using self-service network ? >>>>>>>>> >>>>>>>>> I have tried openvswitch firewall native with openstack queens >>>>>>>>> version using provider network. But It's not working good. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hello Jakub, >>>>>>>>>> I will try again but if there is a bug on queens I do not think >>>>>>>>>> it will be corrected because is going out of support. >>>>>>>>>> Thanks >>>>>>>>>> Ignazio >>>>>>>>>> >>>>>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>>>>> >>>>>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a >>>>>>>>>>> node switched >>>>>>>>>>> > on openvswitch works fine . The problem is this migration >>>>>>>>>>> create the qbr on >>>>>>>>>>> > the mode switched to openvswitch. >>>>>>>>>>> > But when I switch another compute node to openvswitch and I >>>>>>>>>>> try to live >>>>>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not >>>>>>>>>>> work because >>>>>>>>>>> > the qbr presence. >>>>>>>>>>> > I verified on nova logs. >>>>>>>>>>> > Ignazio >>>>>>>>>>> >>>>>>>>>>> Hi Ignazio, >>>>>>>>>>> >>>>>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>>>>> should >>>>>>>>>>> not create the qbr on the target node. It sounds like a bug - >>>>>>>>>>> IIRC the >>>>>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> > >>>>>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>>>>> ha scritto: >>>>>>>>>>> > >>>>>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>>>>> iptables_hybrid >>>>>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>>>>> which require >>>>>>>>>>> >>> openvswitch firewall. >>>>>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 >>>>>>>>>>> without >>>>>>>>>>> >> problems. >>>>>>>>>>> >>> Once the compute node 2 was empty, I migrated it to >>>>>>>>>>> openvswitch. >>>>>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>>>>> because it >>>>>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>>>>> >>> >>>>>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>>>>> iptables_hybrid >>>>>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap >>>>>>>>>>> under br-int as >>>>>>>>>>> >>> requested by openvswich firewall, but qbr is still present >>>>>>>>>>> on compute >>>>>>>>>>> >> node >>>>>>>>>>> >>> 1. >>>>>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>>>>>> compute >>>>>>>>>>> >> node 1 >>>>>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>>>>> >>> So I think I should moving on the fly tap interfaces from >>>>>>>>>>> qbr to br-int >>>>>>>>>>> >> on >>>>>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is >>>>>>>>>>> a huge work >>>>>>>>>>> >> on >>>>>>>>>>> >>> a lot of instances. >>>>>>>>>>> >>> >>>>>>>>>>> >>> Any workaround, please ? >>>>>>>>>>> >>> >>>>>>>>>>> >>> Ignazio >>>>>>>>>>> >>> >>>>>>>>>>> >> >>>>>>>>>>> >> I may be a little outdated here but to the best of my >>>>>>>>>>> knowledge there >>>>>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>>>>> >> >>>>>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you >>>>>>>>>>> care about >>>>>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>>>>> start using >>>>>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>>>>> trigger a >>>>>>>>>>> >> mechanism that deletes iptables rules and starts using >>>>>>>>>>> openflow rules. >>>>>>>>>>> >> It will leave the intermediate bridge there but except the >>>>>>>>>>> extra hop in >>>>>>>>>>> >> networking stack, it doesn't mind. >>>>>>>>>>> >> >>>>>>>>>>> >> 2) With multiple-port binding feature, what you described >>>>>>>>>>> above should >>>>>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>>>>> perhaps he >>>>>>>>>>> >> has more information about which release it should be >>>>>>>>>>> functional at, I >>>>>>>>>>> >> think it was Queens. Not sure if any Nova work was required >>>>>>>>>>> to make it >>>>>>>>>>> >> work. >>>>>>>>>>> >> >>>>>>>>>>> >> Hope that helps. >>>>>>>>>>> >> Kuba >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Sa Pham Dang >>>>>>>>> Skype: great_bn >>>>>>>>> Phone/Telegram: 0986.849.582 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sa Pham Dang >>>>>>> Skype: great_bn >>>>>>> Phone/Telegram: 0986.849.582 >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Sa Pham Dang >>>>> Skype: great_bn >>>>> Phone/Telegram: 0986.849.582 >>>>> >>>>> >>>>> >>> >>> -- >>> Sa Pham Dang >>> Skype: great_bn >>> Phone/Telegram: 0986.849.582 >>> >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sun Mar 22 16:46:28 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 22 Mar 2020 17:46:28 +0100 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Hello Sa, have you solved ? Ignazio Il Sab 21 Mar 2020, 16:35 Sa Pham ha scritto: > Which configuration did you use? Or You configured log plugin in neutron > node? > > On Sat, Mar 21, 2020 at 10:02 PM Ignazio Cassano > wrote: > >> Sorry, I mean I added ssh access and then I removed it >> Openviswitch is a requirement for security group logs. >> So , if you read at the documentation, it suggests to modify >> iptables_hybrid on neutron node as well. >> >> 1 month ago I addes a compute node with openvswitch on an openstack with >> iptables_hybrid on neutron node: it did not worked until I modified the >> neutron node. I do not know why >> >> >> >> Il giorno sab 21 mar 2020 alle ore 15:57 Sa Pham ha >> scritto: >> >>> I just use Openvswitch for firewall driver. I did not use log plugin. >>> >>> You said you conffigured sec group rules to allow and deny. As I know, >>> Security group cannot add deny rule. >>> >>> On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano < >>> ignaziocassano at gmail.com> wrote: >>> >>>> Sa, have you modified only the compute node side ? >>>> I've modified also the controller node (neutron node) side ad reported >>>> in documentation for enabling security groups logs. >>>> >>>> https://docs.openstack.org/neutron/queens/admin/config-logging.html >>>> >>>> Ignazio >>>> >>>> >>>> >>>> Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham >>>> ha scritto: >>>> >>>>> One problem which I got few days ago. >>>>> >>>>> I have existing openstack with iptables_hybrid. I changed the firewall >>>>> driver to openvswitch then restart neutron-openvswitch-agent. >>>>> I couldn't reach that VM any more. I tried to reboot or hard reboot >>>>> that VM but It didn't work. >>>>> >>>>> >>>>> >>>>> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano < >>>>> ignaziocassano at gmail.com> wrote: >>>>> >>>>>> Sure, Sa. >>>>>> I have tested it 2 minutes ago. >>>>>> It works . >>>>>> I also changed security groups rules to allow/deny ssh access . It >>>>>> works also after hard reboot >>>>>> Ignazio >>>>>> >>>>>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>>>>> ha scritto: >>>>>> >>>>>>> With VM uses provider network directly, When I hard reboot that VM, >>>>>>> I cannot reach that VM again. Can you test in your environment? >>>>>>> >>>>>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>> >>>>>>>> Hello Sa, I am using self service and provider networks.It works >>>>>>>> fine in both cases. The problem is the migration from iptables hybrid to >>>>>>>> openvswitch without rebooting instanes. >>>>>>>> Do you mean security groups do not work on provider networks ? >>>>>>>> Ignazio >>>>>>>> >>>>>>>> >>>>>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>>>>> >>>>>>>>> Hello Ignazio, >>>>>>>>> >>>>>>>>> Does your openstack environment using self-service network ? >>>>>>>>> >>>>>>>>> I have tried openvswitch firewall native with openstack queens >>>>>>>>> version using provider network. But It's not working good. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hello Jakub, >>>>>>>>>> I will try again but if there is a bug on queens I do not think >>>>>>>>>> it will be corrected because is going out of support. >>>>>>>>>> Thanks >>>>>>>>>> Ignazio >>>>>>>>>> >>>>>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>>>>> >>>>>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a >>>>>>>>>>> node switched >>>>>>>>>>> > on openvswitch works fine . The problem is this migration >>>>>>>>>>> create the qbr on >>>>>>>>>>> > the mode switched to openvswitch. >>>>>>>>>>> > But when I switch another compute node to openvswitch and I >>>>>>>>>>> try to live >>>>>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not >>>>>>>>>>> work because >>>>>>>>>>> > the qbr presence. >>>>>>>>>>> > I verified on nova logs. >>>>>>>>>>> > Ignazio >>>>>>>>>>> >>>>>>>>>>> Hi Ignazio, >>>>>>>>>>> >>>>>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>>>>> should >>>>>>>>>>> not create the qbr on the target node. It sounds like a bug - >>>>>>>>>>> IIRC the >>>>>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> > >>>>>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>>>>> ha scritto: >>>>>>>>>>> > >>>>>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>>>>> iptables_hybrid >>>>>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>>>>> which require >>>>>>>>>>> >>> openvswitch firewall. >>>>>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 >>>>>>>>>>> without >>>>>>>>>>> >> problems. >>>>>>>>>>> >>> Once the compute node 2 was empty, I migrated it to >>>>>>>>>>> openvswitch. >>>>>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>>>>> because it >>>>>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>>>>> >>> >>>>>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>>>>> iptables_hybrid >>>>>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap >>>>>>>>>>> under br-int as >>>>>>>>>>> >>> requested by openvswich firewall, but qbr is still present >>>>>>>>>>> on compute >>>>>>>>>>> >> node >>>>>>>>>>> >>> 1. >>>>>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration from >>>>>>>>>>> compute >>>>>>>>>>> >> node 1 >>>>>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>>>>> >>> So I think I should moving on the fly tap interfaces from >>>>>>>>>>> qbr to br-int >>>>>>>>>>> >> on >>>>>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is >>>>>>>>>>> a huge work >>>>>>>>>>> >> on >>>>>>>>>>> >>> a lot of instances. >>>>>>>>>>> >>> >>>>>>>>>>> >>> Any workaround, please ? >>>>>>>>>>> >>> >>>>>>>>>>> >>> Ignazio >>>>>>>>>>> >>> >>>>>>>>>>> >> >>>>>>>>>>> >> I may be a little outdated here but to the best of my >>>>>>>>>>> knowledge there >>>>>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>>>>> >> >>>>>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you >>>>>>>>>>> care about >>>>>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>>>>> start using >>>>>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>>>>> trigger a >>>>>>>>>>> >> mechanism that deletes iptables rules and starts using >>>>>>>>>>> openflow rules. >>>>>>>>>>> >> It will leave the intermediate bridge there but except the >>>>>>>>>>> extra hop in >>>>>>>>>>> >> networking stack, it doesn't mind. >>>>>>>>>>> >> >>>>>>>>>>> >> 2) With multiple-port binding feature, what you described >>>>>>>>>>> above should >>>>>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>>>>> perhaps he >>>>>>>>>>> >> has more information about which release it should be >>>>>>>>>>> functional at, I >>>>>>>>>>> >> think it was Queens. Not sure if any Nova work was required >>>>>>>>>>> to make it >>>>>>>>>>> >> work. >>>>>>>>>>> >> >>>>>>>>>>> >> Hope that helps. >>>>>>>>>>> >> Kuba >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Sa Pham Dang >>>>>>>>> Skype: great_bn >>>>>>>>> Phone/Telegram: 0986.849.582 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sa Pham Dang >>>>>>> Skype: great_bn >>>>>>> Phone/Telegram: 0986.849.582 >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Sa Pham Dang >>>>> Skype: great_bn >>>>> Phone/Telegram: 0986.849.582 >>>>> >>>>> >>>>> >>> >>> -- >>> Sa Pham Dang >>> Skype: great_bn >>> Phone/Telegram: 0986.849.582 >>> >>> >>> > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsafrono at redhat.com Sun Mar 22 16:10:58 2020 From: rsafrono at redhat.com (Roman Safronov) Date: Sun, 22 Mar 2020 18:10:58 +0200 Subject: [tempest] what is a proper way to install a package into a vm instance spawned by tempest? Message-ID: Hi, I wrote a tempest test which requires keepalived to be running inside vm instance. The test tries to install keepalived package inside vm by running "apt install/yum install". However as I see in upstream gates logs this does not work while the same code works perfectly in our downstream CI using the same image. Do vm instances spawned on upstream gates during tests have a route to the internet? Is there an alternative way that I can use in order to install a package? Thanks in advance -- ROMAN SAFRONOV SENIOR QE, OPENSTACK NETWORKING Red Hat Israel M: +972545433957 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hongbin034 at gmail.com Sun Mar 22 17:28:01 2020 From: hongbin034 at gmail.com (Hongbin Lu) Date: Sun, 22 Mar 2020 13:28:01 -0400 Subject: [k8s][zun] Introduce a new feature in Ussuri - CRI integration Message-ID: Hi all, As we are approaching the end of Ussuri cycle, I would like to take a chance to introduce a new feature the Zun team implemented in this cycle - CRI integration [1]. As known by people, Zun is an OpenStack Container service. It provides API for users to create and manage application containers in an OpenStack cloud. The main concepts in Zun are "Container" and "Capsule". A container is a single container, while a capsule is a group of co-located and co-scheduled containers (basically the same as k8s pod). Probably, the "Container" concept is more widely used. People can use the /containers API endpoint to create and manage a single container. Under the hook, a container is a Docker container in a compute node. What is special is that each Docker container is given a Neutron port so the container is connected to a tenant network in Neutron. Kuryr-libnetwork is the Docker network plugin we use to perform the Neutron port binding which basically connects the container to the virtual switch managed by Neutron. As mentioned before, the concept of "Capsule" in Zun is basically the same as pod in k8s. We introduced this concept mainly for k8s integration. Roughly speaking, the Zun-k8s integration is achieved by (i) registering a special node in k8s, (ii) watching the k8s API for pods being scheduled to this node and (iii) invoking Zun's /capsules API endpoint to create a capsule for each incoming pod. The (i) and (ii) is done by a CNCF sandbox project called Virtual Kubelet [2]. The (iii) is achieved by providing an OpenStack provider [3] for Virtual Kubelet. The special node registered by Virtual Kubelet is called a virtual node because the node doesn't physically exist. Pods being scheduled to the virtual node is basically offloaded from the current k8s cluster, eventually landed on an external platform such as an OpenStack cloud. In high level, what is offered to end-users is a "serverless kubernetes pod" [4]. This term basically means the ability to run pods on demand without planing the capacity (i.e. nodes) upfront. An example of that is AWS EKS on Fargate [5]. In comparison, the traditional approach is to create an entire k8s cluster upfront in order to run the workload. Let's give a simple example. Suppose you want to run a pod, the traditional approach is to provision a k8s cluster with a worker node. Then, run the pod on the worker node. In contract, the "serverless" approach is to create a k8s cluster without any worker node and the pod is offloaded to a cloud provider that provisions the pods at runtime. This approach works well for applications who have fluctuated workloads so it is hard to provision a cluster with the right size for them. Furthermore, from cloud provider's perspective, if all tenant users offloads their pods to the cloud, the cloud provider might be able to pack the workload better (i.e. with fewer physical nodes) thus saving cost. Under the hook, a capsule is a podsandbox with one or more containers in a CRI runtime (i.e. containerd). Compared to Docker, a CRI runtime has a better support for the pod concept so we chose it to implement capsule. A caveat is that CRI requires a CNI plugin for the networking, so we need to implement a CNI plugin for Zun (called zun-cni). The role of CNI plugin is similar as kuryr-libnetwork that we are using for Docker except it implements a different networking model (CNI). I summaries it as below: +--------------+------------------------+---------------+ | Concept | Container | Capsule (Pod) | +--------------+------------------------+---------------+ | API endpoint | /containers | /capsules | | Engine | Docker | CRI runtime | | Network | kuryr-libnetwork (CNM) | zun-cni (CNI) | +--------------+------------------------+---------------+ Typically, a CRI runtime works well with Kata Container which provides hypervisor-based isolation for neighboring containers in the same node. As a result, it is secure to consolidate pods from different tenants into a single node which increases the resource utilization. For deployment, a typical stack looks like below: +----------------------------------------------+ | k8s control plane | +----------------------------------------------+ | Virtual Kubelet (OpenStack provider) | +----------------------------------------------+ | OpenStack control plane (Zun, Neutron, etc.) | +----------------------------------------------+ | OpenStack data plane | | (Zun compute agent, Neutron OVS agent, etc.) | +----------------------------------------------+ | Containerd (with CRI plugin) | +----------------------------------------------+ | Kata Container | +----------------------------------------------+ In this stack, if a user creates a deployment or pod in k8s, the k8s scheduler will schedule the pod to the virtual node registered by Virtual Kubelet. Virtual Kubelet will pick up the pod and let the configured cloud provider to handle it. The cloud provider invokes Zun API to create a capsule. Upon receiving the API request to create a capsule, Zun scheduler will schedule the capsule to a compute node. The Zun compute agent in that node will provision the capsule using a CRI runtime (containerd in this example). The Zun-CRI runtime communication is done via a gRPC protocol through a unix socket. The CRI runtime will first create the pod in Kata Container (or runc as an alternative) that realizes the pod using a lightweight VM. Furthermore, the CRI runtime will use a CNI plugin, which is the zun-cni binary, to setup the network. The zun-cni binary is a thin executable that dispatches the CNI command to a daemon service called zun-cni-daemon. The community is via HTTP within localhost. The zun-cni-daemon will look up the Neutron port information from DB and perform the port binding. In conclusion, starting from Ussuri, Zun adds support for CRI-compatible runtime. Zun uses CRI runtime to realize the concept of pod. Using this feature together with Virtual Kubelet and Kata Container, we can offer "serverless kubernetes pod" service which is comparable with AWS EKS with Fargate. [1] https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime [2] https://github.com/virtual-kubelet/virtual-kubelet [3] https://github.com/virtual-kubelet/openstack-zun [4] https://aws.amazon.com/about-aws/whats-new/2019/12/run-serverless-kubernetes-pods-using-amazon-eks-and-aws-fargate/ [5] https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Mon Mar 23 00:32:44 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sun, 22 Mar 2020 19:32:44 -0500 Subject: [cinder][qa] propose Liang Fang for devstack-plugin-open-cas-core In-Reply-To: References: Message-ID: <17104ce5a07.11f85a429418920.2702189657941393021@ghanshyammann.com> ---- On Fri, 20 Mar 2020 09:15:22 -0500 Brian Rosmaita wrote ---- > Liang Fang is driving the effort to create the devstack-plugin-open-cas > and use it to test the cinder/nova volume-local-cache feature with > tempest. Since he's our SME on open-cas, I propose that he be made a > member of devstack-plugin-open-cas-core. (Currently, the only members > of that group are the members of cinder-core and devstack-core.) > > In the absence of objections, I'll make the change before the next > Cinder team meeting (Wednesday, 25 March at 1400 UTC in its new location > of #openstack-meeting-alt). > +1 on adding him. Though repo is without code for now but he mentioned to start it soon. As cinder-core group, you will be able to add him to the list, just in case any issue ping me on qa channel. -gmann > > cheers, > brian > > From gmann at ghanshyammann.com Mon Mar 23 00:36:01 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sun, 22 Mar 2020 19:36:01 -0500 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> References: <1584638630.12170.41@est.tech> <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> Message-ID: <17104d159c9.af6a95dd418933.7211743314444144094@ghanshyammann.com> ---- On Thu, 19 Mar 2020 14:06:51 -0500 Monty Taylor wrote ---- > > > > On Mar 19, 2020, at 12:23 PM, Balázs Gibizer wrote: > > > > Hi, > > > > Nova has an unwritten rule that requires to have at least two companies involved in any new feature development (or even bugfix?). In the current Nova core diversity situation this rule puts extra burden to the remaining non Red Hat cores and I guess it also makes any Red Hat driven feature development harder. In parallel to working on increasing the size of the core team I suggest to reconsider dropping this rule. > > > > Some discussion happened already on the today's meeting[1] > > I think that’s a great idea. FWIW I’ve never liked that rule, because it assume that developers from a company are putting employer requirements over project requirements when acting in their capacity as a core reviewer - which is contrary to our general expectation of how a core reviewer behaves. > > I think it’s a great idea to get rid of this policy - and then if anyone is behaving in a manner that abuses the trust of the rest of the core reviewers, such as slamming through a new feature that other people obviously have misgivings about … that can be dealt with the same way any other breach of trust would happen. +1, Well explained by Monty. -gmann > > > Cheers, > > gibi > > > > > > [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 > > > > > > > > > > > > > From saphi070 at gmail.com Mon Mar 23 01:15:54 2020 From: saphi070 at gmail.com (Sa Pham) Date: Mon, 23 Mar 2020 08:15:54 +0700 Subject: [qeeens][neutron] migrating from iptables_hybrid to openvswitch In-Reply-To: References: <07051a9a-db1b-1bb1-c166-bad6d173893a@redhat.com> Message-ID: Hello Ignazio, I havent tried it yet. I will test this case on this week. On Sun, Mar 22, 2020 at 11:46 PM Ignazio Cassano wrote: > Hello Sa, have you solved ? > Ignazio > > Il Sab 21 Mar 2020, 16:35 Sa Pham ha scritto: > >> Which configuration did you use? Or You configured log plugin in neutron >> node? >> >> On Sat, Mar 21, 2020 at 10:02 PM Ignazio Cassano < >> ignaziocassano at gmail.com> wrote: >> >>> Sorry, I mean I added ssh access and then I removed it >>> Openviswitch is a requirement for security group logs. >>> So , if you read at the documentation, it suggests to modify >>> iptables_hybrid on neutron node as well. >>> >>> 1 month ago I addes a compute node with openvswitch on an openstack with >>> iptables_hybrid on neutron node: it did not worked until I modified the >>> neutron node. I do not know why >>> >>> >>> >>> Il giorno sab 21 mar 2020 alle ore 15:57 Sa Pham >>> ha scritto: >>> >>>> I just use Openvswitch for firewall driver. I did not use log plugin. >>>> >>>> You said you conffigured sec group rules to allow and deny. As I know, >>>> Security group cannot add deny rule. >>>> >>>> On Sat, Mar 21, 2020 at 9:53 PM Ignazio Cassano < >>>> ignaziocassano at gmail.com> wrote: >>>> >>>>> Sa, have you modified only the compute node side ? >>>>> I've modified also the controller node (neutron node) side ad reported >>>>> in documentation for enabling security groups logs. >>>>> >>>>> https://docs.openstack.org/neutron/queens/admin/config-logging.html >>>>> >>>>> Ignazio >>>>> >>>>> >>>>> >>>>> Il giorno sab 21 mar 2020 alle ore 15:49 Sa Pham >>>>> ha scritto: >>>>> >>>>>> One problem which I got few days ago. >>>>>> >>>>>> I have existing openstack with iptables_hybrid. I changed the >>>>>> firewall driver to openvswitch then restart neutron-openvswitch-agent. >>>>>> I couldn't reach that VM any more. I tried to reboot or hard reboot >>>>>> that VM but It didn't work. >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Mar 21, 2020 at 9:41 PM Ignazio Cassano < >>>>>> ignaziocassano at gmail.com> wrote: >>>>>> >>>>>>> Sure, Sa. >>>>>>> I have tested it 2 minutes ago. >>>>>>> It works . >>>>>>> I also changed security groups rules to allow/deny ssh access . It >>>>>>> works also after hard reboot >>>>>>> Ignazio >>>>>>> >>>>>>> Il giorno sab 21 mar 2020 alle ore 14:22 Sa Pham >>>>>>> ha scritto: >>>>>>> >>>>>>>> With VM uses provider network directly, When I hard reboot that VM, >>>>>>>> I cannot reach that VM again. Can you test in your environment? >>>>>>>> >>>>>>>> On Sat, Mar 21, 2020 at 7:33 PM Ignazio Cassano < >>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hello Sa, I am using self service and provider networks.It works >>>>>>>>> fine in both cases. The problem is the migration from iptables hybrid to >>>>>>>>> openvswitch without rebooting instanes. >>>>>>>>> Do you mean security groups do not work on provider networks ? >>>>>>>>> Ignazio >>>>>>>>> >>>>>>>>> >>>>>>>>> Il Sab 21 Mar 2020, 12:38 Sa Pham ha scritto: >>>>>>>>> >>>>>>>>>> Hello Ignazio, >>>>>>>>>> >>>>>>>>>> Does your openstack environment using self-service network ? >>>>>>>>>> >>>>>>>>>> I have tried openvswitch firewall native with openstack queens >>>>>>>>>> version using provider network. But It's not working good. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Mar 19, 2020 at 11:12 PM Ignazio Cassano < >>>>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hello Jakub, >>>>>>>>>>> I will try again but if there is a bug on queens I do not think >>>>>>>>>>> it will be corrected because is going out of support. >>>>>>>>>>> Thanks >>>>>>>>>>> Ignazio >>>>>>>>>>> >>>>>>>>>>> Il giorno gio 19 mar 2020 alle ore 13:54 Jakub Libosvar < >>>>>>>>>>> jlibosva at redhat.com> ha scritto: >>>>>>>>>>> >>>>>>>>>>>> On 13/03/2020 08:24, Ignazio Cassano wrote: >>>>>>>>>>>> > Hu Jakub, migrating vm from a not with hybrid_itatabes ti a >>>>>>>>>>>> node switched >>>>>>>>>>>> > on openvswitch works fine . The problem is this migration >>>>>>>>>>>> create the qbr on >>>>>>>>>>>> > the mode switched to openvswitch. >>>>>>>>>>>> > But when I switch another compute node to openvswitch and I >>>>>>>>>>>> try to live >>>>>>>>>>>> > migrate the same vm (openvswitch to qopenswitch) it does not >>>>>>>>>>>> work because >>>>>>>>>>>> > the qbr presence. >>>>>>>>>>>> > I verified on nova logs. >>>>>>>>>>>> > Ignazio >>>>>>>>>>>> >>>>>>>>>>>> Hi Ignazio, >>>>>>>>>>>> >>>>>>>>>>>> I think the first step - migrating from hybrid_iptables to ovs >>>>>>>>>>>> should >>>>>>>>>>>> not create the qbr on the target node. It sounds like a bug - >>>>>>>>>>>> IIRC the >>>>>>>>>>>> libvirt domxml should not have the qbr when migrating. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>>>>> > Il Gio 12 Mar 2020, 23:15 Jakub Libosvar >>>>>>>>>>>> ha scritto: >>>>>>>>>>>> > >>>>>>>>>>>> >> On 12/03/2020 11:38, Ignazio Cassano wrote: >>>>>>>>>>>> >>> Hello All, I am facing some problems migrating from >>>>>>>>>>>> iptables_hybrid >>>>>>>>>>>> >>> frirewall to openvswitch firewall on centos 7 queens, >>>>>>>>>>>> >>> I am doing this because I want enable security groups logs >>>>>>>>>>>> which require >>>>>>>>>>>> >>> openvswitch firewall. >>>>>>>>>>>> >>> I would like to migrate without restarting my instances. >>>>>>>>>>>> >>> I startded moving all instances from compute node 1. >>>>>>>>>>>> >>> Then I configured openvswitch firewall on compute node 1, >>>>>>>>>>>> >>> Instances migrated from compute node 2 to compute node 1 >>>>>>>>>>>> without >>>>>>>>>>>> >> problems. >>>>>>>>>>>> >>> Once the compute node 2 was empty, I migrated it to >>>>>>>>>>>> openvswitch. >>>>>>>>>>>> >>> But now instances does not migrate from node 1 to node 2 >>>>>>>>>>>> because it >>>>>>>>>>>> >>> requires the presence of qbr bridge on node 2 >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> This happened because migrating instances from node2 with >>>>>>>>>>>> iptables_hybrid >>>>>>>>>>>> >>> to compute node 1 with openvswitch, does not put the tap >>>>>>>>>>>> under br-int as >>>>>>>>>>>> >>> requested by openvswich firewall, but qbr is still present >>>>>>>>>>>> on compute >>>>>>>>>>>> >> node >>>>>>>>>>>> >>> 1. >>>>>>>>>>>> >>> Once I enabled openvswitch on compute node 2, migration >>>>>>>>>>>> from compute >>>>>>>>>>>> >> node 1 >>>>>>>>>>>> >>> fails because it exprects qbr on compute node 2 . >>>>>>>>>>>> >>> So I think I should moving on the fly tap interfaces from >>>>>>>>>>>> qbr to br-int >>>>>>>>>>>> >> on >>>>>>>>>>>> >>> compute node 1 before migrating to compute node 2 but it is >>>>>>>>>>>> a huge work >>>>>>>>>>>> >> on >>>>>>>>>>>> >>> a lot of instances. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> Any workaround, please ? >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> Ignazio >>>>>>>>>>>> >>> >>>>>>>>>>>> >> >>>>>>>>>>>> >> I may be a little outdated here but to the best of my >>>>>>>>>>>> knowledge there >>>>>>>>>>>> >> are two ways how to migrate from iptables to openvswitch. >>>>>>>>>>>> >> >>>>>>>>>>>> >> 1) If you don't mind the intermediate linux bridge and you >>>>>>>>>>>> care about >>>>>>>>>>>> >> logs, you can just change the config file on compute node to >>>>>>>>>>>> start using >>>>>>>>>>>> >> openvswitch firewall and restart the ovs agent. That should >>>>>>>>>>>> trigger a >>>>>>>>>>>> >> mechanism that deletes iptables rules and starts using >>>>>>>>>>>> openflow rules. >>>>>>>>>>>> >> It will leave the intermediate bridge there but except the >>>>>>>>>>>> extra hop in >>>>>>>>>>>> >> networking stack, it doesn't mind. >>>>>>>>>>>> >> >>>>>>>>>>>> >> 2) With multiple-port binding feature, what you described >>>>>>>>>>>> above should >>>>>>>>>>>> >> be working. I know Miguel spent some time working on that so >>>>>>>>>>>> perhaps he >>>>>>>>>>>> >> has more information about which release it should be >>>>>>>>>>>> functional at, I >>>>>>>>>>>> >> think it was Queens. Not sure if any Nova work was required >>>>>>>>>>>> to make it >>>>>>>>>>>> >> work. >>>>>>>>>>>> >> >>>>>>>>>>>> >> Hope that helps. >>>>>>>>>>>> >> Kuba >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Sa Pham Dang >>>>>>>>>> Skype: great_bn >>>>>>>>>> Phone/Telegram: 0986.849.582 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Sa Pham Dang >>>>>>>> Skype: great_bn >>>>>>>> Phone/Telegram: 0986.849.582 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> Sa Pham Dang >>>>>> Skype: great_bn >>>>>> Phone/Telegram: 0986.849.582 >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> Sa Pham Dang >>>> Skype: great_bn >>>> Phone/Telegram: 0986.849.582 >>>> >>>> >>>> >> >> -- >> Sa Pham Dang >> Skype: great_bn >> Phone/Telegram: 0986.849.582 >> >> >> -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Mon Mar 23 01:16:10 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sun, 22 Mar 2020 20:16:10 -0500 Subject: [all][privsep] Migrate from oslo.rootwrap to oslo.privsep In-Reply-To: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> References: <79f8e1a0e41ebda7dc9271e44c03197ec3c04f85.camel@redhat.com> Message-ID: <17104f61cf3.b245abf6419066.5934766632954977113@ghanshyammann.com> ---- On Thu, 19 Mar 2020 10:45:23 -0500 Rodolfo Alonso wrote ---- > Hello all: > > With this mail I would like to propose the goal to move away from oslo.rootwrap and migrate to > oslo.privsep. The last one offers a superior security model, faster and more secure. During the last > cycles and since privsep was released, the Community has encouraged the usage of privsep and the > deprecation of any existing code still using rootwrap. > > For any developer willing to collaborate, there are plenty of code examples, as I’ll provide later, > implementing and using privsep for new methods and migrations. > > If this goal is approved, I'll open a Story (https://storyboard.openstack.org/) and any developer > will be able to add a task for each patch or set of them related. This would be the tracker for this > common effort. Thanks Rodolfo for taking initiative on this effort, much appreciated. Just to be clear, In this ML, we are checking this as a possible goal candidate for V cycle[1]. I have mentioned this ML link in etherpad: https://etherpad.openstack.org/p/YVR-v-series-goals Once we get the heads up from ML and sort out the open questions, the next step will be to propose this as a goal in governance repo governance/tree/master/goals/proposed. After this goal and its definition are accepted then we select this for cycle goal [2]. On/after it is selected as a cycle goal, then you as goal champion can start the storyboard for tracking. [1] https://etherpad.openstack.org/p/YVR-v-series-goals [2] https://governance.openstack.org/tc/goals/#process-details -gmann > > > PROJECTS TO MIGRATE. > -------------------- > Projects that are still using rootwrap: > http://codesearch.openstack.org/?q=rootwrap&i=nope&files=.*.py&repos= > neutron > os-brick > designate > cinder > ironic-inspector > neutron-vpnaas > nova > solum > glance_store > ironic > zun > magnum > manila > networking-bagpipe > sahara > ceilometer > cinderlib > freezer > ironic-lib > monasca-agent > tacker > tripleo-common > > > USAGE DOCUMENTATION ABOUT PRIVSEP. > ---------------------------------- > How to create a privsep context, assign privileges and use it as a decorator: > https://docs.openstack.org/oslo.privsep/latest/user/index.html > > > HOW TO MIGRATE FROM ROOTWRAP TO PRIVSEP. > ---------------------------------------- > From the same link provided previously, in the section “Converting from rootwrap to privsep”: > https://docs.openstack.org/oslo.privsep/latest/user/index.html#converting-from-rootwrap-to-privsep > > oslo.privsep provides a class, PrivContext, that can be used to create a decorator function. The > instance created is a context of execution and has defined a list of capabilities, matching the > Linux capabilities. The privsep context decorator should contain the minimum needed capabilities to > execute the decorated function. > > For example: > > default = priv_context.PrivContext( > __name__, > cfg_section='privsep', > pypath=__name__ + '.default', > capabilities=[caps.CAP_SYS_ADMIN, > caps.CAP_NET_ADMIN, > caps.CAP_DAC_OVERRIDE, > caps.CAP_DAC_READ_SEARCH, > caps.CAP_SYS_PTRACE], > ) > > > The function “entrypoint” of this instance can be used as a decorator for another function: > > @privileged.default.entrypoint > def delete_interface(ifname, namespace, **kwargs): > _run_iproute_link("del", ifname, namespace, **kwargs) > > > As commented in the given link, a straight 1:1 filter:function replacement generally results in > functions that are still too broad for good security. It is better to replace each chmod rootwrap > call with a narrow privsep function that will limit it to specific files. > > > MIGRATION EXAMPLES. > ------------------- > Nova: > https://review.opendev.org/#/q/project:openstack/nova+branch:master+topic:my-own-personal-alternative-universe > Neutron: > https://review.opendev.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bug/1492714 > os-vif: https://review.opendev.org/#/c/287725/ > > > Thank you and regards. > > > > From thierry at openstack.org Mon Mar 23 09:31:38 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 23 Mar 2020 10:31:38 +0100 Subject: [k8s][zun] Introduce a new feature in Ussuri - CRI integration In-Reply-To: References: Message-ID: <2a8f670b-036b-7015-d7cd-63c4fcc32331@openstack.org> Hongbin Lu wrote: > [...] > In conclusion, starting from Ussuri, Zun adds support for CRI-compatible > runtime. Zun uses CRI runtime to realize the concept of pod. Using this > feature together with Virtual Kubelet and Kata Container, we can offer > "serverless kubernetes pod" service which is comparable with AWS > EKS with Fargate. That's very exciting integration work, Hongbin! Don't forget to mention it in your cycle-highlights, as I expect it to be picked up in Ussuri release communications :) -- Thierry From thierry at openstack.org Mon Mar 23 10:04:43 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 23 Mar 2020 11:04:43 +0100 Subject: [largescale-sig] Next meeting: Mar 25, 9utc Message-ID: <7e2bb4ea-0e50-d92d-10a3-9e1d51862a1d@openstack.org> Hi everyone, The Large Scale SIG will have a meeting this week on Wednesday, Mar 25 at 9 UTC[1] in #openstack-meeting on IRC: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200325T09 Feel free to add topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting A reminder of the TODOs we had from last meeting, in case you have time to make progress on them: - amorin to create a wiki page for large scale documentation - amorin to propose patch against Nova doc - all to check/comment on https://etherpad.openstack.org/p/LargeScaleOps_OpenDev - all to review new patchset of oslo.metrics spec https://review.opendev.org/#/c/704733/ - oneswig to contribute a scaling story on bare metal cluster scaling Talk to you all on Wednesday, -- Thierry Carrez From sbauza at redhat.com Mon Mar 23 11:04:02 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Mon, 23 Mar 2020 12:04:02 +0100 Subject: [nova] FYI for out-of-tree virt drivers : new argument for finish_migration() and finish_revert_migration() Message-ID: I wrote two changes [1] for passing down target allocations to virt drivers given we need to use them for closing some bugfix [2] Given the Nova contributor policy is saying that the public API is not related to virt drivers, we won't support virt drivers that won't accept this new argument. If you're a contributor that works on any out-of-tree Nova virt driver, please make sure that you modify your own two above methods. Thanks, -Sylvain [1] https://review.opendev.org/#/c/589085/7 and https://review.opendev.org/#/c/712118/2 [2] https://bugs.launchpad.net/nova/+bug/1778563 -------------- next part -------------- An HTML attachment was scrubbed... URL: From geguileo at redhat.com Mon Mar 23 11:13:20 2020 From: geguileo at redhat.com (Gorka Eguileor) Date: Mon, 23 Mar 2020 12:13:20 +0100 Subject: [cinder][qa] propose Liang Fang for devstack-plugin-open-cas-core In-Reply-To: References: Message-ID: <20200323111320.xnp4slfonqjfh3ti@localhost> On 20/03, Brian Rosmaita wrote: > Liang Fang is driving the effort to create the devstack-plugin-open-cas and > use it to test the cinder/nova volume-local-cache feature with tempest. > Since he's our SME on open-cas, I propose that he be made a member of > devstack-plugin-open-cas-core. (Currently, the only members of that group > are the members of cinder-core and devstack-core.) > > In the absence of objections, I'll make the change before the next Cinder > team meeting (Wednesday, 25 March at 1400 UTC in its new location of > #openstack-meeting-alt). > > > cheers, > brian > +1 Great idea! From alifshit at redhat.com Mon Mar 23 12:41:28 2020 From: alifshit at redhat.com (Artom Lifshitz) Date: Mon, 23 Mar 2020 08:41:28 -0400 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: References: <1584638630.12170.41@est.tech> Message-ID: On Thu, Mar 19, 2020 at 4:51 PM melanie witt wrote: > > On 3/19/20 10:23, Balázs Gibizer wrote: > > Hi, > > > > Nova has an unwritten rule that requires to have at least two companies > > involved in any new feature development (or even bugfix?). In the > > current Nova core diversity situation this rule puts extra burden to the > > remaining non Red Hat cores and I guess it also makes any Red Hat driven > > feature development harder. In parallel to working on increasing the > > size of the core team I suggest to reconsider dropping this rule. > > I have historically always appreciated the two-company approval rule > because it ensures each patch has been reviewed with at least two > different and diverse company perspectives. Same here. It was never really a matter of preventing abuse by a single company - I trust the RH cores to not have ill intentions - it was more to guarantee a diverse set of perspectives on any given patch. Sometimes we wear blinders without even knowing, and it's good to get a reality check. > However I recognize that the situation has changed over time and we may > not have the luxury any longer to ensure diverse company approvals. I do > not wish to overburden the remaining non Red Hat cores with review > requests for approvals. Speaking of reality, we have to recognize it as it is, not as we'd like it to be. And it's just not realistic to have every single RH patch go through gibi or Alex. > So, given the evolved landscape, I am understanding and open to adapt > our process to drop the two-company rule if there is such consensus in > the rest of the team. FWIW I agree the two company rule no longer makes sense. > Cheers, > -melanie > > > Some discussion happened already on the today's meeting[1]. > > > > Cheers, > > gibi > > > > > > [1] > > http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-03-19-16.00.log.html#l-90 > > > > > > > > > > > > From skaplons at redhat.com Mon Mar 23 13:59:20 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Mon, 23 Mar 2020 14:59:20 +0100 Subject: [neutron] Week 13 - bug deputy report Message-ID: <20200323135920.hkulk753abehwj3r@skaplons-mac> Hi, Last week I was on bug deputy. Below is my summary of it. *Critical* None *High* https://bugs.launchpad.net/neutron/+bug/1867936 - test_update_delete_extra_route failing due to timeout when creating subnets - HIGH as it’s gate failure This has no assignment yet. For me it looks like generall issue which we have from time to time with some very slow responses from API but it has to be verified. https://bugs.launchpad.net/neutron/+bug/1868237 - [tempest] Error in "test_multicast_between_vms_on_same_network" test case - HIGH as this is gate-failure No assignment yet. https://bugs.launchpad.net/neutron/+bug/1868110 - [OVN] neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_ovn_db_sync.TestOvnNbSyncOverTcp.test_ovn_nb_sync_log randomly fails - HIGH as it’s gate failure maciejjozefczyk is working on it *Medium* https://bugs.launchpad.net/neutron/+bug/1868100 - ncat process isn't always spawned on guest vm in scenario tests - MEDIUM Already fixed https://bugs.launchpad.net/neutron/+bug/1868125 - [ovn] Metadata agent spawns haproxy quickly twice on a single new port binding - MEDIUM Already fixed https://bugs.launchpad.net/neutron/+bug/1868137 - default geneve overhead is set not enough - MEDIUM Someone from ovn subteam can take a look *Low* https://bugs.launchpad.net/neutron/+bug/1867869 - [OVN] Remove dependency on fip_object - LOW Patch proposed https://review.opendev.org/#/c/713585/ *Incomplete* https://bugs.launchpad.net/neutron/+bug/1868098 - Neutron openvswitch agent error - INCOMPLETE But would be good if someone can take a look *Stadium projects* https://bugs.launchpad.net/neutron/+bug/1868515 - vpnaas ike policy is not allowed to be updated No assignment yet. Needs someone from vpnaas team to take a look. -- Slawek Kaplonski Senior software engineer Red Hat From balazs.gibizer at est.tech Mon Mar 23 14:28:55 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 23 Mar 2020 15:28:55 +0100 Subject: [nova] Proposing Lee Yarwood as nova core Message-ID: Hi, I'm proposing Lee to the core team. He is around for a long time and he became really active recently not only on the stable branches. I think he would be a great addition to the team. Cores, please vote (-1, 0, +1) before next Monday (03.30.) Cheers, gibi From openstack at fried.cc Mon Mar 23 15:18:52 2020 From: openstack at fried.cc (Eric Fried) Date: Mon, 23 Mar 2020 10:18:52 -0500 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: <6017c774-c3fd-6ce3-a7e7-3248e546be32@fried.cc> Alumnus +1 On 3/23/20 9:28 AM, Balázs Gibizer wrote: > Hi, > > I'm proposing Lee to the core team. He is around for a long time and he > became really active recently not only on the stable branches. I think > he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > Cheers, > gibi > > > From cboylan at sapwetik.org Mon Mar 23 15:23:48 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 23 Mar 2020 08:23:48 -0700 Subject: =?UTF-8?Q?Re:_[tempest]_what_is_a_proper_way_to_install_a_package_into_a?= =?UTF-8?Q?_vm_instance_spawned_by_tempest=3F?= In-Reply-To: References: Message-ID: On Sun, Mar 22, 2020, at 9:10 AM, Roman Safronov wrote: > Hi, > > I wrote a tempest test > which requires keepalived to be running inside vm instance. > The test tries to install keepalived package inside vm by running "apt > install/yum install". > However as I see in upstream gates logs this does not work while the > same code works perfectly in our downstream CI using the same image. > > Do vm instances spawned on upstream gates during tests have a route to > the internet? > Is there an alternative way that I can use in order to install a > package? By default the tempest jobs use a cirros image. This is a very small, quick to boot linux without a package manager. If you need services to be running in the image typically you'll need to boot a more typical linux installation. Keep in mind that nested virt isn't on by default as it isn't available everywhere and has been flaky in the past. This makes these types of installations very small which may make reliable VRRP testing difficult. Thinking out loud here, I'm not sure how much benefit there is to testing VRRP failures in this manner. Do we think that OpenStack sitting on top of libvirt and OVS will somehow produce different results with VRRP than using those tools as is? To put this another way: are we testing OpenStack or are we testing OVS and libvirt? One option here may be to construct this as a Neutron functional test and run VRRP on Neutron networks without virtualization mixed in. > > Thanks in advance > > -- > ROMAN SAFRONOV From kchamart at redhat.com Mon Mar 23 15:33:28 2020 From: kchamart at redhat.com (Kashyap Chamarthy) Date: Mon, 23 Mar 2020 16:33:28 +0100 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> References: <1584638630.12170.41@est.tech> <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> Message-ID: <20200323153328.GA10790@paraplu> On Thu, Mar 19, 2020 at 02:06:51PM -0500, Monty Taylor wrote: > > > > On Mar 19, 2020, at 12:23 PM, Balázs Gibizer wrote: > > > > Hi, > > > > Nova has an unwritten rule that requires to have at least two > > companies involved in any new feature development (or even bugfix?). > > In the current Nova core diversity situation this rule puts extra > > burden to the remaining non Red Hat cores and I guess it also makes > > any Red Hat driven feature development harder. In parallel to > > working on increasing the size of the core team I suggest to > > reconsider dropping this rule. > > > > Some discussion happened already on the today's meeting[1] > > I think that’s a great idea. FWIW I’ve never liked that rule, because > it assume that developers from a company are putting employer > requirements over project requirements when acting in their capacity > as a core reviewer - which is contrary to our general expectation of > how a core reviewer behaves. Yes, I'm with Monty here — on both it not being a stellar idea from the start, and removing the rule now, being forced by the current reality. We've talked about this ad nauseum before; so I'll just to link to my reasoning in this thread, after Denver-II PTG): http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006080.html – [nova][all][ptg] Summary: Same-Company Approvals > I think it’s a great idea to get rid of this policy - and then if > anyone is behaving in a manner that abuses the trust of the rest of > the core reviewers, such as slamming through a new feature that other > people obviously have misgivings about … that can be dealt with the > same way any other breach of trust would happen. Yep, exactly. To quote myself from the above e-mail thread: [...] I'm of course all for diverse set of opinions and reviews from different companies as much as posisble, which I consider super healthy. So long as there are no overly iron-clad "rules" that are "unbendable". What _should_ raise a red flag is a _pattern_. E.g. Developer-A from the company Pied Piper uploads a complex change, within a couple of days (or worse, even shorter), two more upstream "core" reivewers from Pied Piper, who are in the know about the change, pile on it and approve without giving sufficient time for other community reviewers to catch-up. (Because: "hey, we need to get Pied Piper's 'priority feature' into the current release, to get that one-up against the competitor!") *That* kind of behaviour should be called out and rebuked. [...] -- /kashyap From stephenfin at redhat.com Mon Mar 23 15:47:39 2020 From: stephenfin at redhat.com (Stephen Finucane) Date: Mon, 23 Mar 2020 15:47:39 +0000 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: On Mon, 2020-03-23 at 15:28 +0100, Balázs Gibizer wrote: > Hi, > > I'm proposing Lee to the core team. He is around for a long time and he > became really active recently not only on the stable branches. I think > he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) +1 from me. > Cheers, > gibi From mdulko at redhat.com Mon Mar 23 15:48:28 2020 From: mdulko at redhat.com (mdulko at redhat.com) Date: Mon, 23 Mar 2020 16:48:28 +0100 Subject: [k8s][zun] Introduce a new feature in Ussuri - CRI integration In-Reply-To: References: Message-ID: On Sun, 2020-03-22 at 13:28 -0400, Hongbin Lu wrote: > Hi all, > > As we are approaching the end of Ussuri cycle, I would like to take a > chance to introduce a new feature the Zun team implemented in this > cycle - CRI integration [1]. > > As known by people, Zun is an OpenStack Container service. It > provides API for users to create and manage application containers in > an OpenStack cloud. The main concepts in Zun are "Container" and > "Capsule". A container is a single container, while a capsule is a > group of co-located and co-scheduled containers (basically the same > as k8s pod). > > Probably, the "Container" concept is more widely used. People can use > the /containers API endpoint to create and manage a single container. > Under the hook, a container is a Docker container in a compute node. > What is special is that each Docker container is given a Neutron port > so the container is connected to a tenant network in Neutron. Kuryr- > libnetwork is the Docker network plugin we use to perform the Neutron > port binding which basically connects the container to the virtual > switch managed by Neutron. > > As mentioned before, the concept of "Capsule" in Zun is basically the > same as pod in k8s. We introduced this concept mainly for k8s > integration. Roughly speaking, the Zun-k8s integration is achieved by > (i) registering a special node in k8s, (ii) watching the k8s API for > pods being scheduled to this node and (iii) invoking Zun's /capsules > API endpoint to create a capsule for each incoming pod. The (i) and > (ii) is done by a CNCF sandbox project called Virtual Kubelet [2]. > The (iii) is achieved by providing an OpenStack provider [3] for > Virtual Kubelet. The special node registered by Virtual Kubelet is > called a virtual node because the node doesn't physically exist. Pods > being scheduled to the virtual node is basically offloaded from the > current k8s cluster, eventually landed on an external platform such > as an OpenStack cloud. > > In high level, what is offered to end-users is a "serverless > kubernetes pod" [4]. This term basically means the ability to run > pods on demand without planing the capacity (i.e. nodes) upfront. An > example of that is AWS EKS on Fargate [5]. In comparison, the > traditional approach is to create an entire k8s cluster upfront in > order to run the workload. Let's give a simple example. Suppose you > want to run a pod, the traditional approach is to provision a k8s > cluster with a worker node. Then, run the pod on the worker node. In > contract, the "serverless" approach is to create a k8s cluster > without any worker node and the pod is offloaded to a cloud provider > that provisions the pods at runtime. This approach works well for > applications who have fluctuated workloads so it is hard to provision > a cluster with the right size for them. Furthermore, from cloud > provider's perspective, if all tenant users offloads their pods to > the cloud, the cloud provider might be able to pack the workload > better (i.e. with fewer physical nodes) thus saving cost. > > Under the hook, a capsule is a podsandbox with one or more containers > in a CRI runtime (i.e. containerd). Compared to Docker, a CRI runtime > has a better support for the pod concept so we chose it to implement > capsule. A caveat is that CRI requires a CNI plugin for the > networking, so we need to implement a CNI plugin for Zun (called zun- > cni). The role of CNI plugin is similar as kuryr-libnetwork that we > are using for Docker except it implements a different networking > model (CNI). I summaries it as below: Hi, I noticed that Zun's CNI plugin [1] is basically a simplified version of kuryr-kubernetes code. While it's totally fine you've copied that, I wonder what modifications had been made to make it suit Zun? Is there a chance to converge this to make Zun use kuryr-kubernetes directly so that we won't develop two versions of that code in parallel? Thanks, Michał [1] https://github.com/openstack/zun/tree/master/zun/cni > +--------------+------------------------+---------------+ > | Concept | Container | Capsule (Pod) | > +--------------+------------------------+---------------+ > | API endpoint | /containers | /capsules | > | Engine | Docker | CRI runtime | > | Network | kuryr-libnetwork (CNM) | zun-cni (CNI) | > +--------------+------------------------+---------------+ > > Typically, a CRI runtime works well with Kata Container which > provides hypervisor-based isolation for neighboring containers in the > same node. As a result, it is secure to consolidate pods from > different tenants into a single node which increases the resource > utilization. For deployment, a typical stack looks like below: > > +----------------------------------------------+ > | k8s control plane | > +----------------------------------------------+ > | Virtual Kubelet (OpenStack provider) | > +----------------------------------------------+ > | OpenStack control plane (Zun, Neutron, etc.) | > +----------------------------------------------+ > | OpenStack data plane | > | (Zun compute agent, Neutron OVS agent, etc.) | > +----------------------------------------------+ > | Containerd (with CRI plugin) | > +----------------------------------------------+ > | Kata Container | > +----------------------------------------------+ > > In this stack, if a user creates a deployment or pod in k8s, the k8s > scheduler will schedule the pod to the virtual node registered by > Virtual Kubelet. Virtual Kubelet will pick up the pod and let the > configured cloud provider to handle it. The cloud provider invokes > Zun API to create a capsule. Upon receiving the API request to create > a capsule, Zun scheduler will schedule the capsule to a compute node. > The Zun compute agent in that node will provision the capsule using a > CRI runtime (containerd in this example). The Zun-CRI runtime > communication is done via a gRPC protocol through a unix socket. The > CRI runtime will first create the pod in Kata Container (or runc as > an alternative) that realizes the pod using a lightweight VM. > Furthermore, the CRI runtime will use a CNI plugin, which is the zun- > cni binary, to setup the network. The zun-cni binary is a thin > executable that dispatches the CNI command to a daemon service called > zun-cni-daemon. The community is via HTTP within localhost. The zun- > cni-daemon will look up the Neutron port information from DB and > perform the port binding. > > In conclusion, starting from Ussuri, Zun adds support for CRI- > compatible runtime. Zun uses CRI runtime to realize the concept of > pod. Using this feature together with Virtual Kubelet and Kata > Container, we can offer "serverless kubernetes pod" service which is > comparable with AWS EKS with Fargate. > > [1] https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime > [2] https://github.com/virtual-kubelet/virtual-kubelet > [3] https://github.com/virtual-kubelet/openstack-zun > [4] https://aws.amazon.com/about-aws/whats-new/2019/12/run-serverless-kubernetes-pods-using-amazon-eks-and-aws-fargate/ > [5] https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ From hongbin034 at gmail.com Mon Mar 23 16:17:06 2020 From: hongbin034 at gmail.com (Hongbin Lu) Date: Mon, 23 Mar 2020 12:17:06 -0400 Subject: [k8s][zun] Introduce a new feature in Ussuri - CRI integration In-Reply-To: References: Message-ID: On Mon, Mar 23, 2020 at 11:48 AM wrote: > On Sun, 2020-03-22 at 13:28 -0400, Hongbin Lu wrote: > > Hi all, > > > > As we are approaching the end of Ussuri cycle, I would like to take a > > chance to introduce a new feature the Zun team implemented in this > > cycle - CRI integration [1]. > > > > As known by people, Zun is an OpenStack Container service. It > > provides API for users to create and manage application containers in > > an OpenStack cloud. The main concepts in Zun are "Container" and > > "Capsule". A container is a single container, while a capsule is a > > group of co-located and co-scheduled containers (basically the same > > as k8s pod). > > > > Probably, the "Container" concept is more widely used. People can use > > the /containers API endpoint to create and manage a single container. > > Under the hook, a container is a Docker container in a compute node. > > What is special is that each Docker container is given a Neutron port > > so the container is connected to a tenant network in Neutron. Kuryr- > > libnetwork is the Docker network plugin we use to perform the Neutron > > port binding which basically connects the container to the virtual > > switch managed by Neutron. > > > > As mentioned before, the concept of "Capsule" in Zun is basically the > > same as pod in k8s. We introduced this concept mainly for k8s > > integration. Roughly speaking, the Zun-k8s integration is achieved by > > (i) registering a special node in k8s, (ii) watching the k8s API for > > pods being scheduled to this node and (iii) invoking Zun's /capsules > > API endpoint to create a capsule for each incoming pod. The (i) and > > (ii) is done by a CNCF sandbox project called Virtual Kubelet [2]. > > The (iii) is achieved by providing an OpenStack provider [3] for > > Virtual Kubelet. The special node registered by Virtual Kubelet is > > called a virtual node because the node doesn't physically exist. Pods > > being scheduled to the virtual node is basically offloaded from the > > current k8s cluster, eventually landed on an external platform such > > as an OpenStack cloud. > > > > In high level, what is offered to end-users is a "serverless > > kubernetes pod" [4]. This term basically means the ability to run > > pods on demand without planing the capacity (i.e. nodes) upfront. An > > example of that is AWS EKS on Fargate [5]. In comparison, the > > traditional approach is to create an entire k8s cluster upfront in > > order to run the workload. Let's give a simple example. Suppose you > > want to run a pod, the traditional approach is to provision a k8s > > cluster with a worker node. Then, run the pod on the worker node. In > > contract, the "serverless" approach is to create a k8s cluster > > without any worker node and the pod is offloaded to a cloud provider > > that provisions the pods at runtime. This approach works well for > > applications who have fluctuated workloads so it is hard to provision > > a cluster with the right size for them. Furthermore, from cloud > > provider's perspective, if all tenant users offloads their pods to > > the cloud, the cloud provider might be able to pack the workload > > better (i.e. with fewer physical nodes) thus saving cost. > > > > Under the hook, a capsule is a podsandbox with one or more containers > > in a CRI runtime (i.e. containerd). Compared to Docker, a CRI runtime > > has a better support for the pod concept so we chose it to implement > > capsule. A caveat is that CRI requires a CNI plugin for the > > networking, so we need to implement a CNI plugin for Zun (called zun- > > cni). The role of CNI plugin is similar as kuryr-libnetwork that we > > are using for Docker except it implements a different networking > > model (CNI). I summaries it as below: > > Hi, > > I noticed that Zun's CNI plugin [1] is basically a simplified version > of kuryr-kubernetes code. While it's totally fine you've copied that, I > wonder what modifications had been made to make it suit Zun? Is there a > chance to converge this to make Zun use kuryr-kubernetes directly so > that we won't develop two versions of that code in parallel? > Right. I also investigated the possibilities of reusing the kuryr-kubernetes codebase as well. Definitely, some codes are common among two projects. If we can move the common code to a library (i.e. kuryr-lib), Zun should be able to directly consume the code. In particular, I am interesting to directly consume the CNI binding code (kuryr_kubernetes/cni/binding/) and the VIF versioned object (kuryr_kubernetes/objects). Most parts of kuryr-kubernetes code is coupling with the "list-and-watch" logic against k8s API. Zun is not able to reuse those part of code. However, I do advocate to move all the common code to kuryr-lib so Zun can reuse it whenever it is appropriate. > > Thanks, > Michał > > [1] https://github.com/openstack/zun/tree/master/zun/cni > > > +--------------+------------------------+---------------+ > > | Concept | Container | Capsule (Pod) | > > +--------------+------------------------+---------------+ > > | API endpoint | /containers | /capsules | > > | Engine | Docker | CRI runtime | > > | Network | kuryr-libnetwork (CNM) | zun-cni (CNI) | > > +--------------+------------------------+---------------+ > > > > Typically, a CRI runtime works well with Kata Container which > > provides hypervisor-based isolation for neighboring containers in the > > same node. As a result, it is secure to consolidate pods from > > different tenants into a single node which increases the resource > > utilization. For deployment, a typical stack looks like below: > > > > +----------------------------------------------+ > > | k8s control plane | > > +----------------------------------------------+ > > | Virtual Kubelet (OpenStack provider) | > > +----------------------------------------------+ > > | OpenStack control plane (Zun, Neutron, etc.) | > > +----------------------------------------------+ > > | OpenStack data plane | > > | (Zun compute agent, Neutron OVS agent, etc.) | > > +----------------------------------------------+ > > | Containerd (with CRI plugin) | > > +----------------------------------------------+ > > | Kata Container | > > +----------------------------------------------+ > > > > In this stack, if a user creates a deployment or pod in k8s, the k8s > > scheduler will schedule the pod to the virtual node registered by > > Virtual Kubelet. Virtual Kubelet will pick up the pod and let the > > configured cloud provider to handle it. The cloud provider invokes > > Zun API to create a capsule. Upon receiving the API request to create > > a capsule, Zun scheduler will schedule the capsule to a compute node. > > The Zun compute agent in that node will provision the capsule using a > > CRI runtime (containerd in this example). The Zun-CRI runtime > > communication is done via a gRPC protocol through a unix socket. The > > CRI runtime will first create the pod in Kata Container (or runc as > > an alternative) that realizes the pod using a lightweight VM. > > Furthermore, the CRI runtime will use a CNI plugin, which is the zun- > > cni binary, to setup the network. The zun-cni binary is a thin > > executable that dispatches the CNI command to a daemon service called > > zun-cni-daemon. The community is via HTTP within localhost. The zun- > > cni-daemon will look up the Neutron port information from DB and > > perform the port binding. > > > > In conclusion, starting from Ussuri, Zun adds support for CRI- > > compatible runtime. Zun uses CRI runtime to realize the concept of > > pod. Using this feature together with Virtual Kubelet and Kata > > Container, we can offer "serverless kubernetes pod" service which is > > comparable with AWS EKS with Fargate. > > > > [1] https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime > > [2] https://github.com/virtual-kubelet/virtual-kubelet > > [3] https://github.com/virtual-kubelet/openstack-zun > > [4] > https://aws.amazon.com/about-aws/whats-new/2019/12/run-serverless-kubernetes-pods-using-amazon-eks-and-aws-fargate/ > > [5] > https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Mon Mar 23 16:32:37 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 23 Mar 2020 09:32:37 -0700 Subject: [baremetal-sig][ironic] Lets finish the whitepaper doodle Message-ID: Greetings fellow humans! During today's ironic meeting, the idea of just getting on a video call and hammering out the remainder of the baremetal SIG whitepaper was raised. After some quick discussion in the community, I'm sending out a doodle[0] for us to find a few time windows that works, where we can try to gather together and hammer out words, cut/paste/massage content, and overall just try and get it done. In the next 24-48 hours, I'll contact the respondents to and schedule a call, where I'll also send out call details. Please respond to the doodle quickly. -Julia [0] https://doodle.com/poll/k4wnmuay34mvh94v From balazs.gibizer at est.tech Mon Mar 23 16:34:15 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 23 Mar 2020 17:34:15 +0100 Subject: [nova] Proposing Ghanshyam Mann as nova core Message-ID: Hi, I'm proposing Ghanshyam Mann to the core team. He is a long time OpenStack and nova contributor. He is one of our API and QA expert. I think he would be a great addition to the team. Cores, please vote (-1, 0, +1) before next Monday (03.30.) Cheers, gibi From openstack at fried.cc Mon Mar 23 16:37:52 2020 From: openstack at fried.cc (Eric Fried) Date: Mon, 23 Mar 2020 11:37:52 -0500 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: References: Message-ID: <485f7915-a9d3-7986-adac-650abf1cb558@fried.cc> Alumnus +1 On 3/23/20 11:34 AM, Balázs Gibizer wrote: > Hi, > > I'm proposing Ghanshyam Mann to the core team. He is a long time > OpenStack and nova contributor. He is one of our API and QA expert. I > think he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > Cheers, > gibi > > > > From juliaashleykreger at gmail.com Mon Mar 23 17:06:38 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 23 Mar 2020 10:06:38 -0700 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! Message-ID: Greetings Humans, Vulcans, AIs, and any other form of life! Today in the Ironic team meeting, we decided that we needed to have an un-conference! In the theme of an un-conference, I think we will start with two simple questions: * How is everyone? * What ironic (or non-ironic) things shall we discuss? >From there, we will identify topics of discussion, and possibly jump into separate video calls for ~15 minutes each, depending on the number of topics and general insanity. During the last ?15? minutes or so, we shall reconvene into the same video call to enjoy a little time sipping coffee Coffee, Tea or other delicious beverages while we determine who is wearing the silliest hat! So if you have a steampunk hat with gears or even some kind of hat you would wear to a fancy horse race, go right ahead! Only have a rose from your growing quarantine garden! You can wear that too! So what do you need to do! 1) Fill out the doodle[0] so we can actually schedule a time that works for a group of people! 2) Brainstorm some ideas of things you might want to talk about/present or LEARN! 3) Dust off your silly hats! NB: I know the windows on this doodle overlap with Baremetal SIG doodle that I sent out a little while ago. We will figure it out. -Julia [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 From james.denton at rackspace.com Mon Mar 23 17:11:42 2020 From: james.denton at rackspace.com (James Denton) Date: Mon, 23 Mar 2020 17:11:42 +0000 Subject: [neutron] OVS inactivity probes Message-ID: Hello all, Like others, we have seen an increase in the amount of messages like those below, followed up by disconnects: 2020-03-19T07:19:47.414Z|01614|rconn|ERR|br-int<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting 2020-03-19T07:19:47.414Z|01615|rconn|ERR|br-ex<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting 2020-03-19T07:19:47.414Z|01616|rconn|ERR|br-tun<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting We have since increased the value of of_inactivity_probe (https://bugs.launchpad.net/neutron/+bug/1817022) and confirmed the Controller connection reflects this new value: --- # ovs-vsctl list Controller ... _uuid : b4814677-c6f3-4afc-9c9e-999d5a5ac78f connection_mode : out-of-band controller_burst_limit: [] controller_rate_limit: [] enable_async_messages: [] external_ids : {} inactivity_probe : 60000 is_connected : true local_gateway : [] local_ip : [] local_netmask : [] max_backoff : [] other_config : {} role : other status : {last_error="Connection refused", sec_since_connect="1420", sec_since_disconnect="1423", state=ACTIVE} target : "tcp:127.0.0.1:6633" --- However, we also see disconnects on the manager side, which the config option does not address: 2020-03-23T11:01:02.871Z|00443|reconnect|ERR|tcp:127.0.0.1:50098: no response to inactivity probe after 5 seconds, disconnecting This bug (https://bugs.launchpad.net/neutron/+bug/1627106) and related commit (https://opendev.org/openstack/neutron/commit/1698bee770b84a2663ba940a6ded5d4b9733101a) appear to leverage the ovs_vsctl_timeout value (since renamed to ovsdb_timeout), but the inactivity_probe for the Manager connection does not appear to be implemented. Honestly, I'm not sure if that code path is used. --- # ovs-vsctl list Manager _uuid : d61519ba-93fc-4fe5-b05c-b630778a44b0 connection_mode : [] external_ids : {} inactivity_probe : [] is_connected : true max_backoff : [] other_config : {} status : {bound_port="6640", n_connections="2", sec_since_connect="0", sec_since_disconnect="0"} target : "ptcp:6640:127.0.0.1" --- Running this by hand sets the inactivity_probe timeout on manager connection, but we'd prefer to use a built-in method, if possible: # ovs-vsctl set manager d61519ba-93fc-4fe5-b05c-b630778a44b0 inactivity_probe=30000 Any suggestions? Thanks, James From amy at demarco.com Mon Mar 23 17:12:39 2020 From: amy at demarco.com (Amy Marrich) Date: Mon, 23 Mar 2020 12:12:39 -0500 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: References: Message-ID: Great idea Julia! Maybe other projects should try something like this/ Amy (spotz) On Mon, Mar 23, 2020 at 12:09 PM Julia Kreger wrote: > Greetings Humans, Vulcans, AIs, and any other form of life! > > Today in the Ironic team meeting, we decided that we needed to have an > un-conference! > > In the theme of an un-conference, I think we will start with two > simple questions: > * How is everyone? > * What ironic (or non-ironic) things shall we discuss? > > From there, we will identify topics of discussion, and possibly jump > into separate video calls for ~15 minutes each, depending on the > number of topics and general insanity. > > During the last ?15? minutes or so, we shall reconvene into the same > video call to enjoy a little time sipping coffee Coffee, Tea or other > delicious beverages while we determine who is wearing the silliest > hat! So if you have a steampunk hat with gears or even some kind of > hat you would wear to a fancy horse race, go right ahead! Only have a > rose from your growing quarantine garden! You can wear that too! > > So what do you need to do! > 1) Fill out the doodle[0] so we can actually schedule a time that > works for a group of people! > 2) Brainstorm some ideas of things you might want to talk > about/present or LEARN! > 3) Dust off your silly hats! > > NB: I know the windows on this doodle overlap with Baremetal SIG > doodle that I sent out a little while ago. We will figure it out. > > -Julia > > [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Arkady.Kanevsky at dell.com Mon Mar 23 17:45:32 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Mon, 23 Mar 2020 17:45:32 +0000 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: References: Message-ID: <887037c725f7479b9c5c1c0a2d407279@AUSX13MPS308.AMER.DELL.COM> +2 From: Amy Marrich Sent: Monday, March 23, 2020 12:13 PM To: Julia Kreger Cc: openstack-discuss Subject: Re: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! [EXTERNAL EMAIL] Great idea Julia! Maybe other projects should try something like this/ Amy (spotz) On Mon, Mar 23, 2020 at 12:09 PM Julia Kreger > wrote: Greetings Humans, Vulcans, AIs, and any other form of life! Today in the Ironic team meeting, we decided that we needed to have an un-conference! In the theme of an un-conference, I think we will start with two simple questions: * How is everyone? * What ironic (or non-ironic) things shall we discuss? From there, we will identify topics of discussion, and possibly jump into separate video calls for ~15 minutes each, depending on the number of topics and general insanity. During the last ?15? minutes or so, we shall reconvene into the same video call to enjoy a little time sipping coffee Coffee, Tea or other delicious beverages while we determine who is wearing the silliest hat! So if you have a steampunk hat with gears or even some kind of hat you would wear to a fancy horse race, go right ahead! Only have a rose from your growing quarantine garden! You can wear that too! So what do you need to do! 1) Fill out the doodle[0] so we can actually schedule a time that works for a group of people! 2) Brainstorm some ideas of things you might want to talk about/present or LEARN! 3) Dust off your silly hats! NB: I know the windows on this doodle overlap with Baremetal SIG doodle that I sent out a little while ago. We will figure it out. -Julia [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsafrono at redhat.com Mon Mar 23 17:51:04 2020 From: rsafrono at redhat.com (Roman Safronov) Date: Mon, 23 Mar 2020 19:51:04 +0200 Subject: [tempest] what is a proper way to install a package into a vm instance spawned by tempest? In-Reply-To: References: Message-ID: Actually we need such test because we are testing Openstack as a whole product (including Neutron+openvswitch+OVN+Nova+libvirt+Octavia etc.) that's why I think neutron functional test would be not enough. We are creating tests covering scenarios that our customers tried to use and encountered issues. For example this is a bug reported downstream on an issue happened in this scenario: https://bugzilla.redhat.com/show_bug.cgi?id=1707241 There were more reported issues on similar use case and we would like to catch such issues before the product is released. Anyway, as I said this specific test runs stable in downstream CI on virtual multi node environments with nested virtualization. It usually runs with RHEL8 image but I also tried it with standard Ubuntu-18.04 guest image used by upstream CI gates. The only issue is that keepalive package installation by 'apt install' for some reason does not work when running on upstream gates causing the test to be skipped. I just would like to understand if running 'apt install/yum install' inside VMs spawned by upstream tempest tests is not acceptable at all or I am missing something (maybe proxy settings?). On Mon, Mar 23, 2020 at 5:36 PM Clark Boylan wrote: > On Sun, Mar 22, 2020, at 9:10 AM, Roman Safronov wrote: > > Hi, > > > > I wrote a tempest test > > < > https://review.opendev.org/#/c/710975/https://review.opendev.org/#/c/710975/)> > which requires keepalived to be running inside vm instance. > > The test tries to install keepalived package inside vm by running "apt > > install/yum install". > > However as I see in upstream gates logs this does not work while the > > same code works perfectly in our downstream CI using the same image. > > > > Do vm instances spawned on upstream gates during tests have a route to > > the internet? > > Is there an alternative way that I can use in order to install a > > package? > > By default the tempest jobs use a cirros image. This is a very small, > quick to boot linux without a package manager. If you need services to be > running in the image typically you'll need to boot a more typical linux > installation. Keep in mind that nested virt isn't on by default as it isn't > available everywhere and has been flaky in the past. This makes these types > of installations very small which may make reliable VRRP testing difficult. > > Thinking out loud here, I'm not sure how much benefit there is to testing > VRRP failures in this manner. Do we think that OpenStack sitting on top of > libvirt and OVS will somehow produce different results with VRRP than using > those tools as is? To put this another way: are we testing OpenStack or are > we testing OVS and libvirt? > > One option here may be to construct this as a Neutron functional test and > run VRRP on Neutron networks without virtualization mixed in. > > > > > Thanks in advance > > > > -- > > ROMAN SAFRONOV > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Mon Mar 23 18:03:13 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 23 Mar 2020 11:03:13 -0700 Subject: =?UTF-8?Q?Re:_[tempest]_what_is_a_proper_way_to_install_a_package_into_a?= =?UTF-8?Q?_vm_instance_spawned_by_tempest=3F?= In-Reply-To: References: Message-ID: On Mon, Mar 23, 2020, at 10:51 AM, Roman Safronov wrote: > Actually we need such test because we are testing Openstack as a whole > product (including Neutron+openvswitch+OVN+Nova+libvirt+Octavia etc.) > that's why I think neutron functional test would be not enough. We are > creating tests covering scenarios that our customers tried to use and > encountered issues. > For example this is a bug reported downstream on an issue happened in > this scenario: https://bugzilla.redhat.com/show_bug.cgi?id=1707241 > There were more reported issues on similar use case and we would like > to catch such issues before the product is released. Absolutely do what you need to test these use cases. I'm simply suggesting that the most reliable and effective way to do that may be to avoid virtualization because we don't have nested virt. > > Anyway, as I said this specific test runs stable in downstream CI on > virtual multi node environments with nested virtualization. It usually > runs with RHEL8 image but I also tried it with standard Ubuntu-18.04 > guest image used by upstream CI gates. The only issue is that keepalive > package installation by 'apt install' for some reason does not work > when running on upstream gates causing the test to be skipped. I just > would like to understand if running 'apt install/yum install' inside > VMs spawned by upstream tempest tests is not acceptable at all or I am > missing something (maybe proxy settings?). Our test nodes should have full access to the Internet. We typically advise against relying on Internet resources as they can be flaky. For this reason we do run package mirrors within each cloud region that hosts our test nodes. My hunch is that the network topology set up by devstack and neutron has VMs on rfc 1918 subnets for floating IPs. These floating IPs are then not further NAT'd by the host's IP address so any outbound traffic originating from those IPs will have no route back to home. > > On Mon, Mar 23, 2020 at 5:36 PM Clark Boylan wrote: > > On Sun, Mar 22, 2020, at 9:10 AM, Roman Safronov wrote: > > > Hi, > > > > > > I wrote a tempest test > > > )> which requires keepalived to be running inside vm instance. > > > The test tries to install keepalived package inside vm by running "apt > > > install/yum install". > > > However as I see in upstream gates logs this does not work while the > > > same code works perfectly in our downstream CI using the same image. > > > > > > Do vm instances spawned on upstream gates during tests have a route to > > > the internet? > > > Is there an alternative way that I can use in order to install a > > > package? > > > > By default the tempest jobs use a cirros image. This is a very small, quick to boot linux without a package manager. If you need services to be running in the image typically you'll need to boot a more typical linux installation. Keep in mind that nested virt isn't on by default as it isn't available everywhere and has been flaky in the past. This makes these types of installations very small which may make reliable VRRP testing difficult. > > > > Thinking out loud here, I'm not sure how much benefit there is to testing VRRP failures in this manner. Do we think that OpenStack sitting on top of libvirt and OVS will somehow produce different results with VRRP than using those tools as is? To put this another way: are we testing OpenStack or are we testing OVS and libvirt? > > > > One option here may be to construct this as a Neutron functional test and run VRRP on Neutron networks without virtualization mixed in. > > > > > > > > Thanks in advance > > > > > > -- > > > ROMAN SAFRONOV > > From tim.bell at cern.ch Mon Mar 23 18:29:29 2020 From: tim.bell at cern.ch (Tim Bell) Date: Mon, 23 Mar 2020 19:29:29 +0100 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: References: Message-ID: <0BDD1F91-A4C2-4F76-A194-65E060E05114@cern.ch> Can anyone join ? Time to get out the Buffy T-Shirt https://www.lookhuman.com/design/26889-there-s-nothing-we-can-t-face-except-for-bunnies/6010-heathered_gray_nl-md Tim > On 23 Mar 2020, at 18:06, Julia Kreger wrote: > > Greetings Humans, Vulcans, AIs, and any other form of life! > > Today in the Ironic team meeting, we decided that we needed to have an > un-conference! > > In the theme of an un-conference, I think we will start with two > simple questions: > * How is everyone? > * What ironic (or non-ironic) things shall we discuss? > > From there, we will identify topics of discussion, and possibly jump > into separate video calls for ~15 minutes each, depending on the > number of topics and general insanity. > > During the last ?15? minutes or so, we shall reconvene into the same > video call to enjoy a little time sipping coffee Coffee, Tea or other > delicious beverages while we determine who is wearing the silliest > hat! So if you have a steampunk hat with gears or even some kind of > hat you would wear to a fancy horse race, go right ahead! Only have a > rose from your growing quarantine garden! You can wear that too! > > So what do you need to do! > 1) Fill out the doodle[0] so we can actually schedule a time that > works for a group of people! > 2) Brainstorm some ideas of things you might want to talk > about/present or LEARN! > 3) Dust off your silly hats! > > NB: I know the windows on this doodle overlap with Baremetal SIG > doodle that I sent out a little while ago. We will figure it out. > > -Julia > > [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Mon Mar 23 18:34:46 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 23 Mar 2020 13:34:46 -0500 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <170829c0eb9.118671b1a178111.1346517061236524495@ghanshyammann.com> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <170829c0eb9.118671b1a178111.1346517061236524495@ghanshyammann.com> Message-ID: <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> Hello Everyone. Please find the latest updates on V cycle goals: Etherpad: https://etherpad.openstack.org/p/YVR-v-series-goals We have two more candidates for V cycle goals: 1. Migrate from oslo.rootwrap to oslo.privsep Thanks to Rodolfo who agreed to drive this work. He started the ML thread and initial discussions in happening there. ML: http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013409.html 2. Finish moving legacy python-*client CLIs to python-openstackclient This had been tried as a goal in the Train cycle also and one of the important requirements from users. One option we are thinking as multi-cycle goal. Artem agreed to drive this and will give another try by propose as mutli-cycle goal. Selected goals till now or V cycle: 1. Switch legacy Zuul jobs to native - https://governance.openstack.org/tc/goals/selected/victoria/index.html - Champion: tosky -gmann & njohnston ---- On Wed, 26 Feb 2020 11:47:10 -0600 Ghanshyam Mann wrote ---- > ---- On Wed, 05 Feb 2020 12:39:17 -0600 Ghanshyam Mann wrote ---- > > Hello everyone, > > > > We are in R14 week of Ussuri cycle which means It's time to start the > > discussions about community-wide goals ideas for the V series. > > > > Community-wide goals are important in term of solving and improving a technical > > area across OpenStack as a whole. It has lot more benefits to be considered from > > users as well from a developers perspective. See [1] for more details about > > community-wide goals and process. > > > > We have the Zuulv3 migration goal already accepted and pre-selected for v cycle. > > If you are interested in proposing a goal, please write down the idea on this etherpad[2] > > - https://etherpad.openstack.org/p/YVR-v-series-goals > > > > Accordingly, we will start the separate ML discussion over each goal idea. > > > > Also, you can refer to the backlogs of community-wide goals from this[3] and ussuri > > cycle goals[4]. > > Updates: > We have zuulv3 migration goal selected for V cycle[1] and waiting for more goals > proposal. > > For V cycle, we need one more goal to select, please do not wait or hesitate to > propose something you think we should solve in OpenStack as a whole community. > You can write your idea on this etherpad > - https://etherpad.openstack.org/p/YVR-v-series-goals > > NOTE: As per the new process[2], we can have as many goals proposed and > we pick two goals per cycle. Do not limit yourself to propose your goal ideas > which can be selected for V or future cyle. > > > [1] https://governance.openstack.org/tc/goals/selected/victoria/index.html > [2] https://governance.openstack.org/tc/goals/#process-details > > -gmann & njohnston > > > > > NOTE: TC has defined the goal process schedule[5] to streamlined the process and > > ready with goals for projects to plan/implement at the start of the cycle. I am > > hoping to start that schedule for W cycle goals. > > > > [1] https://governance.openstack.org/tc/goals/index.html > > [2] https://etherpad.openstack.org/p/YVR-v-series-goals > > [3] https://etherpad.openstack.org/p/community-goals > > [4] https://etherpad.openstack.org/p/PVG-u-series-goals > > [5] https://governance.openstack.org/tc/goals/#goal-selection-schedule > > > > -gmann > > > > From juliaashleykreger at gmail.com Mon Mar 23 18:40:21 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 23 Mar 2020 11:40:21 -0700 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: <0BDD1F91-A4C2-4F76-A194-65E060E05114@cern.ch> References: <0BDD1F91-A4C2-4F76-A194-65E060E05114@cern.ch> Message-ID: I don't see why not! and ++ on the T-shirt! On Mon, Mar 23, 2020 at 11:29 AM Tim Bell wrote: > > Can anyone join ? > > Time to get out the Buffy T-Shirt > > https://www.lookhuman.com/design/26889-there-s-nothing-we-can-t-face-except-for-bunnies/6010-heathered_gray_nl-md > > Tim > > On 23 Mar 2020, at 18:06, Julia Kreger wrote: > > Greetings Humans, Vulcans, AIs, and any other form of life! > > Today in the Ironic team meeting, we decided that we needed to have an > un-conference! > > In the theme of an un-conference, I think we will start with two > simple questions: > * How is everyone? > * What ironic (or non-ironic) things shall we discuss? > > From there, we will identify topics of discussion, and possibly jump > into separate video calls for ~15 minutes each, depending on the > number of topics and general insanity. > > During the last ?15? minutes or so, we shall reconvene into the same > video call to enjoy a little time sipping coffee Coffee, Tea or other > delicious beverages while we determine who is wearing the silliest > hat! So if you have a steampunk hat with gears or even some kind of > hat you would wear to a fancy horse race, go right ahead! Only have a > rose from your growing quarantine garden! You can wear that too! > > So what do you need to do! > 1) Fill out the doodle[0] so we can actually schedule a time that > works for a group of people! > 2) Brainstorm some ideas of things you might want to talk > about/present or LEARN! > 3) Dust off your silly hats! > > NB: I know the windows on this doodle overlap with Baremetal SIG > doodle that I sent out a little while ago. We will figure it out. > > -Julia > > [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 > > From nate.johnston at redhat.com Mon Mar 23 18:44:24 2020 From: nate.johnston at redhat.com (Nate Johnston) Date: Mon, 23 Mar 2020 14:44:24 -0400 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: <0BDD1F91-A4C2-4F76-A194-65E060E05114@cern.ch> References: <0BDD1F91-A4C2-4F76-A194-65E060E05114@cern.ch> Message-ID: <20200323184424.ppwotidneccsiv6b@firewall> On Mon, Mar 23, 2020 at 07:29:29PM +0100, Tim Bell wrote: > Time to get out the Buffy T-Shirt > > https://www.lookhuman.com/design/26889-there-s-nothing-we-can-t-face-except-for-bunnies/6010-heathered_gray_nl-md Brilliant. --N. From fungi at yuggoth.org Mon Mar 23 19:11:50 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 23 Mar 2020 19:11:50 +0000 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <170829c0eb9.118671b1a178111.1346517061236524495@ghanshyammann.com> <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> Message-ID: <20200323191149.rgeeqkqgucsub745@yuggoth.org> On 2020-03-23 13:34:46 -0500 (-0500), Ghanshyam Mann wrote: [...] > Selected goals till now or V cycle: > 1. Switch legacy Zuul jobs to native > - https://governance.openstack.org/tc/goals/selected/victoria/index.html [...] Please be aware that this goal references the old Zuul v3 Migration Guide which no longer exists except in Git history of the opendev/infra-manual repository (it was removed via https://review.opendev.org/711325 just shy of two years after the Zuul 3.0.0 release). You can still find the old source for it here: -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From ltoscano at redhat.com Mon Mar 23 20:28:41 2020 From: ltoscano at redhat.com (Luigi Toscano) Date: Mon, 23 Mar 2020 21:28:41 +0100 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <20200323191149.rgeeqkqgucsub745@yuggoth.org> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> <20200323191149.rgeeqkqgucsub745@yuggoth.org> Message-ID: <1935919.eo7qFNXsyg@whitebase.usersys.redhat.com> On Monday, 23 March 2020 20:11:50 CET Jeremy Stanley wrote: > On 2020-03-23 13:34:46 -0500 (-0500), Ghanshyam Mann wrote: > [...] > > > Selected goals till now or V cycle: > > 1. Switch legacy Zuul jobs to native > > - https://governance.openstack.org/tc/goals/selected/victoria/index.html > > [...] > > Please be aware that this goal references the old Zuul v3 Migration > Guide which no longer exists except in Git history of the > opendev/infra-manual repository (it was removed via > https://review.opendev.org/711325 just shy of two years after the > Zuul 3.0.0 release). You can still find the old source for it here: > > https://opendev.org/opendev/infra-manual/src/commit/2901f4b87b7506c63aefa34 > ece47106ff3c8a328/doc/source/zuulv3.rst > Why was it removed I think it still important. The amount of jobs still to be converted which lead to this goal should say something. Can it be restored or at least added somewhere else? -- Luigi From melwittt at gmail.com Mon Mar 23 20:37:23 2020 From: melwittt at gmail.com (melanie witt) Date: Mon, 23 Mar 2020 13:37:23 -0700 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: <56325188-a5eb-212b-6094-5c8f4e8c8fee@gmail.com> On 3/23/20 07:28, Balázs Gibizer wrote: > Hi, > > I'm proposing Lee to the core team. He is around for a long time and he > became really active recently not only on the stable branches. I think > he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) +1 From melwittt at gmail.com Mon Mar 23 20:38:08 2020 From: melwittt at gmail.com (melanie witt) Date: Mon, 23 Mar 2020 13:38:08 -0700 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: References: Message-ID: <843713a2-115b-fd92-6140-43787ff8c251@gmail.com> On 3/23/20 09:34, Balázs Gibizer wrote: > Hi, > > I'm proposing Ghanshyam Mann to the core team. He is a long time > OpenStack and nova contributor. He is one of our API and QA expert. I > think he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) +1 From fungi at yuggoth.org Mon Mar 23 20:43:44 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 23 Mar 2020 20:43:44 +0000 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <1935919.eo7qFNXsyg@whitebase.usersys.redhat.com> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> <20200323191149.rgeeqkqgucsub745@yuggoth.org> <1935919.eo7qFNXsyg@whitebase.usersys.redhat.com> Message-ID: <20200323204344.knyijc4eelqm4pfd@yuggoth.org> On 2020-03-23 21:28:41 +0100 (+0100), Luigi Toscano wrote: > On Monday, 23 March 2020 20:11:50 CET Jeremy Stanley wrote: > > On 2020-03-23 13:34:46 -0500 (-0500), Ghanshyam Mann wrote: > > [...] > > > > > Selected goals till now or V cycle: > > > 1. Switch legacy Zuul jobs to native > > > - https://governance.openstack.org/tc/goals/selected/victoria/index.html > > > > [...] > > > > Please be aware that this goal references the old Zuul v3 Migration > > Guide which no longer exists except in Git history of the > > opendev/infra-manual repository (it was removed via > > https://review.opendev.org/711325 just shy of two years after the > > Zuul 3.0.0 release). You can still find the old source for it here: > > > > > https://opendev.org/opendev/infra-manual/src/commit/2901f4b87b7506c63aefa34 > > ece47106ff3c8a328/doc/source/zuulv3.rst > > > Why was it removed I think it still important. The amount of jobs still to be > converted which lead to this goal should say something. > > Can it be restored or at least added somewhere else? The OpenDev Manual maintainers didn't want to continue curating instructions for a migration which should have happened two years ago. The content is CC-BY licensed though, so should be easy to copy into the goal document or similar if the OpenStack project wants to retain it somewhere. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From rosmaita.fossdev at gmail.com Mon Mar 23 21:00:52 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 23 Mar 2020 17:00:52 -0400 Subject: [cinder] reminder: weekly meeting location change Message-ID: <439c46fe-e0ee-c920-401b-748430c2e051@gmail.com> (If you missed last week's meeting, this email may be the first you've heard about this change.) Beginning with this Wednesday (25 March), the Cinder team weekly meeting will be held in: #openstack-meeting-alt The time continues to be 1400 UTC on Wednesdays. cheers, brian From alifshit at redhat.com Mon Mar 23 22:00:40 2020 From: alifshit at redhat.com (Artom Lifshitz) Date: Mon, 23 Mar 2020 18:00:40 -0400 Subject: [tempest][gate] tempest-tox-plugin-sanity-check broken by whitebox-tempest-plugin Message-ID: Hey all, A recent change in the whitebox-tempest-plugin [1] has broken the tempest-tox-plugin-sanity-check job (ex: [2]). The cause is a skipCheck that uses a whitebox-specific config option that had no default value and is obviously not set in any jobs outside of whitebox itself. When attempting to compare a None and an int, a TypeError is thrown. This has been fixed in [3]. The disruption should be minimal, but please don't: 1. recheck tempest until [3] merges 2. blacklist whitebox from the tempest-tox-plugin-sanity-check job :) Cheers! [1] https://review.opendev.org/#/c/699864/ [2] https://review.opendev.org/#/c/711049/8 [3] https://review.opendev.org/#/c/714539/ From gmann at ghanshyammann.com Mon Mar 23 22:04:20 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 23 Mar 2020 17:04:20 -0500 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <20200323204344.knyijc4eelqm4pfd@yuggoth.org> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> <20200323191149.rgeeqkqgucsub745@yuggoth.org> <1935919.eo7qFNXsyg@whitebase.usersys.redhat.com> <20200323204344.knyijc4eelqm4pfd@yuggoth.org> Message-ID: <171096cd6ad.e188118f466591.3103296788189731595@ghanshyammann.com> ---- On Mon, 23 Mar 2020 15:43:44 -0500 Jeremy Stanley wrote ---- > On 2020-03-23 21:28:41 +0100 (+0100), Luigi Toscano wrote: > > On Monday, 23 March 2020 20:11:50 CET Jeremy Stanley wrote: > > > On 2020-03-23 13:34:46 -0500 (-0500), Ghanshyam Mann wrote: > > > [...] > > > > > > > Selected goals till now or V cycle: > > > > 1. Switch legacy Zuul jobs to native > > > > - https://governance.openstack.org/tc/goals/selected/victoria/index.html > > > > > > [...] > > > > > > Please be aware that this goal references the old Zuul v3 Migration > > > Guide which no longer exists except in Git history of the > > > opendev/infra-manual repository (it was removed via > > > https://review.opendev.org/711325 just shy of two years after the > > > Zuul 3.0.0 release). You can still find the old source for it here: > > > > > > > > https://opendev.org/opendev/infra-manual/src/commit/2901f4b87b7506c63aefa34 > > > ece47106ff3c8a328/doc/source/zuulv3.rst > > > > > Why was it removed I think it still important. The amount of jobs still to be > > converted which lead to this goal should say something. > > > > Can it be restored or at least added somewhere else? > > The OpenDev Manual maintainers didn't want to continue curating > instructions for a migration which should have happened two years > ago. The content is CC-BY licensed though, so should be easy to copy > into the goal document or similar if the OpenStack project wants to > retain it somewhere. We need that somewhere until all projects finish the migration. I agree it is a long time for zuulv3 exist but still projects are not migrated to it. What is actual harm of keeping that in opendev manual? -gmann > -- > Jeremy Stanley > From cboylan at sapwetik.org Mon Mar 23 22:42:26 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 23 Mar 2020 15:42:26 -0700 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <171096cd6ad.e188118f466591.3103296788189731595@ghanshyammann.com> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> <20200323191149.rgeeqkqgucsub745@yuggoth.org> <1935919.eo7qFNXsyg@whitebase.usersys.redhat.com> <20200323204344.knyijc4eelqm4pfd@yuggoth.org> <171096cd6ad.e188118f466591.3103296788189731595@ghanshyammann.com> Message-ID: <19262383-38bd-41d2-b48c-41e157cc5cc2@www.fastmail.com> On Mon, Mar 23, 2020, at 3:04 PM, Ghanshyam Mann wrote: > ---- On Mon, 23 Mar 2020 15:43:44 -0500 Jeremy Stanley > wrote ---- > > On 2020-03-23 21:28:41 +0100 (+0100), Luigi Toscano wrote: > > > On Monday, 23 March 2020 20:11:50 CET Jeremy Stanley wrote: > > > > On 2020-03-23 13:34:46 -0500 (-0500), Ghanshyam Mann wrote: > > > > [...] > > > > > > > > > Selected goals till now or V cycle: > > > > > 1. Switch legacy Zuul jobs to native > > > > > - > https://governance.openstack.org/tc/goals/selected/victoria/index.html > > > > > > > > [...] > > > > > > > > Please be aware that this goal references the old Zuul v3 > Migration > > > > Guide which no longer exists except in Git history of the > > > > opendev/infra-manual repository (it was removed via > > > > https://review.opendev.org/711325 just shy of two years after the > > > > Zuul 3.0.0 release). You can still find the old source for it > here: > > > > > > > > > > > > https://opendev.org/opendev/infra-manual/src/commit/2901f4b87b7506c63aefa34 > > > > ece47106ff3c8a328/doc/source/zuulv3.rst > > > > > > > Why was it removed I think it still important. The amount of jobs > still to be > > > converted which lead to this goal should say something. > > > > > > Can it be restored or at least added somewhere else? > > > > The OpenDev Manual maintainers didn't want to continue curating > > instructions for a migration which should have happened two years > > ago. The content is CC-BY licensed though, so should be easy to copy > > into the goal document or similar if the OpenStack project wants to > > retain it somewhere. > > We need that somewhere until all projects finish the migration. I agree > it is a long time > for zuulv3 exist but still projects are not migrated to it. > > What is actual harm of keeping that in opendev manual? We are trying to restructure the documentation to be less OpenStack specific and are moving content into other locations as necessary (all of the zuulv3 migration content is basically openstack or zuul specific). In this specific case much of the content is covered by Zuul's documentation (how to write jobs, job inheritance, secrets, etc). At the time this documentation was written a lot of this was still coming together on the Zuul side, but now that shouldn't be the case. If there are bits not covered by the Zuul docs that should be we can fix that. We might also need to add some openstack specific bits into the openstack contributor guide. I'd also be worried that the doc itself is quite stale as it has been a couple years. In particular I think the document lacks a lot of info on where the devstack jobs have evolved to. From gmann at ghanshyammann.com Mon Mar 23 23:15:59 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 23 Mar 2020 18:15:59 -0500 Subject: [tc][uc][all] Starting community-wide goals ideas for V series In-Reply-To: <19262383-38bd-41d2-b48c-41e157cc5cc2@www.fastmail.com> References: <17016a63ba1.dc0cafe2322988.5181705946513725916@ghanshyammann.com> <17108acfc42.be70ff5e462325.4700344666354487171@ghanshyammann.com> <20200323191149.rgeeqkqgucsub745@yuggoth.org> <1935919.eo7qFNXsyg@whitebase.usersys.redhat.com> <20200323204344.knyijc4eelqm4pfd@yuggoth.org> <171096cd6ad.e188118f466591.3103296788189731595@ghanshyammann.com> <19262383-38bd-41d2-b48c-41e157cc5cc2@www.fastmail.com> Message-ID: <17109ae71d9.129ff09b3467399.6349280198798234025@ghanshyammann.com> ---- On Mon, 23 Mar 2020 17:42:26 -0500 Clark Boylan wrote ---- > On Mon, Mar 23, 2020, at 3:04 PM, Ghanshyam Mann wrote: > > ---- On Mon, 23 Mar 2020 15:43:44 -0500 Jeremy Stanley > > wrote ---- > > > On 2020-03-23 21:28:41 +0100 (+0100), Luigi Toscano wrote: > > > > On Monday, 23 March 2020 20:11:50 CET Jeremy Stanley wrote: > > > > > On 2020-03-23 13:34:46 -0500 (-0500), Ghanshyam Mann wrote: > > > > > [...] > > > > > > > > > > > Selected goals till now or V cycle: > > > > > > 1. Switch legacy Zuul jobs to native > > > > > > - > > https://governance.openstack.org/tc/goals/selected/victoria/index.html > > > > > > > > > > [...] > > > > > > > > > > Please be aware that this goal references the old Zuul v3 > > Migration > > > > > Guide which no longer exists except in Git history of the > > > > > opendev/infra-manual repository (it was removed via > > > > > https://review.opendev.org/711325 just shy of two years after the > > > > > Zuul 3.0.0 release). You can still find the old source for it > > here: > > > > > > > > > > > > > > > > https://opendev.org/opendev/infra-manual/src/commit/2901f4b87b7506c63aefa34 > > > > > ece47106ff3c8a328/doc/source/zuulv3.rst > > > > > > > > > Why was it removed I think it still important. The amount of jobs > > still to be > > > > converted which lead to this goal should say something. > > > > > > > > Can it be restored or at least added somewhere else? > > > > > > The OpenDev Manual maintainers didn't want to continue curating > > > instructions for a migration which should have happened two years > > > ago. The content is CC-BY licensed though, so should be easy to copy > > > into the goal document or similar if the OpenStack project wants to > > > retain it somewhere. > > > > We need that somewhere until all projects finish the migration. I agree > > it is a long time > > for zuulv3 exist but still projects are not migrated to it. > > > > What is actual harm of keeping that in opendev manual? > > We are trying to restructure the documentation to be less OpenStack specific and are moving content into other locations as necessary (all of the zuulv3 migration content is basically openstack or zuul specific). In this specific case much of the content is covered by Zuul's documentation (how to write jobs, job inheritance, secrets, etc). At the time this documentation was written a lot of this was still coming together on the Zuul side, but now that shouldn't be the case. > > If there are bits not covered by the Zuul docs that should be we can fix that. We might also need to add some openstack specific bits into the openstack contributor guide. > > I'd also be worried that the doc itself is quite stale as it has been a couple years. In particular I think the document lacks a lot of info on where the devstack jobs have evolved to. I agree that Zuul documentation has all the details but the migration guide has few helpful sections especially from a legacy job perspective for example 'Legacy Jobs to Projects' which can be quick help to convert those to zuulv3. I am sure many people while working on this goal will find Zuul documentation a little too detail. I remember shanghai PTG discussion of this goal and people asked if we have clear documentation on migration and examples. OpenStack contributor guide is more for how to contribute part not some specific features/changes details. Maybe we can keep it where other OpenStack related infra docs are/will be? -gmann > > From zhangbailin at inspur.com Mon Mar 23 23:18:27 2020 From: zhangbailin at inspur.com (=?gb2312?B?QnJpbiBaaGFuZyjVxbDZwdYp?=) Date: Mon, 23 Mar 2020 23:18:27 +0000 Subject: =?gb2312?B?tPC4tDogW2xpc3RzLm9wZW5zdGFjay5vcme0+reiXVtub3ZhXSBQcm9wb3Np?= =?gb2312?Q?ng_Lee_Yarwood_as_nova_core?= In-Reply-To: References: <706a57d20be28742e119038c7fc02f06@sslemail.net> Message-ID: <8364b8bea8d9460ba4ed9a473bff7765@inspur.com> +1 > 收件人: OpenStack Discuss > 主题: [lists.openstack.org代发][nova] Proposing Lee Yarwood as nova core > > Hi, > > I'm proposing Lee to the core team. He is around for a long time and he > became really active recently not only on the stable branches. I think he would > be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > Cheers, > gibi > > From zhangbailin at inspur.com Mon Mar 23 23:18:14 2020 From: zhangbailin at inspur.com (=?gb2312?B?QnJpbiBaaGFuZyjVxbDZwdYp?=) Date: Mon, 23 Mar 2020 23:18:14 +0000 Subject: =?gb2312?B?tPC4tDogW2xpc3RzLm9wZW5zdGFjay5vcme0+reiXVtub3ZhXSBQcm9wb3Np?= =?gb2312?Q?ng_Ghanshyam_Mann_as_nova_core?= In-Reply-To: References: <9c6cae5d0190cee90b9413973f723d55@sslemail.net> Message-ID: <8b70d07115a64a42bd314a63e67865de@inspur.com> +1 > 收件人: OpenStack Discuss > 主题: [lists.openstack.org代发][nova] Proposing Ghanshyam Mann as nova > core > > Hi, > > I'm proposing Ghanshyam Mann to the core team. He is a long time OpenStack > and nova contributor. He is one of our API and QA expert. I think he would be a > great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > Cheers, > gibi > > > From gmann at ghanshyammann.com Mon Mar 23 23:46:01 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 23 Mar 2020 18:46:01 -0500 Subject: [tempest][gate] tempest-tox-plugin-sanity-check broken by whitebox-tempest-plugin In-Reply-To: References: Message-ID: <17109c9f17c.10093899a467567.2599455527913812913@ghanshyammann.com> ---- On Mon, 23 Mar 2020 17:00:40 -0500 Artom Lifshitz wrote ---- > Hey all, > > A recent change in the whitebox-tempest-plugin [1] has broken the > tempest-tox-plugin-sanity-check job (ex: [2]). The cause is a > skipCheck that uses a whitebox-specific config option that had no > default value and is obviously not set in any jobs outside of whitebox > itself. When attempting to compare a None and an int, a TypeError is > thrown. This has been fixed in [3]. The disruption should be minimal, > but please don't: > > 1. recheck tempest until [3] merges > 2. blacklist whitebox from the tempest-tox-plugin-sanity-check job :) Thanks artom for information. We do not usually balcklist plugins until we do not know who maintain that or cannot be fixed soon. Later is rare case as sanity job just check the installation part of plugins which inclide import and config things. -gmann > > Cheers! > > [1] https://review.opendev.org/#/c/699864/ > [2] https://review.opendev.org/#/c/711049/8 > [3] https://review.opendev.org/#/c/714539/ > > > From alifshit at redhat.com Tue Mar 24 00:02:08 2020 From: alifshit at redhat.com (Artom Lifshitz) Date: Mon, 23 Mar 2020 20:02:08 -0400 Subject: [tempest][gate] tempest-tox-plugin-sanity-check broken by whitebox-tempest-plugin In-Reply-To: <17109c9f17c.10093899a467567.2599455527913812913@ghanshyammann.com> References: <17109c9f17c.10093899a467567.2599455527913812913@ghanshyammann.com> Message-ID: On Mon, Mar 23, 2020 at 7:46 PM Ghanshyam Mann wrote: > > ---- On Mon, 23 Mar 2020 17:00:40 -0500 Artom Lifshitz wrote ---- > > Hey all, > > > > A recent change in the whitebox-tempest-plugin [1] has broken the > > tempest-tox-plugin-sanity-check job (ex: [2]). The cause is a > > skipCheck that uses a whitebox-specific config option that had no > > default value and is obviously not set in any jobs outside of whitebox > > itself. When attempting to compare a None and an int, a TypeError is > > thrown. This has been fixed in [3]. The disruption should be minimal, > > but please don't: > > > > 1. recheck tempest until [3] merges > > 2. blacklist whitebox from the tempest-tox-plugin-sanity-check job :) > > Thanks artom for information. We do not usually balcklist plugins until we do not know who > maintain that or cannot be fixed soon. Later is rare case as sanity job just check the installation > part of plugins which inclide import and config things. Thanks for the clarification! [3] has merged, so things should be back to normal with the tempest gate. > > -gmann > > > > > Cheers! > > > > [1] https://review.opendev.org/#/c/699864/ > > [2] https://review.opendev.org/#/c/711049/8 > > [3] https://review.opendev.org/#/c/714539/ > > > > > > > From melwittt at gmail.com Tue Mar 24 01:55:15 2020 From: melwittt at gmail.com (melanie witt) Date: Mon, 23 Mar 2020 18:55:15 -0700 Subject: Summary: Ironic Mid-cycle at CERN In-Reply-To: References: Message-ID: <8c143c65-a70b-a66b-6fd4-88911b504465@gmail.com> On 3/16/20 16:25, Julia Kreger wrote: > Traits/Scheduling/Flavor Explosion > =========================== > > Arne with CERN raised this topic to bring greater awareness. CERN > presently has greater than 100 flavors representing their hardware > fleet as each physical machine type gets its own flavor. This has > resulted in pain from the lack of flavor specific quotas. What may > help in this area is for Resource Class based quotas, but presently > the state of that work is unknown. The bottom line: A user does not > have clarity into their resource usage. The question then shifted to > being able to report utilization since the current quota model is > based on cores/RAM/instances but not resource_class consumption as a > whole. > > The question largely being “How many am I allowed to create? [before I > eat someone else’s capacity]”. > > https://github.com/stackhpc/os-capacity was raised as a reporting tool > that may help with these sorts of situations with bare metal cloud > operators. Another point raised in this discussion was the lack of > being able to tie consumer and project ID to consumed resources, but > it turns out the allocation list functionality in Placement now has > this functionality. > > In the end of this discussion, there was consensus that this should be > brought back to the Nova community radar. FYI work is in progress to add the ability to have resource class based quota limits as part of the larger effort to add support for unified limits in nova: https://review.opendev.org/#/q/topic:bp/unified-limits-nova+(status:open+OR+status:merged) Specifically, this work-in-progress patch will extract resource classes from a flavor and use them during quota limit enforcement: https://review.opendev.org/615180 Cheers, -melanie From yingjisun at vmware.com Tue Mar 24 03:54:35 2020 From: yingjisun at vmware.com (Yingji Sun) Date: Tue, 24 Mar 2020 03:54:35 +0000 Subject: [oslo][nova][vmware] Replacement for suds library In-Reply-To: References: Message-ID: <3616729F-8084-45BD-AA13-3E5487A4937D@vmware.com> > 在 2020/3/20 下午10:10，“Stephen Finucane” 写入: > > The suds-jurko library used by oslo.vmware is emitting the following > warnings in nova tests. > > /nova/.tox/py36/lib/python3.6/site-packages/suds/resolver.py:89: DeprecationWarning: invalid escape sequence \% > self.splitp = re.compile('({.+})*[^\%s]+' % ps[0]) > /nova/.tox/py36/lib/python3.6/site-packages/suds/wsdl.py:619: DeprecationWarning: invalid escape sequence \s > body.parts = re.split('[\s,]', parts) > > These warnings are going to be errors in Python 3.10 [1]. We have over > 18 months before we need to worry about this [2], but I'd like to see > some movement on this well before then. It seems the suds-jurko fork is > dead [3] and movement on yet another fork, suds-community, is slow [4]. > How difficult would it be to switch over to something that does seem > maintained like zeep [5] and, assuming it's doable, is anyone from > VMWare/SAP able to do this work? > > Stephen Stephen, Thank you very much for pointing this out. Lichao (xuel at vmware.com) and I from VMware will involve into this issue. Do you think zeep is a good alternative of suds ? Or did the replacement already take place on other project ? We would like to make assessment to the zeep first and then make an action plan. Yingji. > [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.python.org%2F3.9%2Fwhatsnew%2F3.9.html%23you-should-check-for-deprecationwarning-in-your-code&data=02%7C01%7Cyingjisun%40vmware.com%7C95008f1ccf0a43198e5a08d7ccd87134%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637203102337393211&sdata=79f%2B3FFTgC275gINmA3aCvWdTe%2BdN8uZ39%2BPM0l85FU%3D&reserved=0 > [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.python.org%2Fdev%2Fpeps%2Fpep-0596%2F&data=02%7C01%7Cyingjisun%40vmware.com%7C95008f1ccf0a43198e5a08d7ccd87134%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637203102337393211&sdata=d0RyU21oygeBi3xxhw20k%2FZTX0xHXQ0Hp7Z2WZb6YEE%3D&reserved=0 > [3] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbitbucket.org%2Fjurko%2Fsuds%2Fsrc%2Fdefault%2F&data=02%7C01%7Cyingjisun%40vmware.com%7C95008f1ccf0a43198e5a08d7ccd87134%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637203102337393211&sdata=lqv3TRF76TL%2B8978gamjson%2FK8B4KnztukYoCNxqSAQ%3D&reserved=0 > [4] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsuds-community%2Fsuds%2Fpull%2F32&data=02%7C01%7Cyingjisun%40vmware.com%7C95008f1ccf0a43198e5a08d7ccd87134%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637203102337393211&sdata=2bWFr5R1e3paJ8Bzrf7fFhkrjKrhYWRJXYpzrZAf45w%3D&reserved=0 > [5] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpython-zeep.readthedocs.io%2Fen%2Fmaster%2F&data=02%7C01%7Cyingjisun%40vmware.com%7C95008f1ccf0a43198e5a08d7ccd87134%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637203102337403198&sdata=G5AtJG%2FZTi2ZgFwZYhKLrfhNren1LliCBEFqa44xcAo%3D&reserved=0 From katonalala at gmail.com Tue Mar 24 05:32:22 2020 From: katonalala at gmail.com (Lajos Katona) Date: Tue, 24 Mar 2020 06:32:22 +0100 Subject: [tempest] what is a proper way to install a package into a vm instance spawned by tempest? In-Reply-To: References: Message-ID: Hi, Perhaps it's better to look at neutron-tempest-plugin: - Your case is more a networking issue as I see. - neutron-tempest-plugin has the option to use other image than cirros with config option advanced_image_ref and in neutron gate that is mostly some ubuntu (ubunt16.04 as I see in latest logs) example: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bf5/713719/3/check/neutron-tempest-plugin-scenario-openvswitch/bf5f249/controller/logs/tempest_conf.txt Lajos Roman Safronov ezt írta (időpont: 2020. márc. 23., H, 18:55): > Actually we need such test because we are testing Openstack as a whole > product (including Neutron+openvswitch+OVN+Nova+libvirt+Octavia etc.) > that's why I think neutron functional test would be not enough. We are > creating tests covering scenarios that our customers tried to use and > encountered issues. > For example this is a bug reported downstream on an issue happened in this > scenario: https://bugzilla.redhat.com/show_bug.cgi?id=1707241 > There were more reported issues on similar use case and we would like to > catch such issues before the product is released. > > Anyway, as I said this specific test runs stable in downstream CI on > virtual multi node environments with nested virtualization. It usually runs > with RHEL8 image but I also tried it with standard Ubuntu-18.04 guest image > used by upstream CI gates. The only issue is that keepalive package > installation by 'apt install' for some reason does not work when running on > upstream gates causing the test to be skipped. I just would like to > understand if running 'apt install/yum install' inside VMs spawned by > upstream tempest tests is not acceptable at all or I am missing something > (maybe proxy settings?). > > On Mon, Mar 23, 2020 at 5:36 PM Clark Boylan wrote: > >> On Sun, Mar 22, 2020, at 9:10 AM, Roman Safronov wrote: >> > Hi, >> > >> > I wrote a tempest test >> > < >> https://review.opendev.org/#/c/710975/https://review.opendev.org/#/c/710975/)> >> which requires keepalived to be running inside vm instance. >> > The test tries to install keepalived package inside vm by running "apt >> > install/yum install". >> > However as I see in upstream gates logs this does not work while the >> > same code works perfectly in our downstream CI using the same image. >> > >> > Do vm instances spawned on upstream gates during tests have a route to >> > the internet? >> > Is there an alternative way that I can use in order to install a >> > package? >> >> By default the tempest jobs use a cirros image. This is a very small, >> quick to boot linux without a package manager. If you need services to be >> running in the image typically you'll need to boot a more typical linux >> installation. Keep in mind that nested virt isn't on by default as it isn't >> available everywhere and has been flaky in the past. This makes these types >> of installations very small which may make reliable VRRP testing difficult. >> >> Thinking out loud here, I'm not sure how much benefit there is to testing >> VRRP failures in this manner. Do we think that OpenStack sitting on top of >> libvirt and OVS will somehow produce different results with VRRP than using >> those tools as is? To put this another way: are we testing OpenStack or are >> we testing OVS and libvirt? >> >> One option here may be to construct this as a Neutron functional test and >> run VRRP on Neutron networks without virtualization mixed in. >> >> > >> > Thanks in advance >> > >> > -- >> > ROMAN SAFRONOV >> >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lpetrut at cloudbasesolutions.com Tue Mar 24 08:07:24 2020 From: lpetrut at cloudbasesolutions.com (Lucian Petrut) Date: Tue, 24 Mar 2020 08:07:24 +0000 Subject: [nova] FYI for out-of-tree virt drivers : new argument for finish_migration() and finish_revert_migration() In-Reply-To: References: Message-ID: Hi, Thanks a lot for the heads up. How far are those patches going to be backported? As soon as they get in, we’ll merge the compute-hyperv one as well: https://review.opendev.org/#/c/714580/ Regards, Lucian Petrut From: Sylvain Bauza Sent: Monday, March 23, 2020 1:06 PM To: OpenStack Discuss Subject: [nova] FYI for out-of-tree virt drivers : new argument for finish_migration() and finish_revert_migration() I wrote two changes [1] for passing down target allocations to virt drivers given we need to use them for closing some bugfix [2] Given the Nova contributor policy is saying that the public API is not related to virt drivers, we won't support virt drivers that won't accept this new argument. If you're a contributor that works on any out-of-tree Nova virt driver, please make sure that you modify your own two above methods. Thanks, -Sylvain [1] https://review.opendev.org/#/c/589085/7 and https://review.opendev.org/#/c/712118/2 [2] https://bugs.launchpad.net/nova/+bug/1778563 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsafrono at redhat.com Tue Mar 24 08:27:29 2020 From: rsafrono at redhat.com (Roman Safronov) Date: Tue, 24 Mar 2020 10:27:29 +0200 Subject: [tempest] what is a proper way to install a package into a vm instance spawned by tempest? In-Reply-To: References: Message-ID: Right, it was ubuntu-16.04 rather than ubuntu-18.04 as I said before by mistake. As Clark said it might be a NAT issue on devstack nodes. On Tue, Mar 24, 2020 at 7:32 AM Lajos Katona wrote: > Hi, > Perhaps it's better to look at neutron-tempest-plugin: > > - Your case is more a networking issue as I see. > - neutron-tempest-plugin has the option to use other image than cirros > with config option advanced_image_ref and in neutron gate that is > mostly some ubuntu (ubunt16.04 as I see in latest logs) > > example: > > https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bf5/713719/3/check/neutron-tempest-plugin-scenario-openvswitch/bf5f249/controller/logs/tempest_conf.txt > > Lajos > > Roman Safronov ezt írta (időpont: 2020. márc. 23., > H, 18:55): > >> Actually we need such test because we are testing Openstack as a whole >> product (including Neutron+openvswitch+OVN+Nova+libvirt+Octavia etc.) >> that's why I think neutron functional test would be not enough. We are >> creating tests covering scenarios that our customers tried to use and >> encountered issues. >> For example this is a bug reported downstream on an issue happened in >> this scenario: https://bugzilla.redhat.com/show_bug.cgi?id=1707241 >> There were more reported issues on similar use case and we would like to >> catch such issues before the product is released. >> >> Anyway, as I said this specific test runs stable in downstream CI on >> virtual multi node environments with nested virtualization. It usually runs >> with RHEL8 image but I also tried it with standard Ubuntu-18.04 guest image >> used by upstream CI gates. The only issue is that keepalive package >> installation by 'apt install' for some reason does not work when running on >> upstream gates causing the test to be skipped. I just would like to >> understand if running 'apt install/yum install' inside VMs spawned by >> upstream tempest tests is not acceptable at all or I am missing something >> (maybe proxy settings?). >> >> On Mon, Mar 23, 2020 at 5:36 PM Clark Boylan >> wrote: >> >>> On Sun, Mar 22, 2020, at 9:10 AM, Roman Safronov wrote: >>> > Hi, >>> > >>> > I wrote a tempest test >>> > < >>> https://review.opendev.org/#/c/710975/https://review.opendev.org/#/c/710975/)> >>> which requires keepalived to be running inside vm instance. >>> > The test tries to install keepalived package inside vm by running "apt >>> > install/yum install". >>> > However as I see in upstream gates logs this does not work while the >>> > same code works perfectly in our downstream CI using the same image. >>> > >>> > Do vm instances spawned on upstream gates during tests have a route to >>> > the internet? >>> > Is there an alternative way that I can use in order to install a >>> > package? >>> >>> By default the tempest jobs use a cirros image. This is a very small, >>> quick to boot linux without a package manager. If you need services to be >>> running in the image typically you'll need to boot a more typical linux >>> installation. Keep in mind that nested virt isn't on by default as it isn't >>> available everywhere and has been flaky in the past. This makes these types >>> of installations very small which may make reliable VRRP testing difficult. >>> >>> Thinking out loud here, I'm not sure how much benefit there is to >>> testing VRRP failures in this manner. Do we think that OpenStack sitting on >>> top of libvirt and OVS will somehow produce different results with VRRP >>> than using those tools as is? To put this another way: are we testing >>> OpenStack or are we testing OVS and libvirt? >>> >>> One option here may be to construct this as a Neutron functional test >>> and run VRRP on Neutron networks without virtualization mixed in. >>> >>> > >>> > Thanks in advance >>> > >>> > -- >>> > ROMAN SAFRONOV >>> >>> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Tue Mar 24 08:57:41 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 24 Mar 2020 09:57:41 +0100 Subject: [neutron] OVS inactivity probes In-Reply-To: References: Message-ID: <20200324085741.7ihlfig4jn5bmxex@skaplons-mac> Hi, I checked it a bit deeper and it seems for me that method add_manager, which is in [1] is not used at all. In the past it was used by function "enable_connection_uri" from neutron.agent.ovsdb.native.helpers module but commit [2] switched it to use helper function from ovsdbapp. So I think that this is simply bug in Neutron which we need to fix. I opened bug for it [3]. [1] https://opendev.org/openstack/neutron/src/branch/master/neutron/agent/common/ovs_lib.py#L122 [2] https://review.opendev.org/#/c/453014/ [3] https://bugs.launchpad.net/neutron/+bug/1868686 On Mon, Mar 23, 2020 at 05:11:42PM +0000, James Denton wrote: > Hello all, > > Like others, we have seen an increase in the amount of messages like those below, followed up by disconnects: > > 2020-03-19T07:19:47.414Z|01614|rconn|ERR|br-int<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting > 2020-03-19T07:19:47.414Z|01615|rconn|ERR|br-ex<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting > 2020-03-19T07:19:47.414Z|01616|rconn|ERR|br-tun<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting > > We have since increased the value of of_inactivity_probe (https://bugs.launchpad.net/neutron/+bug/1817022) and confirmed the Controller connection reflects this new value: > > --- > # ovs-vsctl list Controller > ... > _uuid : b4814677-c6f3-4afc-9c9e-999d5a5ac78f > connection_mode : out-of-band > controller_burst_limit: [] > controller_rate_limit: [] > enable_async_messages: [] > external_ids : {} > inactivity_probe : 60000 > is_connected : true > local_gateway : [] > local_ip : [] > local_netmask : [] > max_backoff : [] > other_config : {} > role : other > status : {last_error="Connection refused", sec_since_connect="1420", sec_since_disconnect="1423", state=ACTIVE} > target : "tcp:127.0.0.1:6633" > --- > > However, we also see disconnects on the manager side, which the config option does not address: > > 2020-03-23T11:01:02.871Z|00443|reconnect|ERR|tcp:127.0.0.1:50098: no response to inactivity probe after 5 seconds, disconnecting > > This bug (https://bugs.launchpad.net/neutron/+bug/1627106) and related commit (https://opendev.org/openstack/neutron/commit/1698bee770b84a2663ba940a6ded5d4b9733101a) appear to leverage the ovs_vsctl_timeout value (since renamed to ovsdb_timeout), but the inactivity_probe for the Manager connection does not appear to be implemented. Honestly, I'm not sure if that code path is used. > > --- > # ovs-vsctl list Manager > _uuid : d61519ba-93fc-4fe5-b05c-b630778a44b0 > connection_mode : [] > external_ids : {} > inactivity_probe : [] > is_connected : true > max_backoff : [] > other_config : {} > status : {bound_port="6640", n_connections="2", sec_since_connect="0", sec_since_disconnect="0"} > target : "ptcp:6640:127.0.0.1" > --- > > Running this by hand sets the inactivity_probe timeout on manager connection, but we'd prefer to use a built-in method, if possible: > > # ovs-vsctl set manager d61519ba-93fc-4fe5-b05c-b630778a44b0 inactivity_probe=30000 > > Any suggestions? > > Thanks, > James > -- Slawek Kaplonski Senior software engineer Red Hat From sbauza at redhat.com Tue Mar 24 09:19:08 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Tue, 24 Mar 2020 10:19:08 +0100 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: On Mon, Mar 23, 2020 at 3:37 PM Balázs Gibizer wrote: > Hi, > > I'm proposing Lee to the core team. He is around for a long time and he > became really active recently not only on the stable branches. I think > he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > +1 Cheers, > gibi > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbauza at redhat.com Tue Mar 24 09:19:41 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Tue, 24 Mar 2020 10:19:41 +0100 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: References: Message-ID: On Mon, Mar 23, 2020 at 5:46 PM Balázs Gibizer wrote: > Hi, > > I'm proposing Ghanshyam Mann to the core team. He is a long time > OpenStack and nova contributor. He is one of our API and QA expert. I > think he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > +1 Cheers, > gibi > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Tue Mar 24 09:49:35 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 24 Mar 2020 10:49:35 +0100 Subject: [nova] feasibility to keep the two-company rule In-Reply-To: <20200323153328.GA10790@paraplu> References: <1584638630.12170.41@est.tech> <43875789-55AC-45A8-B24A-62CDBFDB2F31@inaugust.com> <20200323153328.GA10790@paraplu> Message-ID: Hi, Top posting to summarize public and private discussions. As far as I see overall we agree that the "rule" needs to be less strict. But as far as I understand we have no consensus in the core team to drop the rule entirely. We still would like to keep some of level of control over heavy changes. Where heavy means anything that involves a microversion, service version, rpc version, or database migration. I imagine that these changes need an spec anyhow so the existence of a spec could be a shorthand for when the rule still applies. In the other hand this means that the implementation of specless features, bugfixes, doc enhancements, test enhancements, and refactors can be developed and approved by a single company (but still require two core reviewers). Personally I'm OK with this compromise along with the ongoing effort to add new people to the core team. As this is an unwritten rule I'm not proposing any doc changes to nova to describe this. This mail serves as the official proposal and if there is no major concerns raised against it then this will goes into effect next Tuesday (03.31.) (having an unwritten rule feels so backward to me) Cheers, gibi From stephenfin at redhat.com Tue Mar 24 10:17:58 2020 From: stephenfin at redhat.com (Stephen Finucane) Date: Tue, 24 Mar 2020 10:17:58 +0000 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: References: Message-ID: <012990b86b48e568383b63835617e92c96f0d2be.camel@redhat.com> On Mon, 2020-03-23 at 17:34 +0100, Balázs Gibizer wrote: > Hi, > > I'm proposing Ghanshyam Mann to the core team. He is a long time > OpenStack and nova contributor. He is one of our API and QA expert. I > think he would be a great addition to the team. > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) +1 > Cheers, > gibi > > > > From balazs.gibizer at est.tech Tue Mar 24 10:30:07 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 24 Mar 2020 11:30:07 +0100 Subject: [nova][ptg] PTG participation In-Reply-To: References: <1583947033.12170.37@est.tech> <064cda0d6d2511f3cbccd26fbf1bb5460797fbef.camel@redhat.com> Message-ID: On Thu, Mar 12, 2020 at 15:45, Sylvain Bauza wrote: > > > On Wed, Mar 11, 2020 at 7:11 PM Sean Mooney > wrote: >> On Wed, 2020-03-11 at 17:36 +0000, Sean Mooney wrote: >> > On Wed, 2020-03-11 at 18:17 +0100, Balázs Gibizer wrote: >> > > Hi, >> > > >> > > I've just got the news from my employer that due to COVID19 I >> cannot >> > > travel to Vancouver in June. >> Since then the in-person PTG in Vancuver has been canceled [1]. So we have to come up with a virtual PTG plan that works for most of the people and allows us to discuss nova and cross project related questions regarding the Victoria cycle. >> > >> > From a redhat perspective i dont think a decision has been made >> on if we should attend or not. >> ill clarify that slightly in that we do have guidence that "Red >> Hatters may not travel to attend external events or >> conferences with 1000+ attendees, even within their home country." >> >> in the past when the ptg and summit were combinined and we had the >> devsumit have ment that travel to the openstack even >> would not be allowed. At its current size its kind of in a gray >> zone where its is not banned as a public event but if it >> was an internal event the number of redhat employee that would be >> attending woudl be over the limit we have and the >> physical event would be canceled and converted to a virtual only >> event. >> >> so its tbd if i will be attending too although i have not heard a >> definitive No at this point but i also cant really >> book tickets and flight yet either however the guidance we have >> been given is to try and default to virtual attendance >> were possible. >> >> > last time i spoke to my manager we were still waiting to see how >> thing progress with the assumtion we would attend >> > but we might want to plan for a virtual PTG (via video >> conference, etherpad and email) in the event many cant travel >> > or >> > that things escalate to a point where the PTG event could be >> canceled. >> > >> > i dont think the foundation has indicated that that is likely to >> happen but im sure they are monitoring >> > things closely as our employers will be so having a plan b might >> now be a bad thing in either case. >> > if there isnt a diverse quoram physically at the ptg it would >> limit our ability to make desisions as happend to >> > some extent in china. it would be still good to get operator >> feedback but they may also be under similar travel >> > restrictions. >> > > > > > I personnally have concerns with virtual events that are large like > our PTG : > - which timezone should we use for the virtual PTG ? Anyway, Asia, > Europe or North America folks could have problems with one. > - some folks (like me) have bandwidth issues in their (home) > offices. They could be having problems when trying to discuss > - how can we be sure that all of us can discuss by a virtual meeting > ? Not saying it won't work, but discussions could take more time. I agree that his is a problem. I think we cannot make a 3 days, 8 hours long video meeting due to timezone, bandwidth and other restrictions. Here is what I propose: Until two weeks before the PTG: * Collect discussion topics on the etherpad [2] as usual * you also free to start reflecting on and discussing such topics that are already on the etherpad. Two weeks before the PTG (May 25 - May 29) * I will scan the etherpad and for each open topics I will start a mail thread to have focused discussion. * If the discussion is not finished on a given topic within this week then I will ask the proposer of the topic to organize a real-time meeting (chat, voice, video) either for the week before the PTG (Jun 1 - June 5) or for the week of the PTG (Jun 8 - Jun 12). I think based on the active participants of the mail (or etherpad) discussion the proposer will know who to involve to the real-time discussion. So not every nova devs needs to sit through every video meeting in obscure local times. I'm proposing this longish three weeks schedule to i) allow enough time for an ML discussion to conclude if the topic is straight forward ii) make it possible at the end of the first week to start organizing real time discussions and still have two weeks for those discussion to be scheduled. It might take time to find and agree on a common time slot and there will be popular time slots. So in both case more available days can help. What do you think? > > Anyway, I'm sad for you, gibi. I'm not -2 about a virtual PTG, I just > want to make sure we think about the above concerns before agreeing > with any virtual PTG. I agree. We need to figure out what could work. Cheers, gibi [1] https://www.openstack.org/events/opendev-ptg-2020/ [2] https://etherpad.openstack.org/p/nova-victoria-ptg From akekane at redhat.com Tue Mar 24 10:32:55 2020 From: akekane at redhat.com (Abhishek Kekane) Date: Tue, 24 Mar 2020 16:02:55 +0530 Subject: [glance] weekly meeting IRC channel change Message-ID: Hi All, Beginning with this Thursday (26 March), the Glance team weekly meeting will be held in: #openstack-meeting The time continues to be 1400 UTC on Thursdays. Thanks & Best Regards, Abhishek Kekane -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdulko at redhat.com Tue Mar 24 11:27:57 2020 From: mdulko at redhat.com (mdulko at redhat.com) Date: Tue, 24 Mar 2020 12:27:57 +0100 Subject: [k8s][zun] Introduce a new feature in Ussuri - CRI integration In-Reply-To: References: Message-ID: <668362769fa2171e2c42d58ecb430968fd658b5f.camel@redhat.com> On Mon, 2020-03-23 at 12:17 -0400, Hongbin Lu wrote: > > > On Mon, Mar 23, 2020 at 11:48 AM wrote: > > On Sun, 2020-03-22 at 13:28 -0400, Hongbin Lu wrote: > > > Hi all, > > > > > > As we are approaching the end of Ussuri cycle, I would like to take a > > > chance to introduce a new feature the Zun team implemented in this > > > cycle - CRI integration [1]. > > > Under the hook, a capsule is a podsandbox with one or more containers > > > in a CRI runtime (i.e. containerd). Compared to Docker, a CRI runtime > > > has a better support for the pod concept so we chose it to implement > > > capsule. A caveat is that CRI requires a CNI plugin for the > > > networking, so we need to implement a CNI plugin for Zun (called zun- > > > cni). The role of CNI plugin is similar as kuryr-libnetwork that we > > > are using for Docker except it implements a different networking > > > model (CNI). I summaries it as below: > > > > Hi, > > > > I noticed that Zun's CNI plugin [1] is basically a simplified version > > of kuryr-kubernetes code. While it's totally fine you've copied that, I > > wonder what modifications had been made to make it suit Zun? Is there a > > chance to converge this to make Zun use kuryr-kubernetes directly so > > that we won't develop two versions of that code in parallel? > > Right. I also investigated the possibilities of reusing the kuryr- > kubernetes codebase as well. Definitely, some codes are common among > two projects. If we can move the common code to a library (i.e. > kuryr-lib), Zun should be able to directly consume the code. In > particular, I am interesting to directly consume the CNI binding code > (kuryr_kubernetes/cni/binding/) and the VIF versioned object > (kuryr_kubernetes/objects). > > Most parts of kuryr-kubernetes code is coupling with the "list-and- > watch" logic against k8s API. Zun is not able to reuse those part of > code. However, I do advocate to move all the common code to kuryr-lib > so Zun can reuse it whenever it is appropriate. Uhm, moving more code into kuryr.lib is something Kuryr team would like to avoid. Our tendency is rather to stop depending from it, as kuryr- kubernetes being a CNI plugin is normally consumed as a container image and having any dependencies is a burden there. That's why I was asking about modifications to kuryr-daemon code that Zun required - to see if we can modify kuryr-daemon to be pluggable enough to be consumed by Zun directly. > > Thanks, > > Michał > > > > [1] https://github.com/openstack/zun/tree/master/zun/cni > > > > > +--------------+------------------------+---------------+ > > > | Concept | Container | Capsule (Pod) | > > > +--------------+------------------------+---------------+ > > > | API endpoint | /containers | /capsules | > > > | Engine | Docker | CRI runtime | > > > | Network | kuryr-libnetwork (CNM) | zun-cni (CNI) | > > > +--------------+------------------------+---------------+ > > > > > > Typically, a CRI runtime works well with Kata Container which > > > provides hypervisor-based isolation for neighboring containers in the > > > same node. As a result, it is secure to consolidate pods from > > > different tenants into a single node which increases the resource > > > utilization. For deployment, a typical stack looks like below: > > > > > > +----------------------------------------------+ > > > | k8s control plane | > > > +----------------------------------------------+ > > > | Virtual Kubelet (OpenStack provider) | > > > +----------------------------------------------+ > > > | OpenStack control plane (Zun, Neutron, etc.) | > > > +----------------------------------------------+ > > > | OpenStack data plane | > > > | (Zun compute agent, Neutron OVS agent, etc.) | > > > +----------------------------------------------+ > > > | Containerd (with CRI plugin) | > > > +----------------------------------------------+ > > > | Kata Container | > > > +----------------------------------------------+ > > > > > > In this stack, if a user creates a deployment or pod in k8s, the k8s > > > scheduler will schedule the pod to the virtual node registered by > > > Virtual Kubelet. Virtual Kubelet will pick up the pod and let the > > > configured cloud provider to handle it. The cloud provider invokes > > > Zun API to create a capsule. Upon receiving the API request to create > > > a capsule, Zun scheduler will schedule the capsule to a compute node. > > > The Zun compute agent in that node will provision the capsule using a > > > CRI runtime (containerd in this example). The Zun-CRI runtime > > > communication is done via a gRPC protocol through a unix socket. The > > > CRI runtime will first create the pod in Kata Container (or runc as > > > an alternative) that realizes the pod using a lightweight VM. > > > Furthermore, the CRI runtime will use a CNI plugin, which is the zun- > > > cni binary, to setup the network. The zun-cni binary is a thin > > > executable that dispatches the CNI command to a daemon service called > > > zun-cni-daemon. The community is via HTTP within localhost. The zun- > > > cni-daemon will look up the Neutron port information from DB and > > > perform the port binding. > > > > > > In conclusion, starting from Ussuri, Zun adds support for CRI- > > > compatible runtime. Zun uses CRI runtime to realize the concept of > > > pod. Using this feature together with Virtual Kubelet and Kata > > > Container, we can offer "serverless kubernetes pod" service which is > > > comparable with AWS EKS with Fargate. > > > > > > [1] https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime > > > [2] https://github.com/virtual-kubelet/virtual-kubelet > > > [3] https://github.com/virtual-kubelet/openstack-zun > > > [4] https://aws.amazon.com/about-aws/whats-new/2019/12/run-serverless-kubernetes-pods-using-amazon-eks-and-aws-fargate/ > > > [5] https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ > > > > From elfosardo at gmail.com Tue Mar 24 11:30:05 2020 From: elfosardo at gmail.com (Riccardo Pittau) Date: Tue, 24 Mar 2020 12:30:05 +0100 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: <20200323184424.ppwotidneccsiv6b@firewall> References: <0BDD1F91-A4C2-4F76-A194-65E060E05114@cern.ch> <20200323184424.ppwotidneccsiv6b@firewall> Message-ID: Dusting off my Goofy hat for the special event On Mon, Mar 23, 2020 at 7:49 PM Nate Johnston wrote: > > On Mon, Mar 23, 2020 at 07:29:29PM +0100, Tim Bell wrote: > > > Time to get out the Buffy T-Shirt > > > > https://www.lookhuman.com/design/26889-there-s-nothing-we-can-t-face-except-for-bunnies/6010-heathered_gray_nl-md > > Brilliant. > > --N. > > From james.denton at rackspace.com Tue Mar 24 12:37:03 2020 From: james.denton at rackspace.com (James Denton) Date: Tue, 24 Mar 2020 12:37:03 +0000 Subject: [neutron] OVS inactivity probes In-Reply-To: <20200324085741.7ihlfig4jn5bmxex@skaplons-mac> References: <20200324085741.7ihlfig4jn5bmxex@skaplons-mac> Message-ID: Thank you, Slawek. Appreciate the quick assist! James On 3/24/20, 4:58 AM, "Slawek Kaplonski" wrote: CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hi, I checked it a bit deeper and it seems for me that method add_manager, which is in [1] is not used at all. In the past it was used by function "enable_connection_uri" from neutron.agent.ovsdb.native.helpers module but commit [2] switched it to use helper function from ovsdbapp. So I think that this is simply bug in Neutron which we need to fix. I opened bug for it [3]. [1] https://opendev.org/openstack/neutron/src/branch/master/neutron/agent/common/ovs_lib.py#L122 [2] https://review.opendev.org/#/c/453014/ [3] https://bugs.launchpad.net/neutron/+bug/1868686 On Mon, Mar 23, 2020 at 05:11:42PM +0000, James Denton wrote: > Hello all, > > Like others, we have seen an increase in the amount of messages like those below, followed up by disconnects: > > 2020-03-19T07:19:47.414Z|01614|rconn|ERR|br-int<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting > 2020-03-19T07:19:47.414Z|01615|rconn|ERR|br-ex<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting > 2020-03-19T07:19:47.414Z|01616|rconn|ERR|br-tun<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting > > We have since increased the value of of_inactivity_probe (https://bugs.launchpad.net/neutron/+bug/1817022) and confirmed the Controller connection reflects this new value: > > --- > # ovs-vsctl list Controller > ... > _uuid : b4814677-c6f3-4afc-9c9e-999d5a5ac78f > connection_mode : out-of-band > controller_burst_limit: [] > controller_rate_limit: [] > enable_async_messages: [] > external_ids : {} > inactivity_probe : 60000 > is_connected : true > local_gateway : [] > local_ip : [] > local_netmask : [] > max_backoff : [] > other_config : {} > role : other > status : {last_error="Connection refused", sec_since_connect="1420", sec_since_disconnect="1423", state=ACTIVE} > target : "tcp:127.0.0.1:6633" > --- > > However, we also see disconnects on the manager side, which the config option does not address: > > 2020-03-23T11:01:02.871Z|00443|reconnect|ERR|tcp:127.0.0.1:50098: no response to inactivity probe after 5 seconds, disconnecting > > This bug (https://bugs.launchpad.net/neutron/+bug/1627106) and related commit (https://opendev.org/openstack/neutron/commit/1698bee770b84a2663ba940a6ded5d4b9733101a) appear to leverage the ovs_vsctl_timeout value (since renamed to ovsdb_timeout), but the inactivity_probe for the Manager connection does not appear to be implemented. Honestly, I'm not sure if that code path is used. > > --- > # ovs-vsctl list Manager > _uuid : d61519ba-93fc-4fe5-b05c-b630778a44b0 > connection_mode : [] > external_ids : {} > inactivity_probe : [] > is_connected : true > max_backoff : [] > other_config : {} > status : {bound_port="6640", n_connections="2", sec_since_connect="0", sec_since_disconnect="0"} > target : "ptcp:6640:127.0.0.1" > --- > > Running this by hand sets the inactivity_probe timeout on manager connection, but we'd prefer to use a built-in method, if possible: > > # ovs-vsctl set manager d61519ba-93fc-4fe5-b05c-b630778a44b0 inactivity_probe=30000 > > Any suggestions? > > Thanks, > James > -- Slawek Kaplonski Senior software engineer Red Hat From hongbin034 at gmail.com Tue Mar 24 13:16:47 2020 From: hongbin034 at gmail.com (Hongbin Lu) Date: Tue, 24 Mar 2020 09:16:47 -0400 Subject: [k8s][zun] Introduce a new feature in Ussuri - CRI integration In-Reply-To: <668362769fa2171e2c42d58ecb430968fd658b5f.camel@redhat.com> References: <668362769fa2171e2c42d58ecb430968fd658b5f.camel@redhat.com> Message-ID: On Tue, Mar 24, 2020 at 7:28 AM wrote: > On Mon, 2020-03-23 at 12:17 -0400, Hongbin Lu wrote: > > > > > > On Mon, Mar 23, 2020 at 11:48 AM wrote: > > > On Sun, 2020-03-22 at 13:28 -0400, Hongbin Lu wrote: > > > > Hi all, > > > > > > > > As we are approaching the end of Ussuri cycle, I would like to take a > > > > chance to introduce a new feature the Zun team implemented in this > > > > cycle - CRI integration [1]. > > > > > > > Under the hook, a capsule is a podsandbox with one or more containers > > > > in a CRI runtime (i.e. containerd). Compared to Docker, a CRI runtime > > > > has a better support for the pod concept so we chose it to implement > > > > capsule. A caveat is that CRI requires a CNI plugin for the > > > > networking, so we need to implement a CNI plugin for Zun (called zun- > > > > cni). The role of CNI plugin is similar as kuryr-libnetwork that we > > > > are using for Docker except it implements a different networking > > > > model (CNI). I summaries it as below: > > > > > > Hi, > > > > > > I noticed that Zun's CNI plugin [1] is basically a simplified version > > > of kuryr-kubernetes code. While it's totally fine you've copied that, I > > > wonder what modifications had been made to make it suit Zun? Is there a > > > chance to converge this to make Zun use kuryr-kubernetes directly so > > > that we won't develop two versions of that code in parallel? > > > > Right. I also investigated the possibilities of reusing the kuryr- > > kubernetes codebase as well. Definitely, some codes are common among > > two projects. If we can move the common code to a library (i.e. > > kuryr-lib), Zun should be able to directly consume the code. In > > particular, I am interesting to directly consume the CNI binding code > > (kuryr_kubernetes/cni/binding/) and the VIF versioned object > > (kuryr_kubernetes/objects). > > > > Most parts of kuryr-kubernetes code is coupling with the "list-and- > > watch" logic against k8s API. Zun is not able to reuse those part of > > code. However, I do advocate to move all the common code to kuryr-lib > > so Zun can reuse it whenever it is appropriate. > > Uhm, moving more code into kuryr.lib is something Kuryr team would like > to avoid. Our tendency is rather to stop depending from it, as kuryr- > kubernetes being a CNI plugin is normally consumed as a container image > and having any dependencies is a burden there. > Kuyur-lib is already a dependency for kuryr-kubernetes: https://github.com/openstack/kuryr-kubernetes/blob/master/requirements.txt . Do you mean kuryr-kubernetes is going to remove kuryr-lib as a dependency? And I don't quite get the "container image" justification. Could you explain more? > > That's why I was asking about modifications to kuryr-daemon code that > Zun required - to see if we can modify kuryr-daemon to be pluggable > enough to be consumed by Zun directly. > In theory, you can refactor the code and make it pluggable. Suppose you are able to do that, I would still suggest to move the whole framework out as a library. That is a prerequisite for Zun (or any other projects) to consume it, right? > > > > Thanks, > > > Michał > > > > > > [1] https://github.com/openstack/zun/tree/master/zun/cni > > > > > > > +--------------+------------------------+---------------+ > > > > | Concept | Container | Capsule (Pod) | > > > > +--------------+------------------------+---------------+ > > > > | API endpoint | /containers | /capsules | > > > > | Engine | Docker | CRI runtime | > > > > | Network | kuryr-libnetwork (CNM) | zun-cni (CNI) | > > > > +--------------+------------------------+---------------+ > > > > > > > > Typically, a CRI runtime works well with Kata Container which > > > > provides hypervisor-based isolation for neighboring containers in the > > > > same node. As a result, it is secure to consolidate pods from > > > > different tenants into a single node which increases the resource > > > > utilization. For deployment, a typical stack looks like below: > > > > > > > > +----------------------------------------------+ > > > > | k8s control plane | > > > > +----------------------------------------------+ > > > > | Virtual Kubelet (OpenStack provider) | > > > > +----------------------------------------------+ > > > > | OpenStack control plane (Zun, Neutron, etc.) | > > > > +----------------------------------------------+ > > > > | OpenStack data plane | > > > > | (Zun compute agent, Neutron OVS agent, etc.) | > > > > +----------------------------------------------+ > > > > | Containerd (with CRI plugin) | > > > > +----------------------------------------------+ > > > > | Kata Container | > > > > +----------------------------------------------+ > > > > > > > > In this stack, if a user creates a deployment or pod in k8s, the k8s > > > > scheduler will schedule the pod to the virtual node registered by > > > > Virtual Kubelet. Virtual Kubelet will pick up the pod and let the > > > > configured cloud provider to handle it. The cloud provider invokes > > > > Zun API to create a capsule. Upon receiving the API request to create > > > > a capsule, Zun scheduler will schedule the capsule to a compute node. > > > > The Zun compute agent in that node will provision the capsule using a > > > > CRI runtime (containerd in this example). The Zun-CRI runtime > > > > communication is done via a gRPC protocol through a unix socket. The > > > > CRI runtime will first create the pod in Kata Container (or runc as > > > > an alternative) that realizes the pod using a lightweight VM. > > > > Furthermore, the CRI runtime will use a CNI plugin, which is the zun- > > > > cni binary, to setup the network. The zun-cni binary is a thin > > > > executable that dispatches the CNI command to a daemon service called > > > > zun-cni-daemon. The community is via HTTP within localhost. The zun- > > > > cni-daemon will look up the Neutron port information from DB and > > > > perform the port binding. > > > > > > > > In conclusion, starting from Ussuri, Zun adds support for CRI- > > > > compatible runtime. Zun uses CRI runtime to realize the concept of > > > > pod. Using this feature together with Virtual Kubelet and Kata > > > > Container, we can offer "serverless kubernetes pod" service which is > > > > comparable with AWS EKS with Fargate. > > > > > > > > [1] > https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime > > > > [2] https://github.com/virtual-kubelet/virtual-kubelet > > > > [3] https://github.com/virtual-kubelet/openstack-zun > > > > [4] > https://aws.amazon.com/about-aws/whats-new/2019/12/run-serverless-kubernetes-pods-using-amazon-eks-and-aws-fargate/ > > > > [5] > https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-philippe at evrard.me Tue Mar 24 13:39:07 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Tue, 24 Mar 2020 14:39:07 +0100 Subject: [tc] April 2nd meeting agenda Message-ID: Hello folks, If you have to update the agenda of our next monthly meeting, it's still time! I will send the definitive agenda on the ML Thursday. Regards, JP [1]: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee From kklimonda at syntaxhighlighted.com Tue Mar 24 13:45:42 2020 From: kklimonda at syntaxhighlighted.com (Krzysztof Klimonda) Date: Tue, 24 Mar 2020 14:45:42 +0100 Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment Message-ID: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> Hi, I’ve been spending some time recently thinking about best approach to some sort of multi-region deployment of OpenStack clouds. The two approaches that I’m currently evaluating are shared keystone database (with galera cluster spanning three locations) and shared-nothing approach, where external component is responsible for managing users, projects etc. Shared keystone database seems fairly straightforward from OS point of view (I’m ignoring galera replication over WAN woes for the purpose of this discussion) until I hit 3 regions. Additional regions must either reuse “global” keystone adding latency everywhere, or we need a way to replicate data from “master” galera cluster to “slave” clusters, and route all database write statements back to the master galera cluster, while reading from local asynchronous replica. This has me worried somewhat, as doing that that into eventually-consistent deployment of sort. Services deployed in regions with asynchronous replication can no longer depend on the fact that once transaction is finished, consecutive reads will return up-to-date state. I can imagine scenarios where, as an example, trust is setup for heat, but that fact is not replicated back to the database by the time heat tries to issue a token based on that trust and the process fails. The other approach would be to keep keystone databases completely separate, and have something external to openstack manage all those resources. While not sharing keystone database between regions sidesteps the issue of scalability, and the entire setup seems to be more resilient to failures, it’s not without its own drawbacks: * In this setup Horizon can no longer switch between regions without additional work (see notes below) * There is no longer single API entrypoint to the cloud * Some Keystone API operations would have to be removed from users via custom policy - for example, managing user assignment to projects (for users who have domain admin role) Additional thoughts Could horizon be modified to switch endpoints based on the region selected in the UI? Is the token reissued when region is changed in horizon, or is single token used? I’m assuming it’s the former given my understanding that when projects are changed, a new token is issued - but perhaps the initial token is always used to issue project-scoped tokens for Horizon? In the second scenario, with separate keystone databases, a backend for keystone could be created that proxies some operations (like aforementioned user assignment) back to the external manager so that it can be propagated to other clouds. Does that even make sense? In the end I’m reaching out in hope that someone could chime in based on their experience - perhaps I’m missing a better approach, or making wrong assumptions in my email, especially around asynchronous replication of keystone database and its effect on services in regions that may not have up-to-data view of the databas. Or perhaps trying ot synchronize keystone state by external tool is not really worth the additional effort that would require. -Chris From mark at stackhpc.com Tue Mar 24 16:24:57 2020 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 24 Mar 2020 16:24:57 +0000 Subject: [kolla][uc] Kolla SIG In-Reply-To: References: Message-ID: On Fri, 6 Mar 2020 at 16:39, Mark Goddard wrote: > > Hi, > > I'd like to propose the creation of a Special Interest Group (SIG) [0] > for Kolla. > > The main aim of the group would be to improve communication between > operators and developers. > > The SIG would host regular virtual project onboarding, project update, > and feedback sessions, ideally via video calls. This should remove the > necessity of being physically present at Open Infra Summits for > participation in the project. I like to think of this as the fifth > open [1] (name TBD). > > I propose that in addition to the above sessions, the SIG should host > more informal discussions, probably every 2-4 weeks with the aim of > meeting other community members, discussing successes and failures, > sharing knowledge, and generally getting to know each other a bit > better. These could be via video call, IRC, or a mix. > > Finally - I propose that we build and maintain a list of Kolla users, > including details of their environments and areas of interest and > expertise. Of course this would be opt-in. This would help us to > connect with subject matter experts and interested parties to help > answer queries in IRC, or when making changes to a specific area. > > This is all up for discussion, and subject to sufficient interest. If > you are interested, please add your name and email address to the > Etherpad [2], along with any comments, thoughts or suggestions. > > [0] https://governance.openstack.org/sigs/ > [1] https://www.openstack.org/four-opens/ > [2] https://etherpad.openstack.org/p/kolla-sig > > Cheers, > Mark We have had a good number of people express interest in this group. Based on feedback it will not be a SIG, but an informal group affiliated with the Kolla project. Let's try to schedule a slot for some meetings. I've created a Doodle poll [1] with hour-long slots between 12:00 UTC and 17:00 UTC, for next week and the week after. I suggest we start with meetings every two weeks while we build momentum, but we should probably drop to once per month eventually. [1] https://doodle.com/poll/9g7czxmdngd5zz4t Cheers, Mark From amy at demarco.com Tue Mar 24 16:45:26 2020 From: amy at demarco.com (Amy Marrich) Date: Tue, 24 Mar 2020 11:45:26 -0500 Subject: [kolla][uc] Kolla SIG In-Reply-To: <5E6663ED.5060609@openstack.org> References: <6407c2a8-b778-4163-3fd1-031d9e073209@openstack.org> <5E6663ED.5060609@openstack.org> Message-ID: Just saw this! +2 Amy (spotz) On Mon, Mar 9, 2020 at 10:44 AM Jimmy McArthur wrote: > Kollaborators? > > Thierry Carrez wrote: > > No, I was just suggesting finding a catchy name for those outreach > > activities, one that encourages users to reach out, and make it less > > intimidating than joining the IRC meeting for a project team. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Tue Mar 24 16:50:01 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Tue, 24 Mar 2020 17:50:01 +0100 Subject: [kolla][uc] Kolla SIG In-Reply-To: References: Message-ID: It's worth noting that clocks go forward in Europe this weekend, so you should account for the extra difference with UTC when looking at next week's schedule… or look at your calendar directly in UTC. On Tue, 24 Mar 2020 at 17:35, Mark Goddard wrote: > > On Fri, 6 Mar 2020 at 16:39, Mark Goddard wrote: > > > > Hi, > > > > I'd like to propose the creation of a Special Interest Group (SIG) [0] > > for Kolla. > > > > The main aim of the group would be to improve communication between > > operators and developers. > > > > The SIG would host regular virtual project onboarding, project update, > > and feedback sessions, ideally via video calls. This should remove the > > necessity of being physically present at Open Infra Summits for > > participation in the project. I like to think of this as the fifth > > open [1] (name TBD). > > > > I propose that in addition to the above sessions, the SIG should host > > more informal discussions, probably every 2-4 weeks with the aim of > > meeting other community members, discussing successes and failures, > > sharing knowledge, and generally getting to know each other a bit > > better. These could be via video call, IRC, or a mix. > > > > Finally - I propose that we build and maintain a list of Kolla users, > > including details of their environments and areas of interest and > > expertise. Of course this would be opt-in. This would help us to > > connect with subject matter experts and interested parties to help > > answer queries in IRC, or when making changes to a specific area. > > > > This is all up for discussion, and subject to sufficient interest. If > > you are interested, please add your name and email address to the > > Etherpad [2], along with any comments, thoughts or suggestions. > > > > [0] https://governance.openstack.org/sigs/ > > [1] https://www.openstack.org/four-opens/ > > [2] https://etherpad.openstack.org/p/kolla-sig > > > > Cheers, > > Mark > > We have had a good number of people express interest in this group. > Based on feedback it will not be a SIG, but an informal group > affiliated with the Kolla project. > > Let's try to schedule a slot for some meetings. I've created a Doodle > poll [1] with hour-long slots between 12:00 UTC and 17:00 UTC, for > next week and the week after. I suggest we start with meetings every > two weeks while we build momentum, but we should probably drop to once > per month eventually. > > [1] https://doodle.com/poll/9g7czxmdngd5zz4t > > Cheers, > Mark > From openstack at nemebean.com Tue Mar 24 17:21:51 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 24 Mar 2020 12:21:51 -0500 Subject: [oslo] Feature Freeze is this week Message-ID: Apologies for not mentioning this in the meeting this week, but Oslo Feature Freeze is already this Friday.[0] After Friday, anything that is not a bug fix will need a feature freeze exception in order to merge. If you're wondering why Oslo freezes early, there's a doc[1] for that. If you have any questions not answered there, feel free to reply here or ping me on IRC. Thanks. -Ben 0: https://releases.openstack.org/ussuri/schedule.html 1: https://specs.openstack.org/openstack/oslo-specs/specs/policy/feature-freeze.html From satish.txt at gmail.com Tue Mar 24 19:04:36 2020 From: satish.txt at gmail.com (Satish Patel) Date: Tue, 24 Mar 2020 15:04:36 -0400 Subject: nova-api missing heartbeats to rabbitmq Message-ID: Folks, Recently i am seeing lots of error message in rabbitmq logs saying missing heartbeats from nova-api nodes, I am not seeing any issue at functionality level as everything working fine but just noticed those error and trying to find root cause of it. 172.28.15.125 nova-api server 172.28.15.192 rabbitmq server on rabbit.log 2020-03-24 12:21:41.389 [error] <0.29772.4418> closing AMQP connection <0.29772.4418> (172.28.15.125:42656 -> 172.28.15.192:5671 - uwsgi:32419:9b8a323b-653d-4585-9916-d52b3fd81d59): missed heartbeats from client, timeout: 60s on nova-api.log 2020-03-24 12:19:06.554 32435 ERROR oslo.messaging._drivers.impl_rabbit [-] [4b8adff0-ff9f-4863-a939-537d391e5d9e] AMQP server on 172.28.15.192:5671 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: error: [Errno 104] Connection reset by peer From kennelson11 at gmail.com Tue Mar 24 19:20:17 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 24 Mar 2020 12:20:17 -0700 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: Hey Dmitry :) Would you mind adding that to the etherpad so we have all the info in one place? I looked and didn't think I saw Ironic's testimony in there and it would be super helpful! -Kendall (diablo_rojo) On Wed, Mar 11, 2020 at 7:16 AM Dmitry Tantsur wrote: > Hi, > > Ironic did, I think, 3-4 virtual midcycles, and I think they were quite > successful. > The most positive outcome was to reach out to folks who usually cannot > travel. They really appreciated that. > > We used SIP the first couple of times, but it caused problems for a big > share of participants. Also lack of moderation was problematic. > We switched to Bluejeans later, and that was, IMO, a big improvement: > 1) A participant list with information who is speaking. > 2) An option for a moderator to mute a person or everyone. > 3) Screen sharing. > > Dmitry > > On Tue, Mar 10, 2020 at 1:29 AM Kendall Nelson > wrote: > >> Hello Everyone! >> >> I wanted to collect best practices and pitfalls to avoid wrt projects >> experiences with virtual midcycles. I know of a few projects that have done >> them in the past and with how travel is hard for a lot of people right now, >> I expect more projects to have midcycles. I think it would be helpful to >> have all of the data we can collect in one place for those not just new to >> virtual midcycles but the whole community. >> >> I threw some categories into this etherpad[1] and filled in some options. >> Please add to it :) >> >> -Kendall (diablo_rojo) >> >> [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Mar 24 20:04:25 2020 From: smooney at redhat.com (Sean Mooney) Date: Tue, 24 Mar 2020 20:04:25 +0000 Subject: nova-api missing heartbeats to rabbitmq In-Reply-To: References: Message-ID: <51e5f85e56fa11b07da24c86cd5b16187de5795f.camel@redhat.com> On Tue, 2020-03-24 at 15:04 -0400, Satish Patel wrote: > Folks, > > Recently i am seeing lots of error message in rabbitmq logs saying > missing heartbeats from nova-api nodes, I am not seeing any issue at > functionality level as everything working fine but just noticed those > error and trying to find root cause of it. this is expected and a know issue. a release or two ago we intoduced the use of eventlet monkey patching to the nova api to implemente multi cell scarter gater requests where by we concurrently dispatch request to all cells and then wait for the results instead of doing it serially. a side effect of that change is not that the nova api is monkey patched expcitly, if you execute it via uwsgi or mod_wsgi the heatbeat thread that was previously a full os thread is not jsut a green thread. the wsgi server manges the life time of the api process and can set that tread to sleep or kill it. at presnet there is nothing for the operator to do in regards to this message and you should just ignore bar one caviate. if you are configuring your api you should not scale it useing thread but instead shoudl scale the api using processes. deploying the api as a wsgi applciation with multiple threads per python process can cause issues so threads should always be set to 1 or unset. we have no real agreement on the long term fix. in some environments disableing the heartbeat and relying on the os tcp keepalive config is one option. you can also rever to running the api using the build in python wsgi server instead of uwsgi. if you do this there is a performacne pelenty so we dont really advise people to do that. there have been mail thread on this topic in the past but i do not have them to hand. > > 172.28.15.125 nova-api server > 172.28.15.192 rabbitmq server > > on rabbit.log > > 2020-03-24 12:21:41.389 [error] <0.29772.4418> closing AMQP connection > <0.29772.4418> (172.28.15.125:42656 -> 172.28.15.192:5671 - > uwsgi:32419:9b8a323b-653d-4585-9916-d52b3fd81d59): > missed heartbeats from client, timeout: 60s > > on nova-api.log > > 2020-03-24 12:19:06.554 32435 ERROR > oslo.messaging._drivers.impl_rabbit [-] > [4b8adff0-ff9f-4863-a939-537d391e5d9e] AMQP server on > 172.28.15.192:5671 is unreachable: [Errno 104] Connection reset by > peer. Trying again in 1 seconds.: error: [Errno 104] Connection reset > by peer > From aschultz at redhat.com Tue Mar 24 20:04:53 2020 From: aschultz at redhat.com (Alex Schultz) Date: Tue, 24 Mar 2020 14:04:53 -0600 Subject: nova-api missing heartbeats to rabbitmq In-Reply-To: References: Message-ID: On Tue, Mar 24, 2020 at 1:11 PM Satish Patel wrote: > > Folks, > > Recently i am seeing lots of error message in rabbitmq logs saying > missing heartbeats from nova-api nodes, I am not seeing any issue at > functionality level as everything working fine but just noticed those > error and trying to find root cause of it. > > 172.28.15.125 nova-api server > 172.28.15.192 rabbitmq server > > on rabbit.log > > 2020-03-24 12:21:41.389 [error] <0.29772.4418> closing AMQP connection > <0.29772.4418> (172.28.15.125:42656 -> 172.28.15.192:5671 - > uwsgi:32419:9b8a323b-653d-4585-9916-d52b3fd81d59): > missed heartbeats from client, timeout: 60s > > on nova-api.log > > 2020-03-24 12:19:06.554 32435 ERROR > oslo.messaging._drivers.impl_rabbit [-] > [4b8adff0-ff9f-4863-a939-537d391e5d9e] AMQP server on > 172.28.15.192:5671 is unreachable: [Errno 104] Connection reset by > peer. Trying again in 1 seconds.: error: [Errno 104] Connection reset > by peer > ooo I know this one! See this archive thread: http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005305.html https://bugs.launchpad.net/nova/+bug/1825584 Thanks, -Alex From melwittt at gmail.com Tue Mar 24 20:28:54 2020 From: melwittt at gmail.com (melanie witt) Date: Tue, 24 Mar 2020 13:28:54 -0700 Subject: nova-api missing heartbeats to rabbitmq In-Reply-To: References: Message-ID: On 3/24/20 13:04, Alex Schultz wrote: > On Tue, Mar 24, 2020 at 1:11 PM Satish Patel wrote: >> >> Folks, >> >> Recently i am seeing lots of error message in rabbitmq logs saying >> missing heartbeats from nova-api nodes, I am not seeing any issue at >> functionality level as everything working fine but just noticed those >> error and trying to find root cause of it. >> >> 172.28.15.125 nova-api server >> 172.28.15.192 rabbitmq server >> >> on rabbit.log >> >> 2020-03-24 12:21:41.389 [error] <0.29772.4418> closing AMQP connection >> <0.29772.4418> (172.28.15.125:42656 -> 172.28.15.192:5671 - >> uwsgi:32419:9b8a323b-653d-4585-9916-d52b3fd81d59): >> missed heartbeats from client, timeout: 60s >> >> on nova-api.log >> >> 2020-03-24 12:19:06.554 32435 ERROR >> oslo.messaging._drivers.impl_rabbit [-] >> [4b8adff0-ff9f-4863-a939-537d391e5d9e] AMQP server on >> 172.28.15.192:5671 is unreachable: [Errno 104] Connection reset by >> peer. Trying again in 1 seconds.: error: [Errno 104] Connection reset >> by peer >> > > > ooo I know this one! See this archive thread: > http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005305.html > > https://bugs.launchpad.net/nova/+bug/1825584 One more link for the pile: https://docs.openstack.org/releasenotes/nova/stein.html#known-issues This ^ is linked from the launchpad issue but I link it here directly to make it easier. Cheers, -melanie From juliaashleykreger at gmail.com Tue Mar 24 20:28:58 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Tue, 24 Mar 2020 13:28:58 -0700 Subject: [baremetal-sig][ironic] Lets finish the whitepaper doodle In-Reply-To: References: Message-ID: It appears the two ideal times to meet are: Wednesday of this week at 1 PM UTC And Thursday of this week at 11 AM UTC. I'll send meeting invites to the participants that indicated they will be available. We'll use my bluejeans to chat: https://bluejeans.com/u/jkrger or https://bluejeans.com/5548595878 As a reminder, the whitepaper document an be found at: https://docs.google.com/document/d/1BmB2JL_oG3lWXId_NXT9KWcBJjqgtnbmixIcNsfGooA/edit See you there! -Julia On Mon, Mar 23, 2020 at 9:32 AM Julia Kreger wrote: > > Greetings fellow humans! > > During today's ironic meeting, the idea of just getting on a video > call and hammering out the remainder of the baremetal SIG whitepaper > was raised. After some quick discussion in the community, I'm sending > out a doodle[0] for us to find a few time windows that works, where we > can try to gather together and hammer out words, cut/paste/massage > content, and overall just try and get it done. > > In the next 24-48 hours, I'll contact the respondents to and schedule > a call, where I'll also send out call details. Please respond to the > doodle quickly. > > -Julia > > [0] https://doodle.com/poll/k4wnmuay34mvh94v From juliaashleykreger at gmail.com Tue Mar 24 20:30:12 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Tue, 24 Mar 2020 13:30:12 -0700 Subject: [baremetal-sig][ironic] Lets finish the whitepaper doodle In-Reply-To: References: Message-ID: I hit send too quickly, minor URL correction below! On Tue, Mar 24, 2020 at 1:28 PM Julia Kreger wrote: > > We'll use my bluejeans to chat: https://bluejeans.com/u/jkreger or > https://bluejeans.com/5548595878 From juliaashleykreger at gmail.com Tue Mar 24 20:45:40 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Tue, 24 Mar 2020 13:45:40 -0700 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: References: Message-ID: So we seem to be gaining consensus[0] around 5 PM UTC on Friday! Vote now if you've not voted already! -Julia [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 On Mon, Mar 23, 2020 at 10:06 AM Julia Kreger wrote: > > Greetings Humans, Vulcans, AIs, and any other form of life! > > Today in the Ironic team meeting, we decided that we needed to have an > un-conference! > > In the theme of an un-conference, I think we will start with two > simple questions: > * How is everyone? > * What ironic (or non-ironic) things shall we discuss? > > From there, we will identify topics of discussion, and possibly jump > into separate video calls for ~15 minutes each, depending on the > number of topics and general insanity. > > During the last ?15? minutes or so, we shall reconvene into the same > video call to enjoy a little time sipping coffee Coffee, Tea or other > delicious beverages while we determine who is wearing the silliest > hat! So if you have a steampunk hat with gears or even some kind of > hat you would wear to a fancy horse race, go right ahead! Only have a > rose from your growing quarantine garden! You can wear that too! > > So what do you need to do! > 1) Fill out the doodle[0] so we can actually schedule a time that > works for a group of people! > 2) Brainstorm some ideas of things you might want to talk > about/present or LEARN! > 3) Dust off your silly hats! > > NB: I know the windows on this doodle overlap with Baremetal SIG > doodle that I sent out a little while ago. We will figure it out. > > -Julia > > [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 From colleen at gazlene.net Tue Mar 24 20:53:53 2020 From: colleen at gazlene.net (Colleen Murphy) Date: Tue, 24 Mar 2020 13:53:53 -0700 Subject: =?UTF-8?Q?Re:_[keystone][horizon]_Two_approaches_to_"multi-region"_OpenS?= =?UTF-8?Q?tack_deployment?= In-Reply-To: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> Message-ID: <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> Hi Chris, On Tue, Mar 24, 2020, at 06:45, Krzysztof Klimonda wrote: > Hi, > > I’ve been spending some time recently thinking about best approach to > some sort of multi-region deployment of OpenStack clouds. > > The two approaches that I’m currently evaluating are shared keystone > database (with galera cluster spanning three locations) and > shared-nothing approach, where external component is responsible for > managing users, projects etc. > > Shared keystone database seems fairly straightforward from OS point of > view (I’m ignoring galera replication over WAN woes for the purpose of > this discussion) until I hit 3 regions. Additional regions must either > reuse “global” keystone adding latency everywhere, or we need a way to > replicate data from “master” galera cluster to “slave” clusters, and > route all database write statements back to the master galera cluster, > while reading from local asynchronous replica. > > This has me worried somewhat, as doing that that into > eventually-consistent deployment of sort. Services deployed in regions > with asynchronous replication can no longer depend on the fact that > once transaction is finished, consecutive reads will return up-to-date > state. I can imagine scenarios where, as an example, trust is setup for > heat, but that fact is not replicated back to the database by the time > heat tries to issue a token based on that trust and the process fails. > > The other approach would be to keep keystone databases completely > separate, and have something external to openstack manage all those > resources. > > While not sharing keystone database between regions sidesteps the issue > of scalability, and the entire setup seems to be more resilient to > failures, it’s not without its own drawbacks: > > * In this setup Horizon can no longer switch between regions without > additional work (see notes below) > * There is no longer single API entrypoint to the cloud > * Some Keystone API operations would have to be removed from users via > custom policy - for example, managing user assignment to projects (for > users who have domain admin role) > > Additional thoughts > > Could horizon be modified to switch endpoints based on the region > selected in the UI? Is the token reissued when region is changed in > horizon, or is single token used? I’m assuming it’s the former given my > understanding that when projects are changed, a new token is issued - > but perhaps the initial token is always used to issue project-scoped > tokens for Horizon? > > In the second scenario, with separate keystone databases, a backend for > keystone could be created that proxies some operations (like > aforementioned user assignment) back to the external manager so that it > can be propagated to other clouds. Does that even make sense? > > In the end I’m reaching out in hope that someone could chime in based > on their experience - perhaps I’m missing a better approach, or making > wrong assumptions in my email, especially around asynchronous > replication of keystone database and its effect on services in regions > that may not have up-to-data view of the databas. Or perhaps trying ot > synchronize keystone state by external tool is not really worth the > additional effort that would require. > > -Chris > An alternative to either replicating databases or using an external data syncing tool that the keystone team has been pushing is to federate your keystone deployments. With Keystone-to-Keystone federation, keystone instances act as identity providers to one another, and keystone instances are registered as service providers to one another - which allows horizon to recognize the other keystone instances as alternative sites and allow users to log into them ("switch endpoints" and get a new token) without having prior knowledge of them. The data is not replicated between keystone databases but mapping rules allow you to create identical authorization schemes that gives a uniform user experience on each site. More information can be found in our documentation: https://docs.openstack.org/keystone/latest/admin/federation/federated_identity.html Hope this helps as a starting point, happy to answer further questions. Colleen (cmurphy) From eandersson at blizzard.com Tue Mar 24 21:01:06 2020 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Tue, 24 Mar 2020 21:01:06 +0000 Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment In-Reply-To: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> Message-ID: We have a multi-region (20+ regions) deployment with Keystone. We use the old templated ini format for catalog and ldap for user backend. We then create all our projects using matching uuids (we added custom patch to allow you to provide your own uuid for projects). Initially we replicated the database globally, but some newer features (and/or changes) added to Keystone made this difficult. We also do something similar for Glance with a globally replicated Swift cluster. Best Regards, Erik Olof Gunnar Andersson -----Original Message----- From: Krzysztof Klimonda Sent: Tuesday, March 24, 2020 6:46 AM To: openstack-discuss Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment Hi, I’ve been spending some time recently thinking about best approach to some sort of multi-region deployment of OpenStack clouds. The two approaches that I’m currently evaluating are shared keystone database (with galera cluster spanning three locations) and shared-nothing approach, where external component is responsible for managing users, projects etc. Shared keystone database seems fairly straightforward from OS point of view (I’m ignoring galera replication over WAN woes for the purpose of this discussion) until I hit 3 regions. Additional regions must either reuse “global” keystone adding latency everywhere, or we need a way to replicate data from “master” galera cluster to “slave” clusters, and route all database write statements back to the master galera cluster, while reading from local asynchronous replica. This has me worried somewhat, as doing that that into eventually-consistent deployment of sort. Services deployed in regions with asynchronous replication can no longer depend on the fact that once transaction is finished, consecutive reads will return up-to-date state. I can imagine scenarios where, as an example, trust is setup for heat, but that fact is not replicated back to the database by the time heat tries to issue a token based on that trust and the process fails. The other approach would be to keep keystone databases completely separate, and have something external to openstack manage all those resources. While not sharing keystone database between regions sidesteps the issue of scalability, and the entire setup seems to be more resilient to failures, it’s not without its own drawbacks: * In this setup Horizon can no longer switch between regions without additional work (see notes below) * There is no longer single API entrypoint to the cloud * Some Keystone API operations would have to be removed from users via custom policy - for example, managing user assignment to projects (for users who have domain admin role) Additional thoughts Could horizon be modified to switch endpoints based on the region selected in the UI? Is the token reissued when region is changed in horizon, or is single token used? I’m assuming it’s the former given my understanding that when projects are changed, a new token is issued - but perhaps the initial token is always used to issue project-scoped tokens for Horizon? In the second scenario, with separate keystone databases, a backend for keystone could be created that proxies some operations (like aforementioned user assignment) back to the external manager so that it can be propagated to other clouds. Does that even make sense? In the end I’m reaching out in hope that someone could chime in based on their experience - perhaps I’m missing a better approach, or making wrong assumptions in my email, especially around asynchronous replication of keystone database and its effect on services in regions that may not have up-to-data view of the databas. Or perhaps trying ot synchronize keystone state by external tool is not really worth the additional effort that would require. -Chris From sunny at openstack.org Tue Mar 24 21:08:50 2020 From: sunny at openstack.org (Sunny Cai) Date: Tue, 24 Mar 2020 14:08:50 -0700 Subject: 10 Years of OpenStack Survey Message-ID: Hello Stackers! To celebrate 10 years of OpenStack, we are launching a campaign that not only celebrates the community’s achievement in the past 10 years, but also showcases the top 10 favorite things about OpenStack from our community members. We want to hear from you about what your top 10 favorite things related to OpenStack are! Go to this survey and choose one or more topics to answer. The topics range from your top 10 most memorable moments of OpenStack, your top 10 most used features in OpenStack to your top 10 favorite cities you visited for OpenStack. Top 10 favorites of OpenStack survey link: https://openstackfoundation.formstack.com/forms/10_years_of_openstack Please also share it to your networks including folks who were around at the beginning of OpenStack. We are looking forward to hearing your favorites, and we invite you all to join us and celebrate 10 awesome years of OpenStack! Thanks, Sunny Cai OpenStack Foundation sunny at openstack.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From soulxu at gmail.com Tue Mar 24 23:51:58 2020 From: soulxu at gmail.com (Alex Xu) Date: Wed, 25 Mar 2020 07:51:58 +0800 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: +1 Sylvain Bauza 于2020年3月24日周二 下午5:28写道： > > > On Mon, Mar 23, 2020 at 3:37 PM Balázs Gibizer > wrote: > >> Hi, >> >> I'm proposing Lee to the core team. He is around for a long time and he >> became really active recently not only on the stable branches. I think >> he would be a great addition to the team. >> >> Cores, please vote (-1, 0, +1) before next Monday (03.30.) >> >> > +1 > > Cheers, >> gibi >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From soulxu at gmail.com Tue Mar 24 23:52:11 2020 From: soulxu at gmail.com (Alex Xu) Date: Wed, 25 Mar 2020 07:52:11 +0800 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: <012990b86b48e568383b63835617e92c96f0d2be.camel@redhat.com> References: <012990b86b48e568383b63835617e92c96f0d2be.camel@redhat.com> Message-ID: +1 Stephen Finucane 于2020年3月24日周二 下午6:19写道： > On Mon, 2020-03-23 at 17:34 +0100, Balázs Gibizer wrote: > > Hi, > > > > I'm proposing Ghanshyam Mann to the core team. He is a long time > > OpenStack and nova contributor. He is one of our API and QA expert. I > > think he would be a great addition to the team. > > > > Cores, please vote (-1, 0, +1) before next Monday (03.30.) > > +1 > > > Cheers, > > gibi > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Wed Mar 25 08:32:23 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Wed, 25 Mar 2020 09:32:23 +0100 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: Hi, I actually have, it's just spread over the document :) And somebody (Julia?) has already added more information. Dmitry On Tue, Mar 24, 2020 at 8:20 PM Kendall Nelson wrote: > Hey Dmitry :) > > Would you mind adding that to the etherpad so we have all the info in one > place? I looked and didn't think I saw Ironic's testimony in there and it > would be super helpful! > > -Kendall (diablo_rojo) > > On Wed, Mar 11, 2020 at 7:16 AM Dmitry Tantsur > wrote: > >> Hi, >> >> Ironic did, I think, 3-4 virtual midcycles, and I think they were quite >> successful. >> The most positive outcome was to reach out to folks who usually cannot >> travel. They really appreciated that. >> >> We used SIP the first couple of times, but it caused problems for a big >> share of participants. Also lack of moderation was problematic. >> We switched to Bluejeans later, and that was, IMO, a big improvement: >> 1) A participant list with information who is speaking. >> 2) An option for a moderator to mute a person or everyone. >> 3) Screen sharing. >> >> Dmitry >> >> On Tue, Mar 10, 2020 at 1:29 AM Kendall Nelson >> wrote: >> >>> Hello Everyone! >>> >>> I wanted to collect best practices and pitfalls to avoid wrt projects >>> experiences with virtual midcycles. I know of a few projects that have done >>> them in the past and with how travel is hard for a lot of people right now, >>> I expect more projects to have midcycles. I think it would be helpful to >>> have all of the data we can collect in one place for those not just new to >>> virtual midcycles but the whole community. >>> >>> I threw some categories into this etherpad[1] and filled in some >>> options. Please add to it :) >>> >>> -Kendall (diablo_rojo) >>> >>> [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kklimonda at syntaxhighlighted.com Wed Mar 25 08:49:08 2020 From: kklimonda at syntaxhighlighted.com (Krzysztof Klimonda) Date: Wed, 25 Mar 2020 09:49:08 +0100 Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment In-Reply-To: <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> Message-ID: Hi Colleen, Interesting - I’ve looked at K2K federation initially, but it didn’t seem to fit my requirements. My understanding is that in this setup we have a global keystone (which I’ve described in my initial post) and then all regions are federated to that keystone. Is that correct? Would that work with an existing SSO that we have in place (Keycloak) or does global keystone has to be IdP and not chain to another one? -Chris > On 24 Mar 2020, at 21:53, Colleen Murphy wrote: > > Hi Chris, > > On Tue, Mar 24, 2020, at 06:45, Krzysztof Klimonda wrote: >> Hi, >> >> I’ve been spending some time recently thinking about best approach to >> some sort of multi-region deployment of OpenStack clouds. >> >> The two approaches that I’m currently evaluating are shared keystone >> database (with galera cluster spanning three locations) and >> shared-nothing approach, where external component is responsible for >> managing users, projects etc. >> >> Shared keystone database seems fairly straightforward from OS point of >> view (I’m ignoring galera replication over WAN woes for the purpose of >> this discussion) until I hit 3 regions. Additional regions must either >> reuse “global” keystone adding latency everywhere, or we need a way to >> replicate data from “master” galera cluster to “slave” clusters, and >> route all database write statements back to the master galera cluster, >> while reading from local asynchronous replica. >> >> This has me worried somewhat, as doing that that into >> eventually-consistent deployment of sort. Services deployed in regions >> with asynchronous replication can no longer depend on the fact that >> once transaction is finished, consecutive reads will return up-to-date >> state. I can imagine scenarios where, as an example, trust is setup for >> heat, but that fact is not replicated back to the database by the time >> heat tries to issue a token based on that trust and the process fails. >> >> The other approach would be to keep keystone databases completely >> separate, and have something external to openstack manage all those >> resources. >> >> While not sharing keystone database between regions sidesteps the issue >> of scalability, and the entire setup seems to be more resilient to >> failures, it’s not without its own drawbacks: >> >> * In this setup Horizon can no longer switch between regions without >> additional work (see notes below) >> * There is no longer single API entrypoint to the cloud >> * Some Keystone API operations would have to be removed from users via >> custom policy - for example, managing user assignment to projects (for >> users who have domain admin role) >> >> Additional thoughts >> >> Could horizon be modified to switch endpoints based on the region >> selected in the UI? Is the token reissued when region is changed in >> horizon, or is single token used? I’m assuming it’s the former given my >> understanding that when projects are changed, a new token is issued - >> but perhaps the initial token is always used to issue project-scoped >> tokens for Horizon? >> >> In the second scenario, with separate keystone databases, a backend for >> keystone could be created that proxies some operations (like >> aforementioned user assignment) back to the external manager so that it >> can be propagated to other clouds. Does that even make sense? >> >> In the end I’m reaching out in hope that someone could chime in based >> on their experience - perhaps I’m missing a better approach, or making >> wrong assumptions in my email, especially around asynchronous >> replication of keystone database and its effect on services in regions >> that may not have up-to-data view of the databas. Or perhaps trying ot >> synchronize keystone state by external tool is not really worth the >> additional effort that would require. >> >> -Chris >> > > An alternative to either replicating databases or using an external data syncing tool that the keystone team has been pushing is to federate your keystone deployments. With Keystone-to-Keystone federation, keystone instances act as identity providers to one another, and keystone instances are registered as service providers to one another - which allows horizon to recognize the other keystone instances as alternative sites and allow users to log into them ("switch endpoints" and get a new token) without having prior knowledge of them. The data is not replicated between keystone databases but mapping rules allow you to create identical authorization schemes that gives a uniform user experience on each site. > > More information can be found in our documentation: > > https://docs.openstack.org/keystone/latest/admin/federation/federated_identity.html > > Hope this helps as a starting point, happy to answer further questions. > > Colleen (cmurphy) > From mark at stackhpc.com Wed Mar 25 09:27:27 2020 From: mark at stackhpc.com (Mark Goddard) Date: Wed, 25 Mar 2020 09:27:27 +0000 Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment In-Reply-To: References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> Message-ID: On Wed, 25 Mar 2020 at 08:49, Krzysztof Klimonda wrote: > > Hi Colleen, > > Interesting - I’ve looked at K2K federation initially, but it didn’t seem to fit my requirements. > > My understanding is that in this setup we have a global keystone (which I’ve described in my initial post) > and then all regions are federated to that keystone. Is that correct? Would that work with an existing > SSO that we have in place (Keycloak) or does global keystone has to be IdP and not chain to another one? Hi Chris, You might be interested to read our blog on federated Keystone using Keycloak: https://www.stackhpc.com/federation-and-identity-brokering-using-keycloak.html. Mark > > -Chris > > > On 24 Mar 2020, at 21:53, Colleen Murphy wrote: > > > > Hi Chris, > > > > On Tue, Mar 24, 2020, at 06:45, Krzysztof Klimonda wrote: > >> Hi, > >> > >> I’ve been spending some time recently thinking about best approach to > >> some sort of multi-region deployment of OpenStack clouds. > >> > >> The two approaches that I’m currently evaluating are shared keystone > >> database (with galera cluster spanning three locations) and > >> shared-nothing approach, where external component is responsible for > >> managing users, projects etc. > >> > >> Shared keystone database seems fairly straightforward from OS point of > >> view (I’m ignoring galera replication over WAN woes for the purpose of > >> this discussion) until I hit 3 regions. Additional regions must either > >> reuse “global” keystone adding latency everywhere, or we need a way to > >> replicate data from “master” galera cluster to “slave” clusters, and > >> route all database write statements back to the master galera cluster, > >> while reading from local asynchronous replica. > >> > >> This has me worried somewhat, as doing that that into > >> eventually-consistent deployment of sort. Services deployed in regions > >> with asynchronous replication can no longer depend on the fact that > >> once transaction is finished, consecutive reads will return up-to-date > >> state. I can imagine scenarios where, as an example, trust is setup for > >> heat, but that fact is not replicated back to the database by the time > >> heat tries to issue a token based on that trust and the process fails. > >> > >> The other approach would be to keep keystone databases completely > >> separate, and have something external to openstack manage all those > >> resources. > >> > >> While not sharing keystone database between regions sidesteps the issue > >> of scalability, and the entire setup seems to be more resilient to > >> failures, it’s not without its own drawbacks: > >> > >> * In this setup Horizon can no longer switch between regions without > >> additional work (see notes below) > >> * There is no longer single API entrypoint to the cloud > >> * Some Keystone API operations would have to be removed from users via > >> custom policy - for example, managing user assignment to projects (for > >> users who have domain admin role) > >> > >> Additional thoughts > >> > >> Could horizon be modified to switch endpoints based on the region > >> selected in the UI? Is the token reissued when region is changed in > >> horizon, or is single token used? I’m assuming it’s the former given my > >> understanding that when projects are changed, a new token is issued - > >> but perhaps the initial token is always used to issue project-scoped > >> tokens for Horizon? > >> > >> In the second scenario, with separate keystone databases, a backend for > >> keystone could be created that proxies some operations (like > >> aforementioned user assignment) back to the external manager so that it > >> can be propagated to other clouds. Does that even make sense? > >> > >> In the end I’m reaching out in hope that someone could chime in based > >> on their experience - perhaps I’m missing a better approach, or making > >> wrong assumptions in my email, especially around asynchronous > >> replication of keystone database and its effect on services in regions > >> that may not have up-to-data view of the databas. Or perhaps trying ot > >> synchronize keystone state by external tool is not really worth the > >> additional effort that would require. > >> > >> -Chris > >> > > > > An alternative to either replicating databases or using an external data syncing tool that the keystone team has been pushing is to federate your keystone deployments. With Keystone-to-Keystone federation, keystone instances act as identity providers to one another, and keystone instances are registered as service providers to one another - which allows horizon to recognize the other keystone instances as alternative sites and allow users to log into them ("switch endpoints" and get a new token) without having prior knowledge of them. The data is not replicated between keystone databases but mapping rules allow you to create identical authorization schemes that gives a uniform user experience on each site. > > > > More information can be found in our documentation: > > > > https://docs.openstack.org/keystone/latest/admin/federation/federated_identity.html > > > > Hope this helps as a starting point, happy to answer further questions. > > > > Colleen (cmurphy) > > > > From kklimonda at syntaxhighlighted.com Wed Mar 25 11:00:47 2020 From: kklimonda at syntaxhighlighted.com (Krzysztof Klimonda) Date: Wed, 25 Mar 2020 12:00:47 +0100 Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment In-Reply-To: References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> Message-ID: Hi Mark, Colleen, @Mark Thanks, I remember reading that blog post already :). Just to make sure, in the deployment described in the post you federate keystone with keycloak, but without K2K federation in the middle, right? Also, in the linked blog post there is no mention of regions, so my understanding is that this is a single keycloak->keystone federation. On a related note, in your blog post you show “chained federation” in Keycloak. Does this setup support generating Keystone tokens for API/CLI access? If so, how does the authentication flow look like? Is keystone federated with keycloak using OIDC, and `openstack token issue` opens a web browser to allow for selecting IdP configured in keycloak? “Chained Federation” didn’t seem to work with SAML2 + ECP profile. @Colleen I’m trying to visualise your proposed solution, does this diagram make sense: http://paste.openstack.org/show/791120/ ? In K2K federation, how are regions handled? Are they out of the picture, with federated keystone taking their place instead? I assume all keystone catalogs are separate, and users must use specific cloud auth url to authenticate. Authentication can be then delegated to central keystone, but what about another layer of SSO, i.e. Keycloak? -Chris > On 25 Mar 2020, at 10:27, Mark Goddard wrote: > > On Wed, 25 Mar 2020 at 08:49, Krzysztof Klimonda > wrote: >> >> Hi Colleen, >> >> Interesting - I’ve looked at K2K federation initially, but it didn’t seem to fit my requirements. >> >> My understanding is that in this setup we have a global keystone (which I’ve described in my initial post) >> and then all regions are federated to that keystone. Is that correct? Would that work with an existing >> SSO that we have in place (Keycloak) or does global keystone has to be IdP and not chain to another one? > > Hi Chris, > > You might be interested to read our blog on federated Keystone using > Keycloak: https://www.stackhpc.com/federation-and-identity-brokering-using-keycloak.html. > > Mark > >> >> -Chris >> >>> On 24 Mar 2020, at 21:53, Colleen Murphy wrote: >>> >>> Hi Chris, >>> >>> On Tue, Mar 24, 2020, at 06:45, Krzysztof Klimonda wrote: >>>> Hi, >>>> >>>> I’ve been spending some time recently thinking about best approach to >>>> some sort of multi-region deployment of OpenStack clouds. >>>> >>>> The two approaches that I’m currently evaluating are shared keystone >>>> database (with galera cluster spanning three locations) and >>>> shared-nothing approach, where external component is responsible for >>>> managing users, projects etc. >>>> >>>> Shared keystone database seems fairly straightforward from OS point of >>>> view (I’m ignoring galera replication over WAN woes for the purpose of >>>> this discussion) until I hit 3 regions. Additional regions must either >>>> reuse “global” keystone adding latency everywhere, or we need a way to >>>> replicate data from “master” galera cluster to “slave” clusters, and >>>> route all database write statements back to the master galera cluster, >>>> while reading from local asynchronous replica. >>>> >>>> This has me worried somewhat, as doing that that into >>>> eventually-consistent deployment of sort. Services deployed in regions >>>> with asynchronous replication can no longer depend on the fact that >>>> once transaction is finished, consecutive reads will return up-to-date >>>> state. I can imagine scenarios where, as an example, trust is setup for >>>> heat, but that fact is not replicated back to the database by the time >>>> heat tries to issue a token based on that trust and the process fails. >>>> >>>> The other approach would be to keep keystone databases completely >>>> separate, and have something external to openstack manage all those >>>> resources. >>>> >>>> While not sharing keystone database between regions sidesteps the issue >>>> of scalability, and the entire setup seems to be more resilient to >>>> failures, it’s not without its own drawbacks: >>>> >>>> * In this setup Horizon can no longer switch between regions without >>>> additional work (see notes below) >>>> * There is no longer single API entrypoint to the cloud >>>> * Some Keystone API operations would have to be removed from users via >>>> custom policy - for example, managing user assignment to projects (for >>>> users who have domain admin role) >>>> >>>> Additional thoughts >>>> >>>> Could horizon be modified to switch endpoints based on the region >>>> selected in the UI? Is the token reissued when region is changed in >>>> horizon, or is single token used? I’m assuming it’s the former given my >>>> understanding that when projects are changed, a new token is issued - >>>> but perhaps the initial token is always used to issue project-scoped >>>> tokens for Horizon? >>>> >>>> In the second scenario, with separate keystone databases, a backend for >>>> keystone could be created that proxies some operations (like >>>> aforementioned user assignment) back to the external manager so that it >>>> can be propagated to other clouds. Does that even make sense? >>>> >>>> In the end I’m reaching out in hope that someone could chime in based >>>> on their experience - perhaps I’m missing a better approach, or making >>>> wrong assumptions in my email, especially around asynchronous >>>> replication of keystone database and its effect on services in regions >>>> that may not have up-to-data view of the databas. Or perhaps trying ot >>>> synchronize keystone state by external tool is not really worth the >>>> additional effort that would require. >>>> >>>> -Chris >>>> >>> >>> An alternative to either replicating databases or using an external data syncing tool that the keystone team has been pushing is to federate your keystone deployments. With Keystone-to-Keystone federation, keystone instances act as identity providers to one another, and keystone instances are registered as service providers to one another - which allows horizon to recognize the other keystone instances as alternative sites and allow users to log into them ("switch endpoints" and get a new token) without having prior knowledge of them. The data is not replicated between keystone databases but mapping rules allow you to create identical authorization schemes that gives a uniform user experience on each site. >>> >>> More information can be found in our documentation: >>> >>> https://docs.openstack.org/keystone/latest/admin/federation/federated_identity.html >>> >>> Hope this helps as a starting point, happy to answer further questions. >>> >>> Colleen (cmurphy) >>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Wed Mar 25 11:14:57 2020 From: zigo at debian.org (Thomas Goirand) Date: Wed, 25 Mar 2020 12:14:57 +0100 Subject: [oslo][nova][vmware] Replacement for suds library In-Reply-To: References: Message-ID: On 3/20/20 3:04 PM, Stephen Finucane wrote: > The suds-jurko library used by oslo.vmware is emitting the following > warnings in nova tests. > > /nova/.tox/py36/lib/python3.6/site-packages/suds/resolver.py:89: DeprecationWarning: invalid escape sequence \% > self.splitp = re.compile('({.+})*[^\%s]+' % ps[0]) > /nova/.tox/py36/lib/python3.6/site-packages/suds/wsdl.py:619: DeprecationWarning: invalid escape sequence \s > body.parts = re.split('[\s,]', parts) > > These warnings are going to be errors in Python 3.10 [1]. We have over > 18 months before we need to worry about this [2], but I'd like to see > some movement on this well before then. It seems the suds-jurko fork is > dead [3] and movement on yet another fork, suds-community, is slow [4]. > How difficult would it be to switch over to something that does seem > maintained like zeep [5] and, assuming it's doable, is anyone from > VMWare/SAP able to do this work? > > Stephen Hi Stephen, This IMO should be coordinated with Cinder, who also uses suds. Please make sure that both projects are moving on the same direction, using the same library for these XML stuff. It's been a long time I've been saying we should get rid of that one library which has historically been very annoying. I'm very happy to finally see movement! Cheers, Thomas Goirand (zigo) From thierry at openstack.org Wed Mar 25 11:41:01 2020 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 25 Mar 2020 12:41:01 +0100 Subject: [largescale-sig] Next meeting: Mar 25, 9utc In-Reply-To: <7e2bb4ea-0e50-d92d-10a3-9e1d51862a1d@openstack.org> References: <7e2bb4ea-0e50-d92d-10a3-9e1d51862a1d@openstack.org> Message-ID: <72b5b85d-ac5e-e14e-c6b1-18186e98b3d9@openstack.org> Thierry Carrez wrote: > The Large Scale SIG will have a meeting this week on Wednesday, Mar 25 > at 9 UTC[1] in #openstack-meeting on IRC: > [...] Meeting logs at: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-03-25-09.00.html TODOs: ttx to push for oslo.metric spec approval oneswig to contribute a scaling story on bare metal cluster scaling amorin to create a wiki page for large scale documentation amorin to propose patch against Nova doc Next meeting: April 8, 8:00UTC -- Thierry Carrez (ttx) From sean.mcginnis at gmx.com Wed Mar 25 12:58:42 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 25 Mar 2020 07:58:42 -0500 Subject: [oslo][nova][vmware] Replacement for suds library In-Reply-To: References: Message-ID: <08538812-3634-0952-65e9-35606589ca76@gmx.com> On 3/25/20 6:14 AM, Thomas Goirand wrote: > On 3/20/20 3:04 PM, Stephen Finucane wrote: >> The suds-jurko library used by oslo.vmware is emitting the following >> warnings in nova tests. >> >> /nova/.tox/py36/lib/python3.6/site-packages/suds/resolver.py:89: DeprecationWarning: invalid escape sequence \% >> self.splitp = re.compile('({.+})*[^\%s]+' % ps[0]) >> /nova/.tox/py36/lib/python3.6/site-packages/suds/wsdl.py:619: DeprecationWarning: invalid escape sequence \s >> body.parts = re.split('[\s,]', parts) >> >> These warnings are going to be errors in Python 3.10 [1]. We have over >> 18 months before we need to worry about this [2], but I'd like to see >> some movement on this well before then. It seems the suds-jurko fork is >> dead [3] and movement on yet another fork, suds-community, is slow [4]. >> How difficult would it be to switch over to something that does seem >> maintained like zeep [5] and, assuming it's doable, is anyone from >> VMWare/SAP able to do this work? >> >> Stephen > Hi Stephen, > > This IMO should be coordinated with Cinder, who also uses suds. Please > make sure that both projects are moving on the same direction, using the > same library for these XML stuff. > > It's been a long time I've been saying we should get rid of that one > library which has historically been very annoying. I'm very happy to > finally see movement! > > Cheers, > > Thomas Goirand (zigo) Cinder does not use the suds-jurko library, but it was still in the requirements.txt and lower-constraints.txt files. This was a leftover from a couple of drivers that are no longer in the Cinder tree. https://review.opendev.org/714935 has been proposed to clean that up. Sean From thierry at openstack.org Wed Mar 25 13:46:45 2020 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 25 Mar 2020 14:46:45 +0100 Subject: [Release-job-failures] Release of openstack/python-openstackclient for ref refs/tags/5.1.0 failed In-Reply-To: References: Message-ID: zuul at openstack.org wrote: > Build failed. > > - release-openstack-python https://zuul.opendev.org/t/openstack/build/4519cd03309c4b5e834a9089183f6b3b : SUCCESS in 4m 00s > - announce-release https://zuul.opendev.org/t/openstack/build/7a30f87bded640f0a1f6c418d1dcce66 : FAILURE in 4m 34s > - propose-update-constraints https://zuul.opendev.org/t/openstack/build/eb428894cf24407a9991d22f40de98d7 : SUCCESS in 4m 25s During python-openstackclient 5.1.0 release: Error while sending announcement email: 'Temporary local problem - please try later' It's not the first time this happens, so maybe we should have a retry in there. -- Thierry Carrez (ttx) From rosmaita.fossdev at gmail.com Wed Mar 25 13:56:20 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 25 Mar 2020 09:56:20 -0400 Subject: [cinder][qa] propose Liang Fang for devstack-plugin-open-cas-core In-Reply-To: References: Message-ID: <01d1671c-24d5-6b74-ca97-cb0414e072e0@gmail.com> On 3/20/20 10:15 AM, Brian Rosmaita wrote: > Liang Fang is driving the effort to create the devstack-plugin-open-cas > and use it to test the cinder/nova volume-local-cache feature with > tempest.  Since he's our SME on open-cas, I propose that he be made a > member of devstack-plugin-open-cas-core.  (Currently, the only members > of that group are the members of cinder-core and devstack-core.) > > In the absence of objections, I'll make the change before the next > Cinder team meeting (Wednesday, 25 March at 1400 UTC in its new location > of #openstack-meeting-alt). Having heard only positive responses, I've added Liang Fang to the devstack-plugin-open-cas-core team, with all the privileges and responsibilities pertaining thereto. Congratulations, Liang! > > > cheers, > brian From marie.delavergne at inria.fr Wed Mar 25 14:26:48 2020 From: marie.delavergne at inria.fr (Marie Delavergne) Date: Wed, 25 Mar 2020 15:26:48 +0100 (CET) Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment In-Reply-To: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> Message-ID: <1012518827.19687955.1585146408345.JavaMail.zimbra@inria.fr> Hi Chris, Following your email, I think it makes sense to mention the activities we have been doing initially within the FEMDC SiG [1] and more recently under the Edge Computing WG managed by the OSF., To make a long story short, we [2] tried both approaches you mentionned. Regarding the shared database first, we investigated and compared the performance of Galera and CockroachDB[3]. You can refer to [4][5] for futher details. Regarding the second approach you talked about, It looks rather similar to the OID prototype we introduced during the Berlin Hackathon/summit [6] and we will be more than happy to get your feedback on it. Briefly, our mechanism is larger than delivering collaborations/federations between mutiple instances of keystone, larger in the sense that it can be applied to any openstack service. Considering several OpenStacks running, the idea is to use a DSL to specify which instance of the service (i.e. on which Openstack) the user wants its request to be executed. For instance, to provision a VM on a Nova in Paris using an image from a Glance in New York. You may contact the API either in NewYork or in Paris and perform the following request: "openstack server create --image debian --scope { compute: Paris, image: New York }" A proof of concept is available on GitLab[7]. Obviously, our approach needs to be challenged and additional actions should be done to deliver a production ready system. For instance, we should define a clear semantic for the DSL. This is critical as we are extending the DSL with operators such as AND and OR. Using operators will enable Admin/DevOps to perform requests such as : list all VMs running in Paris, NewYork etc : "openstack server list --scope{compute: Paris & Newyork & Boston}" An academic paper presenting the concept overall has been submitted to the HotEdge topic. We don't know yet whether the paper will be accepted or not (due to the COVID-19 everything is unfortunately delayed). However, we hope to share it ASAP. Feedbacks/questions/comments welcome :-) Have a nice day. [1]https://wiki.openstack.org/wiki/Fog_Edge_Massively_Distributed_Clouds [2] https://beyondtheclouds.github.io/ [3] https://www.cockroachlabs.com/ [4] https://beyondtheclouds.github.io/blog/openstack/cockroachdb/2018/06/04/evaluation-of-openstack-multi-region-keystone-deployments.html This article was never entirely finished because we lack time, but you can find every results we had there. Note also that CockroachDB team did some amazing work to get better perfomance since, so the results would probably be better now. [5] https://www.openstack.org/videos/summits/vancouver-2018/keystone-in-the-context-of-fogedge-massively-distributed-clouds [6] https://www.openstack.org/summit/denver-2019/summit-schedule/events/23352/implementing-localization-into-openstack-cli-for-a-free-collaboration-of-edge-openstack-clouds [7] https://gitlab.inria.fr/discovery/openstackoid Marie Delavergne ----- Mail original ----- > De: "Krzysztof Klimonda" > À: "openstack-discuss" > Envoyé: Mardi 24 Mars 2020 14:45:42 > Objet: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment > Hi, > > I’ve been spending some time recently thinking about best approach to some sort > of multi-region deployment of OpenStack clouds. > > The two approaches that I’m currently evaluating are shared keystone database > (with galera cluster spanning three locations) and shared-nothing approach, > where external component is responsible for managing users, projects etc. > > Shared keystone database seems fairly straightforward from OS point of view (I’m > ignoring galera replication over WAN woes for the purpose of this discussion) > until I hit 3 regions. Additional regions must either reuse “global” keystone > adding latency everywhere, or we need a way to replicate data from “master” > galera cluster to “slave” clusters, and route all database write statements > back to the master galera cluster, while reading from local asynchronous > replica. > > This has me worried somewhat, as doing that that into eventually-consistent > deployment of sort. Services deployed in regions with asynchronous replication > can no longer depend on the fact that once transaction is finished, consecutive > reads will return up-to-date state. I can imagine scenarios where, as an > example, trust is setup for heat, but that fact is not replicated back to the > database by the time heat tries to issue a token based on that trust and the > process fails. > > The other approach would be to keep keystone databases completely separate, and > have something external to openstack manage all those resources. > > While not sharing keystone database between regions sidesteps the issue of > scalability, and the entire setup seems to be more resilient to failures, it’s > not without its own drawbacks: > > * In this setup Horizon can no longer switch between regions without additional > work (see notes below) > * There is no longer single API entrypoint to the cloud > * Some Keystone API operations would have to be removed from users via custom > policy - for example, managing user assignment to projects (for users who have > domain admin role) > > Additional thoughts > > Could horizon be modified to switch endpoints based on the region selected in > the UI? Is the token reissued when region is changed in horizon, or is single > token used? I’m assuming it’s the former given my understanding that when > projects are changed, a new token is issued - but perhaps the initial token is > always used to issue project-scoped tokens for Horizon? > > In the second scenario, with separate keystone databases, a backend for keystone > could be created that proxies some operations (like aforementioned user > assignment) back to the external manager so that it can be propagated to other > clouds. Does that even make sense? > > In the end I’m reaching out in hope that someone could chime in based on their > experience - perhaps I’m missing a better approach, or making wrong assumptions > in my email, especially around asynchronous replication of keystone database > and its effect on services in regions that may not have up-to-data view of the > databas. Or perhaps trying ot synchronize keystone state by external tool is > not really worth the additional effort that would require. > > -Chris From mnaser at vexxhost.com Wed Mar 25 15:27:51 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 25 Mar 2020 11:27:51 -0400 Subject: [openstack-ansible] proposing Erik Berg to openstack-ansible core Message-ID: Hi everyone, I'd like to propose adding Erik Berg to the OSA core team. They've been doing solid contributions with some activity in reviews and I think they'd be an excellent addition to the OSA team If no one opposes, I'll be adding them to the core team in the next few days. Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From amy at demarco.com Wed Mar 25 15:34:21 2020 From: amy at demarco.com (Amy Marrich) Date: Wed, 25 Mar 2020 10:34:21 -0500 Subject: [openstack-ansible] proposing Erik Berg to openstack-ansible core In-Reply-To: References: Message-ID: +2 from me Amy (spotz) On Wed, Mar 25, 2020 at 10:31 AM Mohammed Naser wrote: > Hi everyone, > > I'd like to propose adding Erik Berg to the OSA core team. They've > been doing solid contributions with some activity in reviews and I > think they'd be an excellent addition to the OSA team > > If no one opposes, I'll be adding them to the core team in the next few > days. > > Regards, > Mohammed > > -- > Mohammed Naser — vexxhost > ----------------------------------------------------- > D. 514-316-8872 > D. 800-910-1726 ext. 200 > E. mnaser at vexxhost.com > W. https://vexxhost.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gsteinmuller at vexxhost.com Wed Mar 25 15:39:09 2020 From: gsteinmuller at vexxhost.com (=?UTF-8?Q?Guilherme_Steinm=C3=BCller?=) Date: Wed, 25 Mar 2020 12:39:09 -0300 Subject: [openstack-ansible] proposing Erik Berg to openstack-ansible core In-Reply-To: References: Message-ID: +1 Welcome Erik! Best Regards, Guilherme On Wed, Mar 25, 2020 at 12:33 PM Mohammed Naser wrote: > Hi everyone, > > I'd like to propose adding Erik Berg to the OSA core team. They've > been doing solid contributions with some activity in reviews and I > think they'd be an excellent addition to the OSA team > > If no one opposes, I'll be adding them to the core team in the next few > days. > > Regards, > Mohammed > > -- > Mohammed Naser — vexxhost > ----------------------------------------------------- > D. 514-316-8872 > D. 800-910-1726 ext. 200 > E. mnaser at vexxhost.com > W. https://vexxhost.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.rosser at rd.bbc.co.uk Wed Mar 25 15:45:39 2020 From: jonathan.rosser at rd.bbc.co.uk (Jonathan Rosser) Date: Wed, 25 Mar 2020 15:45:39 +0000 Subject: [openstack-ansible] proposing Erik Berg to openstack-ansible core In-Reply-To: References: Message-ID: <0d12ff74-6d7e-bbcf-dbd0-dfd8f7aa0402@rd.bbc.co.uk> +1 from me. Jon. On 25/03/2020 15:27, Mohammed Naser wrote: > Hi everyone, > > I'd like to propose adding Erik Berg to the OSA core team. They've > been doing solid contributions with some activity in reviews and I > think they'd be an excellent addition to the OSA team > > If no one opposes, I'll be adding them to the core team in the next few days. > > Regards, > Mohammed > From rico.lin.guanyu at gmail.com Wed Mar 25 15:51:15 2020 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Wed, 25 Mar 2020 23:51:15 +0800 Subject: [heat][magnum][tripleo] Official build and test plan for heat container agent In-Reply-To: References: Message-ID: For some back group on what's heat container agents here, We have heat agent hooks [1] which design to be used with os-collect-config to and os-apply-config, to keep watching metadata server and execute any received commands with heat agents. If you build with diskimage-builder, you will generate an image with heat agents and services installed inside. But this will force operators to use large images to run for any application deployments and also add limitations to the application environment (you need also able to install all services above). For that, heat container agents seem to be a better approach if we don't want the above limitations. The concept is simple, just put every service into a container, so whatever the OS operator pick, as long as it supports running containers, it should work. Once we have our container image built, we can deploy/config application after the container with heat agents started in Nova instance. [1] https://opendev.org/openstack/heat-agents On Fri, Mar 13, 2020 at 11:46 AM Rico Lin wrote: > Dear all > > tl;dr: > Shall we build, publish, and test heat container agents on heat-agents > directly?:) > Please help to review > https://review.opendev.org/#/q/topic:build-container-agent+(status:open) > > Current I think we encounter with following issues: > ** No functional/scenario test for heat-agents but only unit tests* > Right now, whatever patch people send up in heat-agents, we only vote > against unit tests. This makes no sense for long term maintenance. Which > makes me thinking of the second issue. > ** No enable scenario tests for software deployment:* > We have tests defined, but since we didn't build any customized image. We > didn't enable it for a long time. To build a container image for > heat-agents (aka heat container agent) is (IMO) the best way for us to > enable these teats. Which we can also use it to cross gating on heat and > heat-agents (or even heat-template). The no much to change for the test > itself but only update the config script to enable the container. > ** Can't guarantee cross-project usage:* > We currently provide no guarantee for any usage of heat agents from other > projects, especially Magnum and TripleO (I didn't personally check if > TripleO still using it, but since they still have experiment CI running) > who uses heat container agents. It will be nice if we provide basic images > for other teams can build their additional requirements. > > To resolve the above issues, I propose [1]. Which a lot of logic was > copied from the current magnum heat container agent scripts [2] so we can > start with a higher possibility to allow magnum to reuse our works. > > For a more detail approach in [1]. The patch defines a build job in > `check` pipeline. And a publish job in `post-check`. Uses zuul secrets to > save docker authN information. As for the account, I created a new > `openstackheat` account [3] for it (I specifically recall there ware > discussion about having an official image hub for OpenStack, but not quite > sure what happened since). > The current approach also can support multiple architectures (like amd64 > arm arm64 ppc64le s390x, etc). But for now, I think we should have tested > before we publish for other architectures. > > You can run the build or publish job easily by running `make > build-images-all-archs` to build images for all architectures or `make > upload-images-all-archs` to build and publish images for all architectures. > > [1] > https://review.opendev.org/#/q/topic:build-container-agent+(status:open) > [2] > https://opendev.org/openstack/magnum/src/branch/master/dockerfiles/heat-container-agent > [3] https://hub.docker.com/r/openstackheat > -- > May The Force of OpenStack Be With You, > > *Rico Lin*irc: ricolin > > -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From whayutin at redhat.com Wed Mar 25 18:10:56 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Wed, 25 Mar 2020 12:10:56 -0600 Subject: [tripleo] rdo registry migration Message-ID: Greetings, FYI.. the rdo registry is migrating from RDO to Vexxhost. You may find some of your jobs hitting "retry_limit" [1] This was caused by a default to log into registries for jobs running in rdo. The default has been updated. Jobs should return to their "GREENISH" status shortly [1] https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset001&job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Wed Mar 25 18:50:39 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 25 Mar 2020 11:50:39 -0700 Subject: [all] Collecting Virtual Midcycle Best Practices In-Reply-To: References: Message-ID: Oh my bad. Never mind! -Kendall (diablo_rojo) On Wed, Mar 25, 2020 at 1:32 AM Dmitry Tantsur wrote: > Hi, > > I actually have, it's just spread over the document :) And somebody > (Julia?) has already added more information. > > Dmitry > > On Tue, Mar 24, 2020 at 8:20 PM Kendall Nelson > wrote: > >> Hey Dmitry :) >> >> Would you mind adding that to the etherpad so we have all the info in one >> place? I looked and didn't think I saw Ironic's testimony in there and it >> would be super helpful! >> >> -Kendall (diablo_rojo) >> >> On Wed, Mar 11, 2020 at 7:16 AM Dmitry Tantsur >> wrote: >> >>> Hi, >>> >>> Ironic did, I think, 3-4 virtual midcycles, and I think they were quite >>> successful. >>> The most positive outcome was to reach out to folks who usually cannot >>> travel. They really appreciated that. >>> >>> We used SIP the first couple of times, but it caused problems for a big >>> share of participants. Also lack of moderation was problematic. >>> We switched to Bluejeans later, and that was, IMO, a big improvement: >>> 1) A participant list with information who is speaking. >>> 2) An option for a moderator to mute a person or everyone. >>> 3) Screen sharing. >>> >>> Dmitry >>> >>> On Tue, Mar 10, 2020 at 1:29 AM Kendall Nelson >>> wrote: >>> >>>> Hello Everyone! >>>> >>>> I wanted to collect best practices and pitfalls to avoid wrt projects >>>> experiences with virtual midcycles. I know of a few projects that have done >>>> them in the past and with how travel is hard for a lot of people right now, >>>> I expect more projects to have midcycles. I think it would be helpful to >>>> have all of the data we can collect in one place for those not just new to >>>> virtual midcycles but the whole community. >>>> >>>> I threw some categories into this etherpad[1] and filled in some >>>> options. Please add to it :) >>>> >>>> -Kendall (diablo_rojo) >>>> >>>> [1] https://etherpad.openstack.org/p/virtual-midcycle-best-practices >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Wed Mar 25 18:53:25 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 25 Mar 2020 11:53:25 -0700 Subject: [all][elections][ptl][tc] Combined PTL/TC Nominations Kickoff Message-ID: Hello Everyone! Nominations for OpenStack PTLs (Project Team Leads) and TC (Technical Committee) positions (5 positions) are now open and will remain open until Mar 31, 2020 23:45 UTC. All nominations must be submitted as a text file to the openstack/election repository as explained at https://governance.openstack.org/election/#how-to-submit-a-candidacy Please make sure to follow the candidacy file naming convention: candidates/victoria// (for example, "candidates/victoria/TC/stacker at example.org"). The name of the file should match an email address for your current OpenStack Foundation Individual Membership. Take this opportunity to ensure that your OSF member profile contains current information: https://www.openstack.org/profile/ Any OpenStack Foundation Individual Member can propose their candidacy for an available, directly-elected seat on the Technical Committee. In order to be an eligible candidate for PTL you must be an OpenStack Foundation Individual Member. PTL candidates must also have contributed to the corresponding team during the Train to Ussuri timeframe, Mar 22, 2019 00:00 UTC - Apr 07, 2020 00:00 UTC. Your Gerrit account must also have a verified email address matching the one used in your candidacy filename. Both PTL and TC elections will be held from Apr 07, 2020 23:45 UTC through to Apr 14, 2020 23:45 UTC. The electorate for the TC election are the OpenStack Foundation Individual Members who have a code contribution to one of the official teams over the Train to Ussuri timeframe, Mar 22, 2019 00:00 UTC - Apr 07, 2020 00:00 UTC, as well as any Extra ATCs who are acknowledged by the TC. The electorate for a PTL election are the OpenStack Foundation Individual Members who have a code contribution over the Train to Ussuri timeframe, Mar 22, 2019 00:00 UTC - Apr 07, 2020 00:00 UTC, in a deliverable repository maintained by the team which the PTL would lead, as well as the Extra ATCs who are acknowledged by the TC for that specific team. The list of project teams can be found at https://governance.openstack.org/tc/reference/projects/ and their individual team pages include lists of corresponding Extra ATCs. Please find below the timeline: nomination starts @ Mar 24, 2020 23:45 UTC nomination ends @ Mar 31, 2020 23:45 UTC campaigning starts @ Mar 31, 2020 23:45 UTC campaigning ends @ Apr 07, 2020 23:45 UTC elections start @ Apr 07, 2020 23:45 UTC elections end @ Apr 14, 2020 23:45 UTC Shortly after election officials approve candidates, they will be listed on the https://governance.openstack.org/election/ page. The electorate is requested to confirm their email addresses in Gerrit prior to 2020-03-31 00:00:00+00:00, so that the emailed ballots are sent to the correct email address. This email address should match one which was provided in your foundation member profile as well. Gerrit account information and OSF member profiles can be updated at https://review.openstack.org/#/settings/contact and https://www.openstack.org/profile/ accordingly. If you have any questions please be sure to either ask them on the mailing list or to the elections officials: https://governance.openstack.org/election/#election-officials -Kendall Nelson (diablo_rojo) & the Election Officials -------------- next part -------------- An HTML attachment was scrubbed... URL: From colleen at gazlene.net Wed Mar 25 19:01:34 2020 From: colleen at gazlene.net (Colleen Murphy) Date: Wed, 25 Mar 2020 12:01:34 -0700 Subject: =?UTF-8?Q?Re:_[keystone][horizon]_Two_approaches_to_"multi-region"_OpenS?= =?UTF-8?Q?tack_deployment?= In-Reply-To: References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> Message-ID: Hi Chris, responses inline: > > On Wed, 25 Mar 2020 at 08:49, Krzysztof Klimonda > > wrote: > >> > >> Hi Colleen, > >> > >> Interesting - I’ve looked at K2K federation initially, but it didn’t seem to fit my requirements. > >> > >> My understanding is that in this setup we have a global keystone (which I’ve described in my initial post) > >> and then all regions are federated to that keystone. Is that correct? Would that work with an existing > >> SSO that we have in place (Keycloak) or does global keystone has to be IdP and not chain to another one? In the setup I described, there doesn't necessarily need to be a global or authoritative keystone. Rather, each keystone could act as both an IdP and an SP to one another. We usually simplify it in documentation to describe a keystone instance as being one or the other, but in practice there's no need for that restriction. In theory a keystone instance could work as a service provider to a Keycloak IdP while also acting as an IdP for other keystones. On Wed, Mar 25, 2020, at 04:00, Krzysztof Klimonda wrote: [snipped] > @Colleen I’m trying to visualise your proposed solution, does this > diagram make sense: http://paste.openstack.org/show/791120/ ? In K2K > federation, how are regions handled? Are they out of the picture, with > federated keystone taking their place instead? I assume all keystone > catalogs are separate, and users must use specific cloud auth url to > authenticate. Authentication can be then delegated to central keystone, > but what about another layer of SSO, i.e. Keycloak? [snipped] Your diagram is a hierarchical tree where it doesn't necessarily need to be, it could be a bidirected graph. But having a single known entrypoint into the cluster of clouds as shown in the diagram may also be a good way to structure it. "Regions" in the OpenStack sense of the word would not really apply because regions are owned by a single keystone instance. This also does indeed mean that the catalogs are separate, so users could only discover other catalogs by knowing the endpoint of one keystone, discovering service providers for that keystone, and then querying for catalogs from the other keystone service providers. The lack of a global or federated catalog for this use case is something that we recognize as a gap. The other, simpler "federation" approach that has been mentioned in this thread is to omit keystone as an IdP and instead use an external non-OpenStack IdP like Keycloak as your source of identity for all sites. This is conceptually easier but still doesn't solve the problem of having no shared catalog between sites. Colleen (cmurphy) From whayutin at redhat.com Wed Mar 25 19:52:28 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Wed, 25 Mar 2020 13:52:28 -0600 Subject: [tripleo] ptg topics Message-ID: Greetings, It's that time of year to start collecting topics for the PTG [1]. The in person PTG has been switched to a virtual meeting [2]. So the bad news is that the world is upside down atm, but the good news is that you may have a better lunch at the sessions ( I kid.. ) You can help plan the virtual ptg by signing up @ https://etherpad.openstack.org/p/Virtual_PTG_Planning Thanks all!! [1] https://etherpad.openstack.org/p/tripleo-victoria-topics [2] http://lists.openstack.org/pipermail/foundation/2020-March/002854.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Thu Mar 26 04:22:19 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 25 Mar 2020 21:22:19 -0700 Subject: [first contact][all] Meeting Frequency & Time Message-ID: Hello Everyone, In our meeting yesterday[1], the first one of the year, we discussed changing our meeting frequency to once a month and maybe shifting the day/time. I think Tuesday still works for me but an hour or two earlier wouldn't go amiss. If no-one objects to this, I will put a patch up to change the meeting time & frequency next week to monthly instead of biweekly and -Kendall (diablo_rojo) [1] Meeting Log: http://eavesdrop.openstack.org/meetings/fc_sig/2020/fc_sig.2020-03-25-02.39.log.html [2] Current Meeting Info: http://eavesdrop.openstack.org/#First_Contact_SIG_Meeting -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tushar.Patil at nttdata.com Thu Mar 26 10:00:44 2020 From: Tushar.Patil at nttdata.com (Patil, Tushar) Date: Thu, 26 Mar 2020 10:00:44 +0000 Subject: [tosca-parser][heat-translator][tacker] - Need new version of heat-translator and tosca-parser libraries for Tacker Message-ID: Hi PTL and Core Reviewers, We are working on implementing ETSI spec in tacker for Ussuri release for which we have pushed couple of patches in heat-translator [1][2] and tosca-parser[3]. Patch [1] is not yet merged so request core reviewers to please take a look at it. We have uploaded many patches in tacker [4] but most of the patches are failing on CI as it's requires changes from heat-translator and tosca-parser. So we need a new release of heat-translator (after patch [1] is merged) and tosca-parser badly. Please let us know when are you planning to release a new version of these libraries. [1] : https://review.opendev.org/#/q/status:merged+project:openstack/heat-translator+branch:master+topic:etsi_nfv-sol001 [2] : https://review.opendev.org/#/q/status:open+project:openstack/heat-translator+branch:master+topic:bp/support-etsi-nfv-specs [3] : https://review.opendev.org/#/c/688633 [4] : https://review.opendev.org/#/q/topic:bp/support-etsi-nfv-specs+(status:open+OR+status:merged) Thank you. Regards, tpatil Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. From john at johngarbutt.com Thu Mar 26 10:34:06 2020 From: john at johngarbutt.com (John Garbutt) Date: Thu, 26 Mar 2020 10:34:06 +0000 Subject: [keystone][horizon] Two approaches to "multi-region" OpenStack deployment In-Reply-To: References: <00311D40-AF54-4E5F-BB57-26CB96D4069C@syntaxhighlighted.com> <40f6c1f7-c6e4-492b-b1ba-05938dc7aecb@www.fastmail.com> Message-ID: On Wed, 25 Mar 2020 at 11:08, Krzysztof Klimonda wrote: > @Mark Thanks, I remember reading that blog post already :). Just to make sure, in the deployment described in the post you federate keystone with keycloak, but without K2K federation in the middle, right? Correct, as colleen confirmed, its without K2K federation. We use OIDC to link Keycloak with OpenStack. And also use OIDC between Keycloak and the acutal IdP, For context, we are using this pattern, using Keycloak as the proxy: https://aarc-project.eu/architecture/ I see that as hub-and-spoke when you compare it to other approaches here: https://wiki.geant.org/display/eduGAIN/Federation+Architectures I should be clear the requirements for this project ruled out K2K from the start. Rather than being multi-region, our aim was allowing people to login to OpenStack using the regular organisation credentials. (In the UK university context, we made use of the existing Edugain federation.) Howerver, we also needed something that also allowed user applications to also make use of the same AAAI system, in particular to project the users web apps like JuypterHub, OpenOnDemand, etc. > On a related note, in your blog post you show “chained federation” in Keycloak. > Does this setup support generating Keystone tokens for API/CLI access? If so, how does the authentication flow look like? We are using Application Credentials for all API and CLI access (including terraform): https://docs.openstack.org/keystone/train/user/application_credentials.html They are keystone region specific (which I see as a positive feature), but they work well. (Assuming your federation mapping doesn't use groups, and instead directly maps users to projects) I hope that helps, johnthetubaguy From john at johngarbutt.com Thu Mar 26 10:36:52 2020 From: john at johngarbutt.com (John Garbutt) Date: Thu, 26 Mar 2020 10:36:52 +0000 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: References: <012990b86b48e568383b63835617e92c96f0d2be.camel@redhat.com> Message-ID: +1 On Tue, 24 Mar 2020 at 23:59, Alex Xu wrote: > > +1 > > Stephen Finucane 于2020年3月24日周二 下午6:19写道： >> >> On Mon, 2020-03-23 at 17:34 +0100, Balázs Gibizer wrote: >> > Hi, >> > >> > I'm proposing Ghanshyam Mann to the core team. He is a long time >> > OpenStack and nova contributor. He is one of our API and QA expert. I >> > think he would be a great addition to the team. >> > >> > Cores, please vote (-1, 0, +1) before next Monday (03.30.) >> >> +1 >> >> > Cheers, >> > gibi >> > >> > >> > >> > >> >> From john at johngarbutt.com Thu Mar 26 10:37:16 2020 From: john at johngarbutt.com (John Garbutt) Date: Thu, 26 Mar 2020 10:37:16 +0000 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: +1 On Tue, 24 Mar 2020 at 23:59, Alex Xu wrote: > > +1 > > Sylvain Bauza 于2020年3月24日周二 下午5:28写道： >> >> >> >> On Mon, Mar 23, 2020 at 3:37 PM Balázs Gibizer wrote: >>> >>> Hi, >>> >>> I'm proposing Lee to the core team. He is around for a long time and he >>> became really active recently not only on the stable branches. I think >>> he would be a great addition to the team. >>> >>> Cores, please vote (-1, 0, +1) before next Monday (03.30.) >>> >> >> +1 >> >>> Cheers, >>> gibi >>> >>> >>> From thierry at openstack.org Thu Mar 26 10:52:21 2020 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 26 Mar 2020 11:52:21 +0100 Subject: [all][elections][ptl][tc] Combined PTL/TC Nominations Kickoff In-Reply-To: References: Message-ID: <30721b58-628b-2567-6b37-d3676c5505fd@openstack.org> Re: TC seats, I'd like to encourage people with operational experience of OpenStack to run for election. We have lots of OpenStack users. Our focus today is more on continuously improving the experience of running OpenStack. Filling the gaps rather than developing radically-new breaking features. Integrating with other open infrastructure software that you run alongside it. So, in governance and key policy decisions, we rely on the feedback of people that have practical experience operating the software. Having more of those directly involved in the TC will facilitate that feedback loop, and make sure we take into account a wide variety of situations. Being on the TC is not as time consuming as you might think. The time-consuming part is trying to stay on top of what's happening with OpenStack: I would say that being on the TC just adds a max of two hours per week (reviewing proposed changes, chiming in on governance discussions on the mailing-list, attending some weekly office hours or the monthly meeting). But it is a great way to make sure the direction of the software is aligned to your needs as an OpenStack user, and the needs of other users in a similar situation. Thanks for considering it! -- Thierry Carrez (ttx) From dougal at redhat.com Thu Mar 26 14:13:25 2020 From: dougal at redhat.com (Dougal Matthews) Date: Thu, 26 Mar 2020 14:13:25 +0000 Subject: [tripleo][mistral] Farewell Message-ID: Hey all, I am leaving Red Hat and my last day is tomorrow. As a result I will be moving on from OpenStack. I just wanted to say my farewells here and say thanks to everyone for all the fun over the years. As my last core duty I will be de-coring myself tomorrow :) If anyone wants to reach me beyond tomorrow, my email is probably best: dougal at dougalmatthews.com Cheers, Dougal -------------- next part -------------- An HTML attachment was scrubbed... URL: From munnaeebd at gmail.com Thu Mar 26 14:55:17 2020 From: munnaeebd at gmail.com (Md. Hejbul Tawhid MUNNA) Date: Thu, 26 Mar 2020 20:55:17 +0600 Subject: Regarding Split network plane for live migration Message-ID: Hi, We are using Openstack Rocky . We are observing internal network is using while we do live migration. our internal network interface is 1G. So interface getting full and migration never getting complete. (nova server-migration-list ) 'Remaining Memory Bytes' increasing and decreasing but not getting completed. We have 10G interface for public/external network. I have tried following solution but no luck. still using internal network https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/split-network-plane-for-live-migration.html Some Information: [DEFAULT] my_ip = Internal_network_IP [libvirt] live_migration_uri=qemu+ssh://nova@%s.external /system?keyfile=/var/lib/nova/.ssh/id_rsa&no_verify=1 Please let us know if any further information is required. Please advice us if we can solve the issue. Regards, Munna -------------- next part -------------- An HTML attachment was scrubbed... URL: From saphi070 at gmail.com Thu Mar 26 15:07:12 2020 From: saphi070 at gmail.com (Sa Pham) Date: Thu, 26 Mar 2020 22:07:12 +0700 Subject: Regarding Split network plane for live migration In-Reply-To: References: Message-ID: I think you can use the live_migration_inbound_addr option in libvirt section. For more information, use [1]. [1] - https://docs.openstack.org/nova/latest/configuration/config.html On Thu, Mar 26, 2020 at 9:57 PM Md. Hejbul Tawhid MUNNA wrote: > Hi, > > We are using Openstack Rocky . We are observing internal network is using > while we do live migration. our internal network interface is 1G. So > interface getting full and migration never getting complete. > > (nova server-migration-list ) 'Remaining Memory Bytes' increasing and > decreasing but not getting completed. > > We have 10G interface for public/external network. I have tried following > solution but no luck. still using internal network > > > https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/split-network-plane-for-live-migration.html > > Some Information: > > [DEFAULT] > > my_ip = Internal_network_IP > > > [libvirt] > > live_migration_uri=qemu+ssh://nova@%s.external > /system?keyfile=/var/lib/nova/.ssh/id_rsa&no_verify=1 > > > Please let us know if any further information is required. > > Please advice us if we can solve the issue. > > Regards, > Munna > -- Sa Pham Dang Skype: great_bn Phone/Telegram: 0986.849.582 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Thu Mar 26 15:07:43 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Thu, 26 Mar 2020 11:07:43 -0400 Subject: [tc][elections] Nomination Message-ID: Hi everyone, I want to run again for the OpenStack technical committee for the upcoming cycle. I have been a long time operator of OpenStack with a focus on never using a local for, which has resulted in my involvement with many different projects by making quality-of-life improvements for these services that can help other deployers. I've contributed and worked with different sizes of teams inside OpenStack. >From our big projects such as Nova to projects providing specific services such as Magnum and pretty much most deployments tools at this point. On the operator side, I've helped run one of the longest-running OpenStack public clouds and many private clouds. I've also interacted with many other OpenStack operators that are not as involved in our community, which gives me a significant exposure to our operator story and what that looks like outside our community of known operators. I believe it's time now for me to start driving significant changes in our community to simplify the operator experience, and I think that should start from the technical committee. We're seeing the world evolve around us with new technologies, but we've remained relatively stable and stuck to our "known and trusted" stack. Operators have historically had a lot of frustrations with systems like RabbitMQ and their clustering issues, which end up making the OpenStack experience disappointing. There are many new ways of doing RPC, which we can leverage and help make it even easier to run OpenStack (things such as gRPC). We've also had a history of making it complicated to use containers to deploy OpenStack and historically refused to ship "official" containers. The OpenDev community has built our a step of tooling that can allow us to ship containers very quickly, with nothing more than a well written bindep.txt file and a simple multi-stage build. It's about time that we start having official images and easy ways to deploy OpenStack to reduce those barriers. We define OpenStack as a "cloud operating system." We struggle to deliver much more than the essential infrastructure components. The drive behind OpenStack was to allow people to get more than the necessary infrastructure components. Still, large amounts of different efforts lead us into complicated ways of having to wrangle VMs to run services. With popular container orchestrators and schedulers, OpenStack has a perfect platform to be able to launch those services. By setting up a proper framework, we can start delivering many services such as database as a service, memory store as a service, and so much more by building out Kubernetes operators. It also opens up a whole domain of possible contributors that can use those operators without OpenStack. In closing, I don't expect everyone to agree with all the different ideas I'm putting out. We may only drive just a few of them, or maybe none of them. However, I want to convince you that we need to change. I can understand they sound like big leaps, and they may seem uncomfortable. However, they are genuinely some of the things we need to do to continue to be relevant and put us at the leading edge of technology like we used to be years ago. Thank you for reading this. Regards, Mohammed From mnaser at vexxhost.com Thu Mar 26 15:08:16 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Thu, 26 Mar 2020 11:08:16 -0400 Subject: [openstack-ansible][elections] Nomination Message-ID: I'd like to announce my candidacy for OpenStack Ansible PTL. As usual, I want to go over all the points I talked about last time and their progress. # Simplifying scenarios for usage and testing We probably haven't progressed much in this, our testing has improved, we did add upgrade testing but unfortuantely that hasn't been as stable. # Using integrated repository for testing and dropping role tests This is pretty much done for the expect of a few corner projects. # Progress on switching all deployments to use Python3 This is also done with some packaging workarounds to make it work with CentOS 7. # Eventual addition of CentOS 8 option We made some progress in this, but it still needs a lot more work, big thanks to jrosser for driving a bunch of this work too. # Reduction in number of config variables (encouraging overrides) This has been done over a few roles now that are using overrides in hopes of minimizing the number of variables to speed up runs. # Increase cooperation with other deployment projects (i.e. TripleO) The Ansible SIG is moving and alive, TripleO uses python_venv_build and os_tempest as well as our connection plugins, so yay. We've worked on getting some new cores as well and making sure the project continues to be sustainable. Thank you for your consideration. From munnaeebd at gmail.com Thu Mar 26 15:12:30 2020 From: munnaeebd at gmail.com (Md. Hejbul Tawhid MUNNA) Date: Thu, 26 Mar 2020 21:12:30 +0600 Subject: Regarding Split network plane for live migration In-Reply-To: References: Message-ID: Hi, I already tried it. But after that its still using internal interface bandwidth. Please find the additional configuration : live_migration_inbound_addr=external_network_IP #live_migration_tunnelled=false [libvirt] live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST,VIR_MIGRATE_TUNNELLED Regards, Munna On Thu, Mar 26, 2020 at 9:07 PM Sa Pham wrote: > I think you can use the live_migration_inbound_addr option in libvirt > section. For more information, use [1]. > > > [1] - https://docs.openstack.org/nova/latest/configuration/config.html > > On Thu, Mar 26, 2020 at 9:57 PM Md. Hejbul Tawhid MUNNA < > munnaeebd at gmail.com> wrote: > >> Hi, >> >> We are using Openstack Rocky . We are observing internal network is using >> while we do live migration. our internal network interface is 1G. So >> interface getting full and migration never getting complete. >> >> (nova server-migration-list ) 'Remaining Memory Bytes' increasing and >> decreasing but not getting completed. >> >> We have 10G interface for public/external network. I have tried following >> solution but no luck. still using internal network >> >> >> https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/split-network-plane-for-live-migration.html >> >> Some Information: >> >> [DEFAULT] >> >> my_ip = Internal_network_IP >> >> >> [libvirt] >> >> live_migration_uri=qemu+ssh://nova@%s.external >> /system?keyfile=/var/lib/nova/.ssh/id_rsa&no_verify=1 >> >> >> Please let us know if any further information is required. >> >> Please advice us if we can solve the issue. >> >> Regards, >> Munna >> > > > -- > Sa Pham Dang > Skype: great_bn > Phone/Telegram: 0986.849.582 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Mar 26 15:33:11 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 26 Mar 2020 15:33:11 +0000 Subject: Regarding Split network plane for live migration In-Reply-To: References: Message-ID: <62e1a9cf3aac71fe500ec222b315bce5e77e6cea.camel@redhat.com> On Thu, 2020-03-26 at 20:55 +0600, Md. Hejbul Tawhid MUNNA wrote: > Hi, > > We are using Openstack Rocky . We are observing internal network is using > while we do live migration. our internal network interface is 1G. So > interface getting full and migration never getting complete. > > (nova server-migration-list ) 'Remaining Memory Bytes' increasing and > decreasing but not getting completed. > > We have 10G interface for public/external network. I have tried following > solution but no luck. still using internal network > > https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/split-network-plane-for-live-migration.html > > Some Information: > > [DEFAULT] > > my_ip = Internal_network_IP > > > [libvirt] > > live_migration_uri=qemu+ssh://nova@%s.external > /system?keyfile=/var/lib/nova/.ssh/id_rsa&no_verify=1 on later release we generally recommend using live_migration_inbound_addr instead. https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_inbound_addr this will be ignored however if you set live_migration_tunnelled the other genral advice we have for live migration is that you set max_concurrent_live_migrations to 1 https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.max_concurrent_live_migrations if you have a NFV workload or something that is activly writing to memory its entirly possible to never have the migration complete. if the rate of change of the dirty pages in the geuss exceed the available bandwith this is going to result in the migration not finishing. there are ways to help with this. for example you can enable live_migration_permit_auto_converge and live_migration_permit_post_copy https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_permit_post_copy https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_permit_auto_converge auto convergance will after a short timout slow start reducing the execution time of the guest to try and allow forward progress to be made. post copy will after a delay copy enough memory form the source to the dest to allow execution to start on the dest while memory is still being copied form the souces. as long as you understand the implciation of autoconvergence(trotteling the guest) and post_copy (remote exectuion over the network) then i would generaly recommend turning both on to help in your case but try using the live_migration_inbound_addr first. > > > Please let us know if any further information is required. > > Please advice us if we can solve the issue. > > Regards, > Munna From smooney at redhat.com Thu Mar 26 15:34:31 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 26 Mar 2020 15:34:31 +0000 Subject: Regarding Split network plane for live migration In-Reply-To: References: Message-ID: <630d8dda3829951bd112445ddc8532bc7b583189.camel@redhat.com> On Thu, 2020-03-26 at 21:12 +0600, Md. Hejbul Tawhid MUNNA wrote: > Hi, > > I already tried it. But after that its still using internal interface > bandwidth. > > Please find the additional configuration : > > live_migration_inbound_addr=external_network_IP > > #live_migration_tunnelled=false > > [libvirt] > live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST,VIR_MI > GRATE_TUNNELLED live_migration_flag is not supported on rocky but if it had been this would have reenabled tunneling and prevented live_migration_inbound addr form working > > Regards, > Munna > > > On Thu, Mar 26, 2020 at 9:07 PM Sa Pham wrote: > > > I think you can use the live_migration_inbound_addr option in libvirt > > section. For more information, use [1]. > > > > > > [1] - https://docs.openstack.org/nova/latest/configuration/config.html > > > > On Thu, Mar 26, 2020 at 9:57 PM Md. Hejbul Tawhid MUNNA < > > munnaeebd at gmail.com> wrote: > > > > > Hi, > > > > > > We are using Openstack Rocky . We are observing internal network is using > > > while we do live migration. our internal network interface is 1G. So > > > interface getting full and migration never getting complete. > > > > > > (nova server-migration-list ) 'Remaining Memory Bytes' increasing and > > > decreasing but not getting completed. > > > > > > We have 10G interface for public/external network. I have tried following > > > solution but no luck. still using internal network > > > > > > > > > https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/split-network-plane-for-live-migration.html > > > > > > Some Information: > > > > > > [DEFAULT] > > > > > > my_ip = Internal_network_IP > > > > > > > > > [libvirt] > > > > > > live_migration_uri=qemu+ssh://nova@%s.external > > > /system?keyfile=/var/lib/nova/.ssh/id_rsa&no_verify=1 > > > > > > > > > Please let us know if any further information is required. > > > > > > Please advice us if we can solve the issue. > > > > > > Regards, > > > Munna > > > > > > > > > -- > > Sa Pham Dang > > Skype: great_bn > > Phone/Telegram: 0986.849.582 > > > > > > From rajiv.mucheli at gmail.com Wed Mar 25 10:56:08 2020 From: rajiv.mucheli at gmail.com (rajiv mucheli) Date: Wed, 25 Mar 2020 16:26:08 +0530 Subject: [ops] need help configuring glance policies Message-ID: Hi there, In regards to https://answers.launchpad.net/glance/+question/689450, i need help configuring glance policies. I did review the excellent videos but had no response from the authors over IRC. The 403 errors occur with admin_role = '' Please suggest how to proceed. Regards, Rajiv -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Thu Mar 26 15:51:16 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 26 Mar 2020 10:51:16 -0500 Subject: [nova] API updates week 20-13 Message-ID: <171178a60f1.f566a87749241.8031265499619469437@ghanshyammann.com> Hello Everyone, Please find the Nova API updates of this week. Please add if I missed any BPs/API related work. API Related BP : ============ COMPLETED: 1. Add image-precache-support spec: - https://blueprints.launchpad.net/nova/+spec/image-precache-support Code Ready for Review: ------------------------------ 1. Nova API policy improvement - Topic: https://review.opendev.org/#/q/topic:bp/policy-defaults-refresh+(status:open+OR+status:merged) - Weekly Progress: Lot of Policies work merged and I am working on remaining ones. - Review guide over ML - http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008504.html 2. Non-Admin user can filter their instance by availability zone: - Topic: https://review.opendev.org/#/q/topic:bp/non-admin-filter-instance-by-az+(status:open+OR+status:merged) - Weekly Progress: Nova change is merged, novaclient change is in-progress - https://review.opendev.org/#/c/713089/ 3. Boot from volume instance rescue - Topic: https://review.opendev.org/#/q/topic:bp/virt-bfv-instance-rescue+(status:open+OR+status:merged) - Weekly Progress: Code Review is in progress. 4. Add action event fault details -Topic: https://review.opendev.org/#/q/topic:bp/action-event-fault-details+(status:open+OR+status:merged) - Weekly Progress: Few discussion on how we can minimize the info leak to non-admin. discussion is going on. Almost good. 5. Support re-configure deleted_on_termination in server -Topic: https://review.opendev.org/#/q/topic:bp/destroy-instance-with-datavolume+(status:open+OR+status:merged) - Weekly Progress: Discussion happened not to use PATCH for this and use existing PUT method. gibi will discuss this in today nova meeting on process and FF wise. Specs are merged and code in-progress: ------------------------------ ------------------ - None Spec Ready for Review or Action from Author: --------------------------------------------------------- -Marking as None here as Ussuri spec deadline is already over. Others: 1. None Bugs: ==== I started fixing policy bugs while working on policy-defaults-refresh BP. 5 bugs have been identified till now and fix up for review. - https://bugs.launchpad.net/nova/+bugs?field.tag=policy-defaults-refresh NOTE- There might be some bug which is not tagged as 'api' or 'api-ref', those are not in the above list. Tag such bugs so that we can keep our eyes. - https://etherpad.openstack.org/p/nova-bug-triage-ussuri -gmann From openstack at nemebean.com Thu Mar 26 16:03:33 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 26 Mar 2020 11:03:33 -0500 Subject: [oslo] PTL Non-Candidacy Message-ID: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> This won't come as a surprise to most of you, but in case you missed the announcement at the beginning of this cycle I won't be able to serve as Oslo PTL for the next one. While I very much wish this were not the case, I do feel good about the current state of the Oslo team and I'm confident it will continue to do good things under someone else's leadership. Thanks to all of the Oslo contributors who have made the past two years a great experience! -Ben From paye600 at gmail.com Thu Mar 26 16:04:56 2020 From: paye600 at gmail.com (Roman Gorshunov) Date: Thu, 26 Mar 2020 17:04:56 +0100 Subject: [Murano-Open-PaaS] Openstack using guacamole In-Reply-To: References: Message-ID: Hello Samuel, Thanks for your email. Yes, Guacamole can be installed as an app via Murano. Discussions about Open PaaS are now in openstack-discuss mailing list. Please use the tag [Murano-Open-PaaS] in the subject line. I'm re-routing your email. Here are docs for the Murano project [0]. [0] https://docs.openstack.org/murano/latest/ Best regards, Roman Gorshunov On Thu, Mar 26, 2020 at 4:37 PM Samuel Abdullah wrote: > Hi, > > Would like to ask if it is possible to run guacamole in openstack > environment? > Seek for help on how would this be possible as i saw that guacamole > component are part of the openstack in your murano project > > Looking forward for your reply > > Best Regards > > -- > > > > www.silverliningsys.com > > > *Abu Bakar Samuel Abdullah* > > Cloud Infrastructure & Operations > > > P: +603-2712-0081 > > M: +60.12.654.5938 > > E: samuel at silverliningsys.com > _______________________________________________ > OpenStack-Infra mailing list > OpenStack-Infra at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra -------------- next part -------------- An HTML attachment was scrubbed... URL: From hberaud at redhat.com Thu Mar 26 16:24:17 2020 From: hberaud at redhat.com (Herve Beraud) Date: Thu, 26 Mar 2020 17:24:17 +0100 Subject: [oslo] PTL Non-Candidacy In-Reply-To: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> References: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> Message-ID: Thanks Ben for all the things you've done, I really appreciate your work as the Oslo PTL! Le jeu. 26 mars 2020 à 17:06, Ben Nemec a écrit : > This won't come as a surprise to most of you, but in case you missed the > announcement at the beginning of this cycle I won't be able to serve as > Oslo PTL for the next one. While I very much wish this were not the > case, I do feel good about the current state of the Oslo team and I'm > confident it will continue to do good things under someone else's > leadership. > > Thanks to all of the Oslo contributors who have made the past two years > a great experience! > > -Ben > > -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Thu Mar 26 19:15:35 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Thu, 26 Mar 2020 12:15:35 -0700 Subject: [ironic] The Sanity Preservation Un-Conference, or SPUC - An Ironic gathering! In-Reply-To: References: Message-ID: The time has been agreed! Tomorrow - 5 PM UTC! Etherpad: https://etherpad.openstack.org/p/SanityPreservationUnConference Video conferencing URL: https://bluejeans.com/u/jkreger On Tue, Mar 24, 2020 at 1:45 PM Julia Kreger wrote: > > So we seem to be gaining consensus[0] around 5 PM UTC on Friday! > > Vote now if you've not voted already! > > -Julia > > [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 > > On Mon, Mar 23, 2020 at 10:06 AM Julia Kreger > wrote: > > > > Greetings Humans, Vulcans, AIs, and any other form of life! > > > > Today in the Ironic team meeting, we decided that we needed to have an > > un-conference! > > > > In the theme of an un-conference, I think we will start with two > > simple questions: > > * How is everyone? > > * What ironic (or non-ironic) things shall we discuss? > > > > From there, we will identify topics of discussion, and possibly jump > > into separate video calls for ~15 minutes each, depending on the > > number of topics and general insanity. > > > > During the last ?15? minutes or so, we shall reconvene into the same > > video call to enjoy a little time sipping coffee Coffee, Tea or other > > delicious beverages while we determine who is wearing the silliest > > hat! So if you have a steampunk hat with gears or even some kind of > > hat you would wear to a fancy horse race, go right ahead! Only have a > > rose from your growing quarantine garden! You can wear that too! > > > > So what do you need to do! > > 1) Fill out the doodle[0] so we can actually schedule a time that > > works for a group of people! > > 2) Brainstorm some ideas of things you might want to talk > > about/present or LEARN! > > 3) Dust off your silly hats! > > > > NB: I know the windows on this doodle overlap with Baremetal SIG > > doodle that I sent out a little while ago. We will figure it out. > > > > -Julia > > > > [0]: https://doodle.com/poll/krtn9p4z7xx9pa29 From allison at openstack.org Thu Mar 26 19:22:13 2020 From: allison at openstack.org (Allison Price) Date: Thu, 26 Mar 2020 14:22:13 -0500 Subject: [all][ptl] How to Contribute to the OpenStack Blog & Superuser Magazine Message-ID: <4C16D078-0753-4EAF-A80B-19F4B5973968@openstack.org> Hey Stackers— We have received more questions recently about where the community can contribute content, especially on official OpenStack or community channels. Accompanied by the amount of events that have been transformed to virtual experiences, the OpenStack Foundation has been working on ways for the community to centrally share learnings online, and create a place where the community can send project, SIG, and overall community updates. Superuser and the OpenStack blog are community channels—for the community, by the community, so we would really like to get more contributors. We even had a user survey completed yesterday from an operator saying they wish the blog had more information on bug fixes and software updates, so here we are. Here’s how you can contribute: OpenStack Blog We are reviving the OpenStack blog! Here, we want to publish OpenStack bug fixes, software updates, and project milestones that would be beneficial for users to know more about that may be more in depth. Often these are posted on the mailing list, which is great, but could also double as blog posts so that they are searchable in the future when operators stumble upon an issue. Once you have an idea of what you would like to share or have questions, please email community at openstack.org . We can also keep track of a running list of ideas / calls for content for people who have questions that would be great to get answered on the blog. PTLs—heading into the Ussuri cycle, this is a great time to share your ideas on the etherpad around what you can write about. The OpenStack Foundation will reach out to folks who put their names or nicks in the etherpad as well as those who proactively reach out. Superuser Superuser has evolved to cover the open infrastructure landscape covering different use cases and open source projects. Whether it’s a video or an article, this content is typically more in depth than what you would see on a project blog and includes things like hands on tutorials, case studies, thought leadership articles, Q&As with open source community leaders, and updates and highlights from open source events around the world. If you have an idea you would like to share, fill out this form and the editorial team will reach out. If you have questions about whether your topic is the right fit, email editor at openstack.org . You can also attend the OSF community meeting next week to learn more about how you can get involved and contribute! Thanks! Allison Allison Price OpenStack Foundation allison at openstack.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungleboyj at gmail.com Thu Mar 26 19:40:44 2020 From: jungleboyj at gmail.com (Jay Bryant) Date: Thu, 26 Mar 2020 14:40:44 -0500 Subject: [oslo] PTL Non-Candidacy In-Reply-To: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> References: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> Message-ID: <5e78cfc7-8de5-ea42-0339-51ec9fecaf78@gmail.com> On 3/26/2020 11:03 AM, Ben Nemec wrote: > This won't come as a surprise to most of you, but in case you missed > the announcement at the beginning of this cycle I won't be able to > serve as Oslo PTL for the next one. While I very much wish this were > not the case, I do feel good about the current state of the Oslo team > and I'm confident it will continue to do good things under someone > else's leadership. > > Thanks to all of the Oslo contributors who have made the past two > years a great experience! > > -Ben > Ben, Thanks for your great leadership of Oslo.  It has been in good hands under your leadership! Jay From sean.mcginnis at gmx.com Thu Mar 26 19:48:58 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 26 Mar 2020 14:48:58 -0500 Subject: [oslo] PTL Non-Candidacy In-Reply-To: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> References: <4224fdd9-54f9-51a0-b266-812cbf429c43@nemebean.com> Message-ID: On 3/26/20 11:03 AM, Ben Nemec wrote: > This won't come as a surprise to most of you, but in case you missed > the announcement at the beginning of this cycle I won't be able to > serve as Oslo PTL for the next one. While I very much wish this were > not the case, I do feel good about the current state of the Oslo team > and I'm confident it will continue to do good things under someone > else's leadership. > > Thanks to all of the Oslo contributors who have made the past two > years a great experience! > > -Ben > Thanks for all you've done Ben! From jungleboyj at gmail.com Thu Mar 26 20:06:29 2020 From: jungleboyj at gmail.com (Jay Bryant) Date: Thu, 26 Mar 2020 15:06:29 -0500 Subject: [all][elections][ptl][tc] Combined PTL/TC Nominations Kickoff In-Reply-To: <30721b58-628b-2567-6b37-d3676c5505fd@openstack.org> References: <30721b58-628b-2567-6b37-d3676c5505fd@openstack.org> Message-ID: On 3/26/2020 5:52 AM, Thierry Carrez wrote: > Re: TC seats, I'd like to encourage people with operational experience > of OpenStack to run for election. > > We have lots of OpenStack users. Our focus today is more on > continuously improving the experience of running OpenStack. Filling > the gaps rather than developing radically-new breaking features. > Integrating with other open infrastructure software that you run > alongside it. > > So, in governance and key policy decisions, we rely on the feedback of > people that have practical experience operating the software. Having > more of those directly involved in the TC will facilitate that > feedback loop, and make sure we take into account a wide variety of > situations. > > Being on the TC is not as time consuming as you might think. The > time-consuming part is trying to stay on top of what's happening with > OpenStack: I would say that being on the TC just adds a max of two > hours per week (reviewing proposed changes, chiming in on governance > discussions on the mailing-list, attending some weekly office hours or > the monthly meeting). But it is a great way to make sure the direction > of the software is aligned to your needs as an OpenStack user, and the > needs of other users in a similar situation. > > Thanks for considering it! > Just wanted to echo Thierry's words here as a new TC Member this year. Would love to see more users/operators joining us and bringing new perspectives. The additional time commitment has not been significant and being a part of the TC has given me a new perspective of the OpenStack community that I feel has been beneficial. So, if you are able, please take the time to consider this opportunity! Jay (IRC: jungleboyj) From skaplons at redhat.com Thu Mar 26 21:48:37 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 26 Mar 2020 22:48:37 +0100 Subject: [neutron] Drivers meeting - 27.03.2020 Message-ID: Hi, Due to lack of agenda for our next drivers meeting I will cancel it. I would like You to check list of opened, not triaged, old RFEs [1] and check if there are maybe some RFEs which You would like to continue work on, or maybe there are some which You think we should close as not needed anymore. Also if RFE on this list belongs to You, please respond to any questions from comments in this RFE or just let us know that You don’t want/don’t have time to continue work on it. Thx in advance [1] https://bugs.launchpad.net/neutron/+bugs?field.tag=rfe — Slawek Kaplonski Senior software engineer Red Hat From changzhi at cn.ibm.com Fri Mar 27 08:51:23 2020 From: changzhi at cn.ibm.com (Zhi CZ Chang) Date: Fri, 27 Mar 2020 08:51:23 +0000 Subject: [nova][cinder] Why nova-api uses volume's size as bdm.volume_size? Message-ID: An HTML attachment was scrubbed... URL: From renat.akhmerov at gmail.com Fri Mar 27 09:08:46 2020 From: renat.akhmerov at gmail.com (Renat Akhmerov) Date: Fri, 27 Mar 2020 16:08:46 +0700 Subject: [tripleo][mistral] Farewell In-Reply-To: References: Message-ID: <365576f0-e287-446d-8076-0cb8bf93673e@Spark> Hey Dougal, Just want to say it’s a bit said that you’re leaving but I hope you’re moving towards being happier :) So I’m really glad for you. I thank you for everything, it was a great pleasure to work with you as a teammate! You are one of the most enthusiastic and talented engineers I’ve met. Good luck! Thanks Renat Akhmerov @Nokia On 26 Mar 2020, 21:25 +0700, Dougal Matthews , wrote: > Hey all, > > I am leaving Red Hat and my last day is tomorrow. As a result I will be moving on from OpenStack. I just wanted to say my farewells here and say thanks to everyone for all the fun over the years. > > As my last core duty I will be de-coring myself tomorrow :) > > If anyone wants to reach me beyond tomorrow, my email is probably best: dougal at dougalmatthews.com > > Cheers, > Dougal -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-philippe at evrard.me Fri Mar 27 09:54:49 2020 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Fri, 27 Mar 2020 10:54:49 +0100 Subject: [all][tc] Meeting Agenda April 2nd Message-ID: Hello everyone, Here are the report topics for the tc next meeting, April 2nd. A few of those are most likely noop, tbh. Feel free to join us! - Report on tc/uc merge (ttx) - Report on the analysis of the 2019 survey (jungleboyj), and survey 2020. - Report on telemetry after Rico's convo with the PTL (ricolin) - report on stable branch policy work: mnaser to push an update to distill what happened on the ML in terms of stable branch policy (mnaser) - report on the community goals for U and V, py2 drop (gmann) - Moving OpenDev discussion forward (clarkb) - Report on the OSF board initiatives (mnaser) Please note that, as usual, the office hours are happening after the meeting, so if you want to chat over new things, it's a perfect place! Regards, Jean-Philippe Evrard From skaplons at redhat.com Fri Mar 27 12:19:24 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 27 Mar 2020 13:19:24 +0100 Subject: [neutron] Final release of non-client libraries Message-ID: <8FDFEFBA-2963-4ED3-9811-8CCACE8C8463@redhat.com> Hi, Next week is deadline for final releases for non-client libraries and I was checking what patches we are still in review and potentially we would need to have in Ussuri. *Neutron-lib* I see only https://review.opendev.org/#/c/712657/ which potentially may be good to get before final release. IMO it’s in good shape, please review it if You have few minutes (it’s small). *Ovsdbapp* There are 3 opened patches on https://review.opendev.org/#/q/project:openstack/ovsdbapp+branch:master+status:open But I think that maybe only https://review.opendev.org/#/c/683355/ may be potentially needed, mostly by ovn subteam. So please check if this is needed and we should update it to make it in before final Ussuri release. Os-ken I don’t see any important patches waiting in the list https://review.opendev.org/#/q/project:openstack/os-ken+branch:master+status:open so it should be good to go. If I missed anything or You have something else which You need to get in before this final release, please ping me by email or on irc. — Slawek Kaplonski Senior software engineer Red Hat From sean.mcginnis at gmx.com Fri Mar 27 15:10:12 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Fri, 27 Mar 2020 10:10:12 -0500 Subject: [release] Release countdown for week R-6, March 30 - April 3 Message-ID: <20200327151012.GA644442@sm-workstation> Development Focus ----------------- Work on libraries should be wrapping up, in preparation for the various library-related deadlines coming up. Now is a good time to make decisions on deferring feature work to the next development cycle in order to be able to focus on finishing already-started feature work. General Information ------------------- We are now getting close to the end of the cycle, and will be gradually freezing feature work on the various deliverables that make up the OpenStack release. This coming week is the deadline for general libraries (except client libraries): their last feature release needs to happen before "Non-client library freeze" on April 2. Only bugfixes releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/ussuri branching request with the review (as an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/os-brick.yaml#n2) In the next weeks we will have deadlines for: * Client libraries (think python-*client libraries), which need to have their last feature release before "Client library freeze" (April 9) * Deliverables following a cycle-with-rc model (that would be most services), which observe a Feature freeze on that same date, April 9. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. As we are getting to the point of creating stable/ussuri branches, this would be a good point for teams to review membership in their $project-stable-maint groups. Once the stable/ussuri branches are cut for a repo, the ability to approve any necessary backports into those branches for ussuri will be limited to the members of that stable team. If there are any questions about stable policy or stable team membership, please reach out in the #openstack-stable channel. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: April 2 (R-6 week) Client library freeze: April 9 (R-5 week) Ussuri-3 milestone (including cycle highlights): April 9 (R-5 week) Cycle Highlights Due: April 9 (R-5 week) Ussuri final release: May 13 From akshata.g at altencalsoftlabs.com Thu Mar 26 17:46:09 2020 From: akshata.g at altencalsoftlabs.com (Akshata Jitendra Gage) Date: Thu, 26 Mar 2020 23:16:09 +0530 (IST) Subject: openstack-dev Tacker Message-ID: <698231858.231822.1585244769615.JavaMail.zimbra@altencalsoftlabs.com> Hello, I am new to Tacker. As mentioned in the installation guide i have installed tacker on my server. I need to know that is there any way available by which we can create "service function chaining" via Horizon? And is there any forum related to tacker on which i can raise my doubts and questions? Thanks & Regards Akshata Gage -------------- next part -------------- An HTML attachment was scrubbed... URL: From samuel at silverliningsys.com Thu Mar 26 18:09:41 2020 From: samuel at silverliningsys.com (Samuel Abdullah) Date: Fri, 27 Mar 2020 02:09:41 +0800 Subject: [Murano-Open-PaaS] Openstack using guacamole In-Reply-To: References: Message-ID: Hi, Does Anyone know how can i install guacamole in openstack environment? Even if its via murano? Any manual guideline? Best Regards Samuel On Fri, 27 Mar 2020, 00:05 Roman Gorshunov, wrote: > Hello Samuel, > > Thanks for your email. Yes, Guacamole can be installed as an app via > Murano. > Discussions about Open PaaS are now in openstack-discuss mailing list. > Please use the tag [Murano-Open-PaaS] in the subject line. I'm re-routing > your email. > > Here are docs for the Murano project [0]. > > [0] https://docs.openstack.org/murano/latest/ > > Best regards, > Roman Gorshunov > > On Thu, Mar 26, 2020 at 4:37 PM Samuel Abdullah < > samuel at silverliningsys.com> wrote: > >> Hi, >> >> Would like to ask if it is possible to run guacamole in openstack >> environment? >> Seek for help on how would this be possible as i saw that guacamole >> component are part of the openstack in your murano project >> >> Looking forward for your reply >> >> Best Regards >> >> -- >> >> >> >> www.silverliningsys.com >> >> >> *Abu Bakar Samuel Abdullah* >> >> Cloud Infrastructure & Operations >> >> >> P: +603-2712-0081 >> >> M: +60.12.654.5938 >> >> E: samuel at silverliningsys.com >> _______________________________________________ >> OpenStack-Infra mailing list >> OpenStack-Infra at lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziaul.ict2018 at gmail.com Fri Mar 27 12:50:24 2020 From: ziaul.ict2018 at gmail.com (Md. Ziaul Haque) Date: Fri, 27 Mar 2020 18:50:24 +0600 Subject: podman pull failed: Trying to pull Image Message-ID: Hi I have faced a problem with overcloud deploy. If anybody faces the same problem, please suggest to me how I shall solve this problem. My triple version is stein Here I have attached the ansible log [root at controller2 ~]# podman pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...ERRO[0000] Error pulling image ref // 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker:// 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client Failed Error: error pulling image " 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker:// 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client =========================================================== [root at controller2 ~]# podman info host: BuildahVersion: 1.9.0 Conmon: package: podman-1.4.4-4.el7.centos.x86_64 path: /usr/libexec/podman/conmon version: 'conmon version 0.3.0, commit: unknown' Distribution: distribution: '"centos"' version: "7" MemFree: 11154231296 MemTotal: 16655982592 OCIRuntime: package: runc-1.0.0-65.rc8.el7.centos.x86_64 path: /usr/bin/runc version: 'runc version spec: 1.0.1-dev' SwapFree: 4294963200 SwapTotal: 4294963200 arch: amd64 cpus: 8 hostname: controller2 kernel: 3.10.0-1062.18.1.el7.x86_64 os: linux rootless: false uptime: 17h 37m 6.86s (Approximately 0.71 days) registries: blocked: null insecure: null search: - registry.access.redhat.com - docker.io - registry.fedoraproject.org - quay.io - registry.centos.org store: ConfigFile: /etc/containers/storage.conf ContainerStore: number: 0 GraphDriverName: overlay GraphOptions: null GraphRoot: /var/lib/containers/storage GraphStatus: Backing Filesystem: xfs Native Overlay Diff: "true" Supports d_type: "true" Using metacopy: "false" ImageStore: number: 0 RunRoot: /var/run/containers/storage VolumePath: /var/lib/containers/storage/volumes If I set insecure registry by manual than it pulls the image Thanks Ziaul -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- "2020-03-27 08:53:49,453 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:45Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Failed", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "", "2020-03-27 08:53:49,454 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:49,762 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:49,763 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:50,030 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,030 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:50,120 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,121 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:50,313 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:47Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,313 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:50,379 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:47Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,379 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:52,620 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:49Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:52,620 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:52,886 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:49Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:52,887 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:53,164 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,165 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:53,268 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,268 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:53,429 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,429 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:53,495 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,496 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:56,241 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:52Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,242 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:56,528 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,528 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:56,620 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,620 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:56,764 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,765 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:56,878 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,879 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:56,936 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,936 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:59,453 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,454 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:59,670 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,671 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:59,779 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,779 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:59,896 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,896 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:00,046 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:00,046 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:00,121 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:57Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:00,121 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:02,688 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,688 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:54:02,688 ERROR: 55141 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:54:02,780 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,780 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:54:02,781 ERROR: 55143 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:54:02,945 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,946 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:54:02,946 ERROR: 55139 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:54:03,053 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,053 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:03,054 ERROR: 55142 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:03,194 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:54:00Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,195 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:03,195 ERROR: 55144 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:03,270 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:54:00Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,271 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:03,271 ERROR: 55140 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:05,829 ERROR: 55141 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-nova_libvirt', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,nova_config,nova_paste_api_ini,libvirtd_config,nova_config,file,libvirt_tls_password', '--env', u'NAME=nova_libvirt', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpIn2bqO:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-nova_libvirt.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:54:02Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", " attempt(s): 1", "2020-03-27 08:54:05,829 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:05,905 ERROR: 55143 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-crond', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron', '--env', u'NAME=crond', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpco_ms_:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-crond.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:54:02Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:05,905 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:06,070 ERROR: 55139 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-ceilometer', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,ceilometer_config', '--env', u'NAME=ceilometer', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpeW3EiB:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-ceilometer.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,071 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:06,205 ERROR: 55142 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-ovn_controller', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', u'NAME=ovn_controller', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpYSZG7j:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/lib/modules:/lib/modules:ro', '--volume', u'/run/openvswitch:/run/openvswitch:shared,z', '--volume', u'/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,205 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:06,345 ERROR: 55144 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-neutron', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,neutron_config,ovn_metadata_agent_config', '--env', u'NAME=neutron', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpR86uEY:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-neutron.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/lib/modules:/lib/modules:ro', '--volume', u'/run/openvswitch:/run/openvswitch:shared,z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,346 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:06,410 ERROR: 55140 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-iscsid', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,iscsid_config', '--env', u'NAME=iscsid', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpf41yRW:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-iscsid.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/etc/iscsi:/etc/iscsi:z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,411 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:08,938 ERROR: 55141 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-nova_libvirt'] run failed after Error: unable to find container container-puppet-nova_libvirt: no container with name or ID container-puppet-nova_libvirt found: no such container", " attempt(s): 2", "2020-03-27 08:54:08,939 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:09,012 ERROR: 55143 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-crond'] run failed after Error: unable to find container container-puppet-crond: no container with name or ID container-puppet-crond found: no such container", "2020-03-27 08:54:09,012 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:09,178 ERROR: 55139 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ceilometer'] run failed after Error: unable to find container container-puppet-ceilometer: no container with name or ID container-puppet-ceilometer found: no such container", "2020-03-27 08:54:09,178 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:09,352 ERROR: 55142 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ovn_controller'] run failed after Error: unable to find container container-puppet-ovn_controller: no container with name or ID container-puppet-ovn_controller found: no such container", "2020-03-27 08:54:09,353 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:09,470 ERROR: 55144 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-neutron'] run failed after Error: unable to find container container-puppet-neutron: no container with name or ID container-puppet-neutron found: no such container", "2020-03-27 08:54:09,470 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:09,536 ERROR: 55140 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-iscsid'] run failed after Error: unable to find container container-puppet-iscsid: no container with name or ID container-puppet-iscsid found: no such container", "2020-03-27 08:54:09,537 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:12,065 ERROR: 55141 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-nova_libvirt'] run failed after Error: unable to find container container-puppet-nova_libvirt: no container with name or ID container-puppet-nova_libvirt found: no such container", " attempt(s): 3", "2020-03-27 08:54:12,065 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:12,065 ERROR: 55141 -- Failed running container for nova_libvirt", "2020-03-27 08:54:12,065 INFO: 55141 -- Finished processing puppet configs for nova_libvirt", "2020-03-27 08:54:12,130 ERROR: 55143 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-crond'] run failed after Error: unable to find container container-puppet-crond: no container with name or ID container-puppet-crond found: no such container", "2020-03-27 08:54:12,130 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:12,130 ERROR: 55143 -- Failed running container for crond", "2020-03-27 08:54:12,130 INFO: 55143 -- Finished processing puppet configs for crond", "2020-03-27 08:54:12,303 ERROR: 55139 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ceilometer'] run failed after Error: unable to find container container-puppet-ceilometer: no container with name or ID container-puppet-ceilometer found: no such container", "2020-03-27 08:54:12,304 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:12,304 ERROR: 55139 -- Failed running container for ceilometer", "2020-03-27 08:54:12,304 INFO: 55139 -- Finished processing puppet configs for ceilometer", "2020-03-27 08:54:12,537 ERROR: 55142 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ovn_controller'] run failed after Error: unable to find container container-puppet-ovn_controller: no container with name or ID container-puppet-ovn_controller found: no such container", "2020-03-27 08:54:12,538 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:12,538 ERROR: 55142 -- Failed running container for ovn_controller", "2020-03-27 08:54:12,538 INFO: 55142 -- Finished processing puppet configs for ovn_controller", "2020-03-27 08:54:12,670 ERROR: 55144 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-neutron'] run failed after Error: unable to find container container-puppet-neutron: no container with name or ID container-puppet-neutron found: no such container", "2020-03-27 08:54:12,671 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:12,671 ERROR: 55144 -- Failed running container for neutron", "2020-03-27 08:54:12,671 INFO: 55144 -- Finished processing puppet configs for neutron", "2020-03-27 08:54:12,728 ERROR: 55140 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-iscsid'] run failed after Error: unable to find container container-puppet-iscsid: no container with name or ID container-puppet-iscsid found: no such container", "2020-03-27 08:54:12,729 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:12,729 ERROR: 55140 -- Failed running container for iscsid", "2020-03-27 08:54:12,729 INFO: 55140 -- Finished processing puppet configs for iscsid", "2020-03-27 08:54:12,730 ERROR: 55136 -- ERROR configuring ceilometer", "2020-03-27 08:54:12,730 ERROR: 55136 -- ERROR configuring iscsid", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring nova_libvirt", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring ovn_controller", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring crond", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring neutron" ] } 2020-03-27 14:56:30,243 p=54331 u=mistral | PLAY RECAP ********************************************************************* 2020-03-27 14:56:30,243 p=54331 u=mistral | compute1 : ok=182 changed=49 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | compute2 : ok=182 changed=49 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller1 : ok=233 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller2 : ok=233 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller3 : ok=234 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,245 p=54331 u=mistral | undercloud : ok=12 changed=7 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 2020-03-27 14:56:30,252 p=54331 u=mistral | Friday 27 March 2020 14:56:30 +0600 (0:00:01.749) 0:12:59.282 ********** 2020-03-27 14:56:30,252 p=54331 u=mistral | =============================================================================== The network path was not found. From skaplons at redhat.com Fri Mar 27 15:51:17 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 27 Mar 2020 16:51:17 +0100 Subject: [election][neutron] PTL nomination Message-ID: Hi, I want to propose my candidacy for Neutron PTL in the Victoria cycle. I have been serving this great team as PTL in the Ussuri cycle already and I would love to continue this in the next cycle as well. Ussuri was my first time as PTL and I learned a lot during this cycle. I think that this new experience and knowledge which I now have will help me serve as PTL in this next cycle as well. In Ussuri I proposed a couple major goals which I wanted to focus on: * merge networking-ovn driver into main Neutron repository - in Ussuri we actually merged networking-ovn code into neutron and made ovn driver as one of the main in-tree drivers for ML2. We also merged the networking-ovn core team into the neutron core team. * Improve overall stability of our CI - we are continuously working to improve CI stability: during this cycle we added couple OVN related jobs to Neutron CI, and we also removed some unnecessary jobs. Unfortunately we didn’t make enough progress with our CI and some jobs still need a lot of work to improve their stability. I think that this is a continuous effort and we will need to focus on it in the next cycle as well. * Improve test coverage for existing features - we introduced many new test cases in the neutron-tempest-plugin repository. Many of them are covering some additional scenarios and advanced functionality: multicast traffic, TCP and UDP port forwarding, and others. I think that this is a continuous effort and we will need to focus on that one in the next cycle as well. * Finish the transition to the new engine facade - this is something which we didn’t make enough progress so far and I would like to treat it as a priority in Victoria. * Improve the stability of Neutron when it is running under uWSGI - recently we promoted our uWSGI jobs to be voting. Those jobs are now pretty stable. I think that in the next cycle we need to finally focus on making uwsgi as the default in Devstack, so that it will be used in most Neutron CI job. In addition to that I wanted to continue Miguel’s way of mentoring potential new core reviewers and contributors, and I think that this has been working pretty fine. In addition to the networking-ovn core team which is now part of neutron core, we also have new core reviewer Lajos Katona and new member of drivers team: Nate Johnston. Our team is now in good shape and we have many active core reviewers and contributors. That’s my summary of what we actually achieved in the Ussuri cycle. Now let’s talk about what are my goals for the Victoria cycle. * Continue adoption of the OVN driver in Neutron - now that the OVN is in-tree, we need to think about making it the default in Devstack. * Finish adoption of new engine facade - it’s unfinished goal from the Ussuri cycle and I think that we should treat it as a priority now, * Metadata over IPv6 - we still have not finished the RFE about support for Metadata service in IPv6 only networks. In the Ussuri cycle we made some progress with the spec [1], and we also had some discussions about it with cloud-init developers. Now I think it’s time to finally get this spec merged and implement it in Neutron, * Continue work on test coverage and CI stability - this is a never ending story and I think that it should be treated with high priority all the time. It is my continuing desire to do my best to help our outstanding team to deliver better software and to grow. [1] https://review.opendev.org/#/c/315604/ — Slawek Kaplonski Senior software engineer Red Hat From gmann at ghanshyammann.com Fri Mar 27 16:07:03 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 27 Mar 2020 11:07:03 -0500 Subject: [tacker]Re: openstack-dev Tacker In-Reply-To: <698231858.231822.1585244769615.JavaMail.zimbra@altencalsoftlabs.com> References: <698231858.231822.1585244769615.JavaMail.zimbra@altencalsoftlabs.com> Message-ID: <1711cbf2cea.f0f3f37190552.2803634146173182326@ghanshyammann.com> ---- On Thu, 26 Mar 2020 12:46:09 -0500 Akshata Jitendra Gage wrote ---- > Hello, > I am new to Tacker. As mentioned in the installation guide i have installed tacker on my server. I need to know that is there any way available by which we can create "service function chaining" via Horizon? > And is there any forum related to tacker on which i can raise my doubts and questions? Hi Akshata, Not sure about Horizon has this feature. tacker-horizon is the plugin for tacker UI things. I have tagged 'tacker' in sub to notify the tacker team. We have IRC channel #tacker where you can find the tacker team and raise your queries. Info on how to join IRC: https://docs.openstack.org/contributors/common/irc.html > Thanks & Regards > Akshata Gage From iurygregory at gmail.com Fri Mar 27 16:11:23 2020 From: iurygregory at gmail.com (Iury Gregory) Date: Fri, 27 Mar 2020 17:11:23 +0100 Subject: [ironic][ops] Stable Ocata and Pike branches are now Unmaintained Message-ID: Greetings Ironicers and Operators, During the ironic upstream meeting held on 09 March 2020, we decided to move stable/ocata and stable/pike to unmaintained [1]. Apologies for sending this email almost 3 weeks after the decision was made. We have documented what unmaintained means in our contributor docs [2]. Feel free to rise any questions here or in the irc channel (#openstack-ironic) [1] http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2020-03-09.log.html#t2020-03-09T15:30:16 [2] https://docs.openstack.org/ironic/latest/contributor/#our-policy-for-stable-branches Best regards, -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Fri Mar 27 16:24:56 2020 From: donny at fortnebula.com (Donny Davis) Date: Fri, 27 Mar 2020 12:24:56 -0400 Subject: [tc][elections] Nomination Message-ID: Hello Openstack Humanoids, My name is Donny Davis and I would like to announce my candidacy for TC. I built my first Openstack cloud by hand on the Grizzly release to provide a training environment for fellow employees who lacked Satellite equipment to train on. I have spent subsequent years of my life building, maintaining, operating, and teaching other people how to Openstack. I contribute Openstack CI resources via Fort Nebula (last cycle), and this cycle via Open Edge. This is a cloud that has been tuned specifically for the Openstack Community CI workload. I support many of the obscure requirements to run non-standard CI jobs on this cloud. This Openstack deployment is all my personal equipment and is 100% privately funded. In the last cycle my commit for test instances was mostly 50 concurrent jobs, and this cycle is mostly 40. As a member of the TC, I bring close to a decade of experience as an Operator. I feel the Openstack community can continue to deliver quality code while focusing on easing the burden of getting Openstack clouds into production. For a long time the focus of this community has been to develop bleeding edge features to deliver technical value to consumers. Now this community should focus on making those features easier to consume. I can view the TC through the eyes of a builder, tinkerer, trainer, and long time operator of Openstack. Thank you for your consideration. -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From rfolco at redhat.com Fri Mar 27 16:44:22 2020 From: rfolco at redhat.com (Rafael Folco) Date: Fri, 27 Mar 2020 13:44:22 -0300 Subject: [tripleo] TripleO CI Summary: Sprint 44 Message-ID: Greetings, The TripleO CI team has just completed Sprint 44 / Unified Sprint 23 (Mar 05 thru Mar 25). The following is a summary of completed work during this sprint cycle: - Migrated stable branches from rdo-cloud to vexxhost - Upstream TripleO migrated to CentOS-8. - Upstream component pipeline done. - RDO content unpinned - First CentOS-8 promotion on March 16th - Promotion server is online and ready for CentOS-8 - Completed development of unified logging with tripleo-quickstart and infrared, roll out in progress Ruck/Rover Notes: - https://hackmd.io/7MBqFHurTA2e5H8kYRwgag The planned work for the next sprint [1] extends the work started in the previous sprint and focuses on the following: - TripleO-Operator-Ansible: contribute w/ code, docs, tests and zuul jobs. - Internal component pipeline: starting work - Unified tempest test skip list The Ruck and Rover for this sprint are Wesley Hayutin (weshay) and Sandeep Yadav (ysandeep). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes to be tracked in etherpad [2]. Thanks, rfolco [1] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-24 [2] https://hackmd.io/XMd_vskSTdGZl7ruZ1UckA -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Fri Mar 27 17:05:30 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Fri, 27 Mar 2020 13:05:30 -0400 Subject: [openstack-ansible] proposing Erik Berg to openstack-ansible core In-Reply-To: <0d12ff74-6d7e-bbcf-dbd0-dfd8f7aa0402@rd.bbc.co.uk> References: <0d12ff74-6d7e-bbcf-dbd0-dfd8f7aa0402@rd.bbc.co.uk> Message-ID: I've added Erik to openstack-ansible-core. Welcome Erik. :) On Wed, Mar 25, 2020 at 11:49 AM Jonathan Rosser wrote: > > +1 from me. > > Jon. > > On 25/03/2020 15:27, Mohammed Naser wrote: > > Hi everyone, > > > > I'd like to propose adding Erik Berg to the OSA core team. They've > > been doing solid contributions with some activity in reviews and I > > think they'd be an excellent addition to the OSA team > > > > If no one opposes, I'll be adding them to the core team in the next few days. > > > > Regards, > > Mohammed > > > -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From ziaul.ict2018 at gmail.com Fri Mar 27 17:39:51 2020 From: ziaul.ict2018 at gmail.com (Md. Ziaul Haque) Date: Fri, 27 Mar 2020 23:39:51 +0600 Subject: Fwd: podman pull failed: Trying to pull Image In-Reply-To: References: Message-ID: Hi I have faced a problem with overcloud deploy. If anybody faces the same problem, please suggest to me how I shall solve this problem. My triple version is stein Here I have attached the ansible log. Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...ERRO[0000] Error pulling image ref // 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker:// 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client Failed Error: error pulling image " 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker:// 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client =========================================================== Thanks Ziaul -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- "2020-03-27 08:53:49,453 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:45Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Failed", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "", "2020-03-27 08:53:49,454 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:49,762 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:49,763 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:50,030 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,030 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:50,120 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,121 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:50,313 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:47Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,313 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:50,379 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:47Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,379 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:52,620 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:49Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:52,620 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:52,886 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:49Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:52,887 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:53,164 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,165 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:53,268 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,268 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:53,429 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,429 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:53,495 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,496 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:56,241 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:52Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,242 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:56,528 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,528 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:56,620 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,620 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:56,764 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,765 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:56,878 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,879 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:56,936 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,936 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:59,453 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,454 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:59,670 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,671 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:59,779 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,779 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:59,896 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,896 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:00,046 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:00,046 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:00,121 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:57Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:00,121 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:02,688 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,688 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:54:02,688 ERROR: 55141 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:54:02,780 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,780 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:54:02,781 ERROR: 55143 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:54:02,945 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,946 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:54:02,946 ERROR: 55139 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:54:03,053 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,053 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:03,054 ERROR: 55142 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:03,194 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:54:00Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,195 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:03,195 ERROR: 55144 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:03,270 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:54:00Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,271 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:03,271 ERROR: 55140 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:05,829 ERROR: 55141 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-nova_libvirt', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,nova_config,nova_paste_api_ini,libvirtd_config,nova_config,file,libvirt_tls_password', '--env', u'NAME=nova_libvirt', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpIn2bqO:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-nova_libvirt.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:54:02Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", " attempt(s): 1", "2020-03-27 08:54:05,829 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:05,905 ERROR: 55143 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-crond', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron', '--env', u'NAME=crond', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpco_ms_:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-crond.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:54:02Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:05,905 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:06,070 ERROR: 55139 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-ceilometer', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,ceilometer_config', '--env', u'NAME=ceilometer', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpeW3EiB:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-ceilometer.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,071 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:06,205 ERROR: 55142 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-ovn_controller', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', u'NAME=ovn_controller', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpYSZG7j:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/lib/modules:/lib/modules:ro', '--volume', u'/run/openvswitch:/run/openvswitch:shared,z', '--volume', u'/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,205 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:06,345 ERROR: 55144 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-neutron', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,neutron_config,ovn_metadata_agent_config', '--env', u'NAME=neutron', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpR86uEY:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-neutron.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/lib/modules:/lib/modules:ro', '--volume', u'/run/openvswitch:/run/openvswitch:shared,z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,346 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:06,410 ERROR: 55140 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-iscsid', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,iscsid_config', '--env', u'NAME=iscsid', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpf41yRW:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-iscsid.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/etc/iscsi:/etc/iscsi:z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,411 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:08,938 ERROR: 55141 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-nova_libvirt'] run failed after Error: unable to find container container-puppet-nova_libvirt: no container with name or ID container-puppet-nova_libvirt found: no such container", " attempt(s): 2", "2020-03-27 08:54:08,939 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:09,012 ERROR: 55143 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-crond'] run failed after Error: unable to find container container-puppet-crond: no container with name or ID container-puppet-crond found: no such container", "2020-03-27 08:54:09,012 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:09,178 ERROR: 55139 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ceilometer'] run failed after Error: unable to find container container-puppet-ceilometer: no container with name or ID container-puppet-ceilometer found: no such container", "2020-03-27 08:54:09,178 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:09,352 ERROR: 55142 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ovn_controller'] run failed after Error: unable to find container container-puppet-ovn_controller: no container with name or ID container-puppet-ovn_controller found: no such container", "2020-03-27 08:54:09,353 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:09,470 ERROR: 55144 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-neutron'] run failed after Error: unable to find container container-puppet-neutron: no container with name or ID container-puppet-neutron found: no such container", "2020-03-27 08:54:09,470 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:09,536 ERROR: 55140 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-iscsid'] run failed after Error: unable to find container container-puppet-iscsid: no container with name or ID container-puppet-iscsid found: no such container", "2020-03-27 08:54:09,537 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:12,065 ERROR: 55141 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-nova_libvirt'] run failed after Error: unable to find container container-puppet-nova_libvirt: no container with name or ID container-puppet-nova_libvirt found: no such container", " attempt(s): 3", "2020-03-27 08:54:12,065 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:12,065 ERROR: 55141 -- Failed running container for nova_libvirt", "2020-03-27 08:54:12,065 INFO: 55141 -- Finished processing puppet configs for nova_libvirt", "2020-03-27 08:54:12,130 ERROR: 55143 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-crond'] run failed after Error: unable to find container container-puppet-crond: no container with name or ID container-puppet-crond found: no such container", "2020-03-27 08:54:12,130 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:12,130 ERROR: 55143 -- Failed running container for crond", "2020-03-27 08:54:12,130 INFO: 55143 -- Finished processing puppet configs for crond", "2020-03-27 08:54:12,303 ERROR: 55139 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ceilometer'] run failed after Error: unable to find container container-puppet-ceilometer: no container with name or ID container-puppet-ceilometer found: no such container", "2020-03-27 08:54:12,304 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:12,304 ERROR: 55139 -- Failed running container for ceilometer", "2020-03-27 08:54:12,304 INFO: 55139 -- Finished processing puppet configs for ceilometer", "2020-03-27 08:54:12,537 ERROR: 55142 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ovn_controller'] run failed after Error: unable to find container container-puppet-ovn_controller: no container with name or ID container-puppet-ovn_controller found: no such container", "2020-03-27 08:54:12,538 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:12,538 ERROR: 55142 -- Failed running container for ovn_controller", "2020-03-27 08:54:12,538 INFO: 55142 -- Finished processing puppet configs for ovn_controller", "2020-03-27 08:54:12,670 ERROR: 55144 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-neutron'] run failed after Error: unable to find container container-puppet-neutron: no container with name or ID container-puppet-neutron found: no such container", "2020-03-27 08:54:12,671 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:12,671 ERROR: 55144 -- Failed running container for neutron", "2020-03-27 08:54:12,671 INFO: 55144 -- Finished processing puppet configs for neutron", "2020-03-27 08:54:12,728 ERROR: 55140 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-iscsid'] run failed after Error: unable to find container container-puppet-iscsid: no container with name or ID container-puppet-iscsid found: no such container", "2020-03-27 08:54:12,729 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:12,729 ERROR: 55140 -- Failed running container for iscsid", "2020-03-27 08:54:12,729 INFO: 55140 -- Finished processing puppet configs for iscsid", "2020-03-27 08:54:12,730 ERROR: 55136 -- ERROR configuring ceilometer", "2020-03-27 08:54:12,730 ERROR: 55136 -- ERROR configuring iscsid", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring nova_libvirt", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring ovn_controller", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring crond", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring neutron" ] } 2020-03-27 14:56:30,243 p=54331 u=mistral | PLAY RECAP ********************************************************************* 2020-03-27 14:56:30,243 p=54331 u=mistral | compute1 : ok=182 changed=49 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | compute2 : ok=182 changed=49 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller1 : ok=233 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller2 : ok=233 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller3 : ok=234 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,245 p=54331 u=mistral | undercloud : ok=12 changed=7 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 2020-03-27 14:56:30,252 p=54331 u=mistral | Friday 27 March 2020 14:56:30 +0600 (0:00:01.749) 0:12:59.282 ********** 2020-03-27 14:56:30,252 p=54331 u=mistral | =============================================================================== The network path was not found. From noonedeadpunk at ya.ru Fri Mar 27 18:45:29 2020 From: noonedeadpunk at ya.ru (Dmitriy Rabotyagov) Date: Fri, 27 Mar 2020 20:45:29 +0200 Subject: [openstack-ansible] proposing Erik Berg to openstack-ansible core In-Reply-To: References: <0d12ff74-6d7e-bbcf-dbd0-dfd8f7aa0402@rd.bbc.co.uk> Message-ID: <2807411585334727@iva4-35f072fa8e4e.qloud-c.yandex.net> An HTML attachment was scrubbed... URL: From mordred at inaugust.com Fri Mar 27 19:18:28 2020 From: mordred at inaugust.com (Monty Taylor) Date: Fri, 27 Mar 2020 14:18:28 -0500 Subject: [sdk][osc][ptl][election] OpenStackSDK PTL Nomination Message-ID: Hi everybody! I'd like to run for PTL of OpenStackSDK again. We just merged the OpenStackClient and OpenStackSDK teams. While on the one hand this is a nice big new challenge, I think it also presents an excellent opportunity to accelerate our long-standing goal of integrating the two more seamlessly. I'm excited to get some motion on that. In this last cycle we finally got started on the long-awaited python-*client-ectomy from OpenStackClient by removing glanceclient. I'm hoping in the next cycle we can at least remove keystoneclient - but I think we can pick up the pace now. We need to streamline how config and arguments are parsed so that it's 100% consistent between OSC and SDK and Ansible. This is going to mean using openstack.config in OpenStackClient more directly - and I think it's going to mean an end to the double-auth dance. That, in turn, will likely tie in to attempting to unwind the plugin structure from entrypoints and laying in an optimization so that we can skip entrypoints calls in most cases. We've recently been more aggressive with adding new project-specific cores and I'd like to continue that. We need to trust people from project teams to help maintain relevant code in SDK and OSC, and as a result get into the habit of getting support for new things into SDK and OSC as a part of shipping them in the service. That leads to our need to get to the point where both SDK and OpenStackClient can speak to the latest microversion for the projects supporting microversions. This has been mentioned as one of the things people from the projects don't like about OSC vs the project-specific CLIs. The thing is, we will already use the most recent microversion we understand - so we already behave like people want us to. The thing we need to get to is understanding the MV as it's added. The momentum is picking up to get a lot accomplished, and I'd love to keep helping to pushing these rocks uphill. Thanks! Monty From mordred at inaugust.com Fri Mar 27 19:24:08 2020 From: mordred at inaugust.com (Monty Taylor) Date: Fri, 27 Mar 2020 14:24:08 -0500 Subject: Upcoming pip changes - stop installing test-requirements in devstack Message-ID: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> Heya, Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us. The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent. I’ve pushed up this: https://review.opendev.org/#/c/715469/ Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now. It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer. Monty From zbitter at redhat.com Fri Mar 27 21:38:26 2020 From: zbitter at redhat.com (Zane Bitter) Date: Fri, 27 Mar 2020 17:38:26 -0400 Subject: [all][tc] Post-mortem of my time on the TC Message-ID: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> As promised last year, I will not be running for a third term as a member of the Technical Committee. I'm stepping down to spend more time with my family (no, really! ;) I thought this would be a good opportunity to talk to those folks who are thinking of running for a seat, or who aren't but should be. The most common question I hear from potential candidates is "what should I actually do on the TC?" I can't tell you what *you* should do, but I can tell you what I did and how that worked out. Maybe that will give you the flavour. And perhaps (he hinted) some of the other outgoing TC members can do the same to provide more of a cross-section. * I drafted and edited the review guide, "How to Review Changes the OpenStack Way", to try to ensure that the best parts of our review culture are evenly distributed across the project: https://docs.openstack.org/project-team-guide/review-the-openstack-way.html (This started as part of an initiative from Julia Kreger to reduce nitpicking in reviews.) In theory anybody could have done this, but in practice I think this is one of those things that only somebody elected to the TC could have driven. I received a lot of feedback afterwards (which is unusual for anything the TC does): several people, and one whole team, mentioned that they had tweaked their reviewing style after this discussion. And hearing their feedback prompted me to tweak my own reviewing style as well. * I drafted and edited the Vision for OpenStack Clouds: https://governance.openstack.org/tc/reference/technical-vision.html. This was a ton of work as it involved consulting with so many people: first, folks in the community who on the surface appeared to have quite different visions for what OpenStack should be, who later turned out to not be so far apart as we all thought. After a lengthy round of reviews, I approached every project team individually to explain the specific relevance to them, and seek their feedback. Finally, I presented it to the OSF Board and held an in-person feedback session at the Forum, after all of which it was approved by the TC. I have some mixed feelings about the success of this. I still believe it was desperately needed... in about 2013. Now the Kubernetes community is succeeding at engaging with actual end users (not just operators), which OpenStack has always failed to do. With contributions to OpenStack dropping, it's not clear that we can, or necessarily should, sustain such a broad vision. It is my hope that if and when the community needs to discuss adjusting our scope, we will do so by amending this document - going through it section by section and deciding which parts to double down on and which to allow to fall by the wayside - and not by a series of ad-hoc actions that leave everybody confused as to what the project is supposed to be once again. On a personal level, the vision has been extremely valuable for me because it required me to spend a lot of time thinking deeply about the various components of OpenStack and how they fit into a greater purpose. A large portion of the value comes from participating in the process, not necessarily the final document. I hope that all of the folks who helped along the way got as much out of it as I did. * I developed the process by which we keep OpenStack up to date with new releases of Python: https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html I also shepherded it through the first couple of release cycles, to try to entrench it as a standard part of each release. (BTW this task needs somebody to take it over.) This mostly involved pointing people back to the resolution and inviting them to formally update it every time someone tried to bikeshed the details, which happened more than I ever believed possible. (To date there have been no changes proposed to the resolution.) * I engaged with the OSF Board to pass on feedback from members in the developer community on the process for adding new Open Infrastructure projects (e.g. Zuul, StarlingX, Airship, Kata). Sometimes the work of the TC is completely invisible! I also tried to convey information the other way, publicising information that was otherwise only available by attending board meetings. (Most TC members usually attend board meetings, including the in-person joint leadership meetings held before Summits.) * In response to feedback from the board, I replaced the "Help Most Wanted" list with the "Upstream Investment Opportunities" list, and kick-started the rewriting of the existing entries in a format that is more targeted at decision-makers: https://governance.openstack.org/tc/reference/upstream-investment-opportunities/index.html (Huge thanks to Tom Barron, who did even more work to complete the job of converting all of the existing entries - proof that you don't need to be a TC member to start contributing to governance activities.) It remains to be seen whether this will have any effect, but this content is now being shared in the community newsletter, and the board is making moves to use it to actively solicit contributions to the community. Previously they felt unable to make use of the Help Most Wanted list because it was written with the wrong audience in mind. * I tweaked the guidelines for the release naming system to better align with our actual practice, because they had diverged somewhat. Believe it or not, this one issue took up more of the TC's time than any other thing that happened in 2019, and I'm very glad that we ended up not needing these tweaks because we eventually replaced the process wholesale, with one that makes it explicit that the TC is ultimately accountable for the release name not causing any international incidents or other snafus. * I researched and documented a design, developed during extensive discussions with many members of the technical community, for a new kind of simplified cloud built partially using OpenStack components but designed to be Kubernetes-native first: Project Teapot https://governance.openstack.org/ideas/ideas/teapot/index.html I do think this is worth building, but if nothing else I really hope this helps bring into focus the parts of OpenStack that remain essential in a Kubernetes-centric world. In writing this I was very much guided by the Vision for OpenStack Clouds as well as the ways in which Kubernetes expects to interact with an underlying cloud. I think we need more of these kinds of exercises, and while they don't have to be sponsored by TC members, I think it helps. If there's a pattern here it's this: take something that's implicitly agreed upon by the community (whether we realise it or not), write it down, seek feedback, and make it explicit so that anyone can refer to it, rather than leaving it as closely held tribal knowledge. Of course it goes without saying that I was involved in many more discussions and reviews that were led by others. I'll leave it to them to talk about those. And it should also go without saying that all of the work listed above was greatly improved by the input of the many talented folks in our community who contributed. In my humble opinion OpenStack is the best open source community of its size ever created. That's because we believe not just in one 'open' (source), but four (https://governance.openstack.org/tc/reference/opens.html); because we're committed to using open tools to build open software; and because, while it's certainly not the case that we can't improve, we have managed to do this while avoiding a widespread culture of bullying. To do all of this on the scale of OpenStack was, as far as I know, completely without precedent. (Perhaps the ASF comes closest, but its projects operate independently, unlike OpenStack subprojects.) All of us should be proud of this achievement. But it didn't happen by accident: it is the product of concerted effort by leaders in our community over the course of a decade. It's been a tremendous privilege to work alongside some of those folks, both before, during and (hopefully) after my time on the TC. I have learned so much from them. Now it is time for others to step up and continue to build that legacy, as the project evolves and adapts over time. The diversification of the OSF also gives us the opportunity to spread our ethos to other projects under the same umbrella, and from there to the open source community as a whole. However, we must remember that having a great community is not enough. To remain relevant, we also need to produce software that meets users' needs, which change over time. By an accident of fate I have approached OpenStack from a different direction than most - I came to OpenStack via the Heat project, which means I've always looked at it from an outside-in standpoint: users are building cloud applications, how can OpenStack provide them with what they need from a cloud through an open API? Many others, by necessity, have approached it from the inside-out: we have all this hardware, how can we expose its capabilities to users? I think it is fair to say that many of these folks and I have found each other's views to be mutually incomprehensible over the years. In writing up the Teapot design, I have learned a lot more about the other perspective. I believe we will need a more people to also grow their understanding in the opposite direction to avoid the trap of most capital-E "Enterprise" software - designed to check the boxes of those who decide what products to buy, but widely hated by all those forced to actually use it. Future TCs will have to grapple with this because nobody else in the community is really empowered to do so. The openstack/ideas repo that JP created is a great place for anybody to start documenting ways that we can improve. For my part, this is not goodbye. However, I have already been splitting my time between OpenStack and metal3.io and I expect that the drift in the other direction will only accelerate now that I no longer have any formal responsibilities here. Please feel free to contact me or ask questions if you think I can help. Stay safe out there, and by "out there" I mean physically isolated in your own homes for the next little while. cheers, Zane. From radoslaw.piliszek at gmail.com Sat Mar 28 08:34:39 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Sat, 28 Mar 2020 09:34:39 +0100 Subject: Upcoming pip changes - stop installing test-requirements in devstack In-Reply-To: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> References: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> Message-ID: Hiya, it's now going in. Dev Folks: if it breaks jobs and you need assistance, please respond to this thread or ping on #openstack-qa -yoctozepto pt., 27 mar 2020 o 20:33 Monty Taylor napisał(a): > > Heya, > > Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us. > > The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent. > > I’ve pushed up this: > > https://review.opendev.org/#/c/715469/ > > Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now. > > It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer. > > Monty From zigo at debian.org Sat Mar 28 09:29:44 2020 From: zigo at debian.org (Thomas Goirand) Date: Sat, 28 Mar 2020 10:29:44 +0100 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages Message-ID: Hi, Sphinx 2.4 has been uploaded to Experimental. I just received bug reports against the OpenStack packages because some of them cannot be built with Sphinx 2.4. Here's the list: - ironic - os-api-ref - aodh - cloudkitty - kombu - panko - ceilometerclient - ceilometermiddleware - doc8 - dracclient - glareclient - murano-pkg-check - os-xenapi - scciclient - sahara - sphinxcontrib-programoutput - tempest-horizon It'd be nice if those affected could be fixed for Ussuri. IMO, it's just the doc, so fixing it wouldn't hurt code quality. In other words: help would be more than welcome writing patch or reviewing them, and testing Sphinx 2.4 compatibility. Cheers, Thomas Goirand (zigo) From iurygregory at gmail.com Sat Mar 28 09:35:44 2020 From: iurygregory at gmail.com (Iury Gregory) Date: Sat, 28 Mar 2020 06:35:44 -0300 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: Hi Thomas, Thanks for raising this, I will take a look at the ironic documentation to make it compatible with Sphinx 2.4. Em sáb., 28 de mar. de 2020 às 06:32, Thomas Goirand escreveu: > Hi, > > Sphinx 2.4 has been uploaded to Experimental. I just received bug > reports against the OpenStack packages because some of them cannot be > built with Sphinx 2.4. Here's the list: > > - ironic > - os-api-ref > - aodh > - cloudkitty > - kombu > - panko > - ceilometerclient > - ceilometermiddleware > - doc8 > - dracclient > - glareclient > - murano-pkg-check > - os-xenapi > - scciclient > - sahara > - sphinxcontrib-programoutput > - tempest-horizon > > It'd be nice if those affected could be fixed for Ussuri. IMO, it's just > the doc, so fixing it wouldn't hurt code quality. In other words: help > would be more than welcome writing patch or reviewing them, and testing > Sphinx 2.4 compatibility. > > Cheers, > > Thomas Goirand (zigo) > > -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziaul.ict2018 at gmail.com Sat Mar 28 15:37:08 2020 From: ziaul.ict2018 at gmail.com (Md. Ziaul Haque) Date: Sat, 28 Mar 2020 21:37:08 +0600 Subject: Fwd: podman pull failed: Trying to pull Image In-Reply-To: References: Message-ID: Hi I have faced a problem with overcloud deploy. If anybody faces the same problem, please suggest to me how I shall solve this problem. My triple version is stein Here I have attached the ansible log. Thanks Ziaul -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- "2020-03-27 08:53:49,453 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:45Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Failed", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "", "2020-03-27 08:53:49,454 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:49,762 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:49,763 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:50,030 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,030 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:50,120 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:46Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,121 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:50,313 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:47Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,313 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:50,379 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:47Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: error pulling image \"192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo\": unable to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:53:50,379 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:52,620 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:49Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:52,620 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:52,886 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:49Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:52,887 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:53,164 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,165 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:53,268 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,268 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:53,429 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,429 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:53,495 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:50Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:53,496 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:56,241 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:52Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,242 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:56,528 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,528 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:56,620 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,620 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:56,764 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,765 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:53:56,878 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,879 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:53:56,936 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:53Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:56,936 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:53:59,453 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,454 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:53:59,670 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,671 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:53:59,779 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,779 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:53:59,896 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:53:59,896 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:00,046 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:53:56Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:00,046 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:00,121 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:53:57Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:00,121 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:02,688 WARNING: 55141 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,688 WARNING: 55141 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:54:02,688 ERROR: 55141 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo", "2020-03-27 08:54:02,780 WARNING: 55143 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,780 WARNING: 55143 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:54:02,781 ERROR: 55143 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo", "2020-03-27 08:54:02,945 WARNING: 55139 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:02,946 WARNING: 55139 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:54:02,946 ERROR: 55139 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo", "2020-03-27 08:54:03,053 WARNING: 55142 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:53:59Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,053 WARNING: 55142 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:03,054 ERROR: 55142 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo", "2020-03-27 08:54:03,194 WARNING: 55144 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:54:00Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,195 WARNING: 55144 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:03,195 ERROR: 55144 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo", "2020-03-27 08:54:03,270 WARNING: 55140 -- podman pull failed: Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:54:00Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "2020-03-27 08:54:03,271 WARNING: 55140 -- retrying pulling image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:03,271 ERROR: 55140 -- Failed to pull image: 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo", "2020-03-27 08:54:05,829 ERROR: 55141 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-nova_libvirt', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,nova_config,nova_paste_api_ini,libvirtd_config,nova_config,file,libvirt_tls_password', '--env', u'NAME=nova_libvirt', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpIn2bqO:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-nova_libvirt.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo...time=\"2020-03-27T08:54:02Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-nova-compute:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", " attempt(s): 1", "2020-03-27 08:54:05,829 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:05,905 ERROR: 55143 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-crond', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron', '--env', u'NAME=crond', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpco_ms_:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-crond.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo...time=\"2020-03-27T08:54:02Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-cron:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:05,905 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:06,070 ERROR: 55139 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-ceilometer', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,ceilometer_config', '--env', u'NAME=ceilometer', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpeW3EiB:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-ceilometer.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ceilometer-central:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,071 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:06,205 ERROR: 55142 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-ovn_controller', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', u'NAME=ovn_controller', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpYSZG7j:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/lib/modules:/lib/modules:ro', '--volume', u'/run/openvswitch:/run/openvswitch:shared,z', '--volume', u'/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-ovn-controller:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,205 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:06,345 ERROR: 55144 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-neutron', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,neutron_config,ovn_metadata_agent_config', '--env', u'NAME=neutron', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpR86uEY:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-neutron.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/lib/modules:/lib/modules:ro', '--volume', u'/run/openvswitch:/run/openvswitch:shared,z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-neutron-server-ovn:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,346 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:06,410 ERROR: 55140 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', u'container-puppet-iscsid', '--env', u'PUPPET_TAGS=file,file_line,concat,augeas,cron,iscsid_config', '--env', u'NAME=iscsid', '--env', u'HOSTNAME=compute2', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpf41yRW:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--log-driver', 'k8s-file', '--log-opt', u'path=/var/log/containers/stdouts/container-puppet-iscsid.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', u'/etc/iscsi:/etc/iscsi:z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', u'192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo'] run failed after Trying to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo...time=\"2020-03-27T08:54:03Z\" level=error msg=\"Error pulling image ref //192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client\"", "Error: unable to pull 192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: unable to pull image: Error initializing source docker://192.168.90.10:8787/tripleostein/centos-binary-iscsid:current-tripleo: pinging docker registry returned: Get https://192.168.90.10:8787/v2/: http: server gave HTTP response to HTTPS client", "2020-03-27 08:54:06,411 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:08,938 ERROR: 55141 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-nova_libvirt'] run failed after Error: unable to find container container-puppet-nova_libvirt: no container with name or ID container-puppet-nova_libvirt found: no such container", " attempt(s): 2", "2020-03-27 08:54:08,939 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:09,012 ERROR: 55143 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-crond'] run failed after Error: unable to find container container-puppet-crond: no container with name or ID container-puppet-crond found: no such container", "2020-03-27 08:54:09,012 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:09,178 ERROR: 55139 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ceilometer'] run failed after Error: unable to find container container-puppet-ceilometer: no container with name or ID container-puppet-ceilometer found: no such container", "2020-03-27 08:54:09,178 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:09,352 ERROR: 55142 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ovn_controller'] run failed after Error: unable to find container container-puppet-ovn_controller: no container with name or ID container-puppet-ovn_controller found: no such container", "2020-03-27 08:54:09,353 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:09,470 ERROR: 55144 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-neutron'] run failed after Error: unable to find container container-puppet-neutron: no container with name or ID container-puppet-neutron found: no such container", "2020-03-27 08:54:09,470 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:09,536 ERROR: 55140 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-iscsid'] run failed after Error: unable to find container container-puppet-iscsid: no container with name or ID container-puppet-iscsid found: no such container", "2020-03-27 08:54:09,537 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:12,065 ERROR: 55141 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-nova_libvirt'] run failed after Error: unable to find container container-puppet-nova_libvirt: no container with name or ID container-puppet-nova_libvirt found: no such container", " attempt(s): 3", "2020-03-27 08:54:12,065 WARNING: 55141 -- Retrying running container: nova_libvirt", "2020-03-27 08:54:12,065 ERROR: 55141 -- Failed running container for nova_libvirt", "2020-03-27 08:54:12,065 INFO: 55141 -- Finished processing puppet configs for nova_libvirt", "2020-03-27 08:54:12,130 ERROR: 55143 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-crond'] run failed after Error: unable to find container container-puppet-crond: no container with name or ID container-puppet-crond found: no such container", "2020-03-27 08:54:12,130 WARNING: 55143 -- Retrying running container: crond", "2020-03-27 08:54:12,130 ERROR: 55143 -- Failed running container for crond", "2020-03-27 08:54:12,130 INFO: 55143 -- Finished processing puppet configs for crond", "2020-03-27 08:54:12,303 ERROR: 55139 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ceilometer'] run failed after Error: unable to find container container-puppet-ceilometer: no container with name or ID container-puppet-ceilometer found: no such container", "2020-03-27 08:54:12,304 WARNING: 55139 -- Retrying running container: ceilometer", "2020-03-27 08:54:12,304 ERROR: 55139 -- Failed running container for ceilometer", "2020-03-27 08:54:12,304 INFO: 55139 -- Finished processing puppet configs for ceilometer", "2020-03-27 08:54:12,537 ERROR: 55142 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-ovn_controller'] run failed after Error: unable to find container container-puppet-ovn_controller: no container with name or ID container-puppet-ovn_controller found: no such container", "2020-03-27 08:54:12,538 WARNING: 55142 -- Retrying running container: ovn_controller", "2020-03-27 08:54:12,538 ERROR: 55142 -- Failed running container for ovn_controller", "2020-03-27 08:54:12,538 INFO: 55142 -- Finished processing puppet configs for ovn_controller", "2020-03-27 08:54:12,670 ERROR: 55144 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-neutron'] run failed after Error: unable to find container container-puppet-neutron: no container with name or ID container-puppet-neutron found: no such container", "2020-03-27 08:54:12,671 WARNING: 55144 -- Retrying running container: neutron", "2020-03-27 08:54:12,671 ERROR: 55144 -- Failed running container for neutron", "2020-03-27 08:54:12,671 INFO: 55144 -- Finished processing puppet configs for neutron", "2020-03-27 08:54:12,728 ERROR: 55140 -- ['/usr/bin/podman', 'start', '-a', u'container-puppet-iscsid'] run failed after Error: unable to find container container-puppet-iscsid: no container with name or ID container-puppet-iscsid found: no such container", "2020-03-27 08:54:12,729 WARNING: 55140 -- Retrying running container: iscsid", "2020-03-27 08:54:12,729 ERROR: 55140 -- Failed running container for iscsid", "2020-03-27 08:54:12,729 INFO: 55140 -- Finished processing puppet configs for iscsid", "2020-03-27 08:54:12,730 ERROR: 55136 -- ERROR configuring ceilometer", "2020-03-27 08:54:12,730 ERROR: 55136 -- ERROR configuring iscsid", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring nova_libvirt", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring ovn_controller", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring crond", "2020-03-27 08:54:12,731 ERROR: 55136 -- ERROR configuring neutron" ] } 2020-03-27 14:56:30,243 p=54331 u=mistral | PLAY RECAP ********************************************************************* 2020-03-27 14:56:30,243 p=54331 u=mistral | compute1 : ok=182 changed=49 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | compute2 : ok=182 changed=49 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller1 : ok=233 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller2 : ok=233 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,244 p=54331 u=mistral | controller3 : ok=234 changed=76 unreachable=0 failed=1 skipped=188 rescued=0 ignored=1 2020-03-27 14:56:30,245 p=54331 u=mistral | undercloud : ok=12 changed=7 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 2020-03-27 14:56:30,252 p=54331 u=mistral | Friday 27 March 2020 14:56:30 +0600 (0:00:01.749) 0:12:59.282 ********** 2020-03-27 14:56:30,252 p=54331 u=mistral | =============================================================================== The network path was not found. From smooney at redhat.com Sat Mar 28 18:57:38 2020 From: smooney at redhat.com (Sean Mooney) Date: Sat, 28 Mar 2020 18:57:38 +0000 Subject: Upcoming pip changes - stop installing test-requirements in devstack In-Reply-To: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> References: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> Message-ID: On Fri, 2020-03-27 at 14:24 -0500, Monty Taylor wrote: > Heya, > > Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to > verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us. > > The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those > at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run > devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state > is not one that is completely consistent. > > I’ve pushed up this: > > https://review.opendev.org/#/c/715469/ > > Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in > devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest > which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on > devstack fail now. i brought up the topic of stoping installing test-requirement in the past as it has caused issue before so im happy to see this change going forward :) > > It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s > worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re > not going to get broken this summer. > > Monty > From amotoki at gmail.com Sat Mar 28 21:05:40 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Sun, 29 Mar 2020 06:05:40 +0900 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: Hi, Find my comments on several repositories. On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand wrote: > > Hi, > > Sphinx 2.4 has been uploaded to Experimental. I just received bug > reports against the OpenStack packages because some of them cannot be > built with Sphinx 2.4. Here's the list: > > - ironic > - os-api-ref I can build the document with sphinx 2.4.4. > - aodh > - cloudkitty > - kombu This is not part of OpenStack and is not found in opendev.org. > - panko > - ceilometerclient > - ceilometermiddleware > - doc8 doc8 was retired Jul 2019. > - dracclient > - glareclient > - murano-pkg-check > - os-xenapi > - scciclient > - sahara > - sphinxcontrib-programoutput This is not part of OpenStack and is not found in opendev.org. > - tempest-horizon sphinx document is not actually used and the content was not changed under the initial cookiecutter commit. I proposed a patch to drop sphinx related stuffs. Hope it works for you. > > It'd be nice if those affected could be fixed for Ussuri. IMO, it's just > the doc, so fixing it wouldn't hurt code quality. In other words: help > would be more than welcome writing patch or reviewing them, and testing > Sphinx 2.4 compatibility. > > Cheers, > > Thomas Goirand (zigo) > From zigo at debian.org Sat Mar 28 22:32:50 2020 From: zigo at debian.org (Thomas Goirand) Date: Sat, 28 Mar 2020 23:32:50 +0100 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: <23c241c6-142a-3e50-7862-3e923d5c111b@debian.org> Hi there! Thanks for your reply. On 3/28/20 10:05 PM, Akihiro Motoki wrote: > On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand wrote: >> Sphinx 2.4 has been uploaded to Experimental. I just received bug >> reports against the OpenStack packages because some of them cannot be >> built with Sphinx 2.4. Here's the list: >> >> - ironic >> - os-api-ref > > I can build the document with sphinx 2.4.4. For Ironic, the bug report is here: https://bugs.debian.org/955069 For os-api-ref, here: https://bugs.debian.org/955098 the bug in os-api-ref is *not* in its documentation, but in os-api-ref itself, unfortunately (failure to run unit tests...). >> - aodh >> - cloudkitty >> - kombu > This is not part of OpenStack and is not found in opendev.org. Kombu, probably, however it's used in many OpenStack projects, like taskflow, oslo.messaging, ceilometer, heat, mistral, murano... The other 2 really are in opendev: https://opendev.org/openstack/cloudkitty https://opendev.org/openstack/aodh >> - dracclient >> - glareclient >> - murano-pkg-check >> - os-xenapi >> - scciclient >> - sahara >> - sphinxcontrib-programoutput > > This is not part of OpenStack and is not found in opendev.org. Does your sentence intend to apply to all what you've quoted? :) Hopefully not ... and assuming not: sphinxcontrib-programoutput is used in many docs in OpenStack, so it is also relevant to have it in the list. >> - tempest-horizon > > sphinx document is not actually used and the content was not changed > under the initial cookiecutter commit. > I proposed a patch to drop sphinx related stuffs. Hope it works for you. Thanks. I'll drop the doc package then. Cheers, Thomas Goirand (zigo) From gmann at ghanshyammann.com Sun Mar 29 01:43:08 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sat, 28 Mar 2020 20:43:08 -0500 Subject: [qa] Proposing Martin Kopec to Tempest core Message-ID: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> Hello Everyone, Martin Kopec (IRC: kopecmartin) has been doing great contribution in Tempest since a long time. He has been doing bugs Triage also in Ussuri cycle and has good understanding of Tempest. I would like to propose him for Tempest Core. You can vote/feedback on this email. If no objection by the end of this week, I will add him to the list. -gmann From masayuki.igawa at gmail.com Sun Mar 29 02:36:45 2020 From: masayuki.igawa at gmail.com (Masayuki Igawa) Date: Sun, 29 Mar 2020 11:36:45 +0900 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> Message-ID: <03d4afe4-e724-430a-851f-be786ea86a37@www.fastmail.com> +1 !!! -- Masayuki Igawa On Sun, Mar 29, 2020, at 10:43, Ghanshyam Mann wrote: > Hello Everyone, > > Martin Kopec (IRC: kopecmartin) has been doing great contribution in > Tempest since a long time. > He has been doing bugs Triage also in Ussuri cycle and has good > understanding of Tempest. > > I would like to propose him for Tempest Core. You can vote/feedback on > this email. If no objection by the end of this week, I will add him to > the list. > > -gmann > > From amotoki at gmail.com Sun Mar 29 03:47:19 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Sun, 29 Mar 2020 12:47:19 +0900 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: <23c241c6-142a-3e50-7862-3e923d5c111b@debian.org> References: <23c241c6-142a-3e50-7862-3e923d5c111b@debian.org> Message-ID: On Sun, Mar 29, 2020 at 7:36 AM Thomas Goirand wrote: > > Hi there! > > Thanks for your reply. > > On 3/28/20 10:05 PM, Akihiro Motoki wrote: > > On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand wrote: > >> Sphinx 2.4 has been uploaded to Experimental. I just received bug > >> reports against the OpenStack packages because some of them cannot be > >> built with Sphinx 2.4. Here's the list: > >> > >> - ironic > >> - os-api-ref > > > > I can build the document with sphinx 2.4.4. > > For Ironic, the bug report is here: > https://bugs.debian.org/955069 > > For os-api-ref, here: > https://bugs.debian.org/955098 > > the bug in os-api-ref is *not* in its documentation, but in os-api-ref > itself, unfortunately (failure to run unit tests...). All os-api-ref unittests succeed in the OpenStack CI. For example, https://zuul.opendev.org/t/openstack/build/af69856854424ef59d7b096f4dbffc1b It uses sphinx 2.4.4 and sphinx-testing 1.0.1. At a quick look of sphinx-testing, os-api-ref tests only works with os-api-ref>=1.0.1, while test-requirements.txt says os-api-ref>=0.7.2. read_text() method is from sphinx_testing.path.path class, but up to sphinx-testing 1.0.0 str is returned as app.outdir. The simplest solution would be to update test-requirements.txt to os-api-ref>=1.0.1 to clarify the requirements. Perhaps Debian testing uses sphinx-testing <1.0.1. > >> - aodh > >> - cloudkitty > >> - kombu > > > This is not part of OpenStack and is not found in opendev.org. > > Kombu, probably, however it's used in many OpenStack projects, like > taskflow, oslo.messaging, ceilometer, heat, mistral, murano... My intention is to refer only kombu, so I used "This". You mentioned "OpenStack packages" so I just would like to clarify it. Of course, it would be nice to fix them. > > The other 2 really are in opendev: > > https://opendev.org/openstack/cloudkitty > https://opendev.org/openstack/aodh > > >> - dracclient > >> - glareclient > >> - murano-pkg-check > >> - os-xenapi > >> - scciclient > >> - sahara > >> - sphinxcontrib-programoutput > > > > This is not part of OpenStack and is not found in opendev.org. > > Does your sentence intend to apply to all what you've quoted? :) > Hopefully not ... and assuming not: > > sphinxcontrib-programoutput is used in many docs in OpenStack, so it is > also relevant to have it in the list. Same as the above. > > >> - tempest-horizon > > > > sphinx document is not actually used and the content was not changed > > under the initial cookiecutter commit. > > I proposed a patch to drop sphinx related stuffs. Hope it works for you. > > Thanks. I'll drop the doc package then. > > > - doc8 > > doc8 was retired Jul 2019. One correction on doc8. The repository was retired, but more precisely, as mentioned at https://opendev.org/x/doc8, it was moved to https://github.com/PyCQA/doc8 Thanks, Akihiro > > Cheers, > > Thomas Goirand (zigo) > From pramchan at yahoo.com Sun Mar 29 05:56:32 2020 From: pramchan at yahoo.com (prakash RAMCHANDRAN) Date: Sun, 29 Mar 2020 05:56:32 +0000 (UTC) Subject: [qa] Proposing Martin Kopec to Tempest core (Ghanshyam Mann) References: <827237721.486518.1585461392049.ref@mail.yahoo.com> Message-ID: <827237721.486518.1585461392049@mail.yahoo.com> +1 Message: 4 Date: Sat, 28 Mar 2020 20:43:08 -0500 From: Ghanshyam Mann To: "openstack-discuss" Subject: [qa] Proposing Martin Kopec to Tempest core Message-ID:     <17123f4f761.126cbb10d108731.5702022165643836936 at ghanshyammann.com> Content-Type: text/plain; charset="UTF-8" Hello Everyone, Martin Kopec (IRC: kopecmartin) has been doing great contribution in Tempest since a long time. He has been doing bugs Triage also in Ussuri cycle and has good understanding of Tempest. I would like to propose him for Tempest Core. You can vote/feedback on this email. If no objection by the end of this week, I will add him to the list. -gmann ------------------------------ Sent from Yahoo Mail on Android -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Sun Mar 29 08:01:16 2020 From: zigo at debian.org (Thomas Goirand) Date: Sun, 29 Mar 2020 10:01:16 +0200 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: <23c241c6-142a-3e50-7862-3e923d5c111b@debian.org> Message-ID: <3aad6891-5b4a-3184-a5a7-46ab556c1463@debian.org> On 3/29/20 5:47 AM, Akihiro Motoki wrote: > On Sun, Mar 29, 2020 at 7:36 AM Thomas Goirand wrote: >> >> Hi there! >> >> Thanks for your reply. >> >> On 3/28/20 10:05 PM, Akihiro Motoki wrote: >>> On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand wrote: >>>> Sphinx 2.4 has been uploaded to Experimental. I just received bug >>>> reports against the OpenStack packages because some of them cannot be >>>> built with Sphinx 2.4. Here's the list: >>>> >>>> - ironic >>>> - os-api-ref >>> >>> I can build the document with sphinx 2.4.4. >> >> For Ironic, the bug report is here: >> https://bugs.debian.org/955069 >> >> For os-api-ref, here: >> https://bugs.debian.org/955098 >> >> the bug in os-api-ref is *not* in its documentation, but in os-api-ref >> itself, unfortunately (failure to run unit tests...). > > All os-api-ref unittests succeed in the OpenStack CI. > For example, https://zuul.opendev.org/t/openstack/build/af69856854424ef59d7b096f4dbffc1b > It uses sphinx 2.4.4 and sphinx-testing 1.0.1. > > At a quick look of sphinx-testing, os-api-ref tests only works with > os-api-ref>=1.0.1, while test-requirements.txt says os-api-ref>=0.7.2. > read_text() method is from sphinx_testing.path.path class, but up to > sphinx-testing 1.0.0 str is returned as app.outdir. > The simplest solution would be to update test-requirements.txt to > os-api-ref>=1.0.1 to clarify the requirements. > Perhaps Debian testing uses sphinx-testing <1.0.1. Oh, this is super useful, thanks! Indeed, sphinx-testing is in version 0.8.1 in Debian Sid. So I guess my next course of action is to reassign the bug to sphinx-testing. One thing less to take care of! :) Cheers, Thomas Goirand (zigo) From donny at fortnebula.com Sun Mar 29 12:55:58 2020 From: donny at fortnebula.com (Donny Davis) Date: Sun, 29 Mar 2020 08:55:58 -0400 Subject: [OpenStack-Infra] [Murano-Open-PaaS] Openstack using guacamole In-Reply-To: References: Message-ID: On Fri, Mar 27, 2020 at 11:19 AM Samuel Abdullah wrote: > Hi, > > Does Anyone know how can i install guacamole in openstack environment? > Even if its via murano? Any manual guideline? > > Best Regards > Samuel > > On Fri, 27 Mar 2020, 00:05 Roman Gorshunov, wrote: > >> Hello Samuel, >> >> Thanks for your email. Yes, Guacamole can be installed as an app via >> Murano. >> Discussions about Open PaaS are now in openstack-discuss mailing list. >> Please use the tag [Murano-Open-PaaS] in the subject line. I'm >> re-routing your email. >> >> Here are docs for the Murano project [0]. >> >> [0] https://docs.openstack.org/murano/latest/ >> >> Best regards, >> Roman Gorshunov >> >> On Thu, Mar 26, 2020 at 4:37 PM Samuel Abdullah < >> samuel at silverliningsys.com> wrote: >> >>> Hi, >>> >>> Would like to ask if it is possible to run guacamole in openstack >>> environment? >>> Seek for help on how would this be possible as i saw that guacamole >>> component are part of the openstack in your murano project >>> >>> Looking forward for your reply >>> >>> Best Regards >>> >>> -- >>> >>> >>> >>> www.silverliningsys.com >>> >>> >>> *Abu Bakar Samuel Abdullah* >>> >>> Cloud Infrastructure & Operations >>> >>> >>> P: +603-2712-0081 >>> >>> M: +60.12.654.5938 >>> >>> E: samuel at silverliningsys.com >>> _______________________________________________ >>> OpenStack-Infra mailing list >>> OpenStack-Infra at lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> _______________________________________________ > OpenStack-Infra mailing list > OpenStack-Infra at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra Samuel, Are you trying to replace the default vnc console with guacamole or are you trying to run it as a service on top of Openstack? Scenario A - built in console I am unaware of any method in the code today to replace the default console with guacamole. Scenario B - Use it as a service I can't imagine it would be too hard if you leverage some kind of automation framework to create your instances. In the example of using ansible to create machines on the cloud - you could first call the playbook that creates your instance and then query the API for the details of that instance. You could then insert the needed parameters to the guacamole server to access this machine. The following modules could be used in this case - mind you this is one of many examples to accomplish this task Create Instance https://docs.ansible.com/ansible/latest/modules/os_server_module.html Query the API https://docs.ansible.com/ansible/latest/modules/os_server_info_module.html And then to update Guacamole you can use a template that includes a method to properly create your backend connections of all of the required instances I am not an expert in Guacamole, so you will have to tinker a bit here. Also dropping infra - this ML is mainly for the actual Openstack project infra - so like CI resources and code hosting resources Thanks and good luck! -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Sun Mar 29 13:33:42 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Sun, 29 Mar 2020 15:33:42 +0200 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: On Sat, Mar 28, 2020 at 10:08 PM Akihiro Motoki wrote: > Hi, > > Find my comments on several repositories. > > On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand wrote: > > > > Hi, > > > > Sphinx 2.4 has been uploaded to Experimental. I just received bug > > reports against the OpenStack packages because some of them cannot be > > built with Sphinx 2.4. Here's the list: > > > > - ironic > > - os-api-ref > > I can build the document with sphinx 2.4.4. > > > - aodh > > - cloudkitty > > - kombu > > This is not part of OpenStack and is not found in opendev.org. > > > - panko > > - ceilometerclient > > - ceilometermiddleware > > - doc8 > > doc8 was retired Jul 2019. > For those like me who just nearly had a heart attack: it wasn't fully retired, it was moved (?) to https://github.com/PyCQA/doc8 > > > - dracclient > > - glareclient > > - murano-pkg-check > > - os-xenapi > > - scciclient > > - sahara > > - sphinxcontrib-programoutput > > This is not part of OpenStack and is not found in opendev.org. > > > - tempest-horizon > > sphinx document is not actually used and the content was not changed > under the initial cookiecutter commit. > I proposed a patch to drop sphinx related stuffs. Hope it works for you. > > > > > It'd be nice if those affected could be fixed for Ussuri. IMO, it's just > > the doc, so fixing it wouldn't hurt code quality. In other words: help > > would be more than welcome writing patch or reviewing them, and testing > > Sphinx 2.4 compatibility. > > > > Cheers, > > > > Thomas Goirand (zigo) > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amotoki at gmail.com Sun Mar 29 13:53:50 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Sun, 29 Mar 2020 22:53:50 +0900 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: On Sun, Mar 29, 2020 at 10:37 PM Dmitry Tantsur wrote: > > > > On Sat, Mar 28, 2020 at 10:08 PM Akihiro Motoki wrote: >> >> Hi, >> >> Find my comments on several repositories. >> >> On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand wrote: >> > >> > Hi, >> > >> > Sphinx 2.4 has been uploaded to Experimental. I just received bug >> > reports against the OpenStack packages because some of them cannot be >> > built with Sphinx 2.4. Here's the list: >> > >> > - ironic >> > - os-api-ref >> >> I can build the document with sphinx 2.4.4. >> >> > - aodh >> > - cloudkitty >> > - kombu >> >> This is not part of OpenStack and is not found in opendev.org. >> >> > - panko >> > - ceilometerclient >> > - ceilometermiddleware >> > - doc8 >> >> doc8 was retired Jul 2019. > > > For those like me who just nearly had a heart attack: it wasn't fully retired, it was moved (?) to https://github.com/PyCQA/doc8 Yeah, my first email was not enough and I sent a follow-up. --- One correction on doc8. The repository was retired, but more precisely, as mentioned at https://opendev.org/x/doc8, it was moved to https://github.com/PyCQA/doc8 --- >> >> >> > - dracclient >> > - glareclient >> > - murano-pkg-check >> > - os-xenapi >> > - scciclient >> > - sahara >> > - sphinxcontrib-programoutput >> >> This is not part of OpenStack and is not found in opendev.org. >> >> > - tempest-horizon >> >> sphinx document is not actually used and the content was not changed >> under the initial cookiecutter commit. >> I proposed a patch to drop sphinx related stuffs. Hope it works for you. >> >> > >> > It'd be nice if those affected could be fixed for Ussuri. IMO, it's just >> > the doc, so fixing it wouldn't hurt code quality. In other words: help >> > would be more than welcome writing patch or reviewing them, and testing >> > Sphinx 2.4 compatibility. >> > >> > Cheers, >> > >> > Thomas Goirand (zigo) >> > >> From hongbin034 at gmail.com Sun Mar 29 16:07:03 2020 From: hongbin034 at gmail.com (Hongbin Lu) Date: Sun, 29 Mar 2020 12:07:03 -0400 Subject: [neutron] Bug deputy report (Mar 23 - Mar 29) Message-ID: Hi all, I am on the bug deputy last week. Below is the bug deputy report: Critical: https://bugs.launchpad.net/neutron/+bug/1868691 neutron-rally-task fails 100% on stable branches https://bugs.launchpad.net/neutron/+bug/1869034 Postgresql periodic job failing every day High: https://bugs.launchpad.net/neutron/+bug/1868724 KeyError 'gateway_ip' in neutron/services/segments/plugin.py Medium: https://bugs.launchpad.net/neutron/+bug/1869057 extension "subnet_dns_publish_fixed_ips" named wrong in the docs Low: https://bugs.launchpad.net/neutron/+bug/1868515 vpnaas ike policy is not allowed to be updated https://bugs.launchpad.net/neutron/+bug/1868686 ovsdb_timeout config option isn't used to set when adding manager https://bugs.launchpad.net/neutron/+bug/1868933 [fwaas] Update the rules for firewall group without ports will cause the status to change from inactive to active https://bugs.launchpad.net/neutron/+bug/1869121 [FWaaS] Can't add rule with destination_port large than source_port https://bugs.launchpad.net/neutron/+bug/1869342 OVNMechanismDriver _ovn_client is a read-only property Cannot triage (need help from lieutenants): https://bugs.launchpad.net/neutron/+bug/1868697 [ovn] transaction error: {"details":"Transaction causes multiple rows in \"MAC_Binding\" table to have identical values https://bugs.launchpad.net/neutron/+bug/1869129 neutron accepts CIDR in security groups that are invalid in ovn https://bugs.launchpad.net/neutron/+bug/1869244 RowNotFound: Cannot find Bridge with name=tbr-XXXXXXXX-X when using trunk bridges with DPDK vhostusermode https://bugs.launchpad.net/neutron/+bug/1869354 FIP isn't properly created for octavia loadbalancers Invalid: https://bugs.launchpad.net/neutron/+bug/1869136 Use public flat network to create vm faild Best regards, Hongbin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungleboyj at gmail.com Sun Mar 29 20:12:29 2020 From: jungleboyj at gmail.com (Jay S. Bryant) Date: Sun, 29 Mar 2020 15:12:29 -0500 Subject: [all][tc] Post-mortem of my time on the TC In-Reply-To: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> References: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> Message-ID: <2d37735d-dd57-f1cc-c2ed-ce9709b01773@gmail.com> Zane, Thanks for this great note and for all you have done as part of the TC! Jay On 3/27/2020 4:38 PM, Zane Bitter wrote: > As promised last year, I will not be running for a third term as a > member of the Technical Committee. I'm stepping down to spend more > time with my family (no, really! ;) > > I thought this would be a good opportunity to talk to those folks who > are thinking of running for a seat, or who aren't but should be. The > most common question I hear from potential candidates is "what should > I actually do on the TC?" I can't tell you what *you* should do, but I > can tell you what I did and how that worked out. Maybe that will give > you the flavour. And perhaps (he hinted) some of the other outgoing TC > members can do the same to provide more of a cross-section. > > * I drafted and edited the review guide, "How to Review Changes the > OpenStack Way", to try to ensure that the best parts of our review > culture are evenly distributed across the project: > https://docs.openstack.org/project-team-guide/review-the-openstack-way.html > (This started as part of an initiative from Julia Kreger to reduce > nitpicking in reviews.) > > In theory anybody could have done this, but in practice I think this > is one of those things that only somebody elected to the TC could have > driven. I received a lot of feedback afterwards (which is unusual for > anything the TC does): several people, and one whole team, mentioned > that they had tweaked their reviewing style after this discussion. And > hearing their feedback prompted me to tweak my own reviewing style as > well. > > * I drafted and edited the Vision for OpenStack Clouds: > https://governance.openstack.org/tc/reference/technical-vision.html. > This was a ton of work as it involved consulting with so many people: > first, folks in the community who on the surface appeared to have > quite different visions for what OpenStack should be, who later turned > out to not be so far apart as we all thought. After a lengthy round of > reviews, I approached every project team individually to explain the > specific relevance to them, and seek their feedback. Finally, I > presented it to the OSF Board and held an in-person feedback session > at the Forum, after all of which it was approved by the TC. > > I have some mixed feelings about the success of this. I still believe > it was desperately needed... in about 2013. Now the Kubernetes > community is succeeding at engaging with actual end users (not just > operators), which OpenStack has always failed to do. With > contributions to OpenStack dropping, it's not clear that we can, or > necessarily should, sustain such a broad vision. It is my hope that if > and when the community needs to discuss adjusting our scope, we will > do so by amending this document - going through it section by section > and deciding which parts to double down on and which to allow to fall > by the wayside - and not by a series of ad-hoc actions that leave > everybody confused as to what the project is supposed to be once > again. On a personal level, the vision has been extremely valuable for > me because it required me to spend a lot of time thinking deeply about > the various components of OpenStack and how they fit into a greater > purpose. A large portion of the value comes from participating in the > process, not necessarily the final document. I hope that all of the > folks who helped along the way got as much out of it as I did. > > * I developed the process by which we keep OpenStack up to date with > new releases of Python: > https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html > > I also shepherded it through the first couple of release cycles, to > try to entrench it as a standard part of each release. (BTW this task > needs somebody to take it over.) This mostly involved pointing people > back to the resolution and inviting them to formally update it every > time someone tried to bikeshed the details, which happened more than I > ever believed possible. (To date there have been no changes proposed > to the resolution.) > > * I engaged with the OSF Board to pass on feedback from members in the > developer community on the process for adding new Open Infrastructure > projects (e.g. Zuul, StarlingX, Airship, Kata). Sometimes the work of > the TC is completely invisible! I also tried to convey information the > other way, publicising information that was otherwise only available > by attending board meetings. (Most TC members usually attend board > meetings, including the in-person joint leadership meetings held > before Summits.) > > * In response to feedback from the board, I replaced the "Help Most > Wanted" list with the "Upstream Investment Opportunities" list, and > kick-started the rewriting of the existing entries in a format that is > more targeted at decision-makers: > https://governance.openstack.org/tc/reference/upstream-investment-opportunities/index.html > (Huge thanks to Tom Barron, who did even more work to complete the job > of converting all of the existing entries - proof that you don't need > to be a TC member to start contributing to governance activities.) > > It remains to be seen whether this will have any effect, but this > content is now being shared in the community newsletter, and the board > is making moves to use it to actively solicit contributions to the > community. Previously they felt unable to make use of the Help Most > Wanted list because it was written with the wrong audience in mind. > > * I tweaked the guidelines for the release naming system to better > align with our actual practice, because they had diverged somewhat. > Believe it or not, this one issue took up more of the TC's time than > any other thing that happened in 2019, and I'm very glad that we ended > up not needing these tweaks because we eventually replaced the process > wholesale, with one that makes it explicit that the TC is ultimately > accountable for the release name not causing any international > incidents or other snafus. > > * I researched and documented a design, developed during extensive > discussions with many members of the technical community, for a new > kind of simplified cloud built partially using OpenStack components > but designed to be Kubernetes-native first: Project Teapot > https://governance.openstack.org/ideas/ideas/teapot/index.html > > I do think this is worth building, but if nothing else I really hope > this helps bring into focus the parts of OpenStack that remain > essential in a Kubernetes-centric world. In writing this I was very > much guided by the Vision for OpenStack Clouds as well as the ways in > which Kubernetes expects to interact with an underlying cloud. I think > we need more of these kinds of exercises, and while they don't have to > be sponsored by TC members, I think it helps. > > > If there's a pattern here it's this: take something that's implicitly > agreed upon by the community (whether we realise it or not), write it > down, seek feedback, and make it explicit so that anyone can refer to > it, rather than leaving it as closely held tribal knowledge. > > Of course it goes without saying that I was involved in many more > discussions and reviews that were led by others. I'll leave it to them > to talk about those. And it should also go without saying that all of > the work listed above was greatly improved by the input of the many > talented folks in our community who contributed. > > > In my humble opinion OpenStack is the best open source community of > its size ever created. That's because we believe not just in one > 'open' (source), but four > (https://governance.openstack.org/tc/reference/opens.html); because > we're committed to using open tools to build open software; and > because, while it's certainly not the case that we can't improve, we > have managed to do this while avoiding a widespread culture of > bullying. To do all of this on the scale of OpenStack was, as far as I > know, completely without precedent. (Perhaps the ASF comes closest, > but its projects operate independently, unlike OpenStack subprojects.) > All of us should be proud of this achievement. But it didn't happen by > accident: it is the product of concerted effort by leaders in our > community over the course of a decade. It's been a tremendous > privilege to work alongside some of those folks, both before, during > and (hopefully) after my time on the TC. I have learned so much from > them. Now it is time for others to step up and continue to build that > legacy, as the project evolves and adapts over time. The > diversification of the OSF also gives us the opportunity to spread our > ethos to other projects under the same umbrella, and from there to the > open source community as a whole. > > However, we must remember that having a great community is not enough. > To remain relevant, we also need to produce software that meets users' > needs, which change over time. By an accident of fate I have > approached OpenStack from a different direction than most - I came to > OpenStack via the Heat project, which means I've always looked at it > from an outside-in standpoint: users are building cloud applications, > how can OpenStack provide them with what they need from a cloud > through an open API? Many others, by necessity, have approached it > from the inside-out: we have all this hardware, how can we expose its > capabilities to users? I think it is fair to say that many of these > folks and I have found each other's views to be mutually > incomprehensible over the years. In writing up the Teapot design, I > have learned a lot more about the other perspective. I believe we will > need a more people to also grow their understanding in the opposite > direction to avoid the trap of most capital-E "Enterprise" software - > designed to check the boxes of those who decide what products to buy, > but widely hated by all those forced to actually use it. Future TCs > will have to grapple with this because nobody else in the community is > really empowered to do so. The openstack/ideas repo that JP created is > a great place for anybody to start documenting ways that we can improve. > > For my part, this is not goodbye. However, I have already been > splitting my time between OpenStack and metal3.io and I expect that > the drift in the other direction will only accelerate now that I no > longer have any formal responsibilities here. Please feel free to > contact me or ask questions if you think I can help. > > Stay safe out there, and by "out there" I mean physically isolated in > your own homes for the next little while. > > cheers, > Zane. > > From feilong at catalyst.net.nz Sun Mar 29 20:27:45 2020 From: feilong at catalyst.net.nz (feilong) Date: Mon, 30 Mar 2020 09:27:45 +1300 Subject: [all][tc] Post-mortem of my time on the TC In-Reply-To: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> References: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> Message-ID: <821d80db-701e-247e-185f-6689977009ed@catalyst.net.nz> Hi Zane, Thank you for your great note and all the wonderful work you have done for the community. On 28/03/20 10:38 AM, Zane Bitter wrote: > As promised last year, I will not be running for a third term as a > member of the Technical Committee. I'm stepping down to spend more > time with my family (no, really! ;) > > I thought this would be a good opportunity to talk to those folks who > are thinking of running for a seat, or who aren't but should be. The > most common question I hear from potential candidates is "what should > I actually do on the TC?" I can't tell you what *you* should do, but I > can tell you what I did and how that worked out. Maybe that will give > you the flavour. And perhaps (he hinted) some of the other outgoing TC > members can do the same to provide more of a cross-section. > > * I drafted and edited the review guide, "How to Review Changes the > OpenStack Way", to try to ensure that the best parts of our review > culture are evenly distributed across the project: > https://docs.openstack.org/project-team-guide/review-the-openstack-way.html > (This started as part of an initiative from Julia Kreger to reduce > nitpicking in reviews.) > > In theory anybody could have done this, but in practice I think this > is one of those things that only somebody elected to the TC could have > driven. I received a lot of feedback afterwards (which is unusual for > anything the TC does): several people, and one whole team, mentioned > that they had tweaked their reviewing style after this discussion. And > hearing their feedback prompted me to tweak my own reviewing style as > well. > > * I drafted and edited the Vision for OpenStack Clouds: > https://governance.openstack.org/tc/reference/technical-vision.html. > This was a ton of work as it involved consulting with so many people: > first, folks in the community who on the surface appeared to have > quite different visions for what OpenStack should be, who later turned > out to not be so far apart as we all thought. After a lengthy round of > reviews, I approached every project team individually to explain the > specific relevance to them, and seek their feedback. Finally, I > presented it to the OSF Board and held an in-person feedback session > at the Forum, after all of which it was approved by the TC. > > I have some mixed feelings about the success of this. I still believe > it was desperately needed... in about 2013. Now the Kubernetes > community is succeeding at engaging with actual end users (not just > operators), which OpenStack has always failed to do. With > contributions to OpenStack dropping, it's not clear that we can, or > necessarily should, sustain such a broad vision. It is my hope that if > and when the community needs to discuss adjusting our scope, we will > do so by amending this document - going through it section by section > and deciding which parts to double down on and which to allow to fall > by the wayside - and not by a series of ad-hoc actions that leave > everybody confused as to what the project is supposed to be once > again. On a personal level, the vision has been extremely valuable for > me because it required me to spend a lot of time thinking deeply about > the various components of OpenStack and how they fit into a greater > purpose. A large portion of the value comes from participating in the > process, not necessarily the final document. I hope that all of the > folks who helped along the way got as much out of it as I did. > > * I developed the process by which we keep OpenStack up to date with > new releases of Python: > https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html > > I also shepherded it through the first couple of release cycles, to > try to entrench it as a standard part of each release. (BTW this task > needs somebody to take it over.) This mostly involved pointing people > back to the resolution and inviting them to formally update it every > time someone tried to bikeshed the details, which happened more than I > ever believed possible. (To date there have been no changes proposed > to the resolution.) > > * I engaged with the OSF Board to pass on feedback from members in the > developer community on the process for adding new Open Infrastructure > projects (e.g. Zuul, StarlingX, Airship, Kata). Sometimes the work of > the TC is completely invisible! I also tried to convey information the > other way, publicising information that was otherwise only available > by attending board meetings. (Most TC members usually attend board > meetings, including the in-person joint leadership meetings held > before Summits.) > > * In response to feedback from the board, I replaced the "Help Most > Wanted" list with the "Upstream Investment Opportunities" list, and > kick-started the rewriting of the existing entries in a format that is > more targeted at decision-makers: > https://governance.openstack.org/tc/reference/upstream-investment-opportunities/index.html > (Huge thanks to Tom Barron, who did even more work to complete the job > of converting all of the existing entries - proof that you don't need > to be a TC member to start contributing to governance activities.) > > It remains to be seen whether this will have any effect, but this > content is now being shared in the community newsletter, and the board > is making moves to use it to actively solicit contributions to the > community. Previously they felt unable to make use of the Help Most > Wanted list because it was written with the wrong audience in mind. > > * I tweaked the guidelines for the release naming system to better > align with our actual practice, because they had diverged somewhat. > Believe it or not, this one issue took up more of the TC's time than > any other thing that happened in 2019, and I'm very glad that we ended > up not needing these tweaks because we eventually replaced the process > wholesale, with one that makes it explicit that the TC is ultimately > accountable for the release name not causing any international > incidents or other snafus. > > * I researched and documented a design, developed during extensive > discussions with many members of the technical community, for a new > kind of simplified cloud built partially using OpenStack components > but designed to be Kubernetes-native first: Project Teapot > https://governance.openstack.org/ideas/ideas/teapot/index.html > > I do think this is worth building, but if nothing else I really hope > this helps bring into focus the parts of OpenStack that remain > essential in a Kubernetes-centric world. In writing this I was very > much guided by the Vision for OpenStack Clouds as well as the ways in > which Kubernetes expects to interact with an underlying cloud. I think > we need more of these kinds of exercises, and while they don't have to > be sponsored by TC members, I think it helps. > > > If there's a pattern here it's this: take something that's implicitly > agreed upon by the community (whether we realise it or not), write it > down, seek feedback, and make it explicit so that anyone can refer to > it, rather than leaving it as closely held tribal knowledge. > > Of course it goes without saying that I was involved in many more > discussions and reviews that were led by others. I'll leave it to them > to talk about those. And it should also go without saying that all of > the work listed above was greatly improved by the input of the many > talented folks in our community who contributed. > > > In my humble opinion OpenStack is the best open source community of > its size ever created. That's because we believe not just in one > 'open' (source), but four > (https://governance.openstack.org/tc/reference/opens.html); because > we're committed to using open tools to build open software; and > because, while it's certainly not the case that we can't improve, we > have managed to do this while avoiding a widespread culture of > bullying. To do all of this on the scale of OpenStack was, as far as I > know, completely without precedent. (Perhaps the ASF comes closest, > but its projects operate independently, unlike OpenStack subprojects.) > All of us should be proud of this achievement. But it didn't happen by > accident: it is the product of concerted effort by leaders in our > community over the course of a decade. It's been a tremendous > privilege to work alongside some of those folks, both before, during > and (hopefully) after my time on the TC. I have learned so much from > them. Now it is time for others to step up and continue to build that > legacy, as the project evolves and adapts over time. The > diversification of the OSF also gives us the opportunity to spread our > ethos to other projects under the same umbrella, and from there to the > open source community as a whole. > > However, we must remember that having a great community is not enough. > To remain relevant, we also need to produce software that meets users' > needs, which change over time. By an accident of fate I have > approached OpenStack from a different direction than most - I came to > OpenStack via the Heat project, which means I've always looked at it > from an outside-in standpoint: users are building cloud applications, > how can OpenStack provide them with what they need from a cloud > through an open API? Many others, by necessity, have approached it > from the inside-out: we have all this hardware, how can we expose its > capabilities to users? I think it is fair to say that many of these > folks and I have found each other's views to be mutually > incomprehensible over the years. In writing up the Teapot design, I > have learned a lot more about the other perspective. I believe we will > need a more people to also grow their understanding in the opposite > direction to avoid the trap of most capital-E "Enterprise" software - > designed to check the boxes of those who decide what products to buy, > but widely hated by all those forced to actually use it. Future TCs > will have to grapple with this because nobody else in the community is > really empowered to do so. The openstack/ideas repo that JP created is > a great place for anybody to start documenting ways that we can improve. > > For my part, this is not goodbye. However, I have already been > splitting my time between OpenStack and metal3.io and I expect that > the drift in the other direction will only accelerate now that I no > longer have any formal responsibilities here. Please feel free to > contact me or ask questions if you think I can help. > > Stay safe out there, and by "out there" I mean physically isolated in > your own homes for the next little while. > > cheers, > Zane. > > -- Cheers & Best regards, Feilong Wang (王飞龙) ------------------------------------------------------ Senior Cloud Software Engineer Tel: +64-48032246 Email: flwang at catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington ------------------------------------------------------ From Arkady.Kanevsky at dell.com Sun Mar 29 22:15:23 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Sun, 29 Mar 2020 22:15:23 +0000 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: <00c0057794c544459e0c94802886dbf0@AUSX13MPS308.AMER.DELL.COM> What is the goal for Sphinx 2.4? From: Dmitry Tantsur Sent: Sunday, March 29, 2020 8:34 AM To: openstack-discuss Subject: Re: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages [EXTERNAL EMAIL] On Sat, Mar 28, 2020 at 10:08 PM Akihiro Motoki > wrote: Hi, Find my comments on several repositories. On Sat, Mar 28, 2020 at 6:34 PM Thomas Goirand > wrote: > > Hi, > > Sphinx 2.4 has been uploaded to Experimental. I just received bug > reports against the OpenStack packages because some of them cannot be > built with Sphinx 2.4. Here's the list: > > - ironic > - os-api-ref I can build the document with sphinx 2.4.4. > - aodh > - cloudkitty > - kombu This is not part of OpenStack and is not found in opendev.org. > - panko > - ceilometerclient > - ceilometermiddleware > - doc8 doc8 was retired Jul 2019. For those like me who just nearly had a heart attack: it wasn't fully retired, it was moved (?) to https://github.com/PyCQA/doc8 > - dracclient > - glareclient > - murano-pkg-check > - os-xenapi > - scciclient > - sahara > - sphinxcontrib-programoutput This is not part of OpenStack and is not found in opendev.org. > - tempest-horizon sphinx document is not actually used and the content was not changed under the initial cookiecutter commit. I proposed a patch to drop sphinx related stuffs. Hope it works for you. > > It'd be nice if those affected could be fixed for Ussuri. IMO, it's just > the doc, so fixing it wouldn't hurt code quality. In other words: help > would be more than welcome writing patch or reviewing them, and testing > Sphinx 2.4 compatibility. > > Cheers, > > Thomas Goirand (zigo) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Sun Mar 29 22:22:21 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Sun, 29 Mar 2020 22:22:21 +0000 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: <00c0057794c544459e0c94802886dbf0@AUSX13MPS308.AMER.DELL.COM> References: <00c0057794c544459e0c94802886dbf0@AUSX13MPS308.AMER.DELL.COM> Message-ID: <20200329222221.iretkuoqec5cg7sj@yuggoth.org> On 2020-03-29 22:15:23 +0000 (+0000), Arkady.Kanevsky at dell.com wrote: > What is the goal for Sphinx 2.4? [...] Lots of software projects rely on Sphinx to build their documentation, and GNU/Linux distributions are typically not going to want to have to find a way to carry more than one version of it. Making sure OpenStack projects' documentation can be packaged with the latest releases of Sphinx helps ease the maintenance burden on distributions and integrators. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From Arkady.Kanevsky at dell.com Sun Mar 29 23:41:03 2020 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Sun, 29 Mar 2020 23:41:03 +0000 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: <20200329222221.iretkuoqec5cg7sj@yuggoth.org> References: <00c0057794c544459e0c94802886dbf0@AUSX13MPS308.AMER.DELL.COM> <20200329222221.iretkuoqec5cg7sj@yuggoth.org> Message-ID: <27fdb596ed0049f9acf95760a8d31c04@AUSX13MPS308.AMER.DELL.COM> Thanks Jeremy. I would assume we want all openstack doc per release follow this, including all drivers? Thanks, Arkady -----Original Message----- From: Jeremy Stanley Sent: Sunday, March 29, 2020 5:22 PM To: openstack-discuss at lists.openstack.org Subject: Re: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages On 2020-03-29 22:15:23 +0000 (+0000), Arkady.Kanevsky at dell.com wrote: > What is the goal for Sphinx 2.4? [...] Lots of software projects rely on Sphinx to build their documentation, and GNU/Linux distributions are typically not going to want to have to find a way to carry more than one version of it. Making sure OpenStack projects' documentation can be packaged with the latest releases of Sphinx helps ease the maintenance burden on distributions and integrators. -- Jeremy Stanley From rosmaita.fossdev at gmail.com Mon Mar 30 00:36:43 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Sun, 29 Mar 2020 20:36:43 -0400 Subject: [election][cinder] PTL candidacy for Victoria Message-ID: <6906a907-8f79-bf3a-4a5e-41cacc0b880f@gmail.com> Hello everyone, I would like to announce my candidacy for Cinder PTL for the Victoria cycle. Given that I'm the current PTL, you already have a pretty good idea of what things will be like on the Cinder project with me in that role and whether or not it would be a good idea for me to continue. So instead of bribing you with campaign promises, I'd like to take this opportunity to look at the current state of the project and what things look like for the Victoria cycle. The Cinder community remained fairly stable from Train to Ussuri, which was great in these downsizing times. Some of the drivers that were marked 'unsupported' for the Train release have been restored to 'supported' status during the Ussuri cycle. Some, however, have not. We relaxed the removal policy to give vendors more time to address CI issues and to make it easier for operators using such drivers. We'll see during the Victoria cycle whether this actually encourages the vendors of drivers that have been declared 'unsupported' to get their third-party CIs running again and having the drivers reinstated. During Ussuri, the team gained more testing expertise, particularly around creating more scenario tests, though this hasn't been reflected yet in committed code. As we get more scenario tests into the cinder-tempest-plugin, we'll be able to catch more bugs that affect multiple drivers, whereas now what mostly happens is that driver maintainers fix these piecemeal as they become aware that they affect their drivers. Thus, increasing the scnario test coverage will continue to be a focus in Victoria. The virtual PTG that we held as a follow-up to the Shanghai PTG was productive, as were the two virtual mid-cycle meetings we had this cycle. So I think we're in good shape to make the most of the fully virtual Victoria PTG. I think as a community, we're all doing good work on the Cinder project. We've got cinderlib to keep us relevant in a container-oriented world, and I think we're doing a solid job keeping the software stable and reliable. I'm sure the driver maintainers would like to see faster review turnaround (as would we all), so it will be a priority early in the cycle to identify contributors interested in becoming members of the cinder-core team and mentoring them along so that we can eventually increase the size of the core team during the cycle. In short, things are good, they could be better, and I'd like the opportunity to help drive the project as Victoria PTL. Thank you for your consideration, Brian Rosmaita (rosmaita) From dangtrinhnt at gmail.com Mon Mar 30 01:09:43 2020 From: dangtrinhnt at gmail.com (Trinh Nguyen) Date: Mon, 30 Mar 2020 10:09:43 +0900 Subject: [all][tc] Post-mortem of my time on the TC In-Reply-To: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> References: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> Message-ID: Zane, I was lucky enough to join the OpenStack community when you're doing all of the things listed above. You've done an amazing job. Thank you. On Sat, Mar 28, 2020 at 6:42 AM Zane Bitter wrote: > As promised last year, I will not be running for a third term as a > member of the Technical Committee. I'm stepping down to spend more time > with my family (no, really! ;) > > I thought this would be a good opportunity to talk to those folks who > are thinking of running for a seat, or who aren't but should be. The > most common question I hear from potential candidates is "what should I > actually do on the TC?" I can't tell you what *you* should do, but I can > tell you what I did and how that worked out. Maybe that will give you > the flavour. And perhaps (he hinted) some of the other outgoing TC > members can do the same to provide more of a cross-section. > > * I drafted and edited the review guide, "How to Review Changes the > OpenStack Way", to try to ensure that the best parts of our review > culture are evenly distributed across the project: > https://docs.openstack.org/project-team-guide/review-the-openstack-way.html > (This started as part of an initiative from Julia Kreger to reduce > nitpicking in reviews.) > > In theory anybody could have done this, but in practice I think this is > one of those things that only somebody elected to the TC could have > driven. I received a lot of feedback afterwards (which is unusual for > anything the TC does): several people, and one whole team, mentioned > that they had tweaked their reviewing style after this discussion. And > hearing their feedback prompted me to tweak my own reviewing style as well. > > * I drafted and edited the Vision for OpenStack Clouds: > https://governance.openstack.org/tc/reference/technical-vision.html. > This was a ton of work as it involved consulting with so many people: > first, folks in the community who on the surface appeared to have quite > different visions for what OpenStack should be, who later turned out to > not be so far apart as we all thought. After a lengthy round of reviews, > I approached every project team individually to explain the specific > relevance to them, and seek their feedback. Finally, I presented it to > the OSF Board and held an in-person feedback session at the Forum, after > all of which it was approved by the TC. > > I have some mixed feelings about the success of this. I still believe it > was desperately needed... in about 2013. Now the Kubernetes community is > succeeding at engaging with actual end users (not just operators), which > OpenStack has always failed to do. With contributions to OpenStack > dropping, it's not clear that we can, or necessarily should, sustain > such a broad vision. It is my hope that if and when the community needs > to discuss adjusting our scope, we will do so by amending this document > - going through it section by section and deciding which parts to double > down on and which to allow to fall by the wayside - and not by a series > of ad-hoc actions that leave everybody confused as to what the project > is supposed to be once again. On a personal level, the vision has been > extremely valuable for me because it required me to spend a lot of time > thinking deeply about the various components of OpenStack and how they > fit into a greater purpose. A large portion of the value comes from > participating in the process, not necessarily the final document. I hope > that all of the folks who helped along the way got as much out of it as > I did. > > * I developed the process by which we keep OpenStack up to date with new > releases of Python: > > https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html > > I also shepherded it through the first couple of release cycles, to try > to entrench it as a standard part of each release. (BTW this task needs > somebody to take it over.) This mostly involved pointing people back to > the resolution and inviting them to formally update it every time > someone tried to bikeshed the details, which happened more than I ever > believed possible. (To date there have been no changes proposed to the > resolution.) > > * I engaged with the OSF Board to pass on feedback from members in the > developer community on the process for adding new Open Infrastructure > projects (e.g. Zuul, StarlingX, Airship, Kata). Sometimes the work of > the TC is completely invisible! I also tried to convey information the > other way, publicising information that was otherwise only available by > attending board meetings. (Most TC members usually attend board > meetings, including the in-person joint leadership meetings held before > Summits.) > > * In response to feedback from the board, I replaced the "Help Most > Wanted" list with the "Upstream Investment Opportunities" list, and > kick-started the rewriting of the existing entries in a format that is > more targeted at decision-makers: > > https://governance.openstack.org/tc/reference/upstream-investment-opportunities/index.html > (Huge thanks to Tom Barron, who did even more work to complete the job > of converting all of the existing entries - proof that you don't need to > be a TC member to start contributing to governance activities.) > > It remains to be seen whether this will have any effect, but this > content is now being shared in the community newsletter, and the board > is making moves to use it to actively solicit contributions to the > community. Previously they felt unable to make use of the Help Most > Wanted list because it was written with the wrong audience in mind. > > * I tweaked the guidelines for the release naming system to better align > with our actual practice, because they had diverged somewhat. Believe it > or not, this one issue took up more of the TC's time than any other > thing that happened in 2019, and I'm very glad that we ended up not > needing these tweaks because we eventually replaced the process > wholesale, with one that makes it explicit that the TC is ultimately > accountable for the release name not causing any international incidents > or other snafus. > > * I researched and documented a design, developed during extensive > discussions with many members of the technical community, for a new kind > of simplified cloud built partially using OpenStack components but > designed to be Kubernetes-native first: Project Teapot > https://governance.openstack.org/ideas/ideas/teapot/index.html > > I do think this is worth building, but if nothing else I really hope > this helps bring into focus the parts of OpenStack that remain essential > in a Kubernetes-centric world. In writing this I was very much guided by > the Vision for OpenStack Clouds as well as the ways in which Kubernetes > expects to interact with an underlying cloud. I think we need more of > these kinds of exercises, and while they don't have to be sponsored by > TC members, I think it helps. > > > If there's a pattern here it's this: take something that's implicitly > agreed upon by the community (whether we realise it or not), write it > down, seek feedback, and make it explicit so that anyone can refer to > it, rather than leaving it as closely held tribal knowledge. > > Of course it goes without saying that I was involved in many more > discussions and reviews that were led by others. I'll leave it to them > to talk about those. And it should also go without saying that all of > the work listed above was greatly improved by the input of the many > talented folks in our community who contributed. > > > In my humble opinion OpenStack is the best open source community of its > size ever created. That's because we believe not just in one 'open' > (source), but four > (https://governance.openstack.org/tc/reference/opens.html); because > we're committed to using open tools to build open software; and because, > while it's certainly not the case that we can't improve, we have managed > to do this while avoiding a widespread culture of bullying. To do all of > this on the scale of OpenStack was, as far as I know, completely without > precedent. (Perhaps the ASF comes closest, but its projects operate > independently, unlike OpenStack subprojects.) All of us should be proud > of this achievement. But it didn't happen by accident: it is the product > of concerted effort by leaders in our community over the course of a > decade. It's been a tremendous privilege to work alongside some of those > folks, both before, during and (hopefully) after my time on the TC. I > have learned so much from them. Now it is time for others to step up and > continue to build that legacy, as the project evolves and adapts over > time. The diversification of the OSF also gives us the opportunity to > spread our ethos to other projects under the same umbrella, and from > there to the open source community as a whole. > > However, we must remember that having a great community is not enough. > To remain relevant, we also need to produce software that meets users' > needs, which change over time. By an accident of fate I have approached > OpenStack from a different direction than most - I came to OpenStack via > the Heat project, which means I've always looked at it from an > outside-in standpoint: users are building cloud applications, how can > OpenStack provide them with what they need from a cloud through an open > API? Many others, by necessity, have approached it from the inside-out: > we have all this hardware, how can we expose its capabilities to users? > I think it is fair to say that many of these folks and I have found each > other's views to be mutually incomprehensible over the years. In writing > up the Teapot design, I have learned a lot more about the other > perspective. I believe we will need a more people to also grow their > understanding in the opposite direction to avoid the trap of most > capital-E "Enterprise" software - designed to check the boxes of those > who decide what products to buy, but widely hated by all those forced to > actually use it. Future TCs will have to grapple with this because > nobody else in the community is really empowered to do so. The > openstack/ideas repo that JP created is a great place for anybody to > start documenting ways that we can improve. > > For my part, this is not goodbye. However, I have already been splitting > my time between OpenStack and metal3.io and I expect that the drift in > the other direction will only accelerate now that I no longer have any > formal responsibilities here. Please feel free to contact me or ask > questions if you think I can help. > > Stay safe out there, and by "out there" I mean physically isolated in > your own homes for the next little while. > > cheers, > Zane. > > > -- *Trinh Nguyen* *www.edlab.xyz * -------------- next part -------------- An HTML attachment was scrubbed... URL: From akekane at redhat.com Mon Mar 30 05:13:35 2020 From: akekane at redhat.com (Abhishek Kekane) Date: Mon, 30 Mar 2020 10:43:35 +0530 Subject: [election][glance] PTL candidacy for Victoria Message-ID: Hi All, I would like to announce my candidacy for Glance PTL for the Victoria cycle. Over the past couple of years we've had a small team looking after Glance and planned the priorities accordingly. During Ussuri cycle we were able to add a couple of important features related edge computing, plugin for image decompression, restored s3 driver support in glance-store, removed deprecated sheepdog driver and fixed some important bugs. Should community to select me as PTL for Victoria cycle I would keep driving the key priorities. Production stability, finishing the work we've had in flight and focus on the feature needs related to Edge computing. Thank you for taking my self nomination into consideration, Abhishek Kekane (abhishekk) -------------- next part -------------- An HTML attachment was scrubbed... URL: From chdzsp at 163.com Mon Mar 30 05:42:06 2020 From: chdzsp at 163.com (Shengping Zhong) Date: Mon, 30 Mar 2020 13:42:06 +0800 (CST) Subject: [openstack-discuss] [puppet] Nominating Takashi Kajinami for core of the Puppet OpenStack modules Message-ID: <2edf6519.7982.17129f619d5.Coremail.chdzsp@163.com> Hey Puppet Cores, I would like to nominate Takashi Kajinami as a Core reviewer for the Puppet OpenStack modules. He is an excellent contributor to our modules over the last several cycles. His stats for the last 90 days can be viewed here[0]. Please response with your +1 or any objections. If there are no objections by April 6 I will add him to the core list. As there were no objections, I have added Takashi to the core list. Keep up the good work. Thanks, Shengping. [0] https://www.stackalytics.com/report/contribution/puppet%20openstack-group/90 -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.urdin at binero.com Mon Mar 30 07:09:31 2020 From: tobias.urdin at binero.com (Tobias Urdin) Date: Mon, 30 Mar 2020 07:09:31 +0000 Subject: [openstack-discuss] [puppet] Nominating Takashi Kajinami for core of the Puppet OpenStack modules In-Reply-To: <2edf6519.7982.17129f619d5.Coremail.chdzsp@163.com> References: <2edf6519.7982.17129f619d5.Coremail.chdzsp@163.com> Message-ID: <1585552171666.54902@binero.com> ?Big +1 ________________________________ From: Shengping Zhong Sent: Monday, March 30, 2020 7:42 AM To: openstack-discuss at lists.openstack.org Subject: [openstack-discuss] [puppet] Nominating Takashi Kajinami for core of the Puppet OpenStack modules Hey Puppet Cores, I would like to nominate Takashi Kajinami as a Core reviewer for the Puppet OpenStack modules. He is an excellent contributor to our modules over the last several cycles. His stats for the last 90 days can be viewed here[0]. Please response with your +1 or any objections. If there are no objections by April 6 I will add him to the core list. As there were no objections, I have added Takashi to the core list. Keep up the good work. Thanks, Shengping. [0] https://www.stackalytics.com/report/contribution/puppet%20openstack-group/90 -------------- next part -------------- An HTML attachment was scrubbed... URL: From renat.akhmerov at gmail.com Mon Mar 30 08:02:19 2020 From: renat.akhmerov at gmail.com (Renat Akhmerov) Date: Mon, 30 Mar 2020 15:02:19 +0700 Subject: [mistral][ptl][election] PTL candidacy for Victoria In-Reply-To: <869ee322-1aa8-4868-8c86-7b2df0dcf6cc@Spark> References: <869ee322-1aa8-4868-8c86-7b2df0dcf6cc@Spark> Message-ID: <67febae7-dbd9-491f-9505-49945daa3918@Spark> Hi, I'm Renat Akhmerov. I'd like to announce my PTL candidacy for Mistral in Victoria cycle. In Ussuri, we kept improving Mistral performance to address use cases that desperately needed it. For the most complex known workflows the execution time has now dropped by 5-6 times. Besides lots of performance changes we also achieved:  * Restructured the documentation completely  * Fixed a number of documentation pages and added several new ones.    There's still a lot to do on that though.  * Made a big refactoring of Mistral actions and moved all OpenStack    actions into mistral-extra repo.  * Added namespaces for actions and workbooks  * Added a new cookiecutter-based template for creating Mistral action    projects.  * Fixed a number of important bugs (for "join" tasks, scheduler etc.).  * And lots of other changes.. For V cycle I'd like to keep making Mistral more usable:  * Consistent and well-structured documentation  * More ways to create Mistral actions (including action providers)  * Additional tools that simplify creating custom actions and functions    for Mistral  * Easier debugging of workflow errors  * Easier and smoother installation process We're already making a good progress with all that and anybody who wants to participate is very welcome onboard. The best way to get in touch with us is IRC channel #openstack-mistral or the openstack-discuss mailing list (with [mistral] tag in email subject). [1] https://review.opendev.org/#/c/715837/ Thanks Renat Akhmerov @Nokia -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Mon Mar 30 08:58:05 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Mon, 30 Mar 2020 10:58:05 +0200 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: <03d4afe4-e724-430a-851f-be786ea86a37@www.fastmail.com> References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> <03d4afe4-e724-430a-851f-be786ea86a37@www.fastmail.com> Message-ID: <07A2E701-4578-46AC-944F-9567B8FE4C62@redhat.com> Hi, I know that my vote don’t count but big +1 from me. Every time when Martin was doing review of my patch or triaging my bug reported to tempest, he had very valuable comments and advices. It would be Great to have him in Tempest core team :) > On 29 Mar 2020, at 04:36, Masayuki Igawa wrote: > > +1 !!! > > -- Masayuki Igawa > > On Sun, Mar 29, 2020, at 10:43, Ghanshyam Mann wrote: >> Hello Everyone, >> >> Martin Kopec (IRC: kopecmartin) has been doing great contribution in >> Tempest since a long time. >> He has been doing bugs Triage also in Ussuri cycle and has good >> understanding of Tempest. >> >> I would like to propose him for Tempest Core. You can vote/feedback on >> this email. If no objection by the end of this week, I will add him to >> the list. >> >> -gmann >> >> > — Slawek Kaplonski Senior software engineer Red Hat From ltoscano at redhat.com Mon Mar 30 09:21:45 2020 From: ltoscano at redhat.com (Luigi Toscano) Date: Mon, 30 Mar 2020 11:21:45 +0200 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: References: Message-ID: <7856294.jXfYcTemz2@whitebase.usersys.redhat.com> On Saturday, 28 March 2020 10:29:44 CEST Thomas Goirand wrote: > Hi, > > Sphinx 2.4 has been uploaded to Experimental. I just received bug > reports against the OpenStack packages because some of them cannot be > built with Sphinx 2.4. Here's the list: > > [...] > - sahara Sahara should have been fixed by: https://review.opendev.org/#/c/710658/ -- Luigi From thierry at openstack.org Mon Mar 30 10:04:35 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 30 Mar 2020 12:04:35 +0200 Subject: [all][tc] Post-mortem of my time on the TC In-Reply-To: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> References: <418ddc03-8134-68d7-29ee-0dd1921aefe4@redhat.com> Message-ID: <9a54ba37-e86f-17a4-ac75-45e200434868@openstack.org> Zane Bitter wrote: > As promised last year, I will not be running for a third term as a > member of the Technical Committee. I'm stepping down to spend more time > with my family (no, really! ;) > > I thought this would be a good opportunity to talk to those folks who > are thinking of running for a seat, or who aren't but should be. The > most common question I hear from potential candidates is "what should I > actually do on the TC?" I can't tell you what *you* should do, but I can > tell you what I did and how that worked out. Maybe that will give you > the flavour. And perhaps (he hinted) some of the other outgoing TC > members can do the same to provide more of a cross-section. Thanks Zane! You achieved a lot during your TC membership, and your perspective was extremely valuable for the group, and beyond for all of the OpenStack community. I agree with everything you said (in that letter). In particular: > If there's a pattern here it's this: take something that's implicitly agreed upon by the community (whether we realise it or not), write it down, seek feedback, and make it explicit so that anyone can refer to it, rather than leaving it as closely held tribal knowledge. I see two sides to the TC work. One is to perceive things/changes that are needed and probably already agreed upon by the community, and clarify and document those. The other side of the coin is to be able to take that step back and consider the good of "OpenStack" as a whole -- a framework that people deploy to get things done, rather than individual git repositories and silo-ed teams. You did well on both fronts :) -- Thierry Carrez (ttx) From pkopec at redhat.com Mon Mar 30 10:15:09 2020 From: pkopec at redhat.com (Piotr Kopec) Date: Mon, 30 Mar 2020 12:15:09 +0200 Subject: [tripleo][nova] New project to support advanced Nova features in TripleO Message-ID: Hello, I'd like to introduce a new project and repository [0] that is intended to host Ansible roles to support advanced Nova features, such as FPGA or NVDIMM. Roles will be as generic as possible but the main use case is to use them with TripleO. This project is intendent to be managed similarly to `tripleo-ipa` [1]. Please, accept the change to add new project and repo. Best regards, Piotr [0]: https://review.opendev.org/#/c/715734/ [1]: https://review.opendev.org/#/c/711114/ From anlin.kong at gmail.com Mon Mar 30 10:54:32 2020 From: anlin.kong at gmail.com (Lingxian Kong) Date: Mon, 30 Mar 2020 23:54:32 +1300 Subject: [LOCI] Failed to build Trove image Message-ID: The command I am using: docker build . \ --build-arg FROM=loci-base:ubuntu \ --build-arg PYTHON3=yes \ --build-arg PROJECT=trove \ --tag loci-trove:ubuntu The error: http://paste.openstack.org/show/791311/ Then I changed pydep.txt: uwsgi [!trove] and ran again, another error occured: http://paste.openstack.org/show/791313/ However, if removed '--build-arg PYTHON3=yes' in the command (even without modifying pydep.txt), the image could be built sucessfully. Does LOCI support to build Python 3 image or I did something wrong? - Best regards, Lingxian Kong Catalyst Cloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmellado at redhat.com Mon Mar 30 11:48:52 2020 From: dmellado at redhat.com (Daniel Mellado) Date: Mon, 30 Mar 2020 13:48:52 +0200 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> Message-ID: Big +1 from my side, congratulations Martin! On 3/29/20 3:43 AM, Ghanshyam Mann wrote: > Hello Everyone, > > Martin Kopec (IRC: kopecmartin) has been doing great contribution in Tempest since a long time. > He has been doing bugs Triage also in Ussuri cycle and has good understanding of Tempest. > > I would like to propose him for Tempest Core. You can vote/feedback on this email. If no objection by the end of this week, I will add him to the list. > > -gmann > From arxcruz at redhat.com Mon Mar 30 12:15:31 2020 From: arxcruz at redhat.com (Arx Cruz) Date: Mon, 30 Mar 2020 14:15:31 +0200 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> Message-ID: +1 Congratulations Martin!!! On Mon, Mar 30, 2020 at 1:56 PM Daniel Mellado wrote: > Big +1 from my side, congratulations Martin! > > On 3/29/20 3:43 AM, Ghanshyam Mann wrote: > > Hello Everyone, > > > > Martin Kopec (IRC: kopecmartin) has been doing great contribution in > Tempest since a long time. > > He has been doing bugs Triage also in Ussuri cycle and has good > understanding of Tempest. > > > > I would like to propose him for Tempest Core. You can vote/feedback on > this email. If no objection by the end of this week, I will add him to the > list. > > > > -gmann > > > > > -- Arx Cruz Software Engineer Red Hat EMEA arxcruz at redhat.com @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Mon Mar 30 12:57:04 2020 From: tkajinam at redhat.com (Takashi Kajinami) Date: Mon, 30 Mar 2020 21:57:04 +0900 Subject: [election][storlets] PTL candidacy for Victoria Message-ID: Hi All, I'd like to announce my candidacy to run PTL role for Storlets project in the next Victoria cycles. Thanks to the great effort by several members, we've completed all python3 compatibility work during this Ussuri cycle. Because we are a really small team now, I'll expect that we will focus on improving operability and maintenability in this cycle, while we will also discuss how we'll keep our project in the future. Managing that development and discussion is my responsibility in Storlets project. Thank you, Takashi Kajinami -------------- next part -------------- An HTML attachment was scrubbed... URL: From whayutin at redhat.com Mon Mar 30 13:54:30 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Mon, 30 Mar 2020 07:54:30 -0600 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> Message-ID: +1 woot! On Mon, Mar 30, 2020 at 6:16 AM Arx Cruz wrote: > +1 Congratulations Martin!!! > > On Mon, Mar 30, 2020 at 1:56 PM Daniel Mellado > wrote: > >> Big +1 from my side, congratulations Martin! >> >> On 3/29/20 3:43 AM, Ghanshyam Mann wrote: >> > Hello Everyone, >> > >> > Martin Kopec (IRC: kopecmartin) has been doing great contribution in >> Tempest since a long time. >> > He has been doing bugs Triage also in Ussuri cycle and has good >> understanding of Tempest. >> > >> > I would like to propose him for Tempest Core. You can vote/feedback on >> this email. If no objection by the end of this week, I will add him to the >> list. >> > >> > -gmann >> > >> >> >> > > -- > > Arx Cruz > > Software Engineer > > Red Hat EMEA > > arxcruz at redhat.com > @RedHat Red Hat > Red Hat > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz at redhat.com Mon Mar 30 14:01:55 2020 From: aschultz at redhat.com (Alex Schultz) Date: Mon, 30 Mar 2020 08:01:55 -0600 Subject: [openstack-discuss] [puppet] Nominating Takashi Kajinami for core of the Puppet OpenStack modules In-Reply-To: <1585552171666.54902@binero.com> References: <2edf6519.7982.17129f619d5.Coremail.chdzsp@163.com> <1585552171666.54902@binero.com> Message-ID: +1 On Mon, Mar 30, 2020 at 1:19 AM Tobias Urdin wrote: > > Big +1 > > > ________________________________ > From: Shengping Zhong > Sent: Monday, March 30, 2020 7:42 AM > To: openstack-discuss at lists.openstack.org > Subject: [openstack-discuss] [puppet] Nominating Takashi Kajinami for core of the Puppet OpenStack modules > > > Hey Puppet Cores, > > > I would like to nominate Takashi Kajinami as a Core reviewer for the > > Puppet OpenStack modules. He is an excellent contributor to our > > modules over the last several cycles. His stats for the last 90 days > > can be viewed here[0]. > > > Please response with your +1 or any objections. If there are no > > objections by April 6 I will add him to the core list. > > > As there were no objections, I have added Takashi to the core list. > > Keep up the good work. > > > Thanks, > > Shengping. > > > [0] https://www.stackalytics.com/report/contribution/puppet%20openstack-group/90 > > > > From gmann at ghanshyammann.com Mon Mar 30 14:03:10 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 30 Mar 2020 09:03:10 -0500 Subject: Upcoming pip changes - stop installing test-requirements in devstack In-Reply-To: References: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> Message-ID: <1712bc0d6cc.b8c1d108148375.1053329105595332432@ghanshyammann.com> ---- On Sat, 28 Mar 2020 13:57:38 -0500 Sean Mooney wrote ---- > On Fri, 2020-03-27 at 14:24 -0500, Monty Taylor wrote: > > Heya, > > > > Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to > > verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us. > > > > The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those > > at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run > > devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state > > is not one that is completely consistent. > > > > I’ve pushed up this: > > > > https://review.opendev.org/#/c/715469/ > > > > Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in > > devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest > > which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on > > devstack fail now. > > i brought up the topic of stoping installing test-requirement in the past as it has caused issue before so im happy > to see this change going forward :) > > > > It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s > > worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re > > not going to get broken this summer. Currently, neutron gate is red for glance_store missing requirement of swift/cinderclients which used to get installed by test-requirements installation. Fixes we are trying: - https://review.opendev.org/#/c/715835/2 - https://review.opendev.org/#/c/715722/1 Swift functional jobs are also failing on py2 things due to stestr drop py2 support so capping of stestr for py2 is up - https://review.opendev.org/#/c/715942/ let us know on this thread or on #openstack-qa if any other project failing. We are trying to merge those asap and if that fails further or take time we will go with temporary revert of test-requirement drop patch -gmann > > > > Monty > > > > > From emilien at redhat.com Mon Mar 30 14:05:13 2020 From: emilien at redhat.com (Emilien Macchi) Date: Mon, 30 Mar 2020 10:05:13 -0400 Subject: [openstack-discuss] [puppet] Nominating Takashi Kajinami for core of the Puppet OpenStack modules In-Reply-To: <1585552171666.54902@binero.com> References: <2edf6519.7982.17129f619d5.Coremail.chdzsp@163.com> <1585552171666.54902@binero.com> Message-ID: yes big +2 as well ! Thanks for your contributions :) On Mon, Mar 30, 2020 at 3:20 AM Tobias Urdin wrote: > ​Big +1 > > > ------------------------------ > *From:* Shengping Zhong > *Sent:* Monday, March 30, 2020 7:42 AM > *To:* openstack-discuss at lists.openstack.org > *Subject:* [openstack-discuss] [puppet] Nominating Takashi Kajinami for > core of the Puppet OpenStack modules > > > Hey Puppet Cores, > > > I would like to nominate Takashi Kajinami as a Core reviewer for the > > Puppet OpenStack modules. He is an excellent contributor to our > > modules over the last several cycles. His stats for the last 90 days > > can be viewed here[0]. > > > Please response with your +1 or any objections. If there are no > > objections by April 6 I will add him to the core list. > > > As there were no objections, I have added Takashi to the core list. > > Keep up the good work. > > > Thanks, > > Shengping. > > > [0] > https://www.stackalytics.com/report/contribution/puppet%20openstack-group/90 > > > > -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From knikolla at bu.edu Mon Mar 30 15:50:24 2020 From: knikolla at bu.edu (Nikolla, Kristi) Date: Mon, 30 Mar 2020 15:50:24 +0000 Subject: [elections][tc] Nomination Message-ID: Hi all, Please consider my candidacy for a seat on the OpenStack Technical Committee. I'm a software engineer at the Mass Open Cloud[0], a collaboration between the five major universities of the Boston area to create and operate a public cloud that can be replicated and federated. I've been involved with OpenStack for the past 5 years as a user, operator, and developer. I'm a core reviewer for the keystone project, and I'm also running for its PTL for the Victoria cycle[1]. I have a previous unsuccessful run for the TC[2]. Lately, I've also been involved with OpenInfra Labs[3], a new project of the OSF. I do not have a platform per se, and honestly I wasn't planning on running at all. I have a lot to learn, and if I were to be elected, pretty big shoes to fill. If you are looking for someone that already has all the answers, look elsewhere. On the other hand, if you're looking for someone who is eager to grow, learn, and give back to the community in any way he can, I hope you will consider my candidacy. I'm deeply grateful to the OpenStack community and all the people I've had the pleasure of working with, being mentored by, and doing karaoke during the PTGs. You are like family. Thank you, Kristi Nikolla Pronouns: he, him, his [0]. https://massopen.cloud [1]. https://opendev.org/openstack/election/raw/branch/master/candidates/victoria/Keystone/knikolla%40bu.edu [2]. https://opendev.org/openstack/election/raw/branch/master/candidates/stein/TC/knikolla at bu.edu [3]. https://openinfralabs.org From openstack at nemebean.com Mon Mar 30 16:01:12 2020 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 30 Mar 2020 11:01:12 -0500 Subject: [oslo] Feature Freeze is this week In-Reply-To: References: Message-ID: And we're now in feature freeze. Please don't merge anything that would require a feature release (I'll be doing final feature releases today), and if something does need to go in ask for a feature freeze exception here on the list. Thanks. On 3/24/20 12:21 PM, Ben Nemec wrote: > Apologies for not mentioning this in the meeting this week, but Oslo > Feature Freeze is already this Friday.[0] After Friday, anything that is > not a bug fix will need a feature freeze exception in order to merge. > > If you're wondering why Oslo freezes early, there's a doc[1] for that. > If you have any questions not answered there, feel free to reply here or > ping me on IRC. > > Thanks. > > -Ben > > 0: https://releases.openstack.org/ussuri/schedule.html > 1: > https://specs.openstack.org/openstack/oslo-specs/specs/policy/feature-freeze.html > From David.Paterson at dell.com Mon Mar 30 16:26:08 2020 From: David.Paterson at dell.com (David.Paterson at dell.com) Date: Mon, 30 Mar 2020 16:26:08 +0000 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> Message-ID: <262984f8419a4fe0b6e5babd6eba0e28@AUSX13MPC102.AMER.DELL.COM> +1 -----Original Message----- From: Ghanshyam Mann Sent: Saturday, March 28, 2020 9:43 PM To: openstack-discuss Subject: [qa] Proposing Martin Kopec to Tempest core [EXTERNAL EMAIL] Hello Everyone, Martin Kopec (IRC: kopecmartin) has been doing great contribution in Tempest since a long time. He has been doing bugs Triage also in Ussuri cycle and has good understanding of Tempest. I would like to propose him for Tempest Core. You can vote/feedback on this email. If no objection by the end of this week, I will add him to the list. -gmann From pbabbar at redhat.com Mon Mar 30 16:36:39 2020 From: pbabbar at redhat.com (Paras Babbar) Date: Mon, 30 Mar 2020 12:36:39 -0400 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: <262984f8419a4fe0b6e5babd6eba0e28@AUSX13MPC102.AMER.DELL.COM> References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> <262984f8419a4fe0b6e5babd6eba0e28@AUSX13MPC102.AMER.DELL.COM> Message-ID: +1 On Mon, Mar 30, 2020 at 12:31 PM wrote: > > +1 > > -----Original Message----- > From: Ghanshyam Mann > Sent: Saturday, March 28, 2020 9:43 PM > To: openstack-discuss > Subject: [qa] Proposing Martin Kopec to Tempest core > > > [EXTERNAL EMAIL] > > Hello Everyone, > > Martin Kopec (IRC: kopecmartin) has been doing great contribution in Tempest since a long time. > He has been doing bugs Triage also in Ussuri cycle and has good understanding of Tempest. > > I would like to propose him for Tempest Core. You can vote/feedback on this email. If no objection by the end of this week, I will add him to the list. > > -gmann > -- Paras Babbar Quality Engineer, OpenStack Red Hat | Westford, MA| M:+1 857-222-7309 | PBabbar at redhat.com From abishop at redhat.com Mon Mar 30 16:43:51 2020 From: abishop at redhat.com (Alan Bishop) Date: Mon, 30 Mar 2020 09:43:51 -0700 Subject: [oslo] Feature Freeze exception request Message-ID: I'd like to request an exception for https://review.opendev.org/710539. CI is fully green now, it has one +2, and an early patch set had +2 from another reviewer. Thanks for your consideration, Alan Bishop (abishop) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr at ham.ie Mon Mar 30 16:45:29 2020 From: gr at ham.ie (Graham Hayes) Date: Mon, 30 Mar 2020 17:45:29 +0100 Subject: [election][tc] Nomination for TC Message-ID: <14b4b205-257f-9c07-6d12-938f26ed1b3c@ham.ie> Hello Everyone, I have been a member of the TC for 2 terms now, and I am running again for another term on the TC. I have been involved in OpenStack as a developer, operator, PTL, and TC member for a long time, and I think that I can continue to contribute to the community on the TC. I worked as the technical lead for the System Operations team in Verizon Cloud Platform, where we ran 90+ OpenStack installs across the world, ranging from 3 compute micro regions where we needed edge style compute, to much larger footprints for core regions in the US. This gave me a unique viewpoint on how people use OpenStack for both small scale, and for large, real time mission critical telecommunications applications. Further back in my career, I was part of a team that ran the DNSaaS service in HP Cloud (based on Designate of course), which combined with my current role in Azure, where I am looking at how people use both on prem and external clouds, gives me a good insight into the needs of a cloud provider. My main focus over the last term has been trying to get a co-ordinated way for organizations to contribute to OpenStack - Graham's soap box at the Board face to face meetings has become a regular event. Thankfully, I think this has helped push this forward, and I know I have seen people getting involved in the projects I keep an eye on. I have also been involved in the design discussions around `Project Teapot`_ which I think is a great way for OpenStack to help people scale and manage their data centers, while providing the services modern applications are starting to use. I firmly believe that something like Teapot is an important focus of development for OpenStack. I think the new `ideas repo`_ is a fantastic idea and encourage people to put what ever they think is a good idea for OpenStack in there. I will definitely be putting up some of the ideas I have talked about over the years in there. As a long time PTL of a project I know the pressures they can be under and can provide insight as we consider the future governance of the OpenStack project, and its sub projects. This has given me a lot of historical knowledge and context to why some things within our community are the way they are, and as we record this to avoid needing something as unreliable as a human brain (rst is a lot better :) ) I feel I can contribute to the discussion. Finally, I feel that over the years I have made it clear that I will speak out even if I know it will cause push back, and this has allowed me to receive frank and honest feedback from people in community who don't always wish to spend the mental energy, or pay the social tax of raising topics that can be controversial. I don't plan on changing this. Lastly, I have really enjoyed working in this community, it has given me a lot both personally and professionally, and I feel that I can still pay that back. If you agree, please vote for me in the up coming elections, if not please ask me questions in the next week or so, but for everyone, please vote for whoever you think will be the best for OpenStack. Thanks for taking the time to read this, - Graham .. _Project Teapot: https://governance.openstack.org/ideas/ideas/teapot/index.html .. _ideas repo: https://opendev.org/openstack/ideas From fungi at yuggoth.org Mon Mar 30 17:14:56 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 30 Mar 2020 17:14:56 +0000 Subject: OpenStack should be buildable with sphinx 2.4: FTBFS in many packages In-Reply-To: <27fdb596ed0049f9acf95760a8d31c04@AUSX13MPS308.AMER.DELL.COM> References: <00c0057794c544459e0c94802886dbf0@AUSX13MPS308.AMER.DELL.COM> <20200329222221.iretkuoqec5cg7sj@yuggoth.org> <27fdb596ed0049f9acf95760a8d31c04@AUSX13MPS308.AMER.DELL.COM> Message-ID: <20200330171455.tffufmqwj65k6zvn@yuggoth.org> On 2020-03-29 23:41:03 +0000 (+0000), Arkady.Kanevsky at dell.com wrote: > I would assume we want all openstack doc per release follow this, > including all drivers? Correct, our upper-constraints.txt[0] file actually specifies which version of Sphinx projects should be using to test their documentation builds, as of this moment Sphinx===2.4.4 for master branch docs jobs if run under Python 3.6 or 3.7 (our "tested runtimes" for the Ussuri cycle[1]). I'm somewhat curious why these problems weren't caught sooner, given that. Are those projects running with a different Python interpreter version? Or without global constraints applied? Or perhaps they're not testing documentation builds at all when merging changes? We've basically been telling projects to test[2] with Sphinx 2.4 since 2.4.0 was first released[3] in February. [0] https://opendev.org/openstack/requirements/src/branch/master/upper-constraints.txt [1] https://governance.openstack.org/tc/reference/runtimes/ussuri.html [2] https://review.opendev.org/705380 [3] https://pypi.org/project/Sphinx/2.4.0/#history -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From sean.mcginnis at gmx.com Mon Mar 30 18:02:26 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 30 Mar 2020 13:02:26 -0500 Subject: [election][plt] Sean McGinnis candidacy for Release Management PTL Message-ID: <20200330180226.GA95119@sm-workstation> Hello everyone, I'd like to run for PTL of the Release Management team again. We have a lot of great automation in place in the releases repo to handle a lot of the release process with, hopefully, minimal overhead. That said, we've made some process changes over the last couple of release cycles that have placed a little more work on the release team's shoulders. We have a lot of these processes mostly automated, but a few things that require a little history and trival knowledge yet. Over the next release cycle I would like to work towards automating a little more of that to make sure if/when anyone else steps into a release manager role, it will be an easy introduction to them to follow our release process documentation and have tooling in place to execute each step without needing to have a very deep understanding of how all the various pieces fit together. Like other projects, we've lost some people, or some of us are not able to spend as much time as we used to on the project. But we've also picked up some great help too. Anything we can do to help anyone to join the team and know what to do and how to do it will be better for us all. Thanks for your consideration! Sean From jeremyfreudberg at gmail.com Mon Mar 30 19:04:07 2020 From: jeremyfreudberg at gmail.com (Jeremy Freudberg) Date: Mon, 30 Mar 2020 15:04:07 -0400 Subject: [tc][elections] my nomination Message-ID: Hello everyone, I hereby announce my candidacy for the OpenStack Technical Committee. If you don't know me already I'm the guy who proposed that OpenStack releases be named after ATCs[0]. Some people thought my proposal was satire, but it wasn't, so if you're hoping to elect a satirist look elsewhere. However, this incident is part of my motivation to run for the TC. We can't have satire without culture, and clearly our community has a strong culture if we suspect, and even hope, that satirists are lurking about. All of this indicates to me that OpenStack is a special place and so I would like to play a role in the success of that special place by serving on the TC. "Immensely qualified" is the opposite of what I am, but I have some strengths: * I am nosy. * I have touched a lot of areas of OpenStack because Sahara touches a lot of areas of OpenStack. * I understand the struggles of working on a minor project (tiny team and limited audience but still big goals) because Sahara is a minor project. * I am an optimist because optimism is a necessary part of being the PTL of Sahara nowadays. * I am young. I have much to learn, and luckily I love learning. I don't have many specific goals other than making our community as vibrant as possible. Out of vibrancy, good software will hopefully come naturally, in order to make OpenStack the best it can be in the current landscape. For me being a TC member would be mostly about facilitating the exchange and synthesis of ideas (perhaps crazy ones) and not so much about being a source of ideas myself. Of course, if I happen to have a good idea (they tend to strike very suddenly), I'll share it! To close, thanks to all of you for making our community what it is. It is really something special and everyone who I have ever interacted with (or merely spied on) has been such a delight in their own unique way. It would be an honor to serve you all. -Jeremy [0] https://review.opendev.org/#/c/677827/ From openstack at nemebean.com Mon Mar 30 20:13:27 2020 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 30 Mar 2020 15:13:27 -0500 Subject: [oslo] Feature Freeze exception request In-Reply-To: References: Message-ID: <2b0e58db-4ab0-a88c-4e5f-349d91fdb731@nemebean.com> In principle I'm okay with doing this. It's an optional feature that's off by default, so it's low risk, and it doesn't require any changes in the consuming projects so there's no concern about running into feature freezes elsewhere. I did leave one question on the review though. It seems like we may need to bump the minimum version of etcd3 to make this work correctly. If anyone else has objections please state them ASAP. Otherwise I will grant the FFE for this once we sort out the etcd3 version question. On 3/30/20 11:43 AM, Alan Bishop wrote: > I'd like to request an exception for https://review.opendev.org/710539. > > CI is fully green now, it has one +2,  and an early patch set had +2 > from another reviewer. > > Thanks for your consideration, > > Alan Bishop (abishop) From whayutin at redhat.com Mon Mar 30 20:55:44 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Mon, 30 Mar 2020 14:55:44 -0600 Subject: [ptl][tripleo][election] Message-ID: Greetings, I would like to propose my candidacy for the TripleO PTL role in the Victoria cycle. This cycle there has been a lot of progress in simplifying the TripleO code base and the steps in the deployment. The team should be very proud about migrating from mistral to ansible, novaless deployment, the creation of tripleo-operator-ansible, removing paunch, and container improvements. We were also able to migrate Ussuri jobs from CentOS-7 to CentOS-8 in the nick of time. Like in the Ussuri cycle I would like to see the project continue to focus on simplification and improving the user experience. * Continue to push forward with tripleo-ansible and the transformation squad. * Work with ansible sig group, kolla, openstack-ansible, tempest and related projects. * Continue to simplify the stack of components where possible to achieve a faster, better and more efficient deployment. * Mature new features like tripleo-operator-ansible. * I would also like to focus on encouraging the project's more recent contributors to take a larger role in active collaboration, participation and leadership. Thank you for the opportunity! https://review.opendev.org/716077 -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Mon Mar 30 21:23:28 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 30 Mar 2020 14:23:28 -0700 Subject: [ironic][ops] Breaking change coming in the Victoria development cycle Message-ID: Greetings everyone, One of the items the ironic team has been focused on is improving security of remote/edge deployments where machines may be deployed on networks where an un-trusted actor could also be present. Our answer to this has been the concept of utilizing a temporary token[0] for the deployment, which we use to validate the agent heartbeat operations, and commands sent back to the agent ramdisk from the conductor. While not a complete solution to all possible attack vectors, it is a step forward and we will be taking more steps during the next cycle. For the Ussuri release, this functionality is always enabled, but is not explicitly required[1]. Deployments, with older ramdisks who choose to require this capability, must update their deployment/rescue/cleaning ramdisks to a version with a newer ironic-python-agent version from Ussuri development cycle. In Victoria, the ironic team will change the default for requirement of agent tokens such that they are required by default. Pre-Ussuri agent ramdisks will no longer work and will need to be updated. Please let us know if you have any questions or concerns. -Julia [0]: https://docs.openstack.org/ironic/latest/admin/agent-token.html [1]: https://docs.openstack.org/ironic/latest/admin/agent-token.html#how-it-works From juliaashleykreger at gmail.com Mon Mar 30 21:36:23 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 30 Mar 2020 14:36:23 -0700 Subject: [elections][ironic] PTL candidacy for the Victoria cycle Message-ID: Greetings my OpenStack family! I realize we are presently in a stressful time. That each of us are worrying, and that many of us are struggling. "Normalizing", and ultimately accepting the things we cannot control is a vital step forward in our personal and community growth. Finding a path forward is critical in this time. Not just as humans in a pandemic, but as also community members in a project. The vital aspect is continuous across both contexts: We cannot do it alone, We must work together! I too have strengths and weaknesses that I am all too well aware of, and there truly is nothing more re-assuring than those around you encouraging you to continue to support them. For we all need sturdy foundations to take the steps forward. To build the next layer! To continue the journey we are on. As such, I hereby announce my candidacy for the role of Project Team Leader for the Ironic project during the Victoria cycle. That is if you will have me. I foresee the upcoming cycle as a time when we will need to be there to support our community and users. This may take the form of advanced features, lending an occasional ear, or going that extra mile to help each other. And if we need a distraction, I believe we have have some difficult problems yet to solve which can have a major impact in not just our code, but the world at large. While not an exhaustive list of items, it is still an impactful list of work we have been discussing and seek to add into Ironic moving forward. * Multi-tenancy, while underway, opens the world to new possibilities and co-ordination of hardware resources with-in an organization or even across organisations. * Persistent Agents allowing for faster re-provision of deployments on the edge and where the networking is in a fixed configuration. * TLS for Agent Communication to help ensure the nodes on the edge remain secure. * kexec could potentially save significant amounts of time between steps of deployment, and enable operators with larger memory footprints to save precious time of a Power-on-Self-Test. * Attestation system integration so we can improve the odds of identifying a machine with modified firmware and prevent it from moving through the hardware life-cycle as a result. * Enhancement of power management, so operators can avoid having to wait for the Power-on-Self-Test, and maybe even power some of those machines off. Onward, together! For I'm fairly sure we've not completely taken over the world... yet. Julia (TheJulia) Kreger From donny at fortnebula.com Mon Mar 30 22:14:41 2020 From: donny at fortnebula.com (Donny Davis) Date: Mon, 30 Mar 2020 18:14:41 -0400 Subject: [ironic][ops] Breaking change coming in the Victoria development cycle In-Reply-To: References: Message-ID: woot woot Security !!! On Mon, Mar 30, 2020 at 5:27 PM Julia Kreger wrote: > Greetings everyone, > > One of the items the ironic team has been focused on is improving > security of remote/edge deployments where machines may be deployed on > networks where an un-trusted actor could also be present. > > Our answer to this has been the concept of utilizing a temporary > token[0] for the deployment, which we use to validate the agent > heartbeat operations, and commands sent back to the agent ramdisk from > the conductor. While not a complete solution to all possible attack > vectors, it is a step forward and we will be taking more steps during > the next cycle. > > For the Ussuri release, this functionality is always enabled, but is > not explicitly required[1]. Deployments, with older ramdisks who > choose to require this capability, must update their > deployment/rescue/cleaning ramdisks to a version with a newer > ironic-python-agent version from Ussuri development cycle. > > In Victoria, the ironic team will change the default for requirement > of agent tokens such that they are required by default. Pre-Ussuri > agent ramdisks will no longer work and will need to be updated. > > Please let us know if you have any questions or concerns. > > -Julia > > [0]: https://docs.openstack.org/ironic/latest/admin/agent-token.html > [1]: > https://docs.openstack.org/ironic/latest/admin/agent-token.html#how-it-works > > -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Mon Mar 30 22:25:49 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Mon, 30 Mar 2020 15:25:49 -0700 Subject: [first contact][all] Meeting Frequency & Time In-Reply-To: References: Message-ID: Here's the patch: https://review.opendev.org/716098 -Kendall (diablo_rojo) On Wed, Mar 25, 2020 at 9:22 PM Kendall Nelson wrote: > Hello Everyone, > > In our meeting yesterday[1], the first one of the year, we discussed > changing our meeting frequency to once a month and maybe shifting the > day/time. > > I think Tuesday still works for me but an hour or two earlier wouldn't go > amiss. If no-one objects to this, I will put a patch up to change the > meeting time & frequency next week to monthly instead of biweekly and > > -Kendall (diablo_rojo) > > [1] Meeting Log: > http://eavesdrop.openstack.org/meetings/fc_sig/2020/fc_sig.2020-03-25-02.39.log.html > [2] Current Meeting Info: > http://eavesdrop.openstack.org/#First_Contact_SIG_Meeting > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Mon Mar 30 22:33:28 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 30 Mar 2020 15:33:28 -0700 Subject: [all] Re: review.opendev.org Downtime March 20, 2020 at 1400UTC In-Reply-To: References: <73cdc1ae-54b1-4427-8329-b31d84355111@www.fastmail.com> <2e3e4b3a-9a48-545e-eb85-be6b8613d090@suse.com> <97152a82a566ee7be2e5cd8386c65d79093a00af.camel@redhat.com> <20200320142816.h2pd4n3fv6nefz4v@yuggoth.org> Message-ID: <3e189427-4702-4f8c-8142-dc0311a926fc@www.fastmail.com> On Fri, Mar 20, 2020, at 7:48 AM, Sean Mooney wrote: > On Fri, 2020-03-20 at 14:28 +0000, Jeremy Stanley wrote: > > On 2020-03-20 14:14:57 +0000 (+0000), Sean Mooney wrote: > > > on a related note i noticed i was getting different contenent from > > > https://review.openstack.org/p/openstack/nova and > > > ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git > > > last week. > > > > > > these were my remotes in my nove repo > > > gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (fetch) > > > gerrit ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git (push) > > > origin https://review.openstack.org/p/openstack/nova (fetch) > > > origin https://review.openstack.org/p/openstack/nova (push) > > > > I definitely don't recommend cloning from review.o.o, better to rely > > on https://opendev.org/openstack/nova in your origin remotes. > ya i normally set it up against either github or > https://opendev.org/openstack/nova > the review.o.o is much larger and take up way more space on disk so i > normally avoid > it but i think i origianly create this repo usign gertty for reviews. > > this was on my local laptop and i normally only use that repo for doing > quick fixes. > > > > > > the gerrit remote when i did a fetch was up to date with > > > https://opendev.org/openstack/nova and with > > > https://github.com/openstack/nova but > > > https://review.openstack.org/p/openstack/nova was behind both of > > > them by a few hours maybe a day i dont rememeber but i just > > > remember tinking it was odd. > > > > That https://review.openstack.org/p/openstack/nova URL is "just > > another replica" like opendev.org and github.com (the /p is directed > > to a local copy Gerrit is replicating to a local copy on the > > server's filesystem). > > > > > is this somethign ye were aware of that could happen? it was if > > > the redirect from review.openstack.org was pointing an out of sync > > > gerrit backend. > > > > [...] > > > > Gerrit can't guarantee consistency for its replication tasks, so it > > can certainly happen. I think we had plans to remove that /p replica > > anyway (it's going to potentially conflict with some URLs for newer > > Gerrit versions), but generally if you notice a discrepancy between > > ssh://sean-k-mooney at review.openstack.org:29418/openstack/nova.git > > and https://opendev.org/openstack/nova do let us know, we've been > > trying to get on top of a race condition in our Gitea updates where > > Gerrit will silently fail to replicate refs to some of the servers > > while they're in the middle of restarting. > > ya if i notice an issue between those too ill let you know but in generall > they seam to mostly be in sync. > To followup on this we think we addressed the major issue with https://review.opendev.org/#/c/711130/. We think the important bit is ensuring that Gerrit can detect that replication is going to fail by turning off the ssh daemon before other Gitea services and starting it last. Gerrit will happily retry replication when it detects that it is in a failure state. Unfortunately, the old restart logic had things stopping together and the ssh side would happily accept data if the actual services was done and that led to problems. Again, do let us know if you notice this in the future, but we're hopeful this recent change addresses the problem. Clark From kennelson11 at gmail.com Mon Mar 30 22:42:10 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Mon, 30 Mar 2020 15:42:10 -0700 Subject: [all] Virtualized PTG Brainstorming Community Meetings Message-ID: Hello Everyone! So! After Marks initial email[1] that announced we would be virtualizing the PTG and collected volunteers[2] to help brainstorm the best way to hold the PTG virtually, we polled the volunteers for the best times to meet. In trying to be the most inclusive we have settled on three meetings to brainstorm together as a community. They are as follows: - April 2nd at 12:00 UTC - April 6th at 17:00 UTC - April 7th at 2:00 UTC We will be using my zoom[3] and continue using the same etherpad [2] that we collected volunteers in for all the notes from all three meetings. All are welcome! And if you can attend more than one meeting, please do! The more overlap we have the more cohesive discussions we will have. Hope to see you all there! -Kendall Nelson (diablo_rojo) [1] http://lists.openstack.org/pipermail/foundation/2020-March/002854.html [2] https://etherpad.openstack.org/p/Virtual_PTG_Planning [3] https://zoom.us/my/diablorojo -------------- next part -------------- An HTML attachment was scrubbed... URL: From gouthampravi at gmail.com Mon Mar 30 23:31:13 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Mon, 30 Mar 2020 16:31:13 -0700 Subject: [election][manila] PTL candidacy for Victoria Message-ID: Greetings Zorillas & other Stackers, I would like to submit my candidacy to be the PTL of Manila for the Victoria cycle. I stepped up to this role for the Ussuri cycle and have had a tremendous learning experience. I achieved numerous things I set out to do with the support and encouragement of my fellow contributors. I'd like to continue in this role and drive the growth of this project and its community. In my role as the liaison for this community, I was able to effectively aid internalizing the "OpenStack way" of development. The project and its many deliverables saw several new contributors in the Ussuri cycle. This included successful Outreachy and Google Summer of Code interns who continue to contribute beyond the completion of their internships. We were able to prune the core maintainers group to active reviewers while mentoring and adding three new individuals. We also had a significant reduction in our bug backlog, while we continue to farm out issues for potential first time contributors. In the Victoria cycle, I plan to continue on this mission of growing the contributor base, and having Manila be a great on-boarding project for the OpenDev community. In the Ussuri cycle, thanks to our Outreachy interns, their mentors and the reviewers, we merged feature functionality to support the OpenStack Client. Similar velocity was achieved in the manila-csi project through the GSoC internship program and the continued investment beyond. We didn't get much merged in the Ussuri cycle in terms of improving the availability and fault tolerance of the core share-manager service. These multi-cycle work items will remain a matter of focus and I'd devote time to drive these further. The Ussuri cycle also saw reducing feature parity between the DHSS=True and DHSS=False driver modes. This was a goal for this community for several releases and it's coming to fruition through vendor-agnostic community driven design. In Victoria, we plan to continue down this patch and eliminate further feature gaps between many deployment and usability choices with Manila. As PTL, I will drive the community through the OpenStack-wide goals during the Victoria cycle, alongside some of our own: - Graduating experimental features: We have had significant soak time for all three features (Share Replication, Share Migration and Share Groups); and we have begun the steps to take them out of their experimental state. - Apply for the "vulnerability-managed" TC tag: As a community, we worked through our first CVE in Ussuri cycle with the aid and support from the OpenStack Vulnerability Management Team. In the Victoria cycle, we'll drive to achieve this TC tag for the project. - Evolve bug triage criteria, and automate it: Seeing an unmanageable number of bugs is a distraction enough to ignore the bug backlog. As a community we discussed the idea of automating some triage tasks on our backlog. - Address testing gaps: We'll continue to add new test case scenarios and address gaps such as the lack of scenario testing for the cephfs-native protocol. While this goal may not have a definition of done, it is here to convey my intention to prioritizing regression testing on par with feature development. So, if you will have me, I wish to serve through Victoria and get things done. Thank you for your support, Goutham Pacha Ravi IRC: gouthamr [1] https://review.opendev.org/#/c/716107/ From Tushar.Patil at nttdata.com Mon Mar 30 23:33:26 2020 From: Tushar.Patil at nttdata.com (Patil, Tushar) Date: Mon, 30 Mar 2020 23:33:26 +0000 Subject: [tosca-parser][heat-translator][tacker] - Need new version of heat-translator and tosca-parser libraries for Tacker In-Reply-To: References: Message-ID: Hi PTL and Core Reviewers, Thank you Bob and Jo for reviewing the heat-translator patch. I have uploaded a new PS [1] after addressing comments given by Jo san. Request you all to please review the newly uploaded PS. [1] : https://review.opendev.org/#/c/714026 Thank you. Regards, Tushar Patil ________________________________________ From: Patil, Tushar Sent: Thursday, March 26, 2020 7:00 PM To: openstack-discuss at lists.openstack.org Subject: [tosca-parser][heat-translator][tacker] - Need new version of heat-translator and tosca-parser libraries for Tacker Hi PTL and Core Reviewers, We are working on implementing ETSI spec in tacker for Ussuri release for which we have pushed couple of patches in heat-translator [1][2] and tosca-parser[3]. Patch [1] is not yet merged so request core reviewers to please take a look at it. We have uploaded many patches in tacker [4] but most of the patches are failing on CI as it's requires changes from heat-translator and tosca-parser. So we need a new release of heat-translator (after patch [1] is merged) and tosca-parser badly. Please let us know when are you planning to release a new version of these libraries. [1] : https://urldefense.com/v3/__https://review.opendev.org/*/q/status:merged*project:openstack/heat-translator*branch:master*topic:etsi_nfv-sol001__;IysrKw!!AKgNyAfk550h-spHnEo!OqpUJDWQwwxEBK76c23JD3R5IL0Mh6OIkqbCswRlGv16AlKf2ELDqMQq-BOR1l3G5bw$ [2] : https://urldefense.com/v3/__https://review.opendev.org/*/q/status:open*project:openstack/heat-translator*branch:master*topic:bp/support-etsi-nfv-specs__;IysrKw!!AKgNyAfk550h-spHnEo!OqpUJDWQwwxEBK76c23JD3R5IL0Mh6OIkqbCswRlGv16AlKf2ELDqMQq-BORRnrUPAI$ [3] : https://urldefense.com/v3/__https://review.opendev.org/*/c/688633__;Iw!!AKgNyAfk550h-spHnEo!OqpUJDWQwwxEBK76c23JD3R5IL0Mh6OIkqbCswRlGv16AlKf2ELDqMQq-BOR4xYmHaY$ [4] : https://urldefense.com/v3/__https://review.opendev.org/*/q/topic:bp/support-etsi-nfv-specs*(status:open*OR*status:merged)__;IysrKw!!AKgNyAfk550h-spHnEo!OqpUJDWQwwxEBK76c23JD3R5IL0Mh6OIkqbCswRlGv16AlKf2ELDqMQq-BORA0-cbDU$ Thank you. Regards, tpatil Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. From gmann at ghanshyammann.com Tue Mar 31 01:30:11 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 30 Mar 2020 20:30:11 -0500 Subject: [qa] Not running for PTL Message-ID: <1712e35d421.c194a0df166333.8068452380243178899@ghanshyammann.com> Hello Everyone, I would like to announce that I am not running for QA PTL for V cycle. Its been 4 cycles for me as the QA PTL and I think it's time for a new leader to take this responsibility. It has been a great experience for me as PTL and keeping the QA program healthy and active as it was from starting. Stabilizing the testing more and adopting new tooling under QA. Apart from PTL role, I will continue my contribution to QA with the same bandwidth as it is now. There is no change in that and you can keep pining me for any issue or discussions. Also, I will be there for a smooth PTL transition. Thanks everyone for making these 2 years a great learning period for me. -gmann From masayuki.igawa at gmail.com Tue Mar 31 02:00:07 2020 From: masayuki.igawa at gmail.com (Masayuki Igawa) Date: Tue, 31 Mar 2020 11:00:07 +0900 Subject: Add Masayuki Igawa candidacy for Quality_Assurance Message-ID: Hi everyone, I'd like to announce my candidacy as the PTL for the Quality Assurance PTL role for the Victoria cycle. First off, I would like to thank you all the contributors, core reviewers, PTLs, and anyone who involves and makes the OpenStack better. Let me introduce myself briefly. I have joined the OpenStack community since 2012 as a developer. Now, I'm a core member of some QA projects such as Tempest, os-testr, OpenStack-health, stackviz, coverage2sql, subunit2sql[1]. And I also played the mentors/instructors role at the upstream training in Japan several times. It was a great experience to know how difficult of telling people how OpenStack community is going. In the Ussuri cycle, we've almost completed "Drop Python 2.7 Support"[2] which is one of the Ussuri goals. We also have been trying to keep the gate stable which is the mission of the team. And we approved one interesting spec - "Whitebox Tempest Plugin"[3] which is one of the tempest plugins. It can access controller nodes and/or compute nodes such as read and write INI files, restart services, examine the database, etc. Tempest does not cover this feature because it isn't the scope. However, it is necessary to expand automated testing. We still have a few more planned work pending for Ussuri, such as removing/ migrating the .testr.conf to .stestr.conf, RBAC testing, removing tempest plugin sanity BLACKLIST, adding new glance tests, improving Tempest cleanup, etc. But I think we will be able to accomplish them in the Ussuri cycle as we still have around one month for Ussuri release. Along with daily QA activities, my priorities for QA for the next cycle will be: * New ideas for more tooling and/or tests project to focus on OpenStack or common communities. (whitebox tempest plugin, for example) * Stability of Tempest plugins. The sanity job is already voting, and there are 57 tempest plugins[4], but 12 plugins are in the blacklist. Some of the plugins are deprecated already, but it's hard to know from the registry page. It needs to be classified, and the page can be improved. * Guiding and motivating more contributors to QA projects, improving documentation, and advertising OpenStack QA projects. * Completing Ussuri priority items if it's still lasting. It's hard to accomplish without our collaboration. So, let's do it together! [1] http://stackalytics.com/?user_id=igawa [2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html [3] https://specs.openstack.org/openstack/qa-specs/specs/other/whitebox-tempest-plugin.html [4] https://docs.openstack.org/tempest/latest/plugin-registry.html#detected-plugins Thanks for your consideration! -- Masayuki Igawa (masayukig) From masayuki.igawa at gmail.com Tue Mar 31 02:05:34 2020 From: masayuki.igawa at gmail.com (Masayuki Igawa) Date: Tue, 31 Mar 2020 11:05:34 +0900 Subject: [election][qa] Add Masayuki Igawa candidacy for Quality_Assurance In-Reply-To: References: Message-ID: <1b1319fd-8a75-4509-a378-a33889538709@www.fastmail.com> I forgot to add tags to the subject, so, sending this again just in case. On Tue, Mar 31, 2020, at 11:00, Masayuki Igawa wrote: > Hi everyone, > > I'd like to announce my candidacy as the PTL for the Quality Assurance PTL > role for the Victoria cycle. First off, I would like to thank you all the > contributors, core reviewers, PTLs, and anyone who involves and makes the > OpenStack better. > > Let me introduce myself briefly. I have joined the OpenStack community since > 2012 as a developer. Now, I'm a core member of some QA projects such as > Tempest, os-testr, OpenStack-health, stackviz, coverage2sql, subunit2sql[1]. > And I also played the mentors/instructors role at the upstream training in > Japan several times. It was a great experience to know how difficult of > telling people how OpenStack community is going. > > In the Ussuri cycle, we've almost completed "Drop Python 2.7 Support"[2] > which is one of the Ussuri goals. We also have been trying to keep the gate > stable which is the mission of the team. And we approved one interesting > spec - "Whitebox Tempest Plugin"[3] which is one of the tempest plugins. > It can access controller nodes and/or compute nodes such as read and write > INI files, restart services, examine the database, etc. Tempest does not cover > this feature because it isn't the scope. However, it is necessary to expand > automated testing. > > We still have a few more planned work pending for Ussuri, such as removing/ > migrating the .testr.conf to .stestr.conf, RBAC testing, removing tempest > plugin sanity BLACKLIST, adding new glance tests, improving Tempest cleanup, > etc. But I think we will be able to accomplish them in the Ussuri cycle as we > still have around one month for Ussuri release. > > Along with daily QA activities, my priorities for QA for the next cycle will > be: > > * New ideas for more tooling and/or tests project to focus on OpenStack or > common communities. (whitebox tempest plugin, for example) > > * Stability of Tempest plugins. The sanity job is already voting, and there > are 57 tempest plugins[4], but 12 plugins are in the blacklist. Some of the > plugins are deprecated already, but it's hard to know from the registry > page. It needs to be classified, and the page can be improved. > > * Guiding and motivating more contributors to QA projects, improving > documentation, and advertising OpenStack QA projects. > > * Completing Ussuri priority items if it's still lasting. > > It's hard to accomplish without our collaboration. So, let's do it together! > > > [1] http://stackalytics.com/?user_id=igawa > [2] > https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html > [3] > https://specs.openstack.org/openstack/qa-specs/specs/other/whitebox-tempest-plugin.html > [4] > https://docs.openstack.org/tempest/latest/plugin-registry.html#detected-plugins > > Thanks for your consideration! > -- Masayuki Igawa (masayukig) > From mark at stackhpc.com Tue Mar 31 08:24:15 2020 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 31 Mar 2020 08:24:15 +0000 Subject: [election][kolla] Message-ID: Hi, I'd like to nominate myself to serve as the Kolla PTL for the Victoria cycle. I have been PTL for the Train and Ussuri cycles, and would like the opportunity to continue to lead the team. I am pleased with the current state of the project. We have an active community, produce interesting and useful features, and regularly receive positive feedback. The Kolla community has always been notable for being vendor-neutral and operator-driven. I recently made a proposal for a Kolla --SIG-- Club [1] with the aim of improving communication between regular contributors and the wider community. We had a good response, and I'm looking forward to seeing how it progresses this cycle. If elected this cycle I'd like to encourage continued work on some familiar themes - improving sustainability, increasing test coverage, and better integrating Kayobe [2]. I'd also like to see us focus on our documentation. Thanks for reading, Mark Goddard (mgoddard) [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013122.html [2] https://docs.openstack.org/kayobe From marcin.juszkiewicz at linaro.org Tue Mar 31 08:42:24 2020 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Tue, 31 Mar 2020 10:42:24 +0200 Subject: [election][kolla] In-Reply-To: References: Message-ID: W dniu 31.03.2020 o 10:24, Mark Goddard pisze: > I'd like to nominate myself to serve as the Kolla PTL for the Victoria cycle. +2 From moreira.belmiro.email.lists at gmail.com Tue Mar 31 10:06:22 2020 From: moreira.belmiro.email.lists at gmail.com (Belmiro Moreira) Date: Tue, 31 Mar 2020 12:06:22 +0200 Subject: [tc][elections] Nomination Message-ID: Greetings, I'd like to announce my candidacy for the OpenStack Technical Committee (TC). I'm involved with OpenStack since 2011 when CERN started to investigate a Cloud Infrastructure solution. I work in the design/deployment/operation of the CERN OpenStack Cloud. My particular interest has been how to scale and operate large OpenStack deployments. I was an active member of the old Large Deployment Team (LDT), a group of operators that shared ideas on how to scale OpenStack. I had the privilege to learn from a talented group of Operators on how they manage/run their infrastructures. Currently, a new group was created (Large Scale SIG) where I continue to have an active role. I'm fortunate enough to have attended most of the OpenStack Summits since San Francisco (2012). I'm a regular speaker talking about my experience operating OpenStack at scale. Also, I serve regularly as a track chair, participated in few OpenStack Days, Ops Meetups and wrote several blog posts. During the last year I served as a member of the OpenStack User Committee (UC). We focused in the Ambassador Program revamp and SIGs support. If elected I will focus to engage the Operator/Users community with the different Development teams and bring my experience as Developer/Operator to the group to help in the oversight on all the technical matters. It would be an honor to be an advocate of this great community and promote its mission. Thank you for your consideration, Belmiro Moreira -------------- next part -------------- An HTML attachment was scrubbed... URL: From donny at fortnebula.com Tue Mar 31 12:22:30 2020 From: donny at fortnebula.com (Donny Davis) Date: Tue, 31 Mar 2020 08:22:30 -0400 Subject: [OpenStack-Infra] [Murano-Open-PaaS] Openstack using guacamole In-Reply-To: References: Message-ID: On Tue, Mar 31, 2020 at 7:52 AM Samuel Abdullah wrote: > Hi Donny, > > Thanks for the reply. > To brief you about the environment in our infrastructure, we already have > openstack running in our organization > i am actually creating a new vm that sits in our openstack, and this vm > will be the guacamole server. Aside from that, i would like to make > guacamole as a gateway or a broker between the user and openstack so that > whenever user access to guacamole, they will only be able to manage and > create instances from guacamole that links to the openstack. Understand > from your email that i will have to connect a proper API to openstack from > guacamole? In that case, a proper extension that is only available from > guacamole end , or i will need to compose a new extension probably > something similar from the link you shared to me? Apologize as im not quite > familiar with open source environment, and i came from a windows > environment however i have a team that could do the job in opensource > (openstack. Linux,) > > Despite that, i do understand in terms of technicality and architecture on > the idea, its just the path to achieve this. > Appreciate much o your response. > > Best Regards > Samuel > > > > On Sun, Mar 29, 2020 at 8:56 PM Donny Davis wrote: > >> >> On Fri, Mar 27, 2020 at 11:19 AM Samuel Abdullah < >> samuel at silverliningsys.com> wrote: >> >>> Hi, >>> >>> Does Anyone know how can i install guacamole in openstack environment? >>> Even if its via murano? Any manual guideline? >>> >>> Best Regards >>> Samuel >>> >>> On Fri, 27 Mar 2020, 00:05 Roman Gorshunov, wrote: >>> >>>> Hello Samuel, >>>> >>>> Thanks for your email. Yes, Guacamole can be installed as an app via >>>> Murano. >>>> Discussions about Open PaaS are now in openstack-discuss mailing list. >>>> Please use the tag [Murano-Open-PaaS] in the subject line. I'm >>>> re-routing your email. >>>> >>>> Here are docs for the Murano project [0]. >>>> >>>> [0] https://docs.openstack.org/murano/latest/ >>>> >>>> Best regards, >>>> Roman Gorshunov >>>> >>>> On Thu, Mar 26, 2020 at 4:37 PM Samuel Abdullah < >>>> samuel at silverliningsys.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> Would like to ask if it is possible to run guacamole in openstack >>>>> environment? >>>>> Seek for help on how would this be possible as i saw that guacamole >>>>> component are part of the openstack in your murano project >>>>> >>>>> Looking forward for your reply >>>>> >>>>> Best Regards >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> www.silverliningsys.com >>>>> >>>>> >>>>> *Abu Bakar Samuel Abdullah* >>>>> >>>>> Cloud Infrastructure & Operations >>>>> >>>>> >>>>> P: +603-2712-0081 >>>>> >>>>> M: +60.12.654.5938 >>>>> >>>>> E: samuel at silverliningsys.com >>>>> _______________________________________________ >>>>> OpenStack-Infra mailing list >>>>> OpenStack-Infra at lists.openstack.org >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >>>> >>>> _______________________________________________ >>> OpenStack-Infra mailing list >>> OpenStack-Infra at lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> >> Samuel, >> Are you trying to replace the default vnc console with guacamole or are >> you trying to run it as a service on top of Openstack? >> >> Scenario A - built in console >> I am unaware of any method in the code today to replace the default >> console with guacamole. >> >> Scenario B - Use it as a service >> >> I can't imagine it would be too hard if you leverage some kind of >> automation framework to create your instances. In the example of using >> ansible to create machines on the cloud - you could first call the playbook >> that creates your instance and then query the API for the details of that >> instance. You could then insert the needed parameters to the guacamole >> server to access this machine. >> The following modules could be used in this case - mind you this is one >> of many examples to accomplish this task >> Create Instance >> https://docs.ansible.com/ansible/latest/modules/os_server_module.html >> >> Query the API >> https://docs.ansible.com/ansible/latest/modules/os_server_info_module.html >> >> And then to update Guacamole you can use a template that includes a >> method to properly create your backend connections of all of the required >> instances >> >> I am not an expert in Guacamole, so you will have to tinker a bit here. >> >> Also dropping infra - this ML is mainly for the actual Openstack project >> infra - so like CI resources and code hosting resources >> >> Thanks and good luck! >> >> -- >> ~/DonnyD >> C: 805 814 6800 >> "No mission too difficult. No sacrifice too great. Duty First" >> > > > -- > > > > www.silverliningsys.com > > > *Abu Bakar Samuel Abdullah* > > Cloud Infrastructure & Operations > > > P: +603-2712-0081 > > M: +60.12.654.5938 > > E: samuel at silverliningsys.com > Samuel, In your case I would think you would have to write an extension for guacamole to add a provision function. It could be something very simple that takes the users creds and then executes the openstacksdk, ansible, terraform, etc to create an instance and then wire it back to the guacamole server for their particular account. Honestly this use case is quite interesting to me, and Openstack should really be able to shine for this. Use cases like this are literally the reason to drive infrastructure with API's. I can see three functions being written: create instance query for instance details add instance details to guacamole I am afraid I am probably at my limit of usefulness, but I am super excited to see what you come up with. Please share with the community and don't be afraid to reach back out with questions. -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbauza at redhat.com Tue Mar 31 12:35:02 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Tue, 31 Mar 2020 14:35:02 +0200 Subject: [nova] FFU : When should we remove reshape support for a feature ? Message-ID: Hola, In Stein, I provided a new feature [1] that was checking inventories from multiple children resource providers (RP). Since in Rocky, those inventories were only on the root RP, I had to provide both : #1 : an upgrade support for RPs that were not having a parent RP (that was the case of Stein compute nodes) [2] #2 : a reshape method in the libvirt driver for moving allocations from a root RP to children RPs [3] Now, we're in the Ussuri timeframe and as I was usually doing, I removed the upgrade support (in #1) since we're 100% sure that all compute nodes are either Train or Ussuri [4] This is cool, but now the reshape code I wrote for #2 no longer works (which is normal, since the 'non-Stein' support was removed) [5] Soooooo. Can I also then remove the reshape support ? That would mean that upgrading from Rocky to Train would work, but Rocky to Ussuri wouldn't. I'm personally OK with this, but I guess other people could have concerns with. Do we have any consensus about how longer we would support a reshape ? I think honestly that a 3-releases time window is enough but I can hear some disagreements about this strawman proposal. If so, please voice. -Sylvain [1] https://blueprints.launchpad.net/nova/+spec/vgpu-stein [2] https://review.opendev.org/#/c/636591/ [3] https://review.opendev.org/#/c/599208/ [4] https://review.opendev.org/#/c/715489/ [5] https://541e2403478ac154d5eb-056bfb946e355d1a1a86dc411a70c5ec.ssl.cf2.rackcdn.com/715489/1/check/nova-tox-functional-py36/481510f/testr_results.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From singh.surya64mnnit at gmail.com Tue Mar 31 12:55:10 2020 From: singh.surya64mnnit at gmail.com (Surya Singh) Date: Tue, 31 Mar 2020 21:55:10 +0900 Subject: [election][kolla] In-Reply-To: References: Message-ID: Great leading of Kolla. +2 On Tue, 31 Mar 2020 at 5:32 PM, Mark Goddard wrote: > Hi, > > I'd like to nominate myself to serve as the Kolla PTL for the Victoria > cycle. > > I have been PTL for the Train and Ussuri cycles, and would like the > opportunity > to continue to lead the team. I am pleased with the current state of the > project. We have an active community, produce interesting and useful > features, > and regularly receive positive feedback. > > The Kolla community has always been notable for being vendor-neutral and > operator-driven. I recently made a proposal for a Kolla --SIG-- Club [1] > with > the aim of improving communication between regular contributors and the > wider > community. We had a good response, and I'm looking forward to seeing how it > progresses this cycle. > > If elected this cycle I'd like to encourage continued work on some familiar > themes - improving sustainability, increasing test coverage, and better > integrating Kayobe [2]. I'd also like to see us focus on our > documentation. > > Thanks for reading, > Mark Goddard (mgoddard) > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013122.html > [2] https://docs.openstack.org/kayobe > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Tue Mar 31 13:15:51 2020 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Tue, 31 Mar 2020 21:15:51 +0800 Subject: [election][heat] PTL candidacy for Victoria Message-ID: Hi all, I would like to announce my candidacy for Heat PTL for Victoria cycle. Over the past couple of cycles, we running Heat dynamically according to requirements. We never stop encouraging people to join us and provide consistent reviews or fixes as much as we can basic on priorities. And if those priorities not working for you, that means you must join the community to help. I believe the goals for the next cycle should be community CI stability, cross-project/community scenario tests to ensure what we deliver to users will get higher quality and container-native support to make sure we provider useable Orchestration services for container environment. I believe in new features, so we will try to encourage more features and review them as many as possible, but will not plan for team feature goals unless we can trigger more discussion and hands-on.Thank you for taking my self-nomination into consideration. Rico Lin -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From amodi at redhat.com Tue Mar 31 13:43:00 2020 From: amodi at redhat.com (Archit Modi) Date: Tue, 31 Mar 2020 09:43:00 -0400 Subject: [qa] Proposing Martin Kopec to Tempest core In-Reply-To: References: <17123f4f761.126cbb10d108731.5702022165643836936@ghanshyammann.com> <262984f8419a4fe0b6e5babd6eba0e28@AUSX13MPC102.AMER.DELL.COM> Message-ID: +1 thanks Martin for all the hard work!! On Mon, Mar 30, 2020 at 12:39 PM Paras Babbar wrote: > +1 > > On Mon, Mar 30, 2020 at 12:31 PM wrote: > > > > +1 > > > > -----Original Message----- > > From: Ghanshyam Mann > > Sent: Saturday, March 28, 2020 9:43 PM > > To: openstack-discuss > > Subject: [qa] Proposing Martin Kopec to Tempest core > > > > > > [EXTERNAL EMAIL] > > > > Hello Everyone, > > > > Martin Kopec (IRC: kopecmartin) has been doing great contribution in > Tempest since a long time. > > He has been doing bugs Triage also in Ussuri cycle and has good > understanding of Tempest. > > > > I would like to propose him for Tempest Core. You can vote/feedback on > this email. If no objection by the end of this week, I will add him to the > list. > > > > -gmann > > > > > -- > > Paras Babbar > > Quality Engineer, OpenStack > > Red Hat | Westford, MA| M:+1 857-222-7309 | PBabbar at redhat.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmellado at redhat.com Tue Mar 31 14:09:22 2020 From: dmellado at redhat.com (Daniel Mellado) Date: Tue, 31 Mar 2020 16:09:22 +0200 Subject: [qa] Not running for PTL In-Reply-To: <1712e35d421.c194a0df166333.8068452380243178899@ghanshyammann.com> References: <1712e35d421.c194a0df166333.8068452380243178899@ghanshyammann.com> Message-ID: <83c87b8e-88ad-d282-afe3-5efee3b04114@redhat.com> Thanks for all your hard work, Ghanshyam! On 3/31/20 3:30 AM, Ghanshyam Mann wrote: > Hello Everyone, > > I would like to announce that I am not running for QA PTL for V cycle. Its > been 4 cycles for me as the QA PTL and I think it's time for a new leader to take > this responsibility. > > It has been a great experience for me as PTL and keeping the QA program > healthy and active as it was from starting. Stabilizing the testing more and > adopting new tooling under QA. > > Apart from PTL role, I will continue my contribution to QA with the same bandwidth as > it is now. There is no change in that and you can keep pining me for any issue or discussions. > Also, I will be there for a smooth PTL transition. > > Thanks everyone for making these 2 years a great learning period for me. > > > -gmann > From pierre at stackhpc.com Tue Mar 31 14:16:15 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Tue, 31 Mar 2020 16:16:15 +0200 Subject: [blazar][election][ptl] PTL candidacy for Victoria Message-ID: Hi, I would like to self-nominate to serve as PTL of Blazar for the Victoria release cycle. I have been PTL since the Stein cycle and I am willing to continue in this role. As with many other projects, we are struggling with a lack of contributions. My focus will be to keep our small community active, help new contributors to be more involved in the project, and deliver functionality that will encourage further adoption. Thank you for your support, Pierre Riteau (priteau) From mark at stackhpc.com Tue Mar 31 14:26:06 2020 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 31 Mar 2020 14:26:06 +0000 Subject: [kolla] Tomorrow's meeting cancelled Message-ID: Hi, I'm on leave tomorrow and several cores are unavailable so let's skip this week's meeting. Please bear in mind we're approaching OpenStack feature freeze, and our own freeze will follow shortly afterwards. As such, please focus on finishing off and reviewing our cycle priorities. Cheers, Mark From ianyrchoi at gmail.com Tue Mar 31 14:26:15 2020 From: ianyrchoi at gmail.com (Ian Y. Choi) Date: Tue, 31 Mar 2020 23:26:15 +0900 Subject: [all][elections][ptl][tc] Conbined PTL/TC Nominations Last Days Message-ID: A quick reminder that we are in the last hours for declaring PTL and TC candidacies. Nominations are open until Mar 31, 2020 23:45 UTC. If you want to stand for election, don't delay, follow the instructions at [1] to make sure the community knows your intentions. Make sure your nomination has been submitted to the openstack/election repository and approved by election officials. Election statistics[2]: Nominations started       @ 2020-03-24 23:45:00 UTC Nominations end           @ 2020-03-31 23:45:00 UTC Nominations duration      : 7 days, 0:00:00 Nominations remaining     : 9:22:53 Nominations progress      :  94.42% --------------------------------------------------- Projects[1]               :    60 Projects with candidates  :    30 ( 50.00%) Projects with election    :     0 (  0.00%) --------------------------------------------------- Need election             : 0 () Need appointment          : 30 (Adjutant Barbican Blazar Cloudkitty Congress Cyborg Designate Freezer Heat I18n Infrastructure Karbor Loci Magnum Manila Masakari Nova Octavia OpenStack_Charms Oslo Packaging_Rpm Placement Quality_Assurance Rally Requirements Searchlight Swift Tacker Tricircle Zaqar) =================================================== Stats gathered            @ 2020-03-31 14:22:07 UTC This means that with approximately 2 days left, 30 projects will be deemed leaderless. In this case the TC will oversee PTL selection as described by [3]. Thank you, [1] https://governance.openstack.org/election/#how-to-submit-a-candidacy [2] Any open reviews at https://review.openstack.org/#/q/is:open+project:openstack/election     have not been factored into these stats. [3] https://governance.openstack.org/resolutions/20141128-elections-process-for-leaderless-programs.html From openstack at fried.cc Tue Mar 31 14:29:00 2020 From: openstack at fried.cc (Eric Fried) Date: Tue, 31 Mar 2020 09:29:00 -0500 Subject: [nova] FFU : When should we remove reshape support for a feature ? In-Reply-To: References: Message-ID: <3139f229-f208-ac6a-b921-66765f03d4b4@fried.cc> It may be worth taking into account that we had a work item to write a FFU hook for this, but it was never done. http://lists.openstack.org/pipermail/openstack-dev/2018-October/136110.html http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004204.html efried_stirs_pot_from_afar On 3/31/20 7:35 AM, Sylvain Bauza wrote: > Hola, > > In Stein, I provided a new feature [1] that was checking inventories > from multiple children resource providers (RP). Since in Rocky, those > inventories were only on the root RP, I had to provide both : >  #1 : an upgrade support for RPs that were not having a parent RP (that > was the case of Stein compute nodes) [2] >  #2 : a reshape method in the libvirt driver for moving allocations > from a root RP to children RPs [3] > > Now, we're in the Ussuri timeframe and as I was usually doing, I removed > the upgrade support (in #1) since we're 100% sure that all compute nodes > are either Train or Ussuri [4] > > This is cool, but now the reshape code I wrote for #2 no longer works > (which is normal, since the 'non-Stein' support was removed) [5] > > Soooooo. Can I also then remove the reshape support ? That would mean > that upgrading from Rocky to Train would work, but Rocky to Ussuri > wouldn't. I'm personally OK with this, but I guess other people could > have concerns with. > > Do we have any consensus about how longer we would support a reshape ? I > think honestly that a 3-releases time window is enough but I can hear > some disagreements about this strawman proposal. If so, please voice. > > -Sylvain > > [1] https://blueprints.launchpad.net/nova/+spec/vgpu-stein > [2] https://review.opendev.org/#/c/636591/ > [3] https://review.opendev.org/#/c/599208/ > [4] https://review.opendev.org/#/c/715489/ > [5] > https://541e2403478ac154d5eb-056bfb946e355d1a1a86dc411a70c5ec.ssl.cf2.rackcdn.com/715489/1/check/nova-tox-functional-py36/481510f/testr_results.html > From mnaser at vexxhost.com Tue Mar 31 14:48:57 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Tue, 31 Mar 2020 10:48:57 -0400 Subject: [nova] FFU : When should we remove reshape support for a feature ? In-Reply-To: References: Message-ID: On Tue, Mar 31, 2020 at 8:38 AM Sylvain Bauza wrote: > > Hola, > > In Stein, I provided a new feature [1] that was checking inventories from multiple children resource providers (RP). Since in Rocky, those inventories were only on the root RP, I had to provide both : > #1 : an upgrade support for RPs that were not having a parent RP (that was the case of Stein compute nodes) [2] > #2 : a reshape method in the libvirt driver for moving allocations from a root RP to children RPs [3] > > Now, we're in the Ussuri timeframe and as I was usually doing, I removed the upgrade support (in #1) since we're 100% sure that all compute nodes are either Train or Ussuri [4] > > This is cool, but now the reshape code I wrote for #2 no longer works (which is normal, since the 'non-Stein' support was removed) [5] > > Soooooo. Can I also then remove the reshape support ? That would mean that upgrading from Rocky to Train would work, but Rocky to Ussuri wouldn't. I'm personally OK with this, but I guess other people could have concerns with. > > Do we have any consensus about how longer we would support a reshape ? I think honestly that a 3-releases time window is enough but I can hear some disagreements about this strawman proposal. If so, please voice. I think that's more than enough from an operators POV and in terms of "making the lives of our devs" easier POV. > -Sylvain > > [1] https://blueprints.launchpad.net/nova/+spec/vgpu-stein > [2] https://review.opendev.org/#/c/636591/ > [3] https://review.opendev.org/#/c/599208/ > [4] https://review.opendev.org/#/c/715489/ > [5] https://541e2403478ac154d5eb-056bfb946e355d1a1a86dc411a70c5ec.ssl.cf2.rackcdn.com/715489/1/check/nova-tox-functional-py36/481510f/testr_results.html > -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From sbauza at redhat.com Tue Mar 31 14:49:28 2020 From: sbauza at redhat.com (Sylvain Bauza) Date: Tue, 31 Mar 2020 16:49:28 +0200 Subject: [nova] FFU : When should we remove reshape support for a feature ? In-Reply-To: <3139f229-f208-ac6a-b921-66765f03d4b4@fried.cc> References: <3139f229-f208-ac6a-b921-66765f03d4b4@fried.cc> Message-ID: On Tue, Mar 31, 2020 at 4:43 PM Eric Fried wrote: > It may be worth taking into account that we had a work item to write a > FFU hook for this, but it was never done. > > http://lists.openstack.org/pipermail/openstack-dev/2018-October/136110.html > > http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004204.html > > OK, I already provided my thoughts for the virtual PTG etherpad in https://etherpad.openstack.org/p/nova-victoria-ptg I'll add those links and see whether I can volunteer for it in the next cycle. -S efried_stirs_pot_from_afar > > On 3/31/20 7:35 AM, Sylvain Bauza wrote: > > Hola, > > > > In Stein, I provided a new feature [1] that was checking inventories > > from multiple children resource providers (RP). Since in Rocky, those > > inventories were only on the root RP, I had to provide both : > > #1 : an upgrade support for RPs that were not having a parent RP (that > > was the case of Stein compute nodes) [2] > > #2 : a reshape method in the libvirt driver for moving allocations > > from a root RP to children RPs [3] > > > > Now, we're in the Ussuri timeframe and as I was usually doing, I removed > > the upgrade support (in #1) since we're 100% sure that all compute nodes > > are either Train or Ussuri [4] > > > > This is cool, but now the reshape code I wrote for #2 no longer works > > (which is normal, since the 'non-Stein' support was removed) [5] > > > > Soooooo. Can I also then remove the reshape support ? That would mean > > that upgrading from Rocky to Train would work, but Rocky to Ussuri > > wouldn't. I'm personally OK with this, but I guess other people could > > have concerns with. > > > > Do we have any consensus about how longer we would support a reshape ? I > > think honestly that a 3-releases time window is enough but I can hear > > some disagreements about this strawman proposal. If so, please voice. > > > > -Sylvain > > > > [1] https://blueprints.launchpad.net/nova/+spec/vgpu-stein > > [2] https://review.opendev.org/#/c/636591/ > > [3] https://review.opendev.org/#/c/599208/ > > [4] https://review.opendev.org/#/c/715489/ > > [5] > > > https://541e2403478ac154d5eb-056bfb946e355d1a1a86dc411a70c5ec.ssl.cf2.rackcdn.com/715489/1/check/nova-tox-functional-py36/481510f/testr_results.html > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Tue Mar 31 14:51:13 2020 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Tue, 31 Mar 2020 22:51:13 +0800 Subject: [tc][election] Add Rico Lin candidacy for Victoria TC role Message-ID: Dear all, It has been a great year for me to work on TC and work as TC vice-chair, and I do like to join TC election again and keep serving this community as TC member if I have the honor. In the past year, I have been working on the following tasks: * TC vice-chair: I thank for the vice-chair roles allows me to dive in deeper on TC activity. As we work on TC's daily support for TC chair. IMO, JP shows some nice example to running TC chair by standardizing routine processes. And I do feel it's great to work with him. And from what I learned, to consistent checking on actions for TCs is something we should keep on. Also ML update and host meetings. What I think we should try in advance are keep pushing TC plus UC merging task, and work on providing better decision system so we can make sure at the end of discussion, we have something put into actions and that's been decided by TCs as a group. * Meta SIG chair: It's a great opportunity to working on SIG govern, which works with coordination between devs, ops, and users. We achieved on items like pushing more new SIGs, create SIG guidelines, track SIG status, and of course general SIG govern stuff. But SIGs still needs more attention. Which is one of the goals I have to push SIG forward (And I'm happy to have others as Meta SIG chair). IMO SIG only successful when it can provide a better bridge across all roles. That's something we need to keep pushing our effort to expose SIGs. * Comparison of Official Group Structures: I also work on providing [1] so whoever join this community, can have better understanding on how we works. Over time, our structure require changes, so we can evolve and provide understandable structures for the whole time (and for old and new members). What we can keep working on is to consistently review the structure flow, to make sure everything makes sense for a community to allow consistency and evolution. Also as mentioned, to combine TC and UC, will require more review on structure to make sure teams all get benefits from the achievement we plan. * Cross culture bridge: This is something I'm reeeally proud of. To help community members reach better communication despite language, culture, and timezone barriers. I get the honor to serve a great number of community members on this task, and I'm proud of it. This also alerts me that we need more members in community to become the bridge. We have some great people to keep working on this, but we simply need more from different languages, cultures, and timezone. So community members can have a better way to reach to TCs. * community goal schedule: gmann doing really good job on keep community-wide goal forward. As we implemented our goal schedule to cover from pre-select, select, and implementation schedule. I'm planning to keep govern that process to make sure we jump in the correct routine. [1] https://governance.openstack.org/tc/reference/comparison-of-official-group-structures.html -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Tue Mar 31 14:53:59 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 31 Mar 2020 16:53:59 +0200 Subject: [election][nova] PTL Candidacy for Victoria Message-ID: Hi, I'd like to announce my candidacy for Nova PTL for Train. I'm a long time Nova contributor and Nova core since 2017. I'm currently serving as temporary Nova PTL since we lost Eric during the Ussuri cycle. We are facing multiple challenges for Victoria. We have to organize the first fully virtual PTG to plan the Victoria cycle due to sever travel restrictions around the word. I have already proposed ways [1] to the Nova team and I will continue working on making the best out of the current situation. In the past months we have lost Matt and Eric, two key contributors and Nova maintainers. So I will work on regrowing the Nova core team to restore our review bandwidth. As a first step I proposed two new Nova core members [2][3]. But I will also look for more possible candidates and I will try to mentor them to further increase the size of the core team. We have many unfinished business from the Ussuri cycle. So as a PTL I have the intention to make the team focus on finishing these work before we start any new big change in Nova. Thanks, Balazs Gibizer (gibi) [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013555.html [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013511.html [3] http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013519.html From balazs.gibizer at est.tech Tue Mar 31 14:58:55 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 31 Mar 2020 16:58:55 +0200 Subject: [nova] Proposing Lee Yarwood as nova core In-Reply-To: References: Message-ID: Hi, Thanks you all for the votes. As I see we have a consensus. @Lee: Welcome in the Nova core team! Cheers, gibi On Thu, Mar 26, 2020 at 10:37, John Garbutt wrote: > +1 > > On Tue, 24 Mar 2020 at 23:59, Alex Xu wrote: >> >> +1 >> >> Sylvain Bauza 于2020年3月24日周二 >> 下午5:28写道： >>> >>> >>> >>> On Mon, Mar 23, 2020 at 3:37 PM Balázs Gibizer >>> wrote: >>>> >>>> Hi, >>>> >>>> I'm proposing Lee to the core team. He is around for a long time >>>> and he >>>> became really active recently not only on the stable branches. I >>>> think >>>> he would be a great addition to the team. >>>> >>>> Cores, please vote (-1, 0, +1) before next Monday (03.30.) >>>> >>> >>> +1 >>> >>>> Cheers, >>>> gibi >>>> >>>> >>>> From balazs.gibizer at est.tech Tue Mar 31 14:59:50 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Tue, 31 Mar 2020 16:59:50 +0200 Subject: [nova] Proposing Ghanshyam Mann as nova core In-Reply-To: References: <012990b86b48e568383b63835617e92c96f0d2be.camel@redhat.com> Message-ID: Hi, Thank you all the votes! I see we have a consensus. @Gmann: Welcome in the Nova core team! Cheers, gibi On Thu, Mar 26, 2020 at 10:36, John Garbutt wrote: > +1 > > On Tue, 24 Mar 2020 at 23:59, Alex Xu wrote: >> >> +1 >> >> Stephen Finucane 于2020年3月24日周二 >> 下午6:19写道： >>> >>> On Mon, 2020-03-23 at 17:34 +0100, Balázs Gibizer wrote: >>> > Hi, >>> > >>> > I'm proposing Ghanshyam Mann to the core team. He is a long time >>> > OpenStack and nova contributor. He is one of our API and QA >>> expert. I >>> > think he would be a great addition to the team. >>> > >>> > Cores, please vote (-1, 0, +1) before next Monday (03.30.) >>> >>> +1 >>> >>> > Cheers, >>> > gibi >>> > >>> > >>> > >>> > >>> >>> From jasonanderson at uchicago.edu Tue Mar 31 15:20:19 2020 From: jasonanderson at uchicago.edu (Jason Anderson) Date: Tue, 31 Mar 2020 15:20:19 +0000 Subject: [blazar][election][ptl] PTL candidacy for Victoria In-Reply-To: References: Message-ID: <6fdac5c2-8593-38b2-a9f6-f16f8eb63aba@uchicago.edu> +1, thanks for your work organizing the last few releases. I hope we can continue to support the Blazar project in the Victoria release and deliver some cool things! /Jason On 3/31/20 9:16 AM, Pierre Riteau wrote: > Hi, > > I would like to self-nominate to serve as PTL of Blazar for the Victoria > release cycle. I have been PTL since the Stein cycle and I am willing to > continue in this role. > > As with many other projects, we are struggling with a lack of contributions. My > focus will be to keep our small community active, help new contributors to be > more involved in the project, and deliver functionality that will encourage > further adoption. > > Thank you for your support, > Pierre Riteau (priteau) > From johnsomor at gmail.com Tue Mar 31 15:20:34 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 31 Mar 2020 08:20:34 -0700 Subject: [election][octavia][ptl] Announcing my PTL candidacy for Octavia Message-ID: My fellow OpenStack community, I would like to nominate myself for Octavia PTL during the Victoria cycle. I have previously led the team during the Stein release, and I would like to continue helping our team provide network load balancing services for OpenStack. Looking forward to Victoria I expect the team to finish some major new features, such as the Jobboard transition and introducing HTTP/2 support. Thank you for your support and your consideration for Victoria, Michael Johnson (johnsom) From kennelson11 at gmail.com Tue Mar 31 15:21:39 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 31 Mar 2020 08:21:39 -0700 Subject: [all][elections][ptl][tc] Conbined PTL/TC Nominations Last Days Message-ID: Hello Everyone! A quick reminder that we are in the last hours for declaring PTL and TC candidacies. Nominations are open until Mar 31, 2020 23:45 UTC. If you want to stand for election, don't delay, follow the instructions at [1] to make sure the community knows your intentions. Make sure your nomination has been submitted to the openstack/election repository and approved by election officials. Election statistics[2]: Nominations started @ 2020-03-24 23:45:00 UTC Nominations end @ 2020-03-31 23:45:00 UTC Nominations duration : 7 days, 0:00:00 Nominations remaining : 8:25:21 Nominations progress : 94.99% --------------------------------------------------- Projects[1] : 60 Projects with candidates : 0 ( 0.00%) Projects with election : 0 ( 0.00%) --------------------------------------------------- Need election : 0 () Need appointment : 60 (Adjutant Barbican Blazar Cinder Cloudkitty Congress Cyborg Designate Ec2_Api Freezer Glance Heat Horizon I18n Infrastructure Ironic Karbor Keystone Kolla Kuryr Loci Magnum Manila Masakari Mistral Monasca Murano Neutron Nova Octavia OpenStackAnsible OpenStackSDK OpenStack_Charms OpenStack_Helm Openstack_Chef Oslo Packaging_Rpm Placement Puppet_OpenStack Qinling Quality_Assurance Rally Release_Management Requirements Sahara Searchlight Senlin Solum Storlets Swift Tacker Telemetry Tricircle Tripleo Trove Vitrage Watcher Winstackers Zaqar Zun) =================================================== Stats gathered @ 2020-03-31 15:19:39 UTC This means that with approximately 2 days left, 60 projects will be deemed leaderless. In this case the TC will oversee PTL selection as described by [3]. Thank you, Kendall Nelson(diablo_rojo) & the Election Officials [1] https://governance.openstack.org/election/#how-to-submit-a-candidacy [2] Any open reviews at https://review.openstack.org/#/q/is:open+project:openstack/election have not been factored into these stats. [3] https://governance.openstack.org/resolutions/20141128-elections-process-for-leaderless-programs.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From frode.nordahl at canonical.com Tue Mar 31 15:47:14 2020 From: frode.nordahl at canonical.com (Frode Nordahl) Date: Tue, 31 Mar 2020 17:47:14 +0200 Subject: [charms][ptl][election] PTL Non-candidacy Message-ID: Hello all, The Ussuri cycle is coming to an end and the contributors to the Charms project has done great progress in the key areas we set afoot at the beginning of the cycle. We have also had good participation from the broader community and I hope to see that flourish further in the coming cycles. I believe rotating leadership for open source projects is a good thing and I am happy to pass on the relay baton for the Victoria cycle. -- Frode Nordahl From tobias.rydberg at citynetwork.eu Tue Mar 31 16:29:56 2020 From: tobias.rydberg at citynetwork.eu (Tobias Rydberg) Date: Tue, 31 Mar 2020 18:29:56 +0200 Subject: [tc][elections] Nomination for TC Message-ID: <140fb28c-e781-ab08-369a-661a9ea4254c@citynetwork.eu> Hi fellow Stackers, My name is Tobias Rydberg and I will by this email announce my candidacy for the OpenStack Technical Committee. For those who (yet) don't know me, I'm employed by the cloud provider City Network and through that I've been around and worked with OpenStack since 2014 when we as City Network started our OpenStack journey. I'm a simple person that speak my mind, love to have fun and get to know new people and technologies - and I have to say that being part of this community ticks a lot of those boxes. During the last couple of years I've been chair for the Public Cloud SIG (previously WG), and in that role you might have seen me hosting Forum sessions at all Summits since the Forum was introduced, spreading the word of why OpenStack is super important for public clouds (the Public Cloud Passport Program is a child of PC SIG) and what is still missing... ;-) Even though I'm a developer from the get go, I haven't done a lot of code contributions over the years, but I have tried to contribute in various other ways where being SIG chair is one, driving discussions in the Forums and being part of organising OpenStack days events (Nordic). So, what can I bring to the table? I believe that my background (and precent) with multiple years of running multiple public and private clouds will bring a lot of operator experience. We've over the years also been very focused to deliver OpenStack to customers with very high regulatory compliance demands, such as banks, health care, insurance industry, and as I see it, that is a pretty hot potato right now and I believe that will not cool down for some time. I my current role I have the blessing of talking a lot with customers building their services on top of OpenStack, listening to their experience and feedback - everything from how hard it can be to find the correct piece of documentation to limitations that hinders them from choosing OpenStack as a platform for their service - valuable feedback that not always reach the community. I think that the continued success of OpenStack (for at least 10 more strong years) lies in the hands of the "operators" - the people and the companies that live and breathe OpenStack - the people and the companies that have the experience how OpenStack operates in all the different use cases you can't predict in a public cloud. This is my first time running for a chair at the TC table, my experience of sitting in such chair is by that pretty limited, but I care a lot about this awesome community and I'm eager to learn and to do my fair share to make this community a success to the years to come... Thanks for taking the time to read and to consider my candidacy! Cheers, Tobias From radoslaw.piliszek at gmail.com Tue Mar 31 16:46:24 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 31 Mar 2020 18:46:24 +0200 Subject: [elections][tc] Nomination Message-ID: Dear Fellow OpenStackers, I am Radosław Piliszek (yoctozepto) and I am proposing myself as a TC candidate starting Victoria cycle. For the record, I have never been in TC nor a PTL... yet. I am affiliated with a relatively small Polish university: University of Białystok, with the local Computing Centre offering services to local academics. My experience is broad in that this work involves being a whole IT department at once. :-) OpenStack piqued my interest in 2018 when I joined the Foundation as an individual member. Since then I made myself COA (Certified OpenStack Administrator), learnt the jump from "administrator" to "operator" while deploying locally various PoC instances of basic OpenStack services and finally started contributing upstream via bug reports, patches and reviews. I was (and am!) really proud to have been recognized Kolla core due to my work with fellow project members. Later on, DevStack core nomination followed for which I am grateful as well. In the Kolla project I work mostly on ensuring sanity, enhancing testing and fixing current issues while helping fellow contributors to deliver their features and fixes. I also managed to deliver a bigger change in the Train cycle: all-around IPv6-only networking support. I enjoyed the cross-project aspect of this and decided to contribute a bit to the development experience as well and this is how I ended up DevStack core (true story). Nowadays I am wearing multiple hats in the wide community and many of you probably recognize me from IRC where I try to be helpful but also ask questions around (often technical ones like: to be or not to be?). One could say I'm inquisitive (hopefully in the good sense of this word) and in fact I value highly a deeper insight into the problem at hand (probably due to bad experience with hasty actions!). I react to issues quite quickly (bothering infra from time to time - I am really sorry, you folks are doing great job!) and work on solutions with fellow OpenStackers (aka you) which brings real joy and makes experience real. I have a few, rather broad, goals (or thoughts?) of mine: 1) To test, test and test. My motto is: "not tested = not working". Unfortunately, this is actually often the case in reality. :-( Hence, I encourage thorough testing, especially at the functional and integration level. I believe we need to be reliable to be trustworthy. 2) To make OpenStack easily consumable by small companies, especially in public academia but not only. Everyone deserves a good private cloud experience with decent learning curve. This involves prompt and current documentation but also reliable base for installs. (1) and (2) culminate in (3) [1+2=3 - could not have been simpler!] 3) Who are we? BUILDERS! What do we want? TO BUILD! (I hope you know the meme!) Ultimately OpenStack is software built for specific purpose and is meant to be consumed by cloud operators. We have plenty of ways to get OpenStack "installed" but officially we still rely on the archaic assumption that everything is thrown in the same environment. This is rarely the case nowadays due to various layers of separation being applied: virtual envs, OCI images. It would be beneficial to be able to rely on prebuilt official images of OpenStack services and their prerequisites, also for CI. We don't need to rebuild/reinstall keystone, glance, nova and swift just to test neutron that one more time, only to discover pip hangs installing dep of dep of not-even- neutron due to random networking issue.* Similarly this is wasted effort in case of multinode testing. Multiply this by the amount of scenarios and... well, you probably get the picture. I know we can do better: deliver more reliability via more reliability. * Order random for example purposes. This is as many relevant bits as I thought necessary to introduce my candidacy and share my mind with you. Thank you for reaching this far. Radosław Piliszek (yoctozepto) https://review.opendev.org/716363 From tburke at nvidia.com Tue Mar 31 18:07:28 2020 From: tburke at nvidia.com (Tim Burke) Date: Tue, 31 Mar 2020 11:07:28 -0700 Subject: [election][ptl][swift] Tim Burke candidacy for Swift PTL Message-ID: <29fb2d83-f5d7-65de-c79f-331e9ffab7b2@nvidia.com> It would be my honor and pleasure to continue serving you as Swift PTL. The world continues to generate and retain ever-increasing amounts of data, and Swift continues to rise to the challenge of storing that data durably and making it highly available. I'm excited to help us engineer now to be ready for next year's data-center demands. To that end, there are several projects we'll be advancing; in general, these are not short-term, though some pieces will surely come in the next cycle. As hard-drive sizes increase, the lots-of-small-files problem has become a lots-of-files problem. Between 16TB hard drives hitting the market and cost concerns pushing us toward ever more drives per box, even historically "reasonable" object sizes will produce noticeable memory pressure that we'd like to reduce. OVH's explorations in this space give us all a leg up that we should take advantage of. As clusters grow, we must ensure Swift can scale with them. Some aspects of that will be relatively straight-forward, like allowing rings to support more than 64k devices. Other challenges are less well-defined; there are likely improvements that could be made in replication and backend protocols, for example, but there is no single way forward. There will likely be some experiments that never land on master -- but as long as they are in the open and we can all learn from them, they will not be failures. All of that must be driven by what we learn by operating real clusters at scale. To do that, we must improve our metrics and monitoring, and find ways to observe the system dynamically. Post-facto log analysis is not tenable when dealing with tens of thousands of requests per second across hundreds of nodes. At the same time, we cannot neglect our client ecosystem. This extends not just to python-swiftclient, but to S3 clients as well. For better or worse, S3 has become the de facto standard interface for object storage, and we must ensure our compatibility is as seamless as possible. This cycle we added S3-compatible versioning, but there is so much more we could be doing, from object life-cycle management to bucket inventories. As always, we will listen to our users and prioritize addressing their pain points. Tim Burke ----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ----------------------------------------------------------------------------------- From ashlee at openstack.org Tue Mar 31 21:34:03 2020 From: ashlee at openstack.org (Ashlee Ferguson) Date: Tue, 31 Mar 2020 16:34:03 -0500 Subject: Virtual OpenDev Update Message-ID: <7DE2209B-2B37-4F95-BFEA-D9F23E2AEEE7@openstack.org> Hi everyone, As we continue to gather feedback and discuss what a virtual OpenDev will look like, we wanted to provide you with an update around the event. While we are still attempting to solve some of the riddles of virtual event world, we did come up with some decisions that give us something to build on, which are based on input and experience from members of the Programming Committee. Decisions: - OpenDev will happen after the Virtual PTG (exact format and timing TBD) - Will not be simultaneous/parallel Tracks so everyone has the opportunity to participate in more than one - 3 Tracks, each on different date: - Hardware Automation - Large Scale Ops - Containers in Production We also discussed the original intent and goals of OpenDev to make sure we’re still heading in a direction that’s beneficial for the community, especially since it will no longer directly precede the upcoming PTG. Similar to past OpenDev events, the goal is to identify the questions we don’t have the answers for, and to use that time to dive in and determine the work ahead of us, working together, sharing use cases, and learning from other people asking those same questions. High-level decisions, goals, and some areas we’re looking for feedback are in the etherpad below. As you continue to experience this new amazing world of virtual events, please add any feedback or ideas to the etherpad below. We’re looking forward to creating virtual OpenDev with you all! https://etherpad.openstack.org/p/Virtual_OpenDev_Planning Ashlee Ashlee Ferguson Community & Events Coordinator OpenStack Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Tue Mar 31 23:46:39 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 31 Mar 2020 16:46:39 -0700 Subject: [all][TC][PTL][election] Nominations Close & Campaigning Begins Message-ID: The PTL and TC Nomination periods are now over. The official candidate lists are available on the election website[0][1]. --PTL Election Details-- There are 16 projects without candidates, so according to this resolution[2], the TC will have to decide how the following projects will proceed: Adjutant Barbican Cloudkitty Congress I18n Infrastructure Loci Masakari Oslo Packaging_Rpm Placement Rally Swift Tacker Tricircle Zaqar There are no projects with more than one candidate so we won't need to hold any runoffs! Congratulations to our new and returning PTLs! [0] --TC Election Details-- The official candidate list is available on the election website[1]. There are will be a TC election following the campaigning period that has now begun and runs till Apr 07, 2020 23:45 UTC when the polling will begin. You are encouraged to ask questions to the candidates on the ML to help inform your decisions when polling begins. Thank you, -Kendall (diablo_rojo) & the Election Officials [0] https://governance.openstack.org/election/#victoria-ptl-candidates [1] https://governance.openstack.org/election/#victoria-tc-candidates [2] https://governance.openstack.org/resolutions/20141128-elections-process-for-leaderless-programs.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Tue Mar 31 16:13:05 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Tue, 31 Mar 2020 11:13:05 -0500 Subject: Upcoming pip changes - stop installing test-requirements in devstack In-Reply-To: <1712bc0d6cc.b8c1d108148375.1053329105595332432@ghanshyammann.com> References: <2DD692B0-BDB2-4AB4-AB7C-BEB13CFA93E9@inaugust.com> <1712bc0d6cc.b8c1d108148375.1053329105595332432@ghanshyammann.com> Message-ID: <171315e25c6.c115e75413870.1790091289213523460@ghanshyammann.com> ---- On Mon, 30 Mar 2020 09:03:10 -0500 Ghanshyam Mann wrote ---- > ---- On Sat, 28 Mar 2020 13:57:38 -0500 Sean Mooney wrote ---- > > On Fri, 2020-03-27 at 14:24 -0500, Monty Taylor wrote: > > > Heya, > > > > > > Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to > > > verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us. > > > > > > The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those > > > at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run > > > devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state > > > is not one that is completely consistent. > > > > > > I’ve pushed up this: > > > > > > https://review.opendev.org/#/c/715469/ > > > > > > Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in > > > devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest > > > which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on > > > devstack fail now. > > > > i brought up the topic of stoping installing test-requirement in the past as it has caused issue before so im happy > > to see this change going forward :) > > > > > > It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s > > > worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re > > > not going to get broken this summer. > > Currently, neutron gate is red for glance_store missing requirement of swift/cinderclients which used to get installed by > test-requirements installation. > Fixes we are trying: > - https://review.opendev.org/#/c/715835/2 > - https://review.opendev.org/#/c/715722/1 > > Swift functional jobs are also failing on py2 things due to stestr drop py2 support so capping of stestr for py2 is up > - https://review.opendev.org/#/c/715942/ You might have noticed, both fixes are merged now, we are good to recheck. -gmann > > let us know on this thread or on #openstack-qa if any other project failing. > > We are trying to merge those asap and if that fails further or take time we will go with temporary revert of test-requirement drop patch > > -gmann > > > > > > > Monty > > > > > > > > > > > From samuel at silverliningsys.com Tue Mar 31 11:52:49 2020 From: samuel at silverliningsys.com (Samuel Abdullah) Date: Tue, 31 Mar 2020 19:52:49 +0800 Subject: [OpenStack-Infra] [Murano-Open-PaaS] Openstack using guacamole In-Reply-To: References: Message-ID: Hi Donny, Thanks for the reply. To brief you about the environment in our infrastructure, we already have openstack running in our organization i am actually creating a new vm that sits in our openstack, and this vm will be the guacamole server. Aside from that, i would like to make guacamole as a gateway or a broker between the user and openstack so that whenever user access to guacamole, they will only be able to manage and create instances from guacamole that links to the openstack. Understand from your email that i will have to connect a proper API to openstack from guacamole? In that case, a proper extension that is only available from guacamole end , or i will need to compose a new extension probably something similar from the link you shared to me? Apologize as im not quite familiar with open source environment, and i came from a windows environment however i have a team that could do the job in opensource (openstack. Linux,) Despite that, i do understand in terms of technicality and architecture on the idea, its just the path to achieve this. Appreciate much o your response. Best Regards Samuel On Sun, Mar 29, 2020 at 8:56 PM Donny Davis wrote: > > On Fri, Mar 27, 2020 at 11:19 AM Samuel Abdullah < > samuel at silverliningsys.com> wrote: > >> Hi, >> >> Does Anyone know how can i install guacamole in openstack environment? >> Even if its via murano? Any manual guideline? >> >> Best Regards >> Samuel >> >> On Fri, 27 Mar 2020, 00:05 Roman Gorshunov, wrote: >> >>> Hello Samuel, >>> >>> Thanks for your email. Yes, Guacamole can be installed as an app via >>> Murano. >>> Discussions about Open PaaS are now in openstack-discuss mailing list. >>> Please use the tag [Murano-Open-PaaS] in the subject line. I'm >>> re-routing your email. >>> >>> Here are docs for the Murano project [0]. >>> >>> [0] https://docs.openstack.org/murano/latest/ >>> >>> Best regards, >>> Roman Gorshunov >>> >>> On Thu, Mar 26, 2020 at 4:37 PM Samuel Abdullah < >>> samuel at silverliningsys.com> wrote: >>> >>>> Hi, >>>> >>>> Would like to ask if it is possible to run guacamole in openstack >>>> environment? >>>> Seek for help on how would this be possible as i saw that guacamole >>>> component are part of the openstack in your murano project >>>> >>>> Looking forward for your reply >>>> >>>> Best Regards >>>> >>>> -- >>>> >>>> >>>> >>>> www.silverliningsys.com >>>> >>>> >>>> *Abu Bakar Samuel Abdullah* >>>> >>>> Cloud Infrastructure & Operations >>>> >>>> >>>> P: +603-2712-0081 >>>> >>>> M: +60.12.654.5938 >>>> >>>> E: samuel at silverliningsys.com >>>> _______________________________________________ >>>> OpenStack-Infra mailing list >>>> OpenStack-Infra at lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >>> >>> _______________________________________________ >> OpenStack-Infra mailing list >> OpenStack-Infra at lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > > Samuel, > Are you trying to replace the default vnc console with guacamole or are > you trying to run it as a service on top of Openstack? > > Scenario A - built in console > I am unaware of any method in the code today to replace the default > console with guacamole. > > Scenario B - Use it as a service > > I can't imagine it would be too hard if you leverage some kind of > automation framework to create your instances. In the example of using > ansible to create machines on the cloud - you could first call the playbook > that creates your instance and then query the API for the details of that > instance. You could then insert the needed parameters to the guacamole > server to access this machine. > The following modules could be used in this case - mind you this is one of > many examples to accomplish this task > Create Instance > https://docs.ansible.com/ansible/latest/modules/os_server_module.html > > Query the API > https://docs.ansible.com/ansible/latest/modules/os_server_info_module.html > > And then to update Guacamole you can use a template that includes a method > to properly create your backend connections of all of the required instances > > I am not an expert in Guacamole, so you will have to tinker a bit here. > > Also dropping infra - this ML is mainly for the actual Openstack project > infra - so like CI resources and code hosting resources > > Thanks and good luck! > > -- > ~/DonnyD > C: 805 814 6800 > "No mission too difficult. No sacrifice too great. Duty First" > -- www.silverliningsys.com *Abu Bakar Samuel Abdullah* Cloud Infrastructure & Operations P: +603-2712-0081 M: +60.12.654.5938 E: samuel at silverliningsys.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From whayutin at redhat.com Tue Mar 31 14:34:37 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Tue, 31 Mar 2020 08:34:37 -0600 Subject: [tripleo] meeting notes from the last two weeks Message-ID: Greetings, I need to see why the openstack meeting bot is not in the #tripleo channel. A record was not automatically kept.. so here is the raw txt from the last two meetings.. I'll get the bot fixed asap. Meeting notes March 24 2020 [08:00:44] #topic agenda [08:00:44] * Review past action items [08:00:44] * One off agenda items [08:00:44] * Squad status [08:00:44] * Bugs & Blueprints [08:00:44] * Projects releases or stable backports [08:00:44] * Specs [08:00:44] * open discussion [08:00:44] Anyone can use the #link, #action and #info commands, not just the moderatorǃ [08:00:44] Hey folks! who's around? [08:00:57] o/ [08:00:59] o/ [08:01:03] o/ [08:01:11] o/ [08:01:18] o/ [08:01:24] o/ [08:01:41] o/ [08:01:45] o/ [08:02:24] ysandeep, o/ [08:02:27] #topic review past action items [08:02:27] http://eavesdrop.openstack.org/meetings/tripleo/2020/ [08:02:42] anyone need to raise anything from a previous meeting? [08:03:07] None. [08:03:07] #topic one off agenda items [08:03:07] #link https://etherpad.openstack.org/p/tripleo-meeting-items [08:03:17] «o/ [08:03:18] I added two items to the agenda.. [08:03:27] please add something if you have it.. [08:03:53] * weshay|ruck notes that rdo-cloud the 3rd party ci platform is in a scheduled maint window today [08:04:03] Emilien Macchi proposed openstack/tripleo-heat-templates master: Refactorize test_tht_ansible_syntax to welcome new roles/modules https://review.opendev.org/713669 [08:04:05] the ctrl plane is moving to vexxhost [08:04:10] Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 [08:04:16] should be back by tomorrow [08:04:18] Emilien Macchi proposed openstack/tripleo-heat-templates master: Deprecate Keepalived service https://review.opendev.org/657067 [08:04:28] o/ [08:04:33] hi [08:04:36] any questions? [08:04:43] EmilienM, stop working :) [08:04:45] o/ [08:04:49] o/ [08:04:50] o/ [08:04:53] o/ [08:05:01] leanderthal, you may want to get the testday on the agenda :) [08:05:12] ooOOOooh good call [08:05:19] ok.. the next topic is around bugs [08:05:34] o/ [08:05:49] we were up to 60 untriaged bugs last week.. a lot of folks do NOT have access to self triage and that's fine [08:06:09] but if are a member of the tripleo org in launchpad.. we'd hope that you do follow the triage process [08:06:15] don't we run a meeting bot? [08:06:21] So Question for the group... [08:06:23] sshnaidm, yes [08:06:38] but maybe it's down.. ? [08:06:40] don't know [08:07:06] Question... Any objects to closing any bugs in marked "New" ( untriaged ) older than 365 days? [08:07:11] objection rather [08:07:17] weshay|ruck: sounds good to me [08:07:25] k [08:07:31] weshay|ruck: but 'latest updated ' date right? [08:07:45] ya.. I'll account for that... good point [08:07:47] weshay|ruck: not filed? maybe doesn't matter 365 is long enough that it either /or [08:07:50] fultonj, fmount https://bugs.launchpad.net/tripleo/+bug/1868731 thanks for looking into that [08:07:51] Launchpad bug 1868731 in tripleo "ceph_ansible_skip_tags is getting ignored during multinode overcloud deploy" [Critical,Triaged] [08:07:56] (wrong window) but yeah, that sounds reasonable [08:08:02] * florianf (~flfuchs at 2a02:8109:9280:1aa8:8bb0:31ca:8557:482c) has joined [08:08:09] chandankumar: thanks [08:08:15] I'll send an email to the list to have folks reopen if they need it [08:08:30] ok.. seeing no objections... we'll go ahead and clean it up :) [08:08:47] #topic one off agenda items [08:08:47] #link https://etherpad.openstack.org/p/tripleo-meeting-items [08:08:47] #topic Active Squad status [08:08:47] ci [08:08:47] #link https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw?both [08:08:47] validations [08:08:47] #link https://etherpad.openstack.org/p/tripleo-validations-squad-status [08:08:47] ceph-integration [08:08:47] #link https://etherpad.openstack.org/p/tripleo-integration-squad-status [08:08:47] transformation [08:08:47] #link https://etherpad.openstack.org/p/tripleo-ansible-agenda [08:08:47] mistral-to-ansible [08:08:47] #link https://etherpad.openstack.org/p/tripleo-mistral-to-ansible [08:09:00] any reviews needed by the squads? [08:09:05] * TrevorV (~TrevorV at 130-45-79-246.dyn.grandenetworks.net) has joined [08:09:17] validations: we have our package under TripleO governance! [08:09:25] it's been merged today. [08:09:50] rdo governance :) [08:09:51] weshay|ruck the following reviews would be great to get some eyes on - https://review.opendev.org/#/q/starredby:cloudnull+status:open,n,z [08:10:12] thanks cloudnull [08:10:41] fultonj, I mentioned this earlier.. but ansible 2.9 is being tested along side the latest ceph-ansible.. I don't have a ton of detail on that [08:11:02] * derekh (~derekh at 93.107.228.126) has joined [08:11:15] ok.. let's move on [08:11:19] weshay|ruck: ack [08:11:26] #topic Moderately Active Squads [08:11:26] Ironic-integration [08:11:26] https://etherpad.openstack.org/p/tripleo-ironic-integration-squad-status [08:11:26] upgrade [08:11:26] #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status [08:11:26] edge [08:11:26] #link https://etherpad.openstack.org/p/tripleo-edge-squad-status [08:11:26] networking [08:11:26] #link https://etherpad.openstack.org/p/tripleo-networking-squad-status [08:11:47] probably nothing new here [08:11:59] #link https://launchpad.net/tripleo/+milestone/ussuri-3 [08:12:31] 5 New, 28 Incomplete, 4 Invalid, 4 Won't Fix, 4 Confirmed, 385 Triaged, 120 In Progress, 2 Fix Committed, 151 Fix Released [08:12:50] not bad on the bug counts.. I guess [08:13:00] Storyboard bugs. [08:13:00] #link https://storyboard.openstack.org/#!/project_group/76 [08:13:48] weshay|ruck we've recently updated the mistral_to_ansible board - https://storyboard.openstack.org/#!/board/208 [08:14:17] very nice.. [08:14:31] k.. will include that in the future [08:14:53] There are two blueprints not started for ussuri [08:14:54] Medium tripleo-firewall-rule-reporting New Not started ussuri [08:14:54] Medium remove-puppet Drafting Not started Emilien Macchi ussuri [08:15:02] * amoralej|lunch is now known as amoralej [08:15:20] EmilienM, maybe we move that to the next release or future at this point... [08:15:30] April 20th is m3 I think [08:15:52] and dsneddon owns https://blueprints.launchpad.net/tripleo/+spec/tripleo-firewall-rule-reporting [08:16:10] #topic specs [08:16:10] #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open [08:16:39] reviews for luke please https://review.opendev.org/#/c/698828/ on [08:16:39] Add spec for AWX integration. [08:16:55] #topic open discussion [08:16:55] Anything else that folks want to bring up to the meeting? [08:17:07] weshay|ruck: i'll probably abandon it for now [08:17:12] k.. thanks [08:17:27] anyone else ? [08:17:30] going once [08:17:36] going twice [08:17:43] weshay|ruck: [08:17:46] #endmeeting tripleo [08:17:49] do we wantto consider virtual ptg [08:17:53] weshay|ruck: ? [08:17:54] I'll save the meeting notes [08:17:57] weshay|ruck: next time then [08:17:58] ... [08:17:58] marios, that's still not clear [08:18:04] but most likely [08:18:06] +1 for virt ptg Meeting notes from March 30 2020 #topic agenda [08:01:02] * Review past action items [08:01:02] * One off agenda items [08:01:02] * Squad status [08:01:02] * Bugs & Blueprints [08:01:02] * Projects releases or stable backports [08:01:02] * Specs [08:01:02] * open discussion [08:01:02] Anyone can use the #link, #action and #info commands, not just the moderatorǃ [08:01:02] Hey folks! who's around? [08:01:08] o/ [08:01:18] o/ [08:01:20] ybe [08:01:22] o/ [08:01:24] o/ [08:01:27] EmilienM, aye.. I'll get into that [08:01:28] *maybe [08:02:28] o/ [08:02:30] o/ [08:02:36] o/ [08:02:42] \o/ [08:04:24] o/ [08:05:05] o/ [08:05:05] #topic review past action items [08:05:05] http://eavesdrop.openstack.org/meetings/tripleo/2020/ [08:05:05] None. [08:05:14] #topic one off agenda items [08:05:14] #link https://etherpad.openstack.org/p/tripleo-meeting-items [08:05:27] ok.. virt ptg.. /me pastes from the agenda some info [08:05:38] * Virtual PTG for TripleO [08:05:38] * https://etherpad.openstack.org/p/tripleo-victoria-topics [08:05:38] * http://lists.openstack.org/pipermail/foundation/2020-March/002854.html [08:05:38] * https://www.openstack.org/events/opendev-ptg-2020/ [08:05:38] * https://etherpad.openstack.org/p/Virtual_PTG_Planning?_ga=2.148734014.820591304.1584991646-2020385751.1570459501 [08:05:38] [08:05:38] * planning email sent out by foundation - dates for ptg virt event not yet know. [08:05:38] * dates for planning [08:05:38] - April 2nd at 12:00 UTC [08:05:38] - April 6th at 17:00 UTC [08:05:38] - April 7th at 2:00 UTC [08:05:38] [08:05:53] o/ [08:06:08] I'm going to join the last two mtgs there on the 6th and 7th... [08:06:38] once we get dates.. we'll be sending out a email to confirm who would like to attend.. just like previous in person ptg's [08:07:05] any questions about the ptg in general before we get into tripleo specific ptg topics [08:07:17] hello [08:07:23] is it me your looking for? [08:07:23] weshay: hav we considered the details ... sessions will be in bluejeans? or irc? [08:07:26] weshay: timezones? [08:07:30] marios maciejjozefczyk Madkiss mandre marbindrakon matbu [08:07:38] weshay: ah i see you have time already there [08:07:43] marios, I assume that will be addressed in these planning sessions [08:08:02] note.. those dates are NOT the ptg.. they are PLANNING events FOR the ptg [08:08:05] :) [08:08:06] o/ [08:08:07] weshay: k (the times you have there are for the planning) [08:08:07] ack [08:08:12] aye [08:08:25] there is a sign up sheet for the planning.. if you are interested [08:08:34] OK.. thanks for responding [08:08:36] marios++ [08:08:37] weshay: perhaps this can just be worked out amongst the folks interested in each session... and we don't need to mandate a way/time [08:08:57] there is no mandate for ptg.. volunteers only :) [08:09:08] so.. TLDR.. details pending on ptg [08:09:15] now for tripleo [08:09:15] https://etherpad.openstack.org/p/tripleo-victoria-topics [08:09:37] as usual.. please add topics to this list, w/o topics we have no agenda [08:09:57] I'll be adding a few myself :) [08:09:59] weshay: we could overview/presentation about the component pipeline & how it affect how we promote now [08:10:05] * markllama (~mark5163 at 72.32.180.178) has joined [08:10:07] marios maciejjozefczyk Madkiss mandre marbindrakon markllama matbu [08:10:13] marios++ [08:10:44] marios, yes.. usually there is an hour for CI at the ptg.. the new dlrn structure and component based ci is a good topic imho [08:11:02] +42 [08:11:42] I would assume that new features.. validation, ansible collections, tripleo-operators should be added to the topic list [08:11:48] tripleo-ansible etc [08:12:09] any questions about tripleo + ptg? [08:12:27] I'll check with my folks about the validation part. [08:12:32] +1 [08:12:39] maybe the container would be nice. [08:13:11] EmilienM, you have anything to add? [08:13:36] not now, indeed we need to come with more topics [08:13:44] I'll try to gather some from the team [08:14:05] ya.. I am looking for parity from previous ptg's.. minus the last one [08:14:32] we did not have a lot of folks in china for the last ptg.. and I would expect a backlog of topics [08:14:45] so we'll be covering that here for a few weeks I'm sure [08:14:47] anyhoo [08:14:50] #topic Active Squad status [08:14:50] ci [08:14:50] #link https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw?both [08:14:50] validations [08:14:50] #link https://etherpad.openstack.org/p/tripleo-validations-squad-status [08:14:50] ceph-integration [08:14:50] #link https://etherpad.openstack.org/p/tripleo-integration-squad-status [08:14:50] transformation [08:14:50] #link https://etherpad.openstack.org/p/tripleo-ansible-agenda [08:14:50] mistral-to-ansible [08:14:50] #link https://etherpad.openstack.org/p/tripleo-mistral-to-ansible [08:14:56] where is the bot? [08:14:57] * haleyb (haleyb at nat/redhat/x-koblohkdhoozsgkh) has joined [08:15:09] dang [08:15:26] second week in a row.. I'll look into why the meeting bot is not active.. [08:15:32] * bogdando has quit (Ping timeout: 256 seconds) [08:15:48] ok.. the point now of calling squads... again... is to facilitate reviews [08:16:02] in a more public fashion [08:16:20] o/ [08:16:27] so just a reminder... guidance is to spend 30-60 minutes per day reviewing other peoples code [08:16:32] * bnemec (~bnemec at 96-41-255-59.dhcp.eucl.wi.charter.com) has joined [08:16:55] if you are not doing that.. you're doing opensource wrong :) [08:17:00] ++ [08:17:32] so.. w/o calling out folks specifically... anything that needs attention? [08:18:12] work on removing mistral is going great: https://review.opendev.org/#/q/starredby:cloudnull+status:open,n,z - these are some reviews that are inflight and we'd greatly appreciate if folks could take a look at them. [08:18:21] ++ [08:18:35] weshay: '3 a day' ;) [08:18:42] ++ [08:18:49] used to be a minimum requirement to be core [08:19:08] ya.. hasn't changed.. [08:19:21] * markllama has quit (Ping timeout: 256 seconds) [08:19:24] weshay: yeah but it used to be enforced more enthusiastically ;) [08:19:26] well.. I can show everyone what we look for in cores.. I'll add that to next weeks agenda [08:19:36] s/enthusiastically/stricly [08:19:41] strictly even you know what i mean [08:19:49] ya.. VERY open in ideas to help facilitate reviews.. [08:20:04] all projects struggle w/ that.. but new ideas are welcome [08:20:18] Rabi Mishra proposed openstack/python-tripleoclient master: Fix _get_image() of GlanceClientAdapter to call find_image() https://review.opendev.org/716277 [08:20:20] ok.. /me ends the lecture [08:20:31] #topic Moderately Active Squads [08:20:31] Ironic-integration [08:20:31] https://etherpad.openstack.org/p/tripleo-ironic-integration-squad-status [08:20:31] upgrade [08:20:31] #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status [08:20:31] edge [08:20:31] #link https://etherpad.openstack.org/p/tripleo-edge-squad-status [08:20:31] networking [08:20:31] #link https://etherpad.openstack.org/p/tripleo-networking-squad-status [08:20:31] Any squad wanting to bring up their status, or a topic for the general public? [08:20:47] #link https://launchpad.net/tripleo/+milestone/ussuri-3 [08:20:58] validations: we're aiming to get our code in train [08:21:08] at least for the new packages - validations-common and validations-libs. [08:21:16] 5 New, 28 Incomplete, 5 Invalid, 5 Won't Fix, 4 Confirmed, 378 Triaged, 120 In Progress, 2 Fix Committed, 189 Fix Released [08:21:26] Tengu++ [08:21:46] doing so will allow to get a clean code and, well, make our life easier on Red Hat side. [08:21:49] Tengu, I see ptg topics in your future :) [08:22:00] * bnemec has quit (Read error: No route to host) [08:22:03] weshay: yep, maybe ;). [08:22:19] (hint: just dropped a mail to my team ;)) [08:22:21] so.. bug triage will continue as I have time... just a reminder anything older than 365 days and in new.. will be closed [08:22:40] Storyboard bugs. [08:22:40] #link https://storyboard.openstack.org/#!/project_group/76 [08:23:02] reminder.. be nice to the infra folks and folks driving storyboard... volunteer effort there.. [08:23:16] Martin Kopec proposed openstack/ansible-role-collect-logs master: Extend infrared molecule tests https://review.opendev.org/705552 [08:24:27] * weshay notes... [08:24:40] now is the time to start thinking about blueprints and specs for the next cycle [08:24:49] here's a dump on ussuri [08:24:50] Medium ansible-certmonger Approved Unknown Cédric Jeanneret ussuri [08:24:50] Medium ironic-overcloud-ci Approved Slow progress Derek Higgins ussuri [08:24:50] Medium multiarch-support Approved Good progress Tony Breeds ussuri [08:24:50] Medium nova-less-deploy Approved Good progress Steve Baker ussuri [08:24:50] Medium replace-keepalived-undercloud Approved Good progress Emilien Macchi ussuri [08:24:50] Medium split-controlplane-workflow Approved Unknown ussuri [08:24:50] Medium remove-puppet Drafting Not started Emilien Macchi ussuri [08:24:50] Medium tripleo-firewall-rule-reporting New Not started ussuri [08:24:50] Undefined tripleo-mistral-to-ansible Approved Good progress Dougal Matthews ussuri [08:25:07] * Tengu hides [08:25:34] Martin Kopec proposed openstack/ansible-role-collect-logs master: Extend infrared molecule tests https://review.opendev.org/705552 [08:25:48] ekultails, looks like progress on https://review.opendev.org/698828 [08:25:56] Add spec for AWX integration. [08:25:58] FYI ^ [08:26:17] replace-keepalived-undercloud is WIP and waiting on HA byd efault, blocked by not having upgrade jobs on centos8 [08:26:29] remove-puppet is not in progress, I'm not working on it [08:26:30] for the ansible-certmonger, it's in the Security capable hands, not mine... I just pushed the idea ^^' [08:26:48] ade_lee: -^^ [08:26:55] aye.. EmilienM working in the background on that for you... getting advice on the priority for train + centos8 [08:27:43] need help reviewing https://review.opendev.org/#/c/715734/ Add new project and repository for tripleo-compute-extras [08:27:45] as well folks [08:28:08] k.. /moving on [08:28:12] #topic open discussion [08:28:12] Anything else that folks want to bring up to the meeting? [08:28:26] FYI.. I've nominated myself for ptl in victoria.. [08:28:32] yay! [08:28:34] weshay: you'll get my vote for ptl :) [08:28:45] +1 [08:28:59] * zzzeek has quit (Quit: bye) [08:29:00] I would like to help others this cycle better understand what is involved.. and the how-to's to the best of my ability :) [08:29:21] so if you are interested in perhaps being ptl.. after victoria.. let me know and I'm happy to walk through it [08:29:50] that's it from me.. anyone else have anything new? [08:30:14] once [08:30:15] twice [08:30:18] #endmeeting tripleo [08:30:23] thanks all :) -------------- next part -------------- An HTML attachment was scrubbed... URL: