February 2020 - openstack-discuss

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[tc][uc][all] Starting community-wide goals ideas for V series
by Ghanshyam Mann 15 Jun '20

15 Jun '20

Hello everyone, We are in R14 week of Ussuri cycle which means It's time to start the discussions about community-wide goals ideas for the V series. Community-wide goals are important in term of solving and improving a technical area across OpenStack as a whole. It has lot more benefits to be considered from users as well from a developers perspective. See [1] for more details about community-wide goals and process. We have the Zuulv3 migration goal already accepted and pre-selected for v cycle. If you are interested in proposing a goal, please write down the idea on this etherpad[2] - https://etherpad.openstack.org/p/YVR-v-series-goals Accordingly, we will start the separate ML discussion over each goal idea. Also, you can refer to the backlogs of community-wide goals from this[3] and ussuri cycle goals[4]. NOTE: TC has defined the goal process schedule[5] to streamlined the process and ready with goals for projects to plan/implement at the start of the cycle. I am hoping to start that schedule for W cycle goals. [1] https://governance.openstack.org/tc/goals/index.html [2] https://etherpad.openstack.org/p/YVR-v-series-goals [3] https://etherpad.openstack.org/p/community-goals [4] https://etherpad.openstack.org/p/PVG-u-series-goals [5] https://governance.openstack.org/tc/goals/#goal-selection-schedule -gmann

10 22

[swift] Adding disks - one by one or all lightly weighted?
by Mark Kirkwood 14 May '20

14 May '20

Hi, We are wanting to increase the number of disks in each of our storage nodes - from 4 to 12. I'm wondering whether it is better to: 1/ Add 1st new disk (with a reduced weight)...increase the weight until full, then repeat for next disk etc 2/ Add 'em all with a (much i.e 1/8 of that in 1/ ) reduced weight...increase the weights until done Thoughts? regards Mark

2 2

[Syntribos] Does anyone still use this?
by Gage Hugo 07 May '20

07 May '20

The Security SIG has recently been looking into updating several sites/documentation and one part we discussed the last couple meetings was the security tools listings[0]. One of the projects listed there, Syntribos, doesn't appear to have much activity[1] outside of zuul/python/docs maintenance and the idea of retiring the project was mentioned. However, there does seem to be an occasional bug-fix submitted, so we are sending this email out to see if anyone is still utilizing Syntribos. If so, please either respond here, reach out to us in the #openstack-security irc channel, or fill out the section in the Security SIG Agenda etherpad[2]. Thanks! [0] https://security.openstack.org/#security-tool-development [1] https://review.opendev.org/#/q/project:openstack/syntribos [2] https://etherpad.openstack.org/p/security-agenda

2 2

[openstack-ansible] dropping suse support
by Mohammed Naser 05 May '20

05 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[manila][ptg] Victoria Cycle PTG in Vancouver - Planning
by Goutham Pacha Ravi 30 Apr '20

30 Apr '20

Hello Zorillas and other interested Stackers, We've requested space to gather at the Open Infrastructure PTG event in Vancouver, BC between June 8th-11th 2020. This event gives us the opportunity to catch up in person (and via telepresence) and discuss the next set of goals for manila and its ecosystem of projects. As has been the norm, there's now a planning etherpad for us to collect topics and vote upon [1]. Depending on how much space/time we are budgeted, we will select and schedule these topics. Please take a look and add your topics to the etherpad. Thanks, Goutham [1] https://etherpad.openstack.org/p/vancouver-ptg-manila-planning

1 1

Re: [nova][cinder][ops] question/confirmation of legacy vol attachment migration
by Brett Milford 22 Apr '20

22 Apr '20

On Tue, Dec 10, 2019 at 11:15 PM Gorka Eguileor <geguileo(a)redhat.com> wrote: > > On 06/12, Brett Milford wrote: > > On Fri, Nov 22, 2019 at 3:54 AM Matt Riedemann <mriedemos(a)gmail.com> wrote: > > > > > > On 10/17/2019 5:24 AM, Gorka Eguileor wrote: > > > > I stand by my initial recommendation, being able to update the existing > > > > attachment to add the connection information from Nova. > > > > > > OK, thanks for the input and thoughtfulness on this. I've abandoned my > > > change since I'm not going to be pushing this boulder anymore but left > > > notes in the change in case someone else wants to pick it up some day. > > > > > > Note to nova cores: this means we'll have legacy volume attachment > > > compat code around forever. > > > > > > -- > > > > > > Thanks, > > > > > > Matt > > > > > > > Hi Groka, Cinder & Nova devs, > > > > Following up this thread from the context of > > https://review.opendev.org/#/c/579004/ > > > > To summarise the discussion: > > - initialize_connection shouldn't be called more than once, as it > > may mess up some drivers. > > - To (safely) refresh connection_info a new api for cinder is required. > > Hi, > > It may be a new API or a modification of one of the existing ones. > > > > - a patch to nova, such as #579004 could make a call to this new api > > to refresh connection info on events such as reboot. > > > > Is there any other context to this issue I've missed or an alternate > > path to solving this? > > > > Does the creation of a new api for cinder have any implications for > > being able to backport this patch for nova? > > > > I don't think backporting it in Nova would do us much good, since the > Cinder code would not be backportable (it's a new feature, not a bug > fix). > > > > What would be the process for me to kick off this work? > > > > You should propose a Cinder spec [1] for your work and you may provide a > WIP patch to Cinder at the same time, though depending on the review you > may have to completely change that code later. > > Cheers, > Gorka. > > [1]: https://opendev.org/openstack/cinder-specs > > > Thanks for your help, > > Brett (bcm) > > -- > > ❯ brett > > > Cheers Matt, Groka, I thought this might be the case. -- ❯ brett

2 2

[cinder][ops][extended-maintenance-sig][public-cloud-sig][enterprise-wg] Cinder to EOL some branches
by Brian Rosmaita 13 Apr '20

13 Apr '20

This is a courtesy notice that having received no responses to my email of 28 October [0] proposing to EOL some currently open Cinder branches, and following the policy articulated in [1], at today's Virtual PTG meeting the Cinder project team has decided to put the following stable branches into the End of Life state: driverfixes/mitaka driverfixes/newton stable/ocata stable/pike I will submit the paperwork to get this process moving one week from today (2 December 2019). cheers, brian [0] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010385.… [1] https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-l…

4 8

[all][PTL][release] Call for Ussuri Cycle Highlights
by Kendall Nelson 08 Apr '20

08 Apr '20

Hello Everyone! Its time to start thinking about calling out 'cycle-highlights' in your deliverables! As PTLs, you probably get many pings towards the end of every release cycle by various parties (marketing, management, journalists, etc) asking for highlights of what is new and what significant changes are coming in the new release. By putting them all in the same place it makes them easy to reference because they get compiled into a pretty website like this from the last few releases: Rocky[1], Stein[2], Train[3]. We don't need a fully fledged marketing message, just a few highlights (3-4 ideally), from each project team. Looking through your release notes might be a good place to start. *The deadline for cycle highlights is the end of the R-5 week [4] on April 10th.* How To Reminder: ------------------------- Simply add them to the deliverables/train/$PROJECT.yaml in the openstack/releases repo like this: cycle-highlights: - Introduced new service to use unused host to mine bitcoin. The formatting options for this tag are the same as what you are probably used to with Reno release notes. Also, you can check on the formatting of the output by either running locally: tox -e docs And then checking the resulting doc/build/html/train/highlights.html file or the output of the build-openstack-sphinx-docs job under html/train/ highlights.html. Can't wait to see you all have accomplished this release! Thanks :) -Kendall Nelson (diablo_rojo) [1] https://releases.openstack.org/rocky/highlights.html [2] https://releases.openstack.org/stein/highlights.html [3] https://releases.openstack.org/train/highlights.html [4] htt <https://releases.openstack.org/train/schedule.html#t-cycle-highlights> https://releases.openstack.org/ussuri/schedule.html

2 5

[neutron][OVN] Multiple mechanism drivers
by Sam Morrison 07 Apr '20

07 Apr '20

We are looking at using OVN and are having some issues with it in our ML2 environment. We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network) Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc. It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks. Am I missing something here? (We’re using Stein version) Thanks, Sam

5 6

[Tempest] OpenSack Powered * vs OpeStack Compatible
by prakash RAMCHANDRAN 16 Mar '20

16 Mar '20

Hi all, I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02. Similarly does any Hardware Driver of Software application seeking OpenStack compatibility Logo? Finally does anyone think that Open Infra Distro like Airship or StarlingX should promote Open Infra Airship or Open Infra StarlingX powered as a new way to promote eco system surrounding them similar to OpenStack compatible drivers and software. Will then Argo, customize, Metal3.io or Ironic be qualified as Open Infra Airship compatible? If so how tempest can help in testing the above comments? Refer to this market place below as how Distos and Products leverage OpenStack logos and branding programs. https://www.openstack.org/marketplace/distros/ Discussions and feedback are welcome. A healthy debate as how k8s modules used in Open Infra can be certified will be a good start. ThanksPrakash Sent from Yahoo Mail on Android

6 11

[tripleo][RDO] Version of Ansible in RDO CentOS8 repository
by Alfredo Moralejo Alonso 16 Mar '20

16 Mar '20

Hi all, During CentOS 8 dependencies preparation we've built ansible 2.9 in RDO dependencies repo which was released on Oct 2019, While testing TripleO with CentOS8 it has been discovered that the latest release of ceph-ansible does not support ansible 2.9 but only 2.8, so I'm opening discussion about the best way to move on in CentOS 8: - Make ceph-ansible 4.0 to work with ansible 2.8 *and* 2.9 so that the same releases can be used in CentOS7 with Stein and Train and CentOS8 Train and Ussuri. - Maintain separated ceph-ansible releases and builds for centos7/ansible 2.8 and centos8/ansible 2.9 able to deploy Nautilus. - Move ansible back to 2.8 in CentOS 8 Ussuri repository. I wonder if TripleO or other projects using ansible from RDO repositories has any requirement or need to move to ansible 2.9 in Ussuri cycle or can stay in 2.8 until next release, any thoughts? Best regards, Alfredo

1 2

[neutron] Why network performance is extremely bad and linearly related with number of VMs?
by Yi Yang (杨�D)-云服务集团 11 Mar '20

11 Mar '20

Hi, All Anybody has noticed network performance between VMs is extremely bad, it is basically linearly related with numbers of VMs in same compute node. In my case, if I launch one VM per compute node and run iperf3 tcp and udp, performance is good, it is about 4Gbps and 1.7Gbps, for 16 bytes small UDP packets, it can reach 180000 pps (packets per second), but if I launch two VMs per compute node (note: they are in the same subnet) and only run pps test case, that will be decrease to about 90000 pps, if I launch 3 VMs per compute node, that will be about 50000 pps, I tried to find out the root cause, other VMs in this subnet (they are in the same compute node as iperf3 client) can receive all the packets iperf3 client VM sent out although destination MAC isn’t broadcast MAC or multicast MAC, actually it is MAC of iperf3 server VM in another compute node, by further check, I did find qemu instances of these VMs have higher CPU utilization and corresponding vhost kernel threads also also higher CPU utilization, to be importantly, I did find ovs was broadcasting these packets because all the ovs bridges didn’t learn this destination MAC. I tried this in Queens and Rocky, the same issue is there. By the way, we’re using linux bridge for security group, so VM tap interface is attached into linux bridge which is connected to br-int by veth pair. Here is output of “ovs-appctl dpif/dump-flows br-int” after I launched many VMs: recirc_id(0),in_port(12),eth(src=fa:16:3e:49:26:51,dst=fa:16:3e:a7:0a:3a),et h_type(0x0800),ipv4(tos=0/0x3,frag=no), packets:11012944, bytes:726983412, used:0.000s, flags:SP., actions:push_vlan(vid=1,pcp=0),2,set(tunnel(tun_id=0x49,src=10.3.2.17,dst=10 .3.2.16,ttl=64,tp_dst=4789,flags(df|key))),pop_vlan,9,8,11,13,14,15,16,17,18 ,19 $ sudo ovs-appctl fdb/show br-floating | grep fa:16:3e:49:26:51 $ sudo ovs-appctl fdb/show br-tun | grep fa:16:3e:49:26:51 $ sudo ovs-appctl fdb/show br-bond1 | grep fa:16:3e:49:26:51 $ sudo ovs-appctl fdb/show br-int | grep fa:16:3e:49:26:51 All the bridges can’t learn this MAC. My question is why ovs bridges can’t learn MACs of other compute nodes, is this common issue of all the Openstack versions? Is there any known existing way to fix it? Look forward to hearing your insights and solutions, thank you in advance and have a good day.

4 5

[manila] share group replication spike/questions
by Dong.Ding＠dell.com 11 Mar '20

11 Mar '20

Hi, guys, As we talked about the topic in a virtual PTG few months ago. https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual (Support promoting several shares in group (DELL EMC: dingdong) I'm trying to write a manila-spec for it. It's my first experience to implement such feature in framework. I need to double check with you something, and hope you can give me some guides like: 1. Where is the extra-spec defined for group/group type, it's in Manila repo, right? (like manila.db.sqlalchemy.models....) 2. The command cli should be implemented for 'python-manilaclinet' repo, right? (I have never touched this repo before) 3. Where is the rest-api should be implemented? 4. And more tips you have? like any other related project should be changed? Just list what I know, and more details questions will be raised when implementing, I think. FYI Thanks, Ding Dong

2 7

Re: [CINDER] Distributed storage alternatives
by Burak Hoban 06 Mar '20

06 Mar '20

Hi guys, We use Dell EMC VxFlex OS, which in its current version allows for free use and commercial (in version 3.5 a licence is needed, but its perpetual). It's similar to Ceph but more geared towards scale and performance etc (it use to be called ScaleIO). Other than that, I know of a couple sites using SAN storage, but a lot of people just seem to use Ceph. Cheers, Burak ------------------------------ Message: 2 Date: Thu, 13 Feb 2020 18:20:29 +0100 From: Ignazio Cassano <ignaziocassano(a)gmail.com> To: Alfredo De Luca <alfredo.deluca(a)gmail.com> Cc: openstack-discuss <openstack-discuss(a)lists.openstack.org> Subject: Re: [CINDER] Distributed storage alternatives Message-ID: <CAB7j8cXLQWh5fx-E9AveUEa6OncDwCL6BOGc-Pm2TX4FKwnUKg(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hello Alfredo, I think best opensource solution is ceph. As far as commercial solutions are concerned we are working with network appliance (netapp) and emc unity. Regards Ignazio Il Gio 13 Feb 2020, 13:48 Alfredo De Luca <alfredo.deluca(a)gmail.com> ha scritto: > Hi all. > we 'd like to explore storage back end alternatives to CEPH for > Openstack > > I am aware of GlusterFS but what would you recommend for distributed > storage like Ceph and specifically for block device provisioning? > Of course must be: > > 1. *Reliable* > 2. *Fast* > 3. *Capable of good performance over WAN given a good network back > end* > > Both open source and commercial technologies and ideas are welcome. > > Cheers > > -- > *Alfredo* > > _____________________________________________________________________ The information transmitted in this message and its attachments (if any) is intended only for the person or entity to which it is addressed. The message may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information, by persons or entities other than the intended recipient is prohibited. If you have received this in error, please contact the sender and delete this e-mail and associated material from any computer. The intended recipient of this e-mail may only use, reproduce, disclose or distribute the information contained in this e-mail and any attached files, with the permission of the sender. This message has been scanned for viruses. _____________________________________________________________________

3 3

[neutron] security group list regression
by James Denton 04 Mar '20

04 Mar '20

Hello all, We recently upgraded an environment from Newton -> Rocky, and have noticed a pretty severe regression in the time it takes the API to return the list of security groups. This environment has roughly 8,000+ security groups, and it takes nearly 75 seconds for the ‘openstack security group list’ command to complete. I don’t have actual data from the same environment running Newton, but was able to replicate this behavior with the following lab environments running a mix of virtual and baremetal machines: Newton (VM) Rocky (BM) Stein (VM) Train (BM) Number of sec grps vs time in seconds: # Newton Rocky Stein Train 200 4.1 3.7 5.4 5.2 500 5.3 7 11 9.4 1000 7.2 12.4 19.2 16 2000 9.2 24.2 35.3 30.7 3000 12.1 36.5 52 44 4000 16.1 47.2 73 58.9 5000 18.4 55 90 69 As you can see (hopefully), the response time increased significantly between Newton and Rocky, and has grown slightly ever since. We don't know, yet, if this behavior can be seen with other 'list' commands or is limited to secgroups. We're currently verifying on some intermediate releases to see where things went wonky. There are some similar recent reports out in the wild with little feedback: https://bugzilla.redhat.com/show_bug.cgi?id=1788749 https://bugzilla.redhat.com/show_bug.cgi?id=1721273 I opened a bug here, too: https://bugs.launchpad.net/neutron/+bug/1865223 Bottom line: Has anyone else experienced similar regressions in recent releases? If so, were you able to address them with any sort of tuning? Thanks in advance, James

4 7

[release][tc][horizon] xstatic repositories marked as deprecated
by Akihiro Motoki 04 Mar '20

04 Mar '20

Hi, I noticed some xstatic repositories [1] under the horizon team are marked as deprecated. They are marked as deprecated in [2] as they were never released, but the actual situation looks different. They are used in horizon requirements and the contents of these repositories are same as those in the corresponding PyPI modules. I guess they were released to PyPI based on the repositories but the horizon team just forgot to push the corresponding tags. I would like to drop 'deprecated' mark from the repositories and move them to the proper situation. Does it make sense? If so, what is the right solution to do this? In my understanding, we need to add tags corresponding to the deliverables and to add deliverable files in the releases repo. What I am not sure is whether we can push a tag to a repository without pushing deliverables to PyPI? Or do we need to create another release to tag them in their repositories? [1] Such xstatic repositories are: - xstatic-bootstrap-datepicker - xstatic-hogan - xstatic-jquery-migrate - xstatic-jquery.quicksearch - xstatic-jquery.tablesorter - xstatic-rickshaw - xstatic-spin [2] https://review.opendev.org/#/c/629889/ Thanks, Akihiro Motoki (irc: amotoki)

5 8

[glance] Different checksum between CLI and curl
by gaetan.trellu＠incloudus.com 03 Mar '20

03 Mar '20

Hi guys, Does anyone know why the md5 checksum is different between the "openstack image save" CLI and "curl" commands? During the image creation a checksum is computed to check the image integrity, using the "openstack" CLI match the checksum generated but when "curl" is used by following the API documentation[1] the checksum change at every "download". Any idea? Thanks, Gaëtan [1] https://docs.openstack.org/api-ref/image/v2/index.html?expanded=download-bi…

12 21

[TaskFlow] running multiple engines on a shared thread
by Sachin Laddha 02 Mar '20

02 Mar '20

Hi, We are using taskflow to execute workflows. Each workflow is executed by a separate thread (using engine.run() method). This is limiting our capability to execute maximum number of workflows that can run in parallel. It is limited by the number of threads there in the thread pool. Most of the time, the workflow tasks are run by agents which could take some time to complete. Each engine is alive and runs on a dedicated thread. Is there any way to reuse or run multiple engines on one thread. The individual tasks of these engines can run in parallel. I came across iter_run method of the engine class. But not sure if that can be used for this purpose. Any help is highly appreciated.

2 3

[all] Moving towards OpenDev
by Mohammed Naser 29 Feb '20

29 Feb '20

Hi everyone, The OpenStack infrastructure team has helped build an extremely powerful set of tools which we use on a daily basis in order to build OpenStack. You may have noticed that Open Infrastructure projects (such as Zuul, StarlingX, Airship, etc) have leveraged these tools as well as they found them to be extremely powerful and useful to build software by leveraging our four opens. As you might have noticed, there has been many references to OpenDev and it's time that we split off the infrastructure team to it's own separate community with governance and the OpenStack infrastructure team will continue to help work with the OpenDev team in order to keep our lights running (and most likely, those two teams will be the same at the start, but the idea is hoping that one or both grow separately). I'd like to invite our community to please look at the following two patches and leave your comments. Regardless how you feel about this, we'd really appreciate if you can leave a review (-1 or +1) just to show some sort of acknowledgment and help signal that the community is aware of this upcoming change. I'm certainly very excited about it. :) https://review.opendev.org/#/c/710020/ https://review.opendev.org/#/c/703488/ Thanks everyone! Regards, Mohammed

3 3

[tripleo] openstack-train mirror for CentOS aarch64
by Lenny Verkhovsky 28 Feb '20

28 Feb '20

Hi, I see that there is no openstack-train repo for aarch64 for CentOS[1] This repo is also used by kolla project. Who maintains it and if there are any plans for adding train repo? [1] http://mirror.centos.org/altarch/7/cloud/aarch64/ Best Regards Lenny Verkhovsky Mellanox Technologies<http://www.mellanox.com/> office:+972 74 712 92 44 irc:lennyb

2 1

[security] Security SIG Newsletter - Feb 2020
by Gage Hugo 28 Feb '20

28 Feb '20

#Month Feb 2020 - Security SIG Meeting Info: http://eavesdrop.openstack.org/#Security_SIG_meeting - Weekly on Thursday at 1500 UTC in #openstack-meeting - Agenda: https://etherpad.openstack.org/p/security-agenda - https://security.openstack.org/ - https://wiki.openstack.org/wiki/Security-SIG #Updates - The Security SIG has recently updated the reporting guidelines for private security bugs to now have a max timeframe of 90 days for embargos (excluding unusual circumstances). We have begun marking all existing private bugs to go public after 90 days from the date that it was updated. - - The Security SIG has reached out about obtaining a room/timeslot for the Vancouver PTG. We will be requesting a spot, likely just a shorter timeslot due to a small amount of given interest. However we should be able to have time to discuss any topics brought forward. The first OSSA of the year was released this month, for more details check out the link below. #VMT Reports - A full list of publicly marked security issues can be found here: https://bugs.launchpad.net/ossa/ - OSSA-2020-001 was released: https://security.openstack.org/ossa/OSSA-2020-001.html

1 0

[kuryr] macvlan driver looking for an owner
by Roman Dobosz 28 Feb '20

28 Feb '20

Hi, Recently we migrated most of the drivers and handler to OpenStackSDK, instead of neutron-client[1], and have a plan to drop neutron-client usage. One of the drivers - MACVLAN based interfaces for nested containers - wasn't migrated due to lack of confidence backed up with sufficient tempest tests. Therefore we are looking for a maintainer, who will take care about both - migration to the openstacksdk (which I can help with) and to provide appropriate environment and tests. In case there is no interest on continuing support for this driver we will deprecate it and remove from the source tree possibly in this release. [1] https://review.opendev.org/#/q/topic:bp/switch-to-openstacksdk -- Cheers, Roman Dobosz gryf@freenode

2 2

[release] Release countdown for week R-10, March 2-6
by Sean McGinnis 28 Feb '20

28 Feb '20

General Information ------------------- The following cycle-with-intermediary deliverables have not done any intermediary release yet during this cycle. The cycle-with-rc release model is more suited for deliverables that plan to be released only once per cycle. As a result, we have proposed[1] to change the release model for the following deliverables: freezer karbor networking-baremetal neutron-fwaas-dashboard, neutron-vpnaas-dashboard senlin-dashboard tacker-horizon [1] https://review.opendev.org/#/q/topic:ussuri-cwi PTLs and release liaisons for each of those deliverables can either +1 the release model change, or propose an intermediary release for that deliverable. In absence of answer by the end of R-10 week we'll consider that the switch to cycle-with-rc is preferable. We are now past the published date for the stable/rocky branch to enter the Extended Maintenance phase: https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… A set of patches have been proposed for all cycle-based deliverables in rocky to tag the last release as "rocky-em". After this point, no more official releases will be done. Please acknowledge any patches for your team if you have not already done so, or let us know if there are some last minute patches on their way to merging that we should hold for. https://review.opendev.org/#/q/status:open+project:openstack/releases+branc… Any patches not acknowledged by March 5 will be assumed to be OK and we will process any remaining patches. We also published a proposed release schedule for the upcoming Victoria cycle. Please check out the separate thread: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012735… Upcoming Deadlines & Dates -------------------------- Non-client library freeze: April 2 (R-6 week) Client library freeze: April 9 (R-5 week) Ussuri-3 milestone: April 9 (R-5 week) OpenDev+PTG Vancouver: June 8-11

1 0

[RDO][tripleo][kolla][puppet] Moving CentOS8 master dependencies repo in RDO
by Alfredo Moralejo Alonso 28 Feb '20

28 Feb '20

Hi, In the next hours we are moving RDO to a new Dependencies repo for CentOS8 master based on new CBS (CentOS Build System) builds. We have tested this new repo for all the projects that we know use it (kolla, packstack, puppet-openstck-integration and tripleo) and we have a plan to make the transition non-disrupting so we expect not to affect gates on any project by modifying [1] in several steps. Let us know if you notice abnormal behavior so that we can fix any issue that may arise or even revert the change if needed. Best regards, Alfredo [1] https://trunk.rdoproject.org/centos8-master/delorean-deps.repo

2 1

[keystone] [keystonemiddleware] [neutron] [keystone_authtoken] auth_url not available via oslo_config
by Justin Cattle 28 Feb '20

28 Feb '20

Hi, I'm reaching out for help with a strange issue I've found. Running openstack queens, on ubuntu xenial. We have a bunch of different sites with the same set-up, recently upgraded from mitaka to queens. However, on this one site, after the upgrade, we cannot start neutron-server. The reason is, that the ml2 plugin throws an error because it can't find auth_url from the keystone_authtoken section of neutron.conf. However, it is there in the file. The ml2 plugin is calico, it fails with this error: 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico [-] Exception in function %s: TypeError: expected string or buffer 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico Traceback (most recent call last): 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico File "/usr/lib/python2.7/dist-packages/networking_calico/logutils.py", line 21, in wrapped 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico return fn(*args, **kwargs) 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico File "/usr/lib/python2.7/dist-packages/networking_calico/plugins/ml2/drivers/calico/mech_calico.py", line 347, in _post_fork_init 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico auth_url=re.sub(r'/v3/?$', '', auth_url) + 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico File "/usr/lib/python2.7/re.py", line 155, in sub 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico return _compile(pattern, flags).sub(repl, string, count) 2020-02-20 20:14:22.495 2964911 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico TypeError: expected string or buffer When you look at the code, this is because neither auth_url or is found in cfg.CONF.keystone_authtoken. The config defintely exists. I have copied the neutron.conf config from a working site, same error. I have copied the entire /etc/neutron directory from a working site, same error. I have check with strace, and /etc/neutron/neutron.conf is the only neutron.conf being parsed. Here is the keystone_authtoken part of the config: [keystone_authtoken] auth_uri=https://api-srv-cloud.host.domain:5000 region_name=openstack memcached_servers=1.2.3.4:11211 auth_type=password auth_url=https://api-srv-cloud.host.domain:5000 username=neutron password=xxxxxxxxxxxxxxxxxxxxxxxxx user_domain_name=Default project_name=services project_domain_name=Default I'm struggling to understand how the auth_url config is really registered in via oslo_config. I found an excellent exchagne on the ML here: https://openstack.nimeyo.com/115150/openstack-keystone-devstack-confusion-a… This seems to indicate auth_url is only registered if a particular auth plugin requires it. But I can't find the plugin code that does it, so I'm not sure how/where to debug it properly. If anyone has any ideas, I would really appreciate some input or pointers. Thanks! Cheers, Just -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

5 7

[keystone] Keystone Team Update - Week of 24 February 2020
by Vishakha Agarwal 28 Feb '20

28 Feb '20

# Keystone Team Update - Week of 24 February 2020 ## News ### User Support and Bug Duty Every week the duty is being rotated between the members. The person-in-charge for bug duty for current and upcoming week can be seen on the etherpad [2] [2] https://etherpad.openstack.org/p/keystone-l1-duty ## Open Specs Ussuri specs: https://bit.ly/2XDdpkU Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 8 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 22 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ### Priority Reviews * Ussuri Roadmap Stories - Groups in keystone SAML assertion https://tree.taiga.io/project/keystone-ussuri-roadmap/us/33 https://review.opendev.org/#/c/588211/ Add openstack_groups to assertion - Add support for modifying resource options to CLI tool https://tree.taiga.io/project/keystone-ussuri-roadmap/us/53 https://review.opendev.org/#/c/697444/ Adding options to user cli ## Milestone Outlook https://releases.openstack.org/ussuri/schedule.html Feature proposal freeze happens in two weeks. Feature freeze follows four weeks after that. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 0

WSREP: referenced FK check fail after switching to Galera
by Albert Braden 27 Feb '20

27 Feb '20

I switched our dev and qa clusters to Galera, and they seem to work fine, but I see this error in the logs: Feb 27 11:24:08 us01odc-dev1-ctrl1 mysqld[361982]: 2020-02-27 11:24:08 484069 [ERROR] InnoDB: WSREP: referenced FK check fail: Lock wait index `PRIMARY` table `neutron`.`provisioningblocks` I googled around and people say that it is a harmless error, but it seems to be time-correlated with creation of VMs that aren't pingable. https://jira.mariadb.org/browse/MDEV-18562 Is this a harmless error? Do I need to look elsewhere for the reason why some VMs fail to return pings?

2 1

Train release notes: deprecated option show_multiple_locations missing?
by Eugen Block 27 Feb '20

27 Feb '20

Hi, I'm currently evaluating our new OpenStack environment (Train), testing the usual scenarios. In the past I've had issues with nova and rbd backend, when the compute nodes used to download rbd images just to re-import them back to the ceph cluster instead of simply cloning an image. The cause was the show_multiple_locations option that was marked deprecated since Pike but in fact it's still necessary. The release notes of Pike, Queens, Rocky and Stein all mention that the workaround is still required: > The show_multiple_locations configuration option remains deprecated > in this release, but it has not been removed. > (It had been > scheduled for removal in the Pike release.) Please keep a watch on > the Glance release notes and the > glance-specs repository to stay > informed about developments on this issue. The Train release notes don't mention this option anymore but my tests show that it's still required to avoid unnecessary rbd exports/imports. Maybe the release notes for Train should be updated to cover this issue. In case I missed any further information that already addresses this I apologize. Best regards, Eugen

2 4

[UX] Contributing to OpenStack's User Interface
by Trang Le 27 Feb '20

27 Feb '20

Dear OpenStack Discussion Team, I am Trang Le, a student at UC Berkeley Extension interested in contributing to OpenStack’s UX/UI. I am currently pursuing a Professional Diploma in UX/UI and would love to contribute to an open-source project and work with experienced engineers. Before, I have also worked at Fujitsu Vietnam in the open-source team, where I learned a lot about OpenStack through training. Let me know if my message could be of interest to you, and I would be happy to discuss further. All the best, Trang Trang Le UC Berkeley Extension - Professional Diploma - UX/UI Design Smith College - Bachelors of Arts - Mathematics and Statistics Phone: +1 (650) 300 9007 Github: https://github.com/trangreyle

5 7

[tripleo]
by Wesley Hayutin 27 Feb '20

27 Feb '20

Greetings, Pending a zuul restart I suggest not rechecking or workflowing +2 patches atm. Probably should lay off submitting patches as well [1]. You can watch the dashboard [2] or follow twitter [3] for the restart notice if you want to. Thanks [1] http://zuul.openstack.org/builds?result=RETRY_LIMIT [2] http://dashboard-ci.tripleo.org/d/cockpit/cockpit?orgId=1&fullscreen&panelI… [3] https://twitter.com/openstackinfra

2 2

[cinder] ussuri virtual mid-cycle 16 march at 12:00 UTC
by Brian Rosmaita 27 Feb '20

27 Feb '20

The poll results are in. As with session one, there was only one time when everyone can meet (apologies to the "if need be" people, thank you for being flexible). Session Two of the Cinder Ussuri virtual mid-cycle will be held: DATE: Monday, 16 March 2020 TIME: 1200-1400 UTC LOCATION: https://bluejeans.com/3228528973 The meeting will be recorded. Please add topics to the planning etherpad: https://etherpad.openstack.org/p/cinder-ussuri-mid-cycle-planning cheers, brian

1 0

[all][PTL][tc] U Community Goal: Project PTL & Contrib Docs Update #3
by Kendall Nelson 27 Feb '20

27 Feb '20

Hello! There is some debate about where the content of the actual docs should live. This grew out of the discussion about how to setup the includes so that the correct info shows up where we want it. 'Correct' in that last sentence being where the debate is. There are two main schools of thought: 1. All the content of the docs should live in the top level CONTRIBUTING.rst and the sphinx glue should live in doc/source/contributor/contributing.rst. A patch has already been merged for this approach[1]. There is also a patch to update the goal to match[2]. This approach keeps all the info in one place so that if things change in the future, its easier to keep things straight. All the content is also more easily discoverable when looking at a repo in GitHub (or similar) or checking out the code because it is at the top most level of the repo and not hidden in the docs sub directory. 2. The new patch[3] says that the content should live in /doc/source/contributor/contributing.rst and a skeleton with only the most important version should live in the top level CONTRIBUTING.rst. This approach argues that people don't want to read a wall of text when viewing the code on GitHub (or similar) or checking it out and looking at the top level CONTRIBUTING.rst and as such only the important details should be kept in that file. These important details being that we don't accept patches in github and where to report bugs (both of which are included in the larger format of the content). So what do people think? Which approach do they prefer? I am anxious to get this settled ASAP so that projects have time to complete the goal in time. Previous updates if you missed them[4][5]. Please feel free to add other ideas or make corrections to my summaries of the approaches if I missed things :) -Kendall (diablo_rojo) [1] Merged Template Patch (school of thought 1): https://review.opendev.org/#/c/708511/ [2] Goal Update Patch: https://review.opendev.org/#/c/707736/ [3] Current Template Patch (school of thought 2): https://review.opendev.org/#/c/708672/ [4] Update #1: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012364… [5] Update #2: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012570…

6 7

[oslo][cache] oslo.cache hardening
by Moises Guimaraes de Medeiros 27 Feb '20

27 Feb '20

Hi all, Whenever deploying a service inside a network, basic security concerns come to mind: Is the network trusted? Can we send data in plaintext? Is the service available only to those intended to use it? Can the service itself or others have access to the data? That is no exception for caching servers and a while ago, me and Lance Bragstad started a discussion about this topic. *Possible solutions* *Protecting data in transit using TLS* Requires a backend with TLS support. Since version 1.5.13, Memcached supports authentication and encryption via TLS. This feature requires: OpenSSL 1.1.0 or later; A Memcached client with TLS support; A Memcached server built using ./configure --enable-tls. Encrypting the traffic protects the data in transit from reading and tampering. The complexity impact is that each Memcached server will need a valid certificate. The performance impact is the TLS overhead itself. Performing client authentication protects the server from unauthorized read and write operations. The complexity impact is that each Memcached client will need a valid certificate. The performance impact is bigger due to the extra steps to authenticate both sides. This approach doesn't protect the data held in memory by Memcached in any other way. *Authentication using SASL* Requires a backend with SASL support. Since version 1.4.3, Memcached supports authentication via SASL. This feature requires: A Memcached client with SASL support; A Memcached server built using ./configure --enable-sasl. This approach protects the server from unauthorized read and write operations. The complexity and performance impact is according to SASL usage. This approach doesn't protect the data in transit or held in memory by Memcached in any other way. *Encrypting data before storing* Requires *NO* extra features in the backend. This approach consists of encrypting the data before sending it to the caching servers. The complexity impact is dealing with key sharing for the encryption/decryption process. The performance impact depends on the algorithms used for encryption. This approach protects the data both in transit and held in memory by caching servers, but the key sharing is more prone to setup errors than the TLS or the SASL approach. --- After considering the possible solutions, we decided to tackle the TLS path first. We did an initial analysis of oslo.cache backends that use Memcached together with Hervé Beraud here: https://etherpad.openstack.org/p/oslo-cache-tls-support-worksheet Since python-binary-memcached already has SASL support, we thought it to be a good first candidate to implement TLS support and last month I had it merged here: https://github.com/jaysonsantos/python-binary-memcached/pull/211 We are now looking for more people interested in the discussion and help to push changes forward. -- Moisés Guimarães Software Engineer Red Hat

1 0

[neutron] Shared tenant network allow duplicate IP's?
by Adam Peacock 27 Feb '20

27 Feb '20

Hey folks, So I caught wind from a friend/colleague that allowing duplicate IP's in each tenant was now only achievable by creating a separate tenant network+subnet and assigning them separately to each individual tenant. This doesn't seem right to me and it doesn't scale. *Looking for this:* - tenant-network-id = abc (shared) - tenant1 - vm1: 10.0.0.10 - tenant2 - vm1: 10.0.0.10 Am I missing something and this setup is no longer supported? I hope I'm wrong but I can't find documentation that speaks to this specifically so would appreciate a link if anyone has it handy. Thanks! //adam *Adam Peacock* Principal Architect Office: +1-916-794-5706

4 4

[ptl][release][stable][EM] Extended Maintenance - Rocky
by Elõd Illés 27 Feb '20

27 Feb '20

Hi, In less than one month Rocky is planned to enter into Extended Maintenance phase [1] (estimated date: 2020-02-24). I have generated the list of *open* and *unreleased* changes in *stable/rocky* for the follows-policy tagged repositories [2]. These lists could help the teams, who are planning to do a final release on Rocky before moving stable/rocky branches to Extended Maintenance. Feel free to edit them! * At the transition date the Release Team will tag the latest (Rocky) releases of repositories with *rocky-em* tag. * After the transition stable/rocky will be still open for bugfixes, but there won't be any official releases. NOTE: teams, please focus on wrapping up your libraries first if there is any concern about changes, in order to avoid any broken releases! Thanks, Előd [1] https://releases.openstack.org/ [2] https://etherpad.openstack.org/p/rocky-final-release-before-em

4 5

[ops] nova wsgi config
by Arnaud Morin 27 Feb '20

27 Feb '20

Hey all, I am trying to configure apache as a WSGI. Is there any other documentation than this: https://docs.openstack.org/nova/stein/user/wsgi.html Is there any recommendations? Thanks in advance! -- Arnaud Morin

4 7

Cinder / Nova - Select cinder backend based on nova availability zone?
by Tony Pearce 27 Feb '20

27 Feb '20

Apologies in advance if this seems trivial but I am looking for some direction on this and I may have found a bug while testing also. Some background - I have 2 physical hosts which I am testing with (host1, host2) and I have 2 separate cinder backends (backend1, backend2). Backend1 can only be utilised by host1. Same for backend2 - it can only be utilised by host2. So they are paired together like: host1:backend1 host2:backend2 So I wanted to select a Cinder storage back-end based on nova availability zone and to do this when creating an instance through horizon (not creating a volume directly). Also I wanted to avoid the use of metadata input on each instance create or by using metadata from images (such as cinder_img_volume_type) [2] . Because I can foresee a necessity to then have a number of images which reference each AZ or backend individually. - Is it possible to select a backend based on nova AZ? If so, could anyone share any resources to me that could help me understand how to achieve it? Because I failed at achieving the above, I then decided to use one way which had worked for me in the past, which was to use the image metadata "cinder_img_volume_type". However I find that this is not working. The “default” volume type is selected (if cinder.conf has it) or if no default, then `__DEFAULT__` is being selected. The link at [2] states that first, a volume type is used based on the volume type selected and if not chosen/set then the 2nd method is "cinder_img_volume_type" from the image metadata and then the 3rd and final is the default from cinder.conf. I have tested with fresh deployment using Kayobe as well as RDO’s packstack. Openstack version is *Train* Steps to reproduce: 1. Install packstack 2. Update cinder.conf with enabled_backends and the [backend] 3. Add the volume type to reference the backend (for reference, I call this volume type `number-1`) 4. Upload an image and add metadata `cinder_img_volume_type` and the name as mentioned in step 3: number-1 5. Try and create an instance using horizon. Source = image and create new volume 6. Result = volume type / backend as chosen in the image metadata is not used and instance goes into error status. After fresh-deploying the RDO Packstack, I enabled debug logs and tested again. In the cinder-api.log I see “"volume_type": null,” and then the next debug log immediately after logged as “Create volume request body:” has “volume_type': None”. I was searching for a list of the supported image metadata, in case it had changed but the pages seem empty one rocky/stein/train [3] or not yet updated. Selecting backend based on nova AZ:: I was searching how to achieve this and I came across this video on the subject of AZs [1]. Although it seems only in the context of creating volumes (not with creating instances with volume from an image, for example). I have tried creating a host aggregate in nova, with AZ name `host1az`. I've also created a backend in Cinder (cinder.conf) with `backend_availability_zone = host1az`. But this does not appear to achieve the desired result, either and the cinder api logs are showing “"availability_zone": null” during the volume create part of the launch instance from Horizon. I also tried setting RESKEY [3] in the volume type, but again similar situation seen; although I dont think this option is the correct context for what I am attempting. Could anyone please nudge me in the right direction on this? Any pointers appreciated at this point. Thanks in advance. References: [1] https://www.youtube.com/watch?v=a5332_Ew9JA [2] https://docs.openstack.org/cinder/latest/cli/cli-manage-volumes.html [3] https://docs.openstack.org/cinder/train/contributor/api/cinder.api.schemas.… [4] https://docs.openstack.org/cinder/rocky/admin/blockstorage-availability-zon… Regards, Tony Pearce

2 4

Networking-vpp 20.01 for VPP 20.01 is now available
by Naveen Joy (najoy) 26 Feb '20

26 Feb '20

Hello All, We'd like to invite you to try out Networking-vpp 20.01. As many of you may already know, VPP is a fast user space forwarder based on the DPDK toolkit. VPP uses vector packet processing algorithms to minimize the CPU time spent on each packet to maximize throughput. Networking-vpp is a ML2 mechanism driver that controls VPP on your control and compute hosts to provide fast L2 forwarding under Neutron. This latest version of Networking-vpp is updated to work with VPP 20.01. In this release, we've made the below changes: - We've dropped support for Python 2.7 and updated the code to work with Python 3.6 or later. - We've updated the code to be compatible with VPP 20.01 API changes. - We've added VPP API versioning support using 2 data files. They are API whitelist file and API CRC manifest file. At startup, the vpp-agent will check to see if the API signature is compatible with the installed VPP. Also, at runtime, only the whitelisted API calls will be allowed. - We've fixed an issue with the eventlet that caused problems when binding tap interfaces into Linux bridge - We've been doing the usual round of bug fixes, clean-ups and updates. The code will work with VPP 20.01 and the OpenStack Stein release. The README [1] explains how you can try out VPP with Networking-vpp using devstack: the devstack plugin will deploy the mechanism driver and VPP and should give you a working system with a minimum of hassle. We will be continuing our development for VPP's 20.05 release. We welcome anyone who would like to come help us. -- Jerome, Ian & Naveen [1] https://opendev.org/x/networking-vpp/src/branch/master/README.rst

1 0

[nova][neutron] nova scheduling support for routed networks
by Balázs Gibizer 26 Feb '20

26 Feb '20

Hi, There are multiple efforts to add scheduler support to the neutron routed network feature. Matt's effort to have it as a scheduler pre-filter [1] Adam's (rm_work) effort to have this as a a new scheduler hint and a new scheduler filter [2] We had a good discussion on #openstack-nova [3] about the possible way forward with Sean and Adam. Here is a summary and my proposed way forward. Currently booting a server in a routed network setup works iff the neutron port is pre-created with ip_allocation=deferred before the server create. Nova then schedule the server without consider anything about the possible multiple network segments of the port. Neutron will do the segment assignment when the port is bond to the compute host. The problem is that once a port bound to a segment it cannot be bound to another segment as that would change the IP of the port. So if a server is migrated nova should select a destination host where the _same segment_ is available. Therefore we need a way to influence the scheduling based on the assigned segment id of the port. Both [1] and [2] does that based on the fact that neutron creates nova host aggregates for each network segment and those aggregates are mirrored to placement as placement aggregates. For me the pre-filter approach[1] seems better as * it avoids introducing a new scheduler hint * it is not affected by the allocation candidate limit configuration that effect scheduler filters * it allows us to iterate towards an approach where neutron provides the required aggregates of each port via the ports' resource_request attribute The downside of the pre-filter approach is that right now it needs to query the segments for each network / port the server has before the each scheduling. I think we limit such performance impact by making this pre-filter turned off by default via configuration. So a deployment without routed networks does not need to pay this cost. Later this cost can be nullified by extending neutron to specify the needed aggregates in the port's resource_request. I'm now planning to take [1] over from Matt and finish it up by adding functional test and documentation to it. I would also like to raise attention to the tempest patch that adds basic CI coverage for the currently working server create scenario [4]. Cheers, gibi [1] https://review.opendev.org/#/c/656885 [2] https://review.opendev.org/#/c/709280 [3] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… [4] https://review.opendev.org/#/c/665155

2 1

Restarting neutron-l3-agent service
by Grant Morley 26 Feb '20

26 Feb '20

Hi all, We are currently seeing an issue where neutron isn't setting up ports correctly on routers. We think we have narrowed it down to the "neutron-l3-agent" on the neutron nodes that is causing the issue. We are wanting to restart the service but was wondering if doing so it would cause any issues with currently running instances? We have 2 neutron nodes in HA mode ( one active and one standby ). Currently the issue is only affecting newly built instances, so we don't want to restart the service in hours if it is going to cause issues with instances that are currently unaffected. We are running on OpenStack Queens and have built the platform using OSA. Many thanks for any help. Regards,

2 1

[all][tc][uc] Uniting the TC and the UC
by Thierry Carrez 26 Feb '20

26 Feb '20

Hi all, When our current project governance was established in 2012, we defined two separate bodies. The Technical Committee represented developers / code contributors to the open source project(s), while the User Committee represented the operators running the resulting software, as well as users of the APIs. That setup served us well in those early days. The focus on the upstream side was strongly around *development* of code, we did not have that many users, and even less users directly involved in upstream development. A separate User Committee resulted in the formation of an engaged community of users, and ensured that our community in general (and our events in particular) took the needs of operators into account. Fast-forward to 2020: the upstream focus is more on maintenance. We now have a lot of users, and thanks to the efforts of the UC they are increasingly directly involved in the open source project development, with several operators leading upstream project teams directly. There is now limited UC-specific activity, and as such it is struggling to get volunteers to step up for it: nobody nominated themselves for the last round of election. Keeping two separate bodies to represent them maintains the illusion that devs and ops are different breeds, and sometimes discourages operators from running for the TC and more directly influence the shape of the software. If anything, we need more ops representation at the TC, and seeing people like mnaser being elected at (and chairing) the TC was a great experience. I discussed the situation with the current UC members, and we decided it's time to consider having a single "community" and a single body to represent it. That body would tackle the traditional TC tasks (open source project governance and stewardship) but also the UC tasks (user survey, ambassador program...). That body would be elected by contributors, in a large sense that includes the AUC definition. I feel like that would help remove the artificial barriers discouraging users to get more directly involved with software development and maintenance. There are multiple ways to achieve that, in increasing order of complexity and bylaws changes needed: 1- No bylaws change As bylaws changes take a lot of time and energy, the simplest approach would be to merge the TC and UC without changing the bylaws at all. The single body (called TC) would incorporate the AUC criteria by adding all AUC members as extra-ATC. It would tackle all aspects of our community. To respect the letter of the bylaws, the TC would formally designate 5 of its members to be the 'UC' and those would select a 'UC chair'. But all tasks would be handled together. 2- Minimal bylaws change Same, but all mentions of the UC would be removed from the bylaws (affecting sections 4.12, 4.14, and Appendix 10). This would require a simple vote by the Board of Directors, and would avoid some confusion and having to formally designate a subset of the TC to respect the letter of the bylaws. 3- Maximal bylaws change Create a new body ("steering committee" for example) replacing the TC and the UC. This would require changing sections 4.1, 4.12, 4.13, 4.14, 4.20, 7.1.b, 7.4, 9.1, 9.2.d, and Appendixes 4 and 10. Section 4.13, 7.1 and 7.4 being heavily protected in the bylaws, they require special votes by each class of Foundation members, probably a multi-year effort. A lot of documentation would also need to be changed in case the TC is renamed. I personally don't think the benefit of the change offsets the large cost of option (3). Given the current vacancy in the UC makeup I would recommend we pursue option 1 immediately (starting at the next TC election in April), and propose minimal bylaws change to the board in parallel (option 2). Thoughts ? -- Thierry Carrez

8 16

Retiring x/devstack-plugin-bdd
by Sean McGinnis 26 Feb '20

26 Feb '20

Hello all, This is to announce that I plan on going through the process to retire the x/devstack-plugin-bdd repo. https://opendev.org/x/devstack-plugin-bdd/ This project was to allow using the Cinder block device driver in Devstack. Support for the block device driver was removed from Cinder many releases ago and there is no expectation that this devstack plugin could be used in any meaningful way. Sean

4 6

[charms] Peter Matulis -> charms-core
by James Page 26 Feb '20

26 Feb '20

Hi Team I'd like to proposed Peter Matulis for membership of the charms-core team. Although he's not be focussed on developing the codebase since he started contributing to the OpenStack Charms he's made a number of significant contributions to our documentation as well as regularly providing feedback on updates to README's and the charm deployment guide. I think he would make a valuable addition to the charms-core review team! Cheers James

5 5

[all][crazy][ideas] Call for "ideas" inside the openstack/ideas repo
by Jean-Philippe Evrard 26 Feb '20

26 Feb '20

Hello everyone, I would like to inform you that the "ideas" repos is now created, and waiting for all your fresh (or non fresh) ideas to change OpenStack! You will be able to find rendered ideas in https://governance.openstack.org/ideas/ . Currently it's lacking a little bit of content... I would love to see this changed soon, to be honest :) Here is a reminder of what the ideas repo is: - It's a place where anyone can propose an idea for changing OpenStack as a whole; - It's a git repository, which allows the versioning (and authoring) of ideas; - It's a way to refer to mailing lists, to know full context, but without having to browse through the history of the multiple mailing lists to find what you are looking for; - It's a repo where everyone can propose ideas to, and there will be no judgement on the ideas for merging them (the only thing asked is that what's written in an idea reflects what has been said on the mailing lists). Thank you for your attention! Your TC chair, Jean-Philippe Evrard (evrardjp)

2 1

[largescale-sig] Meeting summary and next actions
by Thierry Carrez 26 Feb '20

26 Feb '20

Hi everyone, The Large Scale SIG held a meeting today. You can catch up with the summary and logs of the meeting at: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_si… amorin reported on progress documenting better configuration defaults for large scale, as can be seen on the etherpad[1]. He will set up a wiki page under our SIG page[2] to continue solidifying that content, as well as proposing a patch to Nova doc to point to that page. [1] https://etherpad.openstack.org/p/large-scale-sig-documentation [2] https://wiki.openstack.org/wiki/Large_Scale_SIG The story we collected in the scaling stories etherpad[3] proves itself very valuable. We should encourage collecting small bits of information and anecdotes of scaling up, it does not have to be as thorough of a report as the one already collected. [3] https://etherpad.openstack.org/p/scaling-stories masahito reported on progress on the oslo.metrics spec[4]. He plans to post a new patchset of that spec very soon. [4] https://review.opendev.org/#/c/704733/ In other topics, oneswig suggested that we also tackle bare metal cluster scaling. This should be considered a part of the "scaling within one cluster" goal, just for specific types of clusters. It would be great to have a session around bare metal cluster scaling at the upcoming OpenDev event[5] (under the "Large scale usage of open infrastructure" track). belmiro and masahito volunteered to be part of the programming committee for that track. the idea would be to make sure it aligns with the goals of this SIG, and that we recruit new members there. For example, we could have a specific session on scaling-stories, where people can informally talk about scaling anecdotes, and then we'd take notes and follow up with them to submit something more detailed. [5] https://www.openstack.org/events/opendev-ptg-2020/ The next meeting will happen on March 11, at 9:00 UTC on #openstack-meeting. Cheers, -- Thierry Carrez (ttx)

1 0

[tripleo] missing centos-8 rpms for kolla builds
by Wesley Hayutin 26 Feb '20

26 Feb '20

Greetings, I know the ceph repo is in progress. TripleO / RDO is not releasing opendaylight Can the RDO team comment on the rest of the missing packages here please? Thank you!! https://review.opendev.org/#/c/699414/9/kolla/image/build.py NOTE(mgoddard): Mark images with missing dependencies as unbuildable for # CentOS 8. 'centos8': { "barbican-api", # Missing uwsgi-plugin-python3 "ceph-base", # Missing Ceph repo "cinder-base", # Missing Ceph repo "collectd", # Missing collectd-ping and # collectd-sensubility packages "elasticsearch", # Missing elasticsearch repo "etcd", # Missing etcd package "fluentd", # Missing td-agent repo "glance-base", # Missing Ceph repo "gnocchi-base", # Missing Ceph repo "hacluster-base", # Missing hacluster repo "ironic-conductor", # Missing shellinabox package "kibana", # Missing elasticsearch repo "manila-share", # Missing Ceph repo "mongodb", # Missing mongodb and mongodb-server packages "monasca-grafana", # Using python2 "nova-compute", # Missing Ceph repo "nova-libvirt", # Missing Ceph repo "nova-spicehtml5proxy", # Missing spicehtml5 package "opendaylight", # Missing opendaylight repo "ovsdpdk", # Not supported on CentOS "sensu-base", # Missing sensu package "tgtd", # Not supported on CentOS 8 }, 'centos8+source': { "barbican-base", # Missing uwsgi-plugin-python3 "bifrost-base", # Bifrost does not support CentOS 8 "cyborg-agent", # opae-sdk does not support CentOS 8 "freezer-base", # Missing package trickle "masakari-monitors", # Missing hacluster repo "zun-compute", # Missing Ceph repo

9 19

OpenStack with vCenter
by Kevin Puusild 26 Feb '20

26 Feb '20

Hello I'm currently trying to setup DevStack with vCenter (Not ESXi), i found a perfect documentation for this task: https://wiki.openstack.org/wiki/NovaVMware/DeveloperGuide But the problem here is that when i start *stack.sh *with localrc file shown in documentation the installing process fails. Is this documentation out-dated? Is there some up to date documentation? -- Best Regards. Kevin Puusild

2 1

[scientific-sig] IRC Meeting today - 1100 UTC
by Stig Telfer 26 Feb '20

26 Feb '20

Hi All - We have a Scientific SIG meeting on IRC at 1100 UTC in channel #openstack-meeting. Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_February_26th_20… <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_February_26th_20…> Today we’d like to cover SIG participation in a number of upcoming events. The best SIG events are when we get together face-to-face. Cheers, Stig

1 0

[tc] February meeting Agenda
by Jean-Philippe Evrard 26 Feb '20

26 Feb '20

Hello everyone, Our next meeting is happening next week Thursday (the 5th), and the agenda is, as usual, on the wiki! Here is a primer of the agenda for this month: Follow up on past action items Report on tc/uc merge (ttx): See http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012806… Ensuring the analysis of the survey was updated (jungleboyj) Report on telemetry after Rico's convo with the PTL (ricolin) Report on stable branch policy work: mnaser to push an update to distill what happened on the ML in terms of stable branch policy (mnaser) Report on the community goals for U and V, py2 drop (gmann) Report on release naming (mugsie) Dropping side projects - draft guidelines are available for review https://review.opendev.org/#/c/707421/ See you there! Regards, Jean-Philippe Evrard (evrardjp)

1 0

[keystone][middleware] why can not middle-ware support redis?
by guoyongxhzhf＠163.com 25 Feb '20

25 Feb '20

In my environment, there is a redis. I want the keystone client in nova to cache token and use redis as a cache server. But after reading keystone middle-ware code, now keystone middle-ware just support swift and memcached server. Can I just modify keystone middleware code to use dogpile.cache.redis directly?

2 1

[tc] Presence at OpenDev Vancouver in June
by Jean-Philippe Evrard 25 Feb '20

25 Feb '20

Hello, I am filling the survey for the room booking during the OpenDev summit in Vancouver. Do you already know if you will attend, this way we have a first guesstimate of the attendance? Can you answer it here by Thursday, please? FYI: I thought of booking for min 1.5 days, maximum time 2 days, as last time we finished the marathon in two days. For the room format, I like the roundtable (was wondering to add extra seats next to the roundtable for people joining), however it's only around 10 persons per table, which won't be enough for us all if everyone attends. Regards, JP

12 12

Re: [all][tc][uc] Uniting the TC and the UC
by Jimmy McArthur 25 Feb '20

25 Feb '20

> > Thanks a lot, ttx for starting this thread. > > option 1 looks more feasible way but few question/feedback: > - How we will execute the "designate 5 members from TC and select UC chair"? > > If by volunteer call from TC? > I think this can lead to the current situation where very few members are interested to > serve as UC. I am afraid that we will get 5 volunteers and a chair. > > If by force? > I do not think this is or should be an option :). But if then how we make sure the TC > members are ok/good to handle the UC tasks on the non-technical side for example > ambassador program. We have dedicated Foundation Staff that offer support for the Ambassador Program as well as User Survey. I don't have concerns about this falling between the cracks during the transition and it's something we can work with the new UC seats under the TC on. Cheers, Jimmy > > - Will there be any change in TC elections, votes weightage and nomination? by this change > of a new subteam under TC? > > - I think along with the merging proposal, we should distribute the current tasks handled by UC > among TC, Ops group, foundation etc. For example, the ambassador program, local user group > interaction or any other managerial tasks should be excluded from TC scope. > If we would like to merge all then I think we should rename TC to Steering Committee or > something which is option 3. > > - What about merging UC into Ops team, they are more close to users/operators and active > in terms of the meetup and providing feedback etc. > > -gmann > > >> Begin forwarded message: >> >> *From: *Thierry Carrez <thierry(a)openstack.org >> <mailto:thierry@openstack.org>> >> *Subject: **Re: [all][tc][uc] Uniting the TC and the UC* >> *Date: *February 25, 2020 at 11:36:06 AM CST >> *To: *openstack-discuss(a)lists.openstack.org >> <mailto:openstack-discuss@lists.openstack.org> >> >> Tim Bell wrote: >>> Are there some stats on the active electorate sizes ? >>> I have the impression that the voting AUCs were a smaller number of >>> people than the contributors so a single election may result in less >>> total representation compared to today’s UC & TC. >> >> I don't have the exact numbers around, but yes, there are many more >> ATCs (voters in the current TC election) than AUCs (voters in the >> current UC election). >> >> So it's a valid concern that people with more of an operator >> background would have trouble getting elected if the electorate >> contains more people with more of a developer background. >> >> But I think it's a fallacy. There is plenty of overlap. Most engaged >> operators are already ATCs, and more and more contributors have >> operational experience. Data points show that when people with more >> operational background have nominated themselves for the TC in recent >> elections, they got elected. 10 of the 13 TC seats are currently >> occupied by people with some decent amount of operational experience. >> >> So yes, we clearly need to communicate that people with >> operational/usage experience are wanted in that new body. We need to >> communicate that there is a change, and a single body will now >> steward all aspects of the open source projects and not just the >> upstream aspects. But I'm not obsessing on the fact that a single >> election would somehow suppress operator voices... >> >> The problem recently has more been to find enough people willing to >> step up and spend extra time stewarding OpenStack, than to actually >> get elected. >> >> -- >> Thierry Carrez (ttx) >> >

1 1

[openstack-ansible] ptg room
by Mohammed Naser 25 Feb '20

25 Feb '20

Hi everyone, I'd like to ask who is planning to be at the OpenDev event in Vancouver to gauge the amount if interest around having a room for OpenStack-Ansible at the PTG. Thanks, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. https://vexxhost.com

2 1

[ansible-sig][openstack-ansible][tripleo] A new meeting time and place for Openstack Ansible modules discussions
by Sagi Shnaidman 25 Feb '20

25 Feb '20

Hi, all according to our poll about meeting time[1] the winner is: Tuesday 15.00 - 16.00 UTC (3.00 PM - 4.00 PM UTC) Please be aware that we meet in different IRC channel - #openstack-ansible-sig Thanks for voting and waiting for you tomorrow in #openstack-ansible-sig at 15.00 UTC [1] https://xoyondo.com/dp/ITMGRZSvaZaONcz -- Best regards Sagi Shnaidman

1 1

[openstack-helm][Horizon] After Using a customize image of horizon container get fail.
by prashant 25 Feb '20

25 Feb '20

Hi All, We have build the horizon image via kolla and trying to deploy using openstack-helm chart. while doing it could able to pull the image but after start the container it get crashed. below logs we are getting: ------------------------------ + exec apache2 -DFOREGROUND AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.21.237.188. Set the 'ServerName' directive globally to suppress this message no listening sockets available, shutting down AH00015: Unable to open logs ----------------------------------- is any change need to be done for customize image for related to port? Thanks, Prashant

1 0

[Multi-Arch-SIG] Presence at OpenDev Vancouver in June
by Rico Lin 25 Feb '20

25 Feb '20

Hi all, As the deadline for OpenDev Vancouver team survey is near (before this weekend). We need to decide if we would like to reserve a room or not. And since I'm not that positively sure I will get the budget to travel this time. We need to make sure if anyone is going so we can have people to be the moderator. Note that I will not occupy a room if no others are able to be there. *might* going is totally fine as long as we can have people there as moderator -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

[automation-sig][self-healing-sig][auto-scaling-sig] Presence at OpenDev Vancouver in June
by Rico Lin 25 Feb '20

25 Feb '20

Hi all, As the deadline for OpenDev Vancouver team survey is near (before this weekend). We need to decide if we would like to reserve a room or not. And since I'm not that positively sure I will get the budget to travel this time. We need to make sure if anyone is going so we can have people to be the moderator. Note that I will not occupy a room if no others are able to be there. *might* going is totally fine as long as we can have people there as moderator -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

[heat] Presence at OpenDev Vancouver in June
by Rico Lin 25 Feb '20

25 Feb '20

Hi all, As the deadline for OpenDev Vancouver team survey is near (before this weekend). We need to decide if we would like to reserved a room or not. And since I'm not that positively sure I will get the budget to travel this time. We need to make sure if anyone is going so we can have people to be the moderator. Note that I will not occupy a room if no others are able to be there. Or at least *might* be there. -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

[Watcher]We will have IRC meeting at 8:00 UTC tomorrow
by licanwei 24 Feb '20

24 Feb '20

please update the meeting agenda if you have something want to discuss. Thanks | | licanwei_cn | | 邮箱：licanwei_cn(a)163.com | 签名由网易邮箱大师定制 On 02/13/2020 17:01, licanwei wrote: Hi, next time i will send the mail one day in advance. hope to meet you next time! Thanks licanwei | | licanwei_cn | | 邮箱：licanwei_cn(a)163.com | 签名由网易邮箱大师定制 On 02/12/2020 17:58, info(a)dantalion.nl wrote: Hello Lican, Sorry but I am unable to attend at such short notice as in my timezone I won't be awake unless it is for the Watcher meeting. To be able to adjust my transit schedule I will have to know a day in advance. Hope to be there next time. Kind regards, Corne Lukken On 2/12/20 3:31 AM, licanwei wrote: > > > > > licanwei_cn > 邮箱：licanwei_cn(a)163.com > 签名由网易邮箱大师定制 > > | | > licanwei_cn > | > | > 邮箱：licanwei_cn(a)163.com > | > > 签名由网易邮箱大师定制 >

1 0

[neutron] Team meeting 25.02.2020 cancellee
by Slawek Kaplonski 24 Feb '20

24 Feb '20

Hi, I’m on PTO tomorrow and I will not be able to chair our team meeting. So lets cancel it. See You all on the meeting next week. One reminder, if You are planning to attend Vancouver PTG in June, please add Your name to the list in [1]. I need to give number of rough number of attendees by Sunday, March 2nd. Please add Your name there even if it’s not yet confirmed. [1] https://etherpad.openstack.org/p/neutron-victoria-ptg — Slawek Kaplonski Senior software engineer Red Hat

1 0

Neutron metadata service not responding due to rabbitmq dropping messages
by Grant Morley 24 Feb '20

24 Feb '20

Hi all, We have recently come across an issue where our metadata service stops responding. If you try to curl the service from within an instance you get: % curl http://169.254.169.254 <html><body><h1>504 Gateway Time-out</h1> The server didn't respond in time. </body></html> After doing some digging around on our neutron nodes I noticed we were getting loads of RabbitMQ timeout errors whilst trying to process message requests: 2020-02-24 07:28:09.747 26378 ERROR neutron.common.rpc [-] Timeout in RPC method get_ports. Waiting for 26 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: MessagingTimeout: Timed out waiting for a reply to message ID a14c4a1395864cd980c1ec563a5c48aa The servers are fairly busy, however we do not have a massive installation >1500 instances and roughly 850 routers. However if I restart the "neutron-metadata-agent" service and the "neutron-server" service it seems to fix the issue for a while but ultimately it comes back. I did increase the "rpc_timeout" on the netutron nodes to 120 seconds but that seems quite long to me. Likewise the RabbitMQ servers are not overly busy, we seem to get a constant stream of only 40+ messages in the queue at one time and that can spike depending on workload. Does anyone know of any tuning or tweaking we can do to the metadata service in either Neutron or Nova that might help? We are running OpenStack Queens if that helps. Many thanks, Grant

1 0

RHOSP-like installation
by Volodymyr Litovka 24 Feb '20

24 Feb '20

Dear colleagues, while having a good experience with Openstack on Ubuntu, we're facing a plenty of questions re RHOSP installation. The primary requirement for our team re RHOSP is to get a knowledge on RHOSP - how to install it and maintain. As far as I understand, RDO is the closest way to reach this target but which kind of installation it's better to use? - * plain RDO installation as described in generic Openstack guide at https://docs.openstack.org/install-guide/index.html (specifics in RHEL/CentOS sections) * or TripleO installation as described in http://tripleo.org/install/ * or, may be, it is possible to use RHOSP in kind of trial mode to get enough knowledge on this platform? Our lab consists of four servers (64G RAM, 16 cores@2GHz) which we're going to use in "ultraconverged" mode - as both controller and agent (compute/network/storage) nodes (controllers, though, can be in virsh-controlled VMs). In case of TripleO scenario, 4th server can be used for undercloud role. This installation is intended not for production use, but rather for learning purposes, so no special requirements for productivity. The only special requirement - to be functionally as much as close to canonical RHOSP platform. I will highly appreciate your suggestions on this issue. Thank you. -- Volodymyr Litovka "Vision without Execution is Hallucination." -- Thomas Edison

2 1

[oslo] Meeting Time Poll
by Ben Nemec 24 Feb '20

24 Feb '20

Hello again, We have a few regular attendees of the Oslo meeting who have conflicts with the current meeting time. As a result, we would like to find a new time to hold the meeting. I've created a Doodle poll[0] for everyone to give their input on times. It's mostly limited to times that reasonably overlap the working day in the US and Europe since that's where most of our attendees are located (yes, I know, that's a self-fulfilling prophecy). If you attend the Oslo meeting, please fill out the poll so we can hopefully find a time that works better for everyone. Thanks! -Ben /me finally checks this one off the action items for next week :-) 0: https://doodle.com/poll/zmyhrhewtes6x9ty

1 1

[ironic][neutron][ops] Ironic multi-tenant networking with VXLAN Overlay
by Austin Cormier 24 Feb '20

24 Feb '20

We are using Openstack + Ironic to test routing software which requires the use of VLANs. Using VXLan overlays allows the VM's to have nested VLANs without issue as they are encapsulated into a VXLAN and sent across. However, we are struggling to find a path forward in extending the VXLAN overlay out to the hardware device. Unfortunately, the networking-generic-switch driver does not support VXLAN but this seems to fairly easy to extend (given the right switch). The challenge I'm facing is that with Neutron OVS + L2Pop, there seems to be no easy way to extend the L2 population to include an external device. The only path forward that I can see is to use the Linux Bridge driver + VXLAN multicast and find a switch that also supports VXLAN multicast so it can participate in the overlay dynamically. The HPE FlexFabric seems to do this from the documentation: https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c04567545 Can anyone help confirm that the switch above will work and or give alternative suggestions on approaching this problem? -Austin

1 0

[openstack-helm][stein]Cinder pods stuck in init state after deployed in k8s cluster
by Sanjay K 24 Feb '20

24 Feb '20

Hello openstack-helm team, I am trying to deploy stein cinder service in my k8s cluster using persistent volume and persistent volume claim for local storage or NFS storage. However after deploying cinder in my cluster, the cinder pods remains in Init state even though the PV and PVC are created. Please look at my below post on openstack forum and guide me how to resolve this issue. https://ask.openstack.org/en/question/126191/cinder-pods-in-init-state-when… thank you for your help and support on this. best regards, Sanjay

2 3

[largescale-sig] Next meeting: Feb 26, 9utc
by Thierry Carrez 24 Feb '20

24 Feb '20

Hi everyone, The Large Scale SIG will have a meeting this week on Wednesday, Feb 26 at 9 UTC[1] in #openstack-meeting on IRC: [1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200226T09 As always, the agenda for our meeting is available at: https://etherpad.openstack.org/p/large-scale-sig-meeting Feel free to add topics to it! A quick reminder of the TODOs from our last meeting: - masahito to update spec based on initial feedback - everyone to review and comment on https://review.opendev.org/#/c/704733/ Talk to you all on Wednesday, -- Thierry Carrez

1 0

[neutron] Bug deputy report (week starting on 2020-02-17)
by Bernard Cafarelli 24 Feb '20

24 Feb '20

Picking up after amotoki's report [0], here is last week's report. This includes all bugs reported up to 1864374 (included) Some gate issues (python2 is still fighting us back, etc), new "interesting" bugs on different topics. Bugs marked with "***" do not have current assignee Critical * socket.timeout error in dvr CI jobs cause SSH issues - https://bugs.launchpad.net/neutron/+bug/1863858 Under investigation by slaweq - neutron-tempest-plugin-designate-scenario fails on stable branches with "SyntaxError: invalid syntax" installing dnspython - https://bugs.launchpad.net/neutron/+bug/1864015 Patch merged to pin designate plugin - https://review.opendev.org/708825 - "ping" prepended to ip netns commands - https://bugs.launchpad.net/neutron/+bug/1864186 Patch merged: https://review.opendev.org/709100 High * When updating the network segment ID, the project_id should be provided - https://bugs.launchpad.net/neutron/+bug/1863619 Needed parameter to get private ranges belonging to the project, fix in progress: https://review.opendev.org/708196/ * [OVN] Provider driver IPv6 traffic doesn't work - https://bugs.launchpad.net/neutron/+bug/1863892 IPv6 load balancers with OVN provider do not work, in progress by maciejjozefczyk * [OVN] DHCP doesn't work while instance has disabled port security - https://bugs.launchpad.net/neutron/+bug/1864027 Behaviour introduced by a recent fix, root cause may be in core OVN *** IP allocation for stateless IPv6 does not filter on segment when fixed-ips contain a subnet_id - https://bugs.launchpad.net/neutron/+bug/1864225 Title sums it up nicely, stateless IPv6 + segments issue *** When adding another stateless subnet, implicit address allocation happen for port with deferred ip allocation - https://bugs.launchpad.net/neutron/+bug/1864333 IPv6 allocation happens for deffered allocation port Medium *** Unhandled error - WSREP has not yet prepared node for application use - https://bugs.launchpad.net/neutron/+bug/1863579 Unassigned. Transient startup error seen in kolla-ansible upgrade CI when database is not ready, we can probably add a retry here * [neutron-tempest-plugin] test_trunk.TrunkTestInheritJSONBase.test_add_subport fails if unordered - https://bugs.launchpad.net/neutron/+bug/1863707 Results order failure in test, fix in progress: https://review.opendev.org/708305/ * [OVN]Could not support more than one qos rule in one policy - https://bugs.launchpad.net/neutron/+bug/1863852 Fix in progress: https://review.opendev.org/708586 * Missing information in log message for Invalid tunnel type - https://bugs.launchpad.net/neutron/+bug/1863888 Add some log extra information, fix in progress: https://review.opendev.org/708634 * [OVN] OVN LoadBalancer VIP shouldn't have addresses set - https://bugs.launchpad.net/neutron/+bug/1863893 Regression fix, in progress by maciejjozefczyk * [OVN] Remove dependency on port_object - https://bugs.launchpad.net/neutron/+bug/1863987 Suggested fix: https://review.opendev.org/708806/ Low * Default security group rules not created during list of SG rules - https://bugs.launchpad.net/neutron/+bug/1864171 Fix in progress: https://review.opendev.org/709070 *** BGP dynamic routing in neutron - No Info about capability to receiv/learn dynamic routes - https://bugs.launchpad.net/neutron/+bug/1864219 Doc enhancement request on BGP limitations *** ml2 ovs does not flush iptables switching to FW ovs - https://bugs.launchpad.net/neutron/+bug/1864374 Switching from iptables to ovs firewall is not a trivial task, this may end up being documentation bug too Related to neutron * Upgrade from Rocky to Stein, router namespace disappear - https://bugs.launchpad.net/neutron/+bug/1863982 Issue in kolla-ansible upgrade Rocky->Stein, failing to restore iptables rules. It does not seem to be neutron-specific, but opinions welcome [0] http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012610… -- Bernard Cafarelli

1 0

[neutron] Network segment ranges feature
by Rodolfo Alonso 24 Feb '20

24 Feb '20

Hello Neutrinos: First of all, a reference: https://bugs.launchpad.net/neutron/+bug/1863423 I detected some problems with this feature. The service plugin, when enabled and according to the spec [1], should segregate the segmentation range in several ranges. Each one (or many) of those ranges could be assigned to a project. When a network is created in a specific project, should receive a segmentation ID from the project network segment ranges. In case of not having any range assigned, a segmentation ID from a default range will be provided. How the current implementation actually works: When a driver is loaded (VLAN, VXLAN, GRE or Geneve), it creates a default network segment range, based on the static configuration provided in the plugin ("network_vlan_ranges", "vni_ranges", etc). Then the admin can create project specific network segment ranges (remember: always per driver type). Those project specific ranges cannot overlap but the project specific ranges CAN overlap the default one. A valid state could be: VLAN: - Default: 100:1000 - Project1: 100:200 - Project2: 200:300 When assigning segmentation IDs to new networks, the driver will query [2]: - The existing ranges per project. - The default range (always present). The result is that, if the project network segment range runs out of segment IDs, it will retrieve those ones from the default range. That will lead, eventually, to a clash between ranges. Project 1 will be able to allocate a network with a segmentation ID from Project 2 range. E.g.: [3] The problem is how the queries are done: [4] --> select one of each commented line to reproduce "query_project_id" and "query_shared" in [2]. Now we know the feature is not properly working, my question is what the proper behaviour should be. Some alternatives: 1) If a project has one or more network segment ranges, the segmentation ID should be retrieved ONLY from those ranges. If a project does not have any range, then it will retrieve from the shared pool BUT NEVER selecting a segment ID belonging to other project ID (ufff this query is going to be complex). --> IMO, the best solution. 2) If a project has one or more network segment ranges, the segmentation ID should be retrieved first from those ranges and then from the shared one, BUT NEVER selecting a segment ID belonging to other project ID. Same for range-less projects. --> IMO, if the administrator has assigned a range for a project, the project should ONLY use this pool. This is not a good solution. Can I have your feedback? Regards. [1] https://specs.openstack.org/openstack/neutron-specs/specs/stein/network-seg… [2]https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driv… [3]http://paste.openstack.org/show/789872/ [4]http://paste.openstack.org/show/789873/

3 3

[openstack-discuss][neutron] is neutron-openvswitch-agent needed for network connection on provider network?
by Chen CH Ji 24 Feb '20

24 Feb '20

2 2

[neutron][dhcp] "additional" dhcp port in state "reserved_dhcp_port", non existing namespace on network node
by Nicolas Ghirlanda 24 Feb '20

24 Feb '20

Dear mailinglist, We have the following behavour, which occured in about 3% of our networks. We have a default configured of 3 dhcp servers per subnet, but in about 3% of all networks, we have unexpectedly 4, from which one is in state "reserved_dhcp_port", also there is NO namespace on that control/network node. Setup: Openstack Rocky, Networkmode legacy # openstack port list --device-owner network:dhcp --network a2d4605d-997e-4807-9250-c0c80af3183e +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ | 40237ca8-aed6-49ee-xxxx-49acf97797e4 | | fa:16:3e:e1:33:83 | ip_address='172.16.0.4', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | | 6a3898c6-6520-43e7-xxxx-e6b2639f2600 | | fa:16:3e:ff:f0:7b | ip_address='172.16.0.2', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | | 89479d32-881d-45b9-xxxx-1b41ed8fd703 | | fa:16:3e:ef:7d:dc | ip_address='172.16.0.6', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | | eaeb0ef7-48c6-43ea-xxxx-8615ed704e48 | | fa:16:3e:66:7c:96 | ip_address='172.16.0.3', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ full output of one of that port: # openstack port show 89479d32-881d-45b9-xxxx-1b41ed8fd703 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | ctl5 | | binding_profile | | | binding_vif_details | datapath_type='system', ovs_hybrid_plug='True', port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | normal | | created_at | 2020-01-16T11:22:14Z | | data_plane_status | None | | description | | | device_id | reserved_dhcp_port | | device_owner | network:dhcp | | dns_assignment | None | | dns_domain | None | | dns_name | None | | extra_dhcp_opts | | | fixed_ips | ip_address='172.16.0.6', subnet_id='e4380af7-13b4-40f9-xxxx-xxxx' | | id | 89479d32-881d-45b9-xxxx-1b41ed8fd703 | | location | Munch({'project': Munch({'domain_id': None, 'id': u'a772e4ab888e4fxxxx', 'name': 'admin', 'domain_name': 'Default'}), 'cloud': '', 'region_name': 'ch-zh1', 'zone': None}) | | mac_address | fa:16:3e:ef:7d:xx | | name | | | network_id | a2d4605d-997e-4807-xxxx-xxx | | port_security_enabled | False | | project_id | 232ecbeb96fd4663xxxx | | propagate_uplink_status | None | | qos_policy_id | None | | resource_request | None | | revision_number | 8 | | security_group_ids | | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2020-02-14T09:16:57Z | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ctl5:~# ip netns |grep a2d4605d ctl5:~# *Workaround:* I can easily delete those ports manually and everything seems to be ok (openstack delete port <portid>) When I tried figuring out, what happens, when I disable dhcp in an affected subnet, the following happens (different example): The 6f34 is a reserved port (last line) # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-af71-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 23f9e5d7-75a2-440a-xxxx-d82000286fbe | | fa:16:3e:8b:e2:a8 | ip_address='192.168.50.3', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | 352bc1a4-e002-440b-xxxx-0383cfb1632b | | fa:16:3e:29:46:6b | ip_address='192.168.50.5', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | 6ee79608-7834-4015-xxxx-fda3f47b5e59 | | fa:16:3e:0c:b6:60 | ip_address='192.168.50.4', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | 6f3435f8-23c0-427c-xxxx-fca1a6910aff | | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ - I disable dhcp (openstack subnet set --no-dhcp <subnetid>), expected to remove all dhcp ports, but the what happens is that all "healthy" dhcp ports will be removed, the reserved port remains # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-af71-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 6f3435f8-23c0-427c-xxxx-fca1a6910aff | | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ - when I now reenable dhcp, nothing happens, still the single, reserved port is existing, no new ports are created. Just the state went from ACTIVE to DOWN to ACTIVE # openstack subnet set --dhcp 882b360a-eae1-4736-a276-af752f552440 # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 6f3435f8-23c0-427c-xxxx-fca1a6910aff | | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | DOWN | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ s# openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 6f3435f8-23c0-427c-xxxx-fca1a6910aff | | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ - after deleting the port manually and disable/enable dhcp, 3 new working ports are created # openstack port delete 6f3435f8-23c0-427c-xxxx-fca1a6910aff # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 # openstack subnet set --no-dhcp 882b360a-eae1-4736-xxxx-af752f552440 # openstack subnet set --dhcp 882b360a-eae1-4736-xxxx-af752f552440 # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 9fd63011-2b1f-410a-xxxx-40d42779c637 | | fa:16:3e:15:be:fd | ip_address='192.168.50.4', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | cdd32840-7899-47b2-xxxx-7d36eae515a2 | | fa:16:3e:4c:40:5c | ip_address='192.168.50.3', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | dc1eeba2-4afd-4502-xxxx-7370d086e553 | | fa:16:3e:29:29:5d | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ Any ideas where those are coming from? thanks a lot Nicolas --

1 0

[CINDER] Distributed storage alternatives
by Alfredo De Luca 23 Feb '20

23 Feb '20

Hi all. we 'd like to explore storage back end alternatives to CEPH for Openstack I am aware of GlusterFS but what would you recommend for distributed storage like Ceph and specifically for block device provisioning? Of course must be: 1. *Reliable* 2. *Fast* 3. *Capable of good performance over WAN given a good network back end* Both open source and commercial technologies and ideas are welcome. Cheers -- *Alfredo*

3 2

[neutron] do we need neutron-openvswitch-agent running in order to make VMs spawned with provider network?
by Chen CH Ji 23 Feb '20

23 Feb '20

3 2

[TC] W release naming
by Sean McGinnis 23 Feb '20

23 Feb '20

Hello TC (and other interested parties), This is a reminder that we have this week set aside for the 'W' release naming for any necessary discussion, campaigning, or other activities before the official polling starts. https://governance.openstack.org/tc/reference/release-naming.html#polls We have a set of names collected from the community. There may be some trademark concerns, but I think we can leave that for the Foundation review after the election polling completes unless anyone has a strong reason to exclude any now. If so, please state so here before I create the poll for next week. https://wiki.openstack.org/wiki/Release_Naming/W_Proposals As a reminder for everyone, this naming poll is the first that follows our new process of having the electorate being members of the Technical Committee. More details can be found in the governance documentation for release naming: https://governance.openstack.org/tc/reference/release-naming.html#release-n… I will prepare the poll to send out next Monday. Since we have a limited electorate this time, if we collect all votes ahead of the published deadline I will check in with the TC if there is any need to wait, and if not close the poll early and get the results ready to publish. We can then move ahead with the legal review that is required before we can officially declare a winner. Thanks! Sean

8 9

[cloud-provider-openstack] [kubernetes] Migrating from in-tree provider to external openstack-cloud-controller-manager
by Lingxian Kong 23 Feb '20

23 Feb '20

For those who still haven't heard of cloud-provider-openstack[1], cloud-provider-openstack is an implementation of Kubernetes Cloud Controller Manager[2], it's a sub-project under Kubernetes, which provides a couple of binaries/services relevant to OpenStack and Kubernetes integration, more documentation could be found here[3]. The demo below shows how to migrate from in-tree openstack cloud provider to external openstack-cloud-controller-manager. The Kubernetes cluster in the demo was set up using kubeadm. https://asciinema.org/a/303399?speed=2 [1]: https://github.com/kubernetes/cloud-provider-openstack [2]: https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controlle… [3]: https://github.com/kubernetes/cloud-provider-openstack/tree/master/docs - Best regards, Lingxian Kong Catalyst Cloud

1 0

[tripleo] RDO image server migration
by Daniel Pawlik 23 Feb '20

23 Feb '20

Hello, Todays migration of images server to the new cloud provider was not finished. We are planning to continue tomorrow (18th Feb), on 10 AM UTC. What was done today: - moved rhel-8 build base image to new image server What will be done tomorrow: - change DNS record - disable upload images to old host - sync old images (if some are available) Migration should be transparent to the end user. However, you have to keep in mind the unforeseen events that may occur. Write access to the old server will be disabled and until DNS propagation is not done, you could have read-only access. If you have any doubts or concerns, please do not hesitate to contact: - Daniel Pawlik <dpawlik(a)redhat.com> - Javier Pena <jpena(a)redhat.com> Regards, Daniel

3 3

[all][nova][ptl] Another one bites the dust
by Eric Fried 22 Feb '20

22 Feb '20

Dear OpenStack- Due to circumstances beyond my control, my job responsibilities will be changing shortly and I will be leaving the community. I have enjoyed my time here immensely; I have never loved a job, my colleagues, or the tools of the trade more than I have here. My last official day will be March 31st (though some portion of my remaining time will be vacation -- TBD). Unfortunately this means I will need to abdicate my position as Nova PTL mid-cycle. As noted in the last meeting [1], I'm calling for a volunteer to take over for the remainder of Ussuri. Feel free to approach me privately if you prefer. Thanks, efried [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-02-13-21.00.log…

9 8

[kolla]: Openstack kolla
by Mohammed, Moiz 22 Feb '20

22 Feb '20

Hello, I am installing Openstack Kolla on CentOs 7 with multimode configuration and facing some issues during pre-checks.I am using openstack stein release and master built. Command: ./kolla-ansible -i ../../multinode prechecks Error: TASK [haproxy : Checking if kolla_internal_vip_address is in the same network as api_interface on all nodes] ************************************************************************************************** fatal: [moiz-kolla-controller-01]: FAILED! => {"changed": false, "cmd": ["ip", "-o", "addr", "show", "dev", "eth0"], "delta": "0:00:00.002984", "end": "2020-02-21 18:11:42.365444", "failed_when_result": true, "rc": 0, "start": "2020-02-21 18:11:42.362460", "stderr": "", "stderr_lines": [], "stdout": "2: eth0 inet 192.168.130.203/24 brd 192.168.130.255 scope global noprefixroute dynamic eth0\\ valid_lft 81807sec preferred_lft 81807sec\n2: eth0 inet6 fe80::f816:3eff:fe7f:4d2/64 scope link \\ valid_lft forever preferred_lft forever", "stdout_lines": ["2: eth0 inet 192.168.130.203/24 brd 192.168.130.255 scope global noprefixroute dynamic eth0\\ valid_lft 81807sec preferred_lft 81807sec", "2: eth0 inet6 fe80::f816:3eff:fe7f:4d2/64 scope link \\ valid_lft forever preferred_lft forever"]} [cid:image003.png@01D5E8CF.EC6EB410] Moiz Mohammed | Systems Engineer - NSM Compute (704)-378-2934 O | (415) 866-3183 M | Oncall Phone: 1-866-577-0007 Ext 802 7815 Cresent Executive Dr, Floor#2 | Charlotte, NC 28217 [cid:image002.png@01D3AA4F.EFCE7660] E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

2 1

[tripleo] proposal to make Kevin Carter core
by Wesley Hayutin 22 Feb '20

22 Feb '20

Greetings, I'm sure by now you have all seen the contributions from Kevin to the tripleo-ansible project, transformation, mistral to ansible etc. In his short tenure in TripleO Kevin has accomplished a lot and is the number #3 contributor to tripleo for the ussuri cycle! First of all very well done, and thank you for all the contributions! Secondly, I'd like to propose Kevin to core.. Please vote by replying to this email. Thank you +1

17 16

[tripleo] Turning off Paunch by default
by Emilien Macchi 22 Feb '20

22 Feb '20

Hi folks, In the long-term effort of simplification and convergence to Ansible, the replacement of Paunch has passed a big milestone where it has proven to be stable enough to be the default. A lot of testing has happened, including updates and upgrades; all positive so far. The role itself is documented here: https://docs.openstack.org/tripleo-ansible/latest/roles/role-tripleo-contai… It has functional testing (Molecule), testing things like container config updates, idempotency, and many more use cases. You can take a look: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ans… The role itself is using custom filters; which are all unit tested. Some modules were created as well. More testing has to happen on these modules but they have proven to work and even improve the operator experience to cover more scenarios and improve the logging and debugging process. The role doesn't support Docker, and won't at this point; it doesn't fit with our roadmap. With that patch, any new deployment (not using Docker) will now use the new role and not Paunch anymore: https://review.opendev.org/#/c/700738/ Which means: standalone, undercloud, overcloud. If you already have a running deployment, you can either run a minor update which will update THT and roll your cloud; or you can also set EnablePaunch: False and the containers will be restarted with the new config. If for some reason, you need to disable it, here's how: - undercloud.conf: undercloud_enable_paunch = false - standalone/overcloud: EnablePaunch: False It is not supported to roll back to a Paunch deployment once your containers are managed by Ansible; if you try it, an error will raise and the deployment/update/upgrade will stop. If you find any problem, bug or have any feedback for improvement, please let me know and we'll make it better. During the following weeks, we'll watch for performances and see if it degraded or if it's better. Paunch's replacement shouldn't make it faster because Ansible has a lot of overhead; however we want to make sure that it doesn't cause issues at scale. Which is why we're going to observe that now and do more testing at scale now it's the default. I want to thank all the reviewers involved in that effort, and specially Sagi, Kevin and Alex for their great feedback. Thanks, -- Emilien Macchi

2 1

[openstack-helm][neutron] neutron-dhcp-agent stuck in Init state
by Sanjay K 21 Feb '20

21 Feb '20

Hi All, I am trying to deploy openstack services using helm charts in my k8s cluster. My cluster is having 4 nodes - 1 k8s master, 1 controller and 2 compute nodes. I am using helm v3.0.3 for openstack stein. I have already deployed the below charts in my cluster successfully in the respective order - ingress, mariadb, rabbitmq, memcached, keystone, glance and then neutron. For mariadb, rabbitmq and glance, I am using PV and PVC in my local storage while deploying. All are ok until I deploy neutron. After deployment, all neutron pods works well except *neutron-dhcp-agent-default pod *which remains in Init:0/2 state. The pod description shows it has dependencies shown as - DEPENDENCY_SERVICE: openstack:rabbitmq,openstack:neutron-server,openstack:nova-api DEPENDENCY_JOBS: neutron-rabbit-init pods state- NAME READY STATUS RESTARTS AGE glance-api-b575ff7c-qp7wt 1/1 Running 0 132m glance-bootstrap-zjtw9 0/1 Completed 0 132m glance-db-init-86xgk 0/1 Completed 0 132m glance-db-sync-z9p9l 0/1 Completed 0 132m glance-ks-endpoints-wwm59 0/3 Completed 0 132m glance-ks-service-vs92t 0/1 Completed 0 132m glance-ks-user-znn88 0/1 Completed 0 132m glance-metadefs-load-g685b 0/1 Completed 0 132m glance-rabbit-init-648w8 0/1 Completed 0 132m glance-storage-init-jdlx2 0/1 Completed 0 132m ingress-8f98f7d96-llkwl 1/1 Running 0 3h42m ingress-error-pages-84647d8fcb-6j6bz 1/1 Running 0 3h42m keystone-api-5785f4787-lz296 1/1 Running 0 154m keystone-bootstrap-6tcdx 0/1 Completed 0 154m keystone-credential-setup-d9js5 0/1 Completed 0 154m keystone-db-init-zsgbj 0/1 Completed 0 154m keystone-db-sync-z8hfk 0/1 Completed 0 154m keystone-domain-manage-nk48h 0/1 Completed 0 154m keystone-fernet-setup-4pzlj 0/1 Completed 0 154m keystone-rabbit-init-mlpsn 0/1 Completed 0 154m mariadb-ingress-669c67b6b5-w8jxb 1/1 Running 0 3h32m mariadb-ingress-error-pages-d77467d69-9njgt 1/1 Running 0 3h32m mariadb-server-0 1/1 Running 0 3h32m memcached-memcached-7b49f48865-tf6xc 1/1 Running 0 3h11m neutron-db-init-zpl87 0/1 Completed 0 109m neutron-db-sync-plsnc 0/1 Completed 0 109m neutron-dhcp-agent-default-sf76z 0/1 Init:0/2 0 109m neutron-ks-endpoints-jgmdv 0/3 Completed 0 109m neutron-ks-service-nr5w2 0/1 Completed 0 109m neutron-ks-user-lw4p4 0/1 Completed 0 109m neutron-l3-agent-default-lnzk8 1/1 Running 0 109m neutron-lb-agent-default-44v9x 1/1 Running 0 109m neutron-lb-agent-default-94xw4 1/1 Running 0 109m neutron-lb-agent-default-whms2 1/1 Running 0 109m neutron-metadata-agent-default-7rbrh 1/1 Running 0 109m neutron-rabbit-init-qkhqm 0/1 Completed 0 109m neutron-server-964fcffcb-gvv4k 1/1 Running 0 109m rabbitmq-cluster-wait-vjbzq 0/1 Completed 0 3h18m rabbitmq-rabbitmq-0 1/1 Running 0 3h18m Please guide me how to resolve this neutron-dhcp-agent pod so that I can deploy other services after that. I have plan to deploy libvirt, nova, cinder, horizon and ceilometer charts into my cluster. thank you for your help and support. best regards, Sanjay

3 2

[all][dev] reno linting anyone?
by Radosław Piliszek 21 Feb '20

21 Feb '20

Hello fellow OpenStackers! I wonder whether any project applies any linting to release notes? How do you maintain quality of release notes? -yoctozepto

2 1

Re: [infra][neutron] Removing networking-calico from OpenStack governance
by David Comay 21 Feb '20

21 Feb '20

Neil, > networking-calico is the code that integrates Project Calico [1] with > Neutron. It has been an OpenStack project for several years, but we, i.e. > its developers [2], would like now to remove it from OpenStack governance > and instead manage it like the other Project Calico projects under > https://github.com/projectcalico/. My primary concern which isn't really governance would be around making sure the components in `networking-calico` are kept in-sync with the parent classes it inherits from Neutron itself. Is there a plan to keep these in-sync together going forward?

5 8

[all] Proposed release schedule for Victoria
by Sean McGinnis 21 Feb '20

21 Feb '20

Hello everyone, Looking for feedback and increasing awareness for a proposed release schedule for Victoria: https://review.opendev.org/#/c/708471/ Mainly, looking for any feedback on any conflicts with major holidays, external factors, and other things that could potentially be an issue for this schedule. Until the logs are culled, here is a convenient link to see what the schedule looks like: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/… Here are the key dates for the schedule: May 13: Ussuri release June 8-12: OpenDev+PTG June 18: Victoria milestone 1 July 30: Victoria milestone 2 Sept 3: Final non-client lib releases Sept 10: Victoria milestone 3, feature freeze, final lib releases Sept 24: RC1 deadline Oct 8: Final RCs Oct 14: Victoria release Oct 19-23: Open Infrastructure Summit Part of the reason for the slightly shorter schedule was to wrap up prior to the Summit. Another version was looked at to extend past the Summit, but given the various deadlines and crunch that happens around that time, that ended up needed to be extended too far out to avoid conflicts. Please comment on the patch, or reply here, if it looks like there would be any major issues with this release schedule. Thanks! Sean

1 0

[nova][sfe] Support re-configure deleted_on_termination in server
by Brin Zhang(张百林) 21 Feb '20

21 Feb '20

Hi, nova: We would like to have a spec freeze exception for the spec: Support re-configure deleted_on_termination in server [1], and it’s PoC code in [2] I will attend the nova meeting on February 20 2020 1400 UTC as much as possible. [1] https://review.opendev.org/#/c/580336/ [2] https://review.opendev.org/#/c/693828/ brinzhang

2 2

[neutron] networking-bagpipe gate failure
by Akihiro Motoki 21 Feb '20

21 Feb '20

Hi, networking-bagpipe gate is broken now due to its dependencies. The situation is complicated so I am summarizing it and exploring the right solution. # what happens now Examples of the gate failure are [1] and [2], and the exact failure is found at [3]. It fails due to horizon dependency from networking-bgpvpn train release (horizon>=14.0.0<17.0.0) and the upper-constraints.txt master (horizon==18.0.0). The neutron team has not released a beta for ussuri, so requirements.txt tries to install networking-bgpvpn train which has capping of horizon version. The capping of horizon in networking-bgpvpn was introduced in [6] and we cut a release so it started to cause the failure like this. We've explored several workarounds to avoid it including specifying horizon in networking-bagpipe, specify horizon in required-projects in networking-bagpipe and dropping networking-bgpvpn in requirements.txt in networking-bagpipe, but all of them do not work. # possible solutions I am thinking two options. The one is to cut a beta release in neutron stadium for ussuri. The other is to uncap horizon in networking-bgpvpn train and release it. I believe both work but the first one would be better as it is time to release beta for Ussuri. Discussing it in the IRC, we are planning to release beta soon. (ovn-octavia-provider is also waiting for a beta release of neutron.) # Side notes Capping dependencies in stable branches is not what we usually do. Why we don't do this was discussed in the mailing list thread [4] and it is highlighted in [5]. Thanks, Akihiro Motoki (irc: amotoki) [1] https://review.opendev.org/#/c/708829/ [2] https://review.opendev.org/#/c/703949/ [3] https://zuul.opendev.org/t/openstack/build/3bc82305168b4d8cad7e4964c7207e00… [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/thread… [5] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011283… [6] https://review.opendev.org/#/c/699456/

4 3

[cinder] ussuri virtual mid-cycle part 2 poll
by Brian Rosmaita 20 Feb '20

20 Feb '20

The second session of the Cinder virtual mid-cycle will be held the week of 16 March. As with part 1, the length will be 2 hours. There's a poll up to determine a suitable day and time that week: https://doodle.com/poll/eme483iv2faupn6z Please fill out the poll as soon as you can. If all the times are horrible for you, please suggest an alternative in a comment on the poll. The poll closes at 21:00 UTC on Wednesday 26 February 2020. cheers, brian

1 0

[security] Vulnerability Management Policy Changes
by Gage Hugo 20 Feb '20

20 Feb '20

The Vulnerability Management Team (VMT) has recently updated the vulnerability:managed policy[0] here, the key points are: - Softened our #5 policy from a hard requirement to a recommendation - Clarified that the VMT does not track external software components - Defined that a project must tag releases to qualify for VMT oversight, and that the VMT only deals with vulnerabilities in real releases (not pre-releases, release candidates, milestones...) - Private embargo's shall not last more than 90 days, except under unusual circumstances With the VMT policy changes[0] merged, we have also updated the VMT process document[1] to match. The biggest change to note is the new 90 day embargo limit: "If a report is held in embargo for 90 days without a fix, or significant details of the report are disclosed in a public venue, the embargo is terminated by a VMT coordinator at that time and subsequent process switches to the public report workflow instead." We'll be updating all current private reports to let participants know that there is a 90-day deadline (from when we update the report) to make those reports public. [0] https://review.opendev.org/#/c/678426/ [1] https://security.openstack.org/vmt-process.html

1 0

[tripleo][rdo][rdo-dev] Status of RDO Trunk Ussuri on CentOS 7 and transition to CentOS 8
by Alfredo Moralejo Alonso 20 Feb '20

20 Feb '20

Hi, I'd like to open a discussion about the status of RDO Ussuri repositories on CentOS7. As you know RDO and upstream teams (kolla, puppet-openstack, TripleO, TripleO CI, etc...) have been working to switch to CentOS8 during last few weeks. In order to make the transition easier from CentOS 7 to CentOS 8, RDO is still maintaining Trunk repos consistent for both CentOS 7/Python 2 and CentOS 8/Python 3. As OpenStack projects have been dropping support for P ython 2, we've started pinning them to the last commit working with Python 2[1], we were expecting that transition will finish soon but it's still going on. Over time, the number of pinned packages has been growing including services and Oslo libraries where we can't follow upper-constraints anymore [2]. Recently, Kolla has removed support for CentOS 7 so i doubt it makes sense to keep pinning packages to keep RDO Trunk consistent artificially and continue running promotion pipelines on a repo with so many outdated packages. Also, pinning these projects makes that changes needed for CentOS 8 will not be in RDO and would need to be backported manually to each package. My proposal is: - Unpin all packages in Ussuri to follow master trunk, or versions in upper -constraints (for clients and libraries). - RDO Ussuri on CentOS 7 repo consistent link will not move anymore (so no more promotions based on it). - We will keep running centos7-master DLRN builder, so that packages still builing with Python 2 will be available in current repo [3] to be used by teams needing them until migration to CentOS 8 is finished everywhere. - Projects which already have CentOS 8 jobs gating in master branch can remove CentOS 7 ones. We understand this can add some pressure on moving to CentOS8 to the teams working on it, but I'd say it's already a priority and it's justified at this stage. What do you think about this plan?, is there any reason to keep CentOS 7 artificially consistent and promoting at this point of the transition to CentOS 8? Best regards, Alfredo [1] https://review.rdoproject.org/r/#/q/topic:pin-py2 [2] https://review.rdoproject.org/r/#/c/24796/ [3] http://trunk.rdoproject.org/centos7-master/current

1 0

[manila] Core Team additions
by Goutham Pacha Ravi 20 Feb '20

20 Feb '20

Hello Zorillas/Stackers, I'd like to make propose some core team additions. Earlier in the cycle [1], I sought contributors who are interested in helping us maintain and grow manila. I'm happy to report that we had an fairly enthusiastic response. I'd like to propose two individuals who have stepped up to join the core maintainers team. Bear with me as I seek to support my proposals with my personal notes of endorsement: Victoria Martinez de la Cruz - Victoria has been contributing to Manila since it's inception. She has played various roles during this time and has contributed in significant ways to build this community. She's been the go-to person to seek reviews and collaborate on for CephFS integration, python-manilaclient, manila-ui maintenance and support for the OpenStack client. She has also brought onboard and mentored multiple interns on the team (Fun fact: She was recognized as Mentor of Mentors [2] by this community). It gives me great joy that she agreed to help maintain the project as a core maintainer. Carlos Eduardo - Carlos has made significant contributions to Manila for the past two releases. He worked on several feature gaps with the DHSS=True driver mode, and is now working on graduating experimental features that the project has been building since the Newton release. He performs meaningful reviews that drive good design discussions. I am happy to note that he needed little mentoring to start reviewing the OpenStack Way [3] - this is a dead give away to me to spot a dedicated maintainer who cares about growing the community, along with the project. Please give me your +/- 1s for this proposal. Thank you, Goutham Pacha Ravi (gouthamr) [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/009910.… [2] https://superuser.openstack.org/articles/openstack-community-contributor-aw… [3] https://docs.openstack.org/project-team-guide/review-the-openstack-way.html

7 7

[manila][stable] Core Team removals -
by Goutham Pacha Ravi 20 Feb '20

20 Feb '20

Hello Zorillas / Stackers, We sadly bid adieu to two of our core maintainers in Ussuri: Ben Swartzlander and Zhong Jun. Both of them are busy with other projects at the moment, and are able to continue to participate in this community, albeit without core maintenance responsibility. We thanked these awesome individuals in our community meeting today [1] and would love to have them back should they decide to get involved again. I would like help from the stable maintenance team to remove Ben and Zhong jun from the stable core team [2]. Thanks, Goutham Pacha Ravi (gouthamr) [1] http://eavesdrop.openstack.org/meetings/manila/2020/manila.2020-02-20-15.00… [2] https://review.opendev.org/#/admin/groups/1099,members

1 0

[nova][sfe] Support volume local cache
by Fang, Liang A 20 Feb '20

20 Feb '20

Hi We would like to have a spec freeze exception for the spec: Support volume local cache [1]. This is part of cross project contribution, with another spec in cinder [2]. I will attend the Nova meeting on February 20 2020 1400 UTC. [1] https://review.opendev.org/#/c/689070/ [2] https://review.opendev.org/#/c/684556/ Regards Liang

2 1

[nova] Ussuri feature scrub
by Eric Fried 20 Feb '20

20 Feb '20

Nova maintainers and contributors- { Please refer to this ML thread [1][2] for background. } Now that spec freeze has passed, I would like to assess the Design:Approved blueprints and understand how many we could reasonably expect to land in Ussuri. We completed 25 blueprints in Train. However, mriedem is gone, and it is likely that I will drop off the radar after Q1. Obviously all blueprints/releases/reviews/etc. are not created equal, but using stackalytics review numbers as a rough heuristic, I expect about 20 blueprints to get completed in Ussuri. If we figure that 5-ish of the incompletes will be due to factors beyond our control, that would mean we should Direction:Approve about 25. As of this writing: - 30 blueprints are targeted for ussuri [3]. Of these, - 7 are already implemented. Of the remaining 23, - 2 are not yet Design:Approved. These will need an exception if they are to proceed. And - 19 (including the unapproved ones) have code in various stages. I would like to see us cut 5-ish of the 30. I have made an etherpad [4] with the unimplemented blueprints listed with owners and code links. I made notes on some of the ones I would like to see prioritized, and a couple on which I'm more meh. If you have a stake in Nova/Ussuri, I encourage you to weigh in. How will we ultimately decide? Will we actually cut anything? I don't have the answers yet. Let's go through this exercise and see if anything obvious falls out, and then we can figure out the next steps. Thanks, efried [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-September/00983… [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/thread.… [3] https://blueprints.launchpad.net/nova/ussuri [4] https://etherpad.openstack.org/p/nova-ussuri-planning

2 2

[neutron]After update neutron-lbaas, How to not restart neutron-server to let new neutron-lbaas code work?
by Zhengyu Pan 20 Feb '20

20 Feb '20

Hi, I modified neutron-lbaas load balancer code to support our special need. I have updated neutron-lbaas code in the controller node. But i don't want to restart neutron-server in case the customer's business was interrupted. How can i let netron-lbaas code work in the case of not restarting neutrn-sever in controller node? -- Best! ZhengYu！

1 0

[all][neutron][neutron-fwaas] Maintainers needed
by Slawek Kaplonski 20 Feb '20

20 Feb '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for neutron-fwaas project were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the neutron-fwaas code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn’t mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark neutron-fwaas as deprecated and in “V” cycle we will propose to move the project from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

8 14

[nova] context being introduced to nova.virt.driver.ComputeDriver.extend_volume
by Lee Yarwood 20 Feb '20

20 Feb '20

Hello all, I'm looking to introduce the request context to the signature of extend_volume in the following change: virt: Pass request context to extend_volume https://review.opendev.org/#/c/706899/ Any out of tree drivers implementing extend_volume will also need to add the request context to their implementations once this lands. This is part of a wider bugfix series below: https://review.opendev.org/#/q/topic:bug/1861071 Also discussed on the ML in the following thread: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012551… If there are any concerns or issues with this then please let me know! Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76

1 0

[kolla-ansible] External Ceph keyring encryption
by Jason Anderson 20 Feb '20

20 Feb '20

Hi all, My understanding is that KA has dropped support for provisioning Ceph directly, and now requires an external Ceph cluster (side note: we should update the docs[1], which state it is only "sometimes necessary" to use an external cluster--I will try to submit something today). I think this works well, but the handling of keyrings cuts a bit against the grain of KA. The keyring files must be dropped in to the node_custom_config directory. This means that operators who prefer to keep their KA configuration in source control must have some mechanism for securing that, as it is unencrypted. What does everybody think about storing Ceph keyring secrets in passwords.yml instead, similar to how SSH keys are handled? Thanks, /Jason [1]: https://docs.openstack.org/kolla-ansible/latest/reference/storage/external-… -- Jason Anderson Chameleon DevOps Lead Consortium for Advanced Science and Engineering, The University of Chicago Mathematics & Computer Science Division, Argonne National Laboratory

4 6

VPNaaS with multiple endpoint groups
by Eugen Block 20 Feb '20

20 Feb '20

Hi all, is anyone here able to help with a vpn issue? It's not really my strong suit but I'll try to explain. In a Rocky environment (using openvswitch) a customer has setup a VPN service successfully, but that only seems to work if there's only one local and one peer endpoint group. According to the docs it should work with multiple endpoint groups, as far as I could tell the setup looks fine and matches the docs (don't create the subnet when creating the vpn service but use said endpoint groups). What we're seeing is that as soon as the vpn site connection is created with multiple endpoints only one of the destination IPs is reachable. And it seems as if it's always the first in the list of EPs (see below). This seems to be reflected in the iptables where we also only see one of the required IP ranges. Also neutron reports duplicate rules if we try to use both EPs: 2020-02-12 14:14:27.638 16275 WARNING neutron.agent.linux.iptables_manager [req-92ff6f06-3a92-4daa-aeea-9c02dc9a31c3 ba9bf239530d461baea2f6f60bd301e6 850dad648ce94dbaa5c0ea2fb450bbda - - -] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A neutron-l3-agent-POSTROUTING -s X.X.252.0/24 -d Y.Y.0.0/16 -m policy --dir out --pol ipsec -j ACCEPT These are the configured endpoints: root@control:~ # openstack vpn endpoint group list +--------------------------------------+-----------------+--------+------------------------------------------------------------------------------------+ | ID | Name | Type | Endpoints | +--------------------------------------+-----------------+--------+------------------------------------------------------------------------------------+ | 0f853567-e4bf-4019-9290-4cd9f94a9793 | peer-ep-group-1 | cidr | [u'X.X.253.0/24'] | | 152a0f9e-ce49-4769-94f1-bc0bebedd3ec | peer-ep-group-2 | cidr | [u'X.X.253.0/24', u'Y.Y.0.0/16'] | | 791ab8ef-e150-4ba0-ac2c-c044659f509e | local-ep-group1 | subnet | [u'38efad5e-0f1e-4e36-8995-74a611bfef41'] | | 810b0bf2-d258-459b-9b57-ae5b491ea612 | local-ep-group2 | subnet | [u'38efad5e-0f1e-4e36-8995-74a611bfef41', u'9e35d80f-029e-4cc1-a30b-1753f7683e16'] | | b5c79e08-41e4-441c-9ed3-9b02c2654173 | peer-ep-group-3 | cidr | [u'Y.Y.0.0/16'] | +--------------------------------------+-----------------+--------+------------------------------------------------------------------------------------+ Has anyone experience with this and could help me out? Another follow-up question: how can we gather some information regarding the ipsec status? Althoug there is a active tunnel we don't see anything with 'ipsec statusall', I've checked all namespaces on the control node. Any help is highly appreciated! Best regards, Eugen

1 1

[ironic]: Failed to create config drive on disk
by fsbiz＠yahoo.com 19 Feb '20

19 Feb '20

Hi, I'd very much appreciate some help on this. We have a medium-large ironic installation where the baremetal nodes are constantly doing provisionings and deprovisionings.We test a variety of images (Ubuntu, RedHat, Windows, etc.) in both BIOS and UEFI modes. Our approach so far is to configure all the baremetal nodes in CSM UEFI mode so that both BIOS and UEFI images can be run. And things have worked fairly well with this. Lately, I'm having this weird "Failed to create config drive on disk" issue and it is happening only with BIOS images on certain baremetal nodes. Here are the important snippets from the ironic conductor and the IPA that I've managed to narrow down. ================================Ironic conductor (before power-on)2020-02-18 13:06:33.261 DEBUG iDeploy boot mode is uefi for ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 ================================Ironic conductor (power-on)2020-02-18 13:07:18.541 INFO Successfully set node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 power state to power on by rebooting. 020-02-18 13:07:18.560 INFO Node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 moved to provision state "wait call-back" from state "deploying"; target provision state is "active" ================================== Ironic Python Agent: (I can provide the full log on request. Only relevant logs provided here for the sake of brevity). wipefs --force --all Feb 18 21:08:17 ironic-python-agent: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): wipefs --force --all /dev/sda Feb 18 21:08:17 ironic-python-agent: CMD "wipefs --force --all /dev/sda" returned: 0 in 0.023s Feb 18 21:08:17 ironic-python-agentt: Execution completed, command line is "wipefs --force --all /dev/sda" Feb 18 21:08:17 ironic-python-agent: Command stdout is: "" execute /usr/share/ironic-python-agent/venv/lib/python2.7/site-packages/ironic_lib/utils.py:103Feb 18 21:08:17 ironic-python-agent: Command stderr is: "" execute /usr/share/ironic-python-agent/venv/lib/python2.7/site-packages/ironic_lib/utils.py:104 sgdisk -ZFeb 18 21:08:17 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:17.304 15063 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sgdisk -Z /dev/sda Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.321 15063 DEBUG oslo_concurrency.processutils [-] CMD "sgdisk -Z /dev/sda" returned: 0 in 1.017s Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.323 15063 DEBUG ironic_lib.utils [-] Execution completed, command line is "sgdisk -Z /dev/sda" Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.323 15063 DEBUG ironic_lib.utils [-] Command stdout is: "Creating new GPT entries in memory.Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: GPT data structures destroyed! You may now partition the disk using fdisk or other utilities. fuser /dev/sda Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.326 15063 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): fuser /dev/sda Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG oslo_concurrency.processutils [-] CMD "fuser /dev/sda" returned: 0 in 0.006s Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG ironic_lib.utils [-] Execution completed, command line is "fuser /dev/sda"Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG ironic_lib.utils [-] Command stdout is: " 15221" Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG ironic_lib.utils [-] Command stderr is: "/dev/sda: fuser /dev/sda Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.324 15063 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): fuser /dev/sda Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 DEBUG oslo_concurrency.processutils [-] CMD "fuser /dev/sda" returned: 1 in 0.012s Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 DEBUG ironic_lib.utils [-] Execution completed, command line is "fuser /dev/sda" Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 INFO ironic_lib.disk_utils [-] Disk metadata on /dev/sda successfully destroyed for node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 start iscsi:Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 DEBUG ironic_python_agent.extensions.iscsi [-] Starting ISCSI target with iqn iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 on device /dev/sda start_iscsi_targetFeb 18 21:08:19 host-10-33-23-71 kernel: Rounding down aligned max_sectors from 4294967295 to 4294967288Feb 18 21:08:19 host-10-33-23-71 kernel: db_root: cannot open: /etc/targetFeb 18 21:08:19 host-10-33-23-71 WARNING ironic_python_agent.extensions.iscsi [-] Linux-IO is not available, falling back to TGT. Error: Cannot set dbroot to /etc/target. Please check if this directory exists..: RTSLibError: Cannot set dbroot to /etc/target. Please check if this directory exists.Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tgtd Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] CMD "tgtadm --lld iscsi --mode target --op show" returned: 0 in 0.002s Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Execution completed, command line is "tgtadm --lld iscsi --mode target --op show" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stdout is: "" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stderr is: "" Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] CMD "tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97" returned: 0 in 0.002s Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Execution completed, command line is "tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stdout is: "" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stderr is: ""Feb 18 21:08:19 host-10-33-23-71 INFO root [-] Command iscsi.start_iscsi_target completed: Command name: start_iscsi_target, params: {u'wipe_disk_metadata': True, u'iqn': u'iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97', u'portal_port': 3260}, status: SUCCEEDED, result: {'iscsi_target_iqn': u'iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97'}.Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: ::ffff:10.33.24.87 - - [18/Feb/2020 21:08:19] "POST /v1/commands?wait=true HTTP/1.1" 200 386 ====================================Back to Ironic conductor020-02-18 13:08:25.953 346408 DEBUG RPC heartbeat called for node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 heartbeat 2020-02-18 13:08:25.997 346408 DEBUG Heartbeat from node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 heartbeat 2020-02-18 13:08:26.033 346408 Node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 moved to provision state "deploying" from state "wait call-back"; target provision state is "active" Starting iscsi:2020-02-18 13:08:28.385 Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.33.23.71:3260 -T iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 --login 2020-02-18 13:08:28.613 CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.33.23.71:3260 -T iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 --login" returned: 0 in 0.229s 2020-02-18 13:08:28.615 Execution completed, command line is "iscsiadm -m node -p 10.33.23.71:3260 -T iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 --login" 2020-02-18 13:08:28.615 Command stdout is: "Logging in to [iface: default, target: iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97, portal: 10.33.23.71,3260] (multiple) Successful login to iSCSI:Login to [iface: default, target: iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97, portal: 10.33.23.71,3260] successful. qemu-img info2020-02-18 13:08:29.005 346408 DEBUG Running cmd (subprocess): /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 -- env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk 2020-02-18 13:08:29.072 346408 DEBUG CMD "/usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 -- env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk" returned: 0 in 0.067s 2020-02-18 13:08:29.073 346408 DEBUG Execution completed, command line is "env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk" 2020-02-18 13:08:29.074 346408 DEBUG Command stdout is: "image: /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/diskfile format: rawvirtual size: 9.8G (10485760000 bytes)disk size: 5.4G copying image via dd: 2020-02-18 13:08:29.075 Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf dd if=/var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk of=/dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 bs=1M oflag=direct 2020-02-18 13:08:49.020 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf dd if=/var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk of=/dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 bs=1M oflag=direct" returned: 0 in 19.945s 2020-02-18 13:08:49.021 346408 DEBUG Execution completed, command line is "dd if=/var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk of=/dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 bs=1M oflag=direct" 2020-02-18 13:08:49.022 346408 DEBUG Command stdout is: "" execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:1032020-02-18 13:08:49.022 346408 DEBUG Command stderr is: "10000+0 records in 10000+0 records out 10485760000 bytes (10 GB) copied, 19.7712 s, 530 MB/s partprobe2020-02-18 13:08:49.023 346408 DEBUGRunning cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:03.341 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 14.318s 2020-02-18 13:09:03.341 346408 DEBUG Execution completed, command line is "partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:03.342 346408 DEBUG Command stdout is: "" 2020-02-18 13:09:03.342 346408 DEBUG Command stderr is: "" lsblk:2020-02-18 13:09:03.343 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf lsblk -Po name,label /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:03.519 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf lsblk -Po name,label /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.176s 2020-02-18 13:09:03.519 346408 DEBUG Execution completed, command line is "lsblk -Po name,label /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:03.519 346408 DEBUG Command stdout is: "NAME="sdb" LABEL="" NAME="sdb1" LABEL="" " execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:1032020-02-18 13:09:03.520 346408 DEBUG iCommand stderr is: "" Adding configDrive partition:2020-02-18 13:09:03.772 346408 DEBUG Adding config drive partition 64 MiB to device: /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 for node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 create_config_drive_partition blkid2020-02-18 13:09:03.773 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:03.959 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.186s 2020-02-18 13:09:03.960 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Execution completed, command line is "blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:03.960 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stdout is: "dos 2020-02-18 13:09:03.960 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" parted2020-02-18 13:09:03.961 346408 DEBUG oRunning cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:3722020-02-18 13:09:04.136 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" returned: 0 in 0.175s 2020-02-18 13:09:04.137 346408 DEBUG Execution completed, command line is "parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" 2020-02-18 13:09:04.137 346408 DEBUG Command stdout is: "BYT;/dev/sdb:953870MiB:scsi:512:512:msdos:IET VIRTUAL-DISK:;1:1.00MiB:9999MiB:9998MiB:ext4::boot; blkid2020-02-18 13:09:04.138 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:04.321 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.182s 2020-02-18 13:09:04.321 346408 DEBUG iExecution completed, command line is "blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:04.321 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stdout is: "dos2020-02-18 13:09:04.322 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" partprobe2020-02-18 13:09:04.322 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe -d -s /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:04.510 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe -d -s /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.188s 2020-02-18 13:09:04.511 346408 DEBUG Execution completed, command line is "partprobe -d -s /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:04.512 346408 DEBUG Command stdout is: "/dev/sdb: msdos partitions 1 " 2020-02-18 13:09:04.512 346408 DEBUG Command stderr is: "" blockdev --getsize642020-02-18 13:09:04.513 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf blockdev --getsize64 /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:05.045 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf blockdev --getsize64 /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.532s 2020-02-18 13:09:05.046 346408 DEBUG Execution completed, command line is "blockdev --getsize64 /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:05.046 346408 DEBUG Command stdout is: "1000204886016" 2020-02-18 13:09:05.046 346408 DEBUG Command stderr is: "" parted/mkpart2020-02-18 13:09:05.047 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s -- /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 mkpart primary fat32 -64MiB -0 2020-02-18 13:09:05.241 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s -- /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 mkpart primary fat32 -64MiB -0" returned: 0 in 0.194s 2020-02-18 13:09:05.241 346408 DEBUG Execution completed, command line is "parted -a optimal -s -- /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 mkpart primary fat32 -64MiB -0" 2020-02-18 13:09:05.242 346408 DEBUG Command stdout is: "" 2020-02-18 13:09:05.242 346408 DEBUG Command stderr is: "Warning: The resulting partition is not properly aligned for best performance. partprobe2020-02-18 13:09:05.329 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:05.516 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.187s2020-02-18 13:09:05.517 346408 DEBUG Execution completed, command line is "partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:05.517 346408 DEBUG Command stdout is: "" 2020-02-18 13:09:05.518 346408 DEBUG Command stderr is: "" sgdisk -v: IS THERE ANY ISSUE WITH THE BELOW OUTPUT2020-02-18 13:09:05.518 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf sgdisk -v /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:05.707 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf sgdisk -v /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.189s 2020-02-18 13:09:05.708 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Execution completed, command line is "sgdisk -v /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:101 2020-02-18 13:09:05.708 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stdout is: "***************************************************************Found invalid GPT and valid MBR; converting MBR to GPT formatin memory.***************************************************************Warning! Secondary partition table overlaps the last partition by33 blocks!You will need to delete this partition or resize it in another utility.Warning! Secondary partition table overlaps the last partition by33 blocks!You will need to delete this partition or resize it in another utility.Identified 1 problems! 2020-02-18 13:09:05.709 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" parted2020-02-18 13:09:05.709 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print 2020-02-18 13:09:05.899 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" returned: 0 in 0.189s 2020-02-18 13:09:05.899 346408 DEBUG Execution completed, command line is "parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" 2020-02-18 13:09:05.900 346408 DEBUG Command stdout is: "BYT;/dev/sdb:953870MiB:scsi:512:512:msdos:IET VIRTUAL-DISK:;1:1.00MiB:9999MiB:9998MiB:ext4::boot;2:953806MiB:953870MiB:64.0MiB:::lba;" execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:1032020-02-18 13:09:05.900 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" 2020-02-18 13:09:05.927 346408 DEBUG Waiting for the config drive partition /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1-part2 on node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 to be ready for writing. create_config_drive_partition /usr/lib/python2.7/site-packages/ironic_lib/disk_utils.py:8692020-02-18 13:09:05.927 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Running cmd (subprocess): test -e /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1-part2 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:3722020-02-18 13:09:05.945 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] u'test -e /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1-part2' failed. Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:461 The code retries several times followed by the config drive failure error.Again, this happens on a few nodes only and happens only when I try to run BIOS based images. UEFI based images provision just fine. Any help will be appreciated.thanks,Fred.

1 1

[oslo] Core reviewer changes
by Stephen Finucane 19 Feb '20

19 Feb '20

Hi all, Just an update on some recent changes that have been made to oslo-core after discussion among current, active cores. Added: scmcginnis Removed: ChangBo Guo(gcb) Davanum Srinivas (dims) Flavio Percoco Joshua Harlow Mehdi Abaakouk (sileht) Michael Still Victor Stinner lifeless Please let me know if anyone has any concerns about these changes. Cheers, Stephen

5 4