April 2020 - openstack-discuss - lists.openstack.org

[nova] [cinder] [ceph] Questioning deployment choices
by Radosław Piliszek 22 Apr '20

22 Apr '20

Hi Fellow OpenStackers! I have a question regarding nova-cinder-ceph deployment configuration. Most notably ephemeral storage. So - if I configure Nova to use Ceph as ephemeral storage and Cinder's only backend is Ceph as well then how does ephemeral storage differ from persistent storage now except for interface from Cinder rather than Nova (and all that comes with that) and a different pool? Was Nova's Ceph integration added just to be able to avoid Cinder in certain scenarios? Other Cinder backends did not get support from Nova. Looking forward to your insights. -yoctozepto

3 5

[kolla][tripleo][nova] nova scheduler startup errors in Train on CentOS 8
by Mark Goddard 22 Apr '20

22 Apr '20

Hi, Our (kolla-ansible) CentOS 8 binary (RDO) deploy jobs have been failing on stable/train since added. Because the source jobs were passing, I was kind of hoping it was a packaging issue that would right itself, but no such luck yet. The majority of the job succeeds, but we grep our logs for CRITICAL messages, and nova-scheduler has a few when it starts up that cause us to fail: 2020-04-21 08:59:30.125 24 CRITICAL nova [req-86f6dad1-25f2-4c77-9d7d-ac7422801b4e - - - - -] Unhandled error: RuntimeError: dictionary changed size during iteration 2020-04-21 08:59:30.125 24 ERROR nova Traceback (most recent call last): 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/bin/nova-scheduler", line 10, in <module> 2020-04-21 08:59:30.125 24 ERROR nova sys.exit(main()) 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/nova/cmd/scheduler.py", line 53, in main 2020-04-21 08:59:30.125 24 ERROR nova service.serve(server, workers=workers) 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/nova/service.py", line 454, in serve 2020-04-21 08:59:30.125 24 ERROR nova restart_method='mutate') 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 842, in launch 2020-04-21 08:59:30.125 24 ERROR nova launcher.launch_service(service, workers=workers) 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 606, in launch_service 2020-04-21 08:59:30.125 24 ERROR nova self._start_child(wrap) 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 575, in _start_child 2020-04-21 08:59:30.125 24 ERROR nova self.launcher.restart() 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 311, in restart 2020-04-21 08:59:30.125 24 ERROR nova self.conf.mutate_config_files() 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_config/cfg.py", line 3013, in mutate_config_files 2020-04-21 08:59:30.125 24 ERROR nova self._warn_immutability() 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_config/cfg.py", line 3040, in _warn_immutability 2020-04-21 08:59:30.125 24 ERROR nova for info, group in self._all_opt_infos(): 2020-04-21 08:59:30.125 24 ERROR nova File "/usr/lib/python3.6/site-packages/oslo_config/cfg.py", line 2502, in _all_opt_infos 2020-04-21 08:59:30.125 24 ERROR nova for info in self._opts.values(): 2020-04-21 08:59:30.125 24 ERROR nova RuntimeError: dictionary changed size during iteration https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/… The service does appear to start up successfully. It doesn't happen on master. Has anyone seen this before? Cheers, Mark

3 4

[cinder] nominate Luigi Toscano for cinder-tempest-plugin-core
by Brian Rosmaita 22 Apr '20

22 Apr '20

Luigi Toscano (tosky on IRC) has been very helpful this cycle addressing testing questions, doing reviews, and adding code related to testing to the various Cinder project repositories. He's been an active participant at the weekly Cinder meetings and has been good about getting visibility on testing-related patches. I'd like to put him in a position to be able to assign priority to reviews for the cinder-tempest-plugin and approve patches to that repository; hence I am nominating him to be a member of the cinder-tempest-plugin-core team. In the absence of objections, I'll make the change before the next Cinder team meeting (Wednesday, 22 April at 1400 UTC in #openstack-meeting-alt). cheers, brian

8 8

Re: [nova][cinder][ops] question/confirmation of legacy vol attachment migration
by Brett Milford 22 Apr '20

22 Apr '20

On Tue, Dec 10, 2019 at 11:15 PM Gorka Eguileor <geguileo(a)redhat.com> wrote: > > On 06/12, Brett Milford wrote: > > On Fri, Nov 22, 2019 at 3:54 AM Matt Riedemann <mriedemos(a)gmail.com> wrote: > > > > > > On 10/17/2019 5:24 AM, Gorka Eguileor wrote: > > > > I stand by my initial recommendation, being able to update the existing > > > > attachment to add the connection information from Nova. > > > > > > OK, thanks for the input and thoughtfulness on this. I've abandoned my > > > change since I'm not going to be pushing this boulder anymore but left > > > notes in the change in case someone else wants to pick it up some day. > > > > > > Note to nova cores: this means we'll have legacy volume attachment > > > compat code around forever. > > > > > > -- > > > > > > Thanks, > > > > > > Matt > > > > > > > Hi Groka, Cinder & Nova devs, > > > > Following up this thread from the context of > > https://review.opendev.org/#/c/579004/ > > > > To summarise the discussion: > > - initialize_connection shouldn't be called more than once, as it > > may mess up some drivers. > > - To (safely) refresh connection_info a new api for cinder is required. > > Hi, > > It may be a new API or a modification of one of the existing ones. > > > > - a patch to nova, such as #579004 could make a call to this new api > > to refresh connection info on events such as reboot. > > > > Is there any other context to this issue I've missed or an alternate > > path to solving this? > > > > Does the creation of a new api for cinder have any implications for > > being able to backport this patch for nova? > > > > I don't think backporting it in Nova would do us much good, since the > Cinder code would not be backportable (it's a new feature, not a bug > fix). > > > > What would be the process for me to kick off this work? > > > > You should propose a Cinder spec [1] for your work and you may provide a > WIP patch to Cinder at the same time, though depending on the review you > may have to completely change that code later. > > Cheers, > Gorka. > > [1]: https://opendev.org/openstack/cinder-specs > > > Thanks for your help, > > Brett (bcm) > > -- > > ❯ brett > > > Cheers Matt, Groka, I thought this might be the case. -- ❯ brett

2 2

[largescale-sig] Next meeting: Apr 22, 8utc
by Thierry Carrez 22 Apr '20

22 Apr '20

Hi everyone, The Large Scale SIG will have a meeting this week on Wednesday, April 22 at 8 UTC[1] in the #openstack-meeting-3 channel on IRC: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200422T08 Feel free to add topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting A reminder of the TODOs we had from last meeting, in case you have time to make progress on them: - masahito to prepare oslo.metric POC code release - oneswig to contribute a scaling story on bare metal cluster scaling - belmoreira to submit a scaling story to https://etherpad.openstack.org/p/scaling-stories - ttx to copy scaling stories on the wiki for long-term storage - amorin propose patch against Nova doc Talk to you all on Wednesday, -- Thierry Carrez

1 1

[requirements][release][monasca] FFE request for blocking confluent-kafka 1.4.0
by Witek Bedyk 21 Apr '20

21 Apr '20

Hi, I'd like to request FFE for: https://review.opendev.org/721685 The last release of confluent-kafka includes a buggy source package [1] which block our CI. Thanks Witek [1] https://github.com/confluentinc/confluent-kafka-python/issues/830

3 2

[keystone][ptg] Virtual PTG planning
by Colleen Murphy 21 Apr '20

21 Apr '20

Hi team, As discussed in today's meeting, we're collecting input on how to structure the keystone team's virtual PTG sessions. See Kendall's email[1] for the full details of how the gathering will work. I suggest we have two 4-hour sessions during a time that works for the main keystone contributors, and perhaps one or two shorter office-hour sessions at alternate times for ad-hoc discussions or cross-project collaboration. As we refine the agenda, we may decide that we want more or less time, and we can probably adjust accordingly down the line. I've created a poll[2] with two-hour blocks to represent the available session times we could schedule. Please fill out the poll if you're a member of the team or if you have a topic to discuss with the team. Please also add your name and topics to the brainstorming etherpad[3]. Colleen [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.ht… [2] https://doodle.com/poll/8agdsvxy2ve9rr4x [3] https://etherpad.opendev.org/p/yvr-ptg-keystone

1 0

[neutron] Neutron Bug Deputy Report Apr. 13-21
by Ryan Tidwell 21 Apr '20

21 Apr '20

Hello Neutron Team, This is my bug deputy report for the previous week. It was an active week of bug reports, here are the highlights: RFE: -------- Neutron ports dns_assignment does not match the designate DNS records for Neutron port https://bugs.launchpad.net/bugs/1873091 - This could use some follow-up from someone who is acquainted with the neutron-designate integration FWaaS: ------------ The firewall group's status is active when firewall group has no any policy https://bugs.launchpad.net/bugs/1872439 - Could simply be a misunderstanding by the user, some clarification by someone with more FWaaS expertise would be helpful. The firewall group's function is failed in the dvr scene. https://bugs.launchpad.net/bugs/1872407 - On the surface this looks like a DVR-related error. Need more info from the user to be able to debug further. DVR: -------- Internal IP leak to physical interface from qrouter in DVR mode https://bugs.launchpad.net/bugs/1873761 - RST packets seem to not get NAT'ed properly, needs some investigation to understand why. Can not use vrrp in a dvr openstack environment https://bugs.launchpad.net/bugs/1873375 - The reporter references https://review.opendev.org/#/c/716302/, I have posted a follow-up question for clarity. Floating IP was not removed from the router when the port forwarding deletion was completed https://bugs.launchpad.net/bugs/1873708 - This had some initial triage, an effort has been made to reproduce the issue without success CI Issues: --------------- [stable/stein] Neutron-tempest-plugin jobs are failing due to wrong python version https://bugs.launchpad.net/bugs/1873776 - https://review.opendev.org/721277 OVN Bugs Filed: ------------------------- [OVN] External ports creation is not account for VNIC_DIRECT_PHYSICAL https://bugs.launchpad.net/bugs/1874065 - This appears to have an owner Misc. Bugs ----------------- VM can't be accessed, openflow rules are removed and neutron-ovs-agent is always reporting errors after rabbitmq is in split-brain state https://bugs.launchpad.net/bugs/1873265 - Things get into a weird state when rabbitmq gets into a split-brain scenario. Lots of discussion happening in the bug report. vpnaas pluto.pid file path wrong https://bugs.launchpad.net/bugs/1872940 Failing to terminate Windows processes https://bugs.launchpad.net/bugs/1872663 - Fix proposed here: https://review.opendev.org/719765 -Ryan

1 0

[oslo] Virtual PTG etherpad
by Ben Nemec 21 Apr '20

21 Apr '20

As discussed in the meeting this week, I've created an etherpad[0] where people can propose topics for a virtual PTG. If you have anything you'd like to discuss please add it there so we can decide whether to meet. Thanks. -Ben 0: https://etherpad.opendev.org/p/oslo-victoria-topics

1 0

[puppet] broken CI for stable/pike and stable/ocata
by Takashi Kajinami 21 Apr '20

21 Apr '20

Hi All, While I'm updating job configurations in each branch to avoid unexpected conflicts between them[1], I noticed that currently CI for stable/pike and stable/ocata is broken. Pike: https://review.opendev.org/#/c/718725/ Ocata: https://review.opendev.org/#/c/718726/ While I guess we need to update the job script to fix the issue, May I know whether anybody still has an interest in maintaining these branches? If nobody is interested in keeping these branches, I'd propose to make these old branches eol, so that we can reduce the number of open branches. I'd appreciate any feedback about this topic. [1] https://bugs.launchpad.net/puppet-openstack-integration/+bug/1871866 Thank you, Takashi -- ---------- Takashi Kajinami Senior Software Maintenance Engineer Customer Experience and Engagement Red Hat e-mail: tkajinam(a)redhat.com

3 4