November 2020 - openstack-discuss

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[nova][dev] Revisiting qemu emulation where guest arch != host arch
by Apsey, Christopher 05 Jan '22

05 Jan '22

All, A few years ago I asked a question[1] about why nova, when given a hw_architecture property from glance for an image, would not end up using the correct qemu-system-xx binary when starting the guest process on a compute node if that compute nodes architecture did not match the proposed guest architecture. As an example, if we had all x86 hosts, but wanted to run an emulated ppc guest, we should be able to do that given that at least one compute node had qemu-system-ppc already installed and libvirt was successfully reporting that as a supported architecture to nova. It seemed like a heavy lift at the time, so it was put on the back burner. I am now in a position to fund a contract developer to make this happen, so the question is: would this be a useful blueprint that would potentially be accepted? Most of the time when people want to run an emulated guest they would just nest it inside of an already running guest of the native architecture, but that severely limits observability and the task of managing any more than a handful of instances in this manner quickly becomes a tangled nightmare of networking, etc. I see real benefit in allowing this scenario to run natively so all of the tooling that exists for fleet management 'just works'. This would also be a significant differentiator for OpenStack as a whole. Thoughts? [1] http://lists.openstack.org/pipermail/openstack-operators/2018-August/015653… Chris Apsey Director | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 https://www.gacybercenter.org

2 2

Help with eventlet 0.26.1 and dnspython >= 2
by Thomas Goirand 04 Oct '21

04 Oct '21

Hi, As you may know, eventlet is incompatible with dnspython >= 2.0.0.0rc1. See [1] for the details. However, Debian unstable has 2.0.0. Would there be some good soul willing to help me fix this situation? I would need a patch to fix this, but I'm really not sure how to start. Cheers, Thomas Goirand (zigo)

8 15

[simplification] Making ask.openstack.org read-only
by Thierry Carrez 27 May '21

27 May '21

Hi everyone, This has been discussed several times on this mailing list in the past, but we never got to actually pull the plug. Ask.openstack.org was launched in 2013. The reason for hosting our own setup was to be able to support multiple languages, while StackOverflow rejected our proposal to have our own openstack-branded StackExchange site. The Chinese ask.o.o side never really took off. The English side also never really worked perfectly (like email alerts are hopelessly broken), but we figured it would get better with time if a big community formed around it. Fast-forward to 2020 and the instance is lacking volunteers to help run it, while the code (and our customization of it) has become more complicated to maintain. It regularly fails one way or another, and questions there often go unanswered, making us look bad. Of the top 30 users, most have abandoned the platform since 2017, leaving only Bernd Bausch actively engaging and helping moderate questions lately. We have called for volunteers several times, but the offers for help never really materialized. At the same time, people are asking OpenStack questions on StackOverflow, and sometimes getting answers there[1]. The fragmentation of the "questions" space is not helping users getting good answers. I think it's time to pull the plug, make ask.openstack.org read-only (so that links to old answers are not lost) and redirect users to the mailing-list and the "OpenStack" tag on StackOverflow. I picked StackOverflow since it seems to have the most openstack questions (2,574 on SO, 76 on SuperUser and 430 on ServerFault). We discussed that option several times, but I now proposed a change to actually make it happen: https://review.opendev.org/#/c/746497/ It's always a difficult decision to make to kill a resource, but I feel like in this case, consolidation and simplification would help. Thoughts, comments? [1] https://stackoverflow.com/questions/tagged/openstack -- Thierry

10 18

[stein][neutron] gratuitous arp
by Ignazio Cassano 14 May '21

14 May '21

Hello, I have a problem on stein neutron. When a vm migrate from one node to another I cannot ping it for several minutes. If in the vm I put a script that ping the gateway continously, the live migration works fine and I can ping it. Why this happens ? I read something about gratuitous arp. Please, help me ? Any workaround , please ? Best Regards Ignazio

3 33

[ironic] Announcing deprecation of the iSCSI deploy interface
by Dmitry Tantsur 04 May '21

04 May '21

Hi all, Following up to the previous mailing list [1] and virtual meetup [2] discussions, I would like to announce the plans to deprecate the 'iscsi' deploy interface. This is the updated plan discussed on the virtual meetup: 1) In the Victoria cycle (i.e. right now): - Fill in the detected feature gaps [3]. - Switch off the iscsi deploy interface by default. - Change [agent]image_dowload_source to HTTP by default. - Give the direct deploy a higher priority, so that it's used by default unless disabled. - Mark it as deprecated in the code (causing warnings when enabled). - Release a major version of ironic to highlight the defaults changes. 2) In the W cycle: - Keep the iscsi deploy deprecated. - Listen to operators' feedback. 3) In the X cycle - Remove the iscsi deploy completely from ironic and IPA. - Remove support code from ironic-lib with a major version bump. Please let us know if you have any questions or concerns. Dmitry [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016681.h… [2] https://etherpad.opendev.org/p/Ironic-Victoria-midcycle [3] https://storyboard.openstack.org/#!/story/2008075 -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

1 1

[ironic] [infra] Making Glean work with IPA for static IP assignment
by Dmitry Tantsur 18 Mar '21

18 Mar '21

Hi Ian, Given our timezone difference, I've decided to send you an email, adding openstack-discuss in CC for greater exposure. We're trying to make IPA (our agent ramdisk) to work without relying on DHCP for cases like Edge deployments. We've settled on the network_data.json format for the API side and wanted to use Glean on the ramdisk to apply it. You can read more details in [1]. The problem is, I cannot make Glean work with any ramdisk I build. The crux of the problem seems to be that NetworkManager (used by default in RHEL, CentOS, Fedora and Debian at least) starts very early, creates the default connection and ignores whatever files Glean happens to write afterwards. On Debian running `systemctl restart networking` actually helped to pick the new configuration, but I'm not sure we want to do that in Glean. I haven't been able to make NetworkManager pick up the changes on RH systems so far. I build ramdisks using IPA-builder [2] by adding the simple-init element. I've tried removing dhcp-all-interfaces (which we depend on by default) to no effect. I've tried disabling the DHCP server, ended up with no IP connectivity at all. I haven't tried to shutdown and restart a connection as recommended in [3] since it's not trivial to do via SSH. Do you maybe have any hints how to proceed? I'd be curious to know how static IP assignment works in the infra setup. Do you have images with NetworkManager there? Do you use the simple-init element? Any help is very appreciated. Dmitry [1] https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/L3… [2] https://opendev.org/openstack/ironic-python-agent-builder [3] https://mail.gnome.org/archives/networkmanager-list/2014-January/msg00032.h… -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

4 9

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[tc][heat][tacker] Discusssion about heat-translater and tosca-parser maintenance
by TAKAHASHI TOSHIAKI(高橋　敏明) 15 Feb '21

15 Feb '21

Hi, I'm a tacker team member. I'd like to discuss how to maintain heat-translator[1] and tosca-parser[2] repositories. These 2 repositories have not been updated so actively, but tacker often requires code updates for these repositories. Recently, patch merge progress of tacker and these repo do not sometimes match, which affects the development of tacker. For example, our some functional tests on zuul fail because these master codes are not updated. Therefore, tacker members would like to take part more in these 2 developments. Tacker members have already started code reviews, patch posts, etc. If possible, we want some Tacker members to join the core team in the near future. How about this? [1] https://opendev.org/openstack/heat-translator [2] https://opendev.org/openstack/tosca-parser Regards, Toshiaki

5 9

[largescale-sig] OpenStack DB Archiver
by Arnaud Morin 10 Feb '21

10 Feb '21

Hello large-scalers! TLDR: we opensource a tool to help reducing size of databases. See https://github.com/ovh/osarchiver/ Few months ago, we released a tool, name osarchiver, which we are using on our production environment (at OVH) to help reduce the size of our tables in mariadb (or mysql) In fact, some tables are well know to grow very quickly. We use it, for example, to clean the OpenStack mistral database from old tasks, actions and executions which are older than a year. Another use case could be to archive some data in another table (e.g. with _archived as suffix) if they are 6 months old, and delete this data after 1 year. The source code of this tool is available here: https://github.com/ovh/osarchiver/ We were wondering if some other users would be interested in using the tool, and maybe move it under the opendev governance? Feel free to contact us and/or answer this thread. Cheers, -- Arnaud, Pierre-Samuel and OVH team

12 18

[all] CI test result table in the new gerrit review UI
by Balázs Gibizer 27 Jan '21

27 Jan '21

Hi, I understand that adapting the old CI test result table to the new gerrit review UI is not a simple task. As a temporary measure I hacked together a tampermonkey user script[1] that does a limited version of the CI table injection from the browser side. I tested it with recent chrome and firefox. Let me know if you have problems with it and I'll try to help. Also feel free to adapt it to your own needs. Cheers, gibi [1] https://gist.github.com/gibizer/717e0cb5b0da0d60d5ecf1e18c0c171a

15 34

[ops][nova][designate] Does anyone rely on fully-qualified instance names?
by Stephen Finucane 14 Jan '21

14 Jan '21

When attaching a port to an instance, nova will check for DNS support in neutron and set a 'dns_name' attribute if found. To populate this attribute, nova uses a sanitised version of the instance name, stored in the instance.hostname attribute. This sanitisation simply strips out any unicode characters and replaces underscores and spaces with dashes, before truncating to 63 characters. It does not currently replace periods and this is the cause of bug 1581977 [1], where an instance name such as 'ubuntu20.04' will fail to schedule since neutron identifies '04' as an invalid TLD. The question now is what to do to resolve this. There are two obvious paths available to us. The first is to simply catch these invalid hostnames and replace them with an arbitrary hostname of format 'Server-{serverUUID}'. This is what we currently do for purely unicode instance names and is what I've proposed at [2]. The other option is to strip all periods, or rather replace them with hyphens, when sanitizing the instance name. This is more predictable but breaks the ability to use the instance name as a FQDN. Such usage is something I'm told we've never supported, but I'm concerned that there are users out there who are relying on this all the same and I'd like to get a feel for whether this is the case first. So, the question: does anyone currently rely on this inadvertent "feature"? Cheers, Stephen [1] https://launchpad.net/bugs/1581977 [2] https://review.opendev.org/c/openstack/nova/+/764482

9 23

[qa][tempest] Update language in tempest code base
by Arx Cruz 05 Jan '21

05 Jan '21

Hello, I would like to start a discussion regarding the topic. At this moment in time we have an opportunity to be a more open and inclusive project by eliminating outdated naming conventions from tempest codebase, such as blacklist, whitelist. We should take the opportunity and do our best to replace outdated terms with their more inclusive alternatives. As you can see in [1] the TripleO project is already working on this initiative, and I would like to work on this as well on the tempest side. Any thoughts? Shall I start with a sepc, adding deprecation warnings? [1] https://review.opendev.org/#/c/740013/1/specs/victoria/renaming_rules.rst Kind regards, -- Arx Cruz Software Engineer Red Hat EMEA <https://www.redhat.com> arxcruz(a)redhat.com @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

6 9

[Ussuri] Auto shutdown VM
by Deepa KR 05 Jan '21

05 Jan '21

Hi All We have a Openstack setup with the Ussuri Version and I am regularly facing auto shutdown of a few VMs (ubuntu16.04) randomly . If I restart then the instance is back . From logs I was able to see the messages below . WARNING nova.compute.manager [req-2a21d455-ac04-44aa-b248-4776e5109013 813f3fb52c434e38991bb90aa4771541 10b5279cb6f64ca19871f132a2cee1a3 - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Received unexpected event network-vif-unplugged-e97839a1-bbc4-4d26-af30-768ca3630ce9 for instance with vm_state active and task_state None. INFO nova.compute.manager [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] VM Stopped (Lifecycle Event) INFO nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. syslog:Nov 13 07:01:07 fgshwbucehyp04 nova-compute[2680204]: 2020-11-13 07:01:07.684 2680204 WARNING nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4 nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance is already powered off in the hypervisor when stop is called. nova.virt.libvirt.driver [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance already shutdown. nova.virt.libvirt.driver [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully. nova.compute.manager [req-7a0a0d03-e286-42f0-9e36-38a432f236f3 d9ca03b9d0884d51a26a39b6c82f02eb 304d859c43df4de4944ca5623f7f455c - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Get console output nova.virt.libvirt.driver [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully. I searched a few blogs and forums but couldn't find a solution to it . Few mentioned to add sync_power_state_interval=-1 in /etc/nova/nova.conf .But understood that this will help only when nova stops vm. But in this case vm itself is shutting down (Instance shutdown by itself. Calling the stop API) Also no memory issue in VM nor the hypervisor. Also did apt-get upgrade . It would be great if anyone can shed light to this issue. Regards, Deepa K R Sent from my iPhone

7 15

Re: [placement] Train upgrade warning
by Seth Tunstall 21 Dec '20

21 Dec '20

Hello, As another person has mentioned (Chanu Romain), I am attempting an upgrade from Queens to Train. I have successfully upgraded Keystone and Glance, but while trying to separate Placement's databases out from Nova, I run into the the same issue with placement-status upgrade check: # placement-status upgrade check +-----------------------------------------------------------------+ | Upgrade Check Results | +-----------------------------------------------------------------+ | Check: Missing Root Provider IDs | | Result: Success | | Details: None | +-----------------------------------------------------------------+ | Check: Incomplete Consumers | | Result: Warning | | Details: There are -3511 incomplete consumers table records for | | existing allocations. Run the "placement-manage db | | online_data_migrations" command. | +-----------------------------------------------------------------+ Likewise, running placement-manage db online_data_migrations does nothing. I am running this placement migration on a dev cluster which has had the databases from a Prod cluster restored to it. This has worked well for me previously as a way to run through upgrades without touching the live system. Thanks very much, Seth

3 10

[tc][all][searchlight ] Retiring the Searchlight project
by Ghanshyam Mann 07 Dec '20

07 Dec '20

Hello Everyone, As you know, Searchlight is a leaderless project for the Wallaby cycle, which means there is no PTL candidate to lead it in the Wallaby cycle. 'No PTL' and no liaisons for DPL model is one of the criteria which triggers TC to start checking the health, maintainers of the project for dropping the project from OpenStack Governance[1]. TC discussed the leaderless project in PTG[2] and checked if the project has maintainers and what activities are done in the Victoria development cycle. It seems no functional changes in Searchlight repos except few gate fixes or community goal commits[3]. Based on all these checks and no maintainer for Searchlight, TC decided to drop this project from OpenStack governance in the Wallaby cycle. Ref: Mandatory Repository Retirement resolution [4] and the detailed process is in the project guide docs [5]. If your organization product/customer use/rely on this project then this is the right time to step forward to maintain it otherwise from the Wallaby cycle, Searchlight will move out of OpenStack governance by keeping their repo under OpenStack namespace with an empty master branch with 'Not Maintained' message in README. If someone from old or new maintainers shows interest to continue its development then it can be re-added to OpenStack governance. With that thanks to Searchlight contributors and PTLs for maintaining this project. [1] https://governance.openstack.org/tc/reference/dropping-projects.html [2] https://etherpad.opendev.org/p/tc-wallaby-ptg [3] https://www.stackalytics.com/?release=victoria&module=searchlight-group&met… [4] https://governance.openstack.org/tc/resolutions/20190711-mandatory-reposito… [5] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-re… -gmann

5 8

[tc][all][qinling] Retiring the Qinling project
by Ghanshyam Mann 07 Dec '20

07 Dec '20

Hello Everyone, As you know, Qinling is a leaderless project for the Wallaby cycle, which means there is no PTL candidate to lead it in the Wallaby cycle. 'No PTL' and no liaisons for DPL model is one of the criteria which triggers TC to start checking the health, maintainers of the project for dropping the project from OpenStack Governance[1]. TC discussed the leaderless project in PTG[2] and checked if the project has maintainers and what activities are done in the Victoria development cycle. It seems no functional changes in Qinling repos except few gate fixes or community goal commits[3]. Based on all these checks and no maintainer for Qinling, TC decided to drop this project from OpenStack governance in the Wallaby cycle. Ref: Mandatory Repository Retirement resolution [4] and the detailed process is in the project guide docs [5]. If your organization product/customer use/rely on this project then this is the right time to step forward to maintain it otherwise from the Wallaby cycle, Qinling will move out of OpenStack governance by keeping their repo under OpenStack namespace with an empty master branch with 'Not Maintained' message in README. If someone from old or new maintainers shows interest to continue its development then it can be re-added to OpenStack governance. With that thanks to Qinling contributors and PTLs (especially lxkong ) for maintaining this project. [1] https://governance.openstack.org/tc/reference/dropping-projects.html [2] https://etherpad.opendev.org/p/tc-wallaby-ptg [3] https://www.stackalytics.com/?release=victoria&module=qinling-group&metric=… [4] https://governance.openstack.org/tc/resolutions/20190711-mandatory-reposito… [5] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-re… -gmann

4 6

[largescale-sig] Next meeting: December 2, 15utc
by Thierry Carrez 02 Dec '20

02 Dec '20

Hi everyone, We'll have a Large Scale SIG meeting this Wednesday in #openstack-meeting-3 on IRC, at 15UTC. You can doublecheck how it translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20201202T15 Our main topic will be to discuss how to best collect operator scaling experiences. Feel free to add other topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting Talk to you all later, -- Thierry Carrez

1 1

[neutron] bug deputy report Nov 23-29
by Oleg Bondarev 01 Dec '20

01 Dec '20

Please find new bugs summary for the week Nov 23-27. Nothing critical. 2 RFEs for neutron drivers team to discuss. High: - https://bugs.launchpad.net/neutron/+bug/1905700 - [OVN] ovn-metadata-agent: "RowNotFound: Cannot find Chassis with name..." when starting the agent - Gate failure - fix approved: https://review.opendev.org/c/openstack/neutron/+/764318 - https://bugs.launchpad.net/neutron/+bug/1905726 - Qos plugin performs too many queries - fixes on review: - https://review.opendev.org/c/openstack/neutron/+/764433 - https://review.opendev.org/c/openstack/neutron/+/764454 Medium: - https://bugs.launchpad.net/neutron/+bug/1905271 - [OVS] Polling cycle continuously interrupted by L2 population (when enabled) - fix on review: https://review.opendev.org/c/openstack/neutron/+/755313 - https://bugs.launchpad.net/neutron/+bug/1905551 - functional: test_gateway_chassis_rebalance fails - Unassigned - https://bugs.launchpad.net/neutron/+bug/1905568 - Sanity checks missing port_name while adding tunnel port - fix approved: https://review.opendev.org/c/openstack/neutron/+/764171 - https://bugs.launchpad.net/neutron/+bug/1905611 - OVN.ovsdb_probe_interval takes effect only after initial database dump - Unassigned Low: - https://bugs.launchpad.net/neutron/+bug/1905538 - Some OVS bridges may lack OpenFlow10 protocol - fix on review: https://review.opendev.org/c/openstack/neutron/+/764150 Wishlist: - https://bugs.launchpad.net/neutron/+bug/1905276 - Overriding hypervisor name for resource provider always requires a complete list of interfaces/bridges - fix on review - https://review.opendev.org/c/openstack/neutron/+/763563 - https://bugs.launchpad.net/neutron/+bug/1905268 - port list performance for trunks can be optimized - fix on review - https://review.opendev.org/c/openstack/neutron/+/763777 RFEs: - https://bugs.launchpad.net/neutron/+bug/1905295 - [RFE] Allow multiple external gateways on a router - https://bugs.launchpad.net/neutron/+bug/1905391 - [RFE] VPNaaS support for OVN Won't fix: - https://bugs.launchpad.net/neutron/+bug/1905552 - neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown' - fwaas is not supported Expired Bugs: - https://bugs.launchpad.net/bugs/1896592 - [neutron-tempest-plugin] test_dhcpv6_stateless_* clashing when creating a IPv6 subnet - https://bugs.launchpad.net/bugs/1853632 - designate dns driver does not use domain settings for auth Thanks, Oleg --- Advanced Software Technology Lab Huawei

2 1

[cinder] Project ID in cinder endpoints and system scopes
by Nicolas Parquet 01 Dec '20

01 Dec '20

Hello there, Wondering if anyone has ongoing work or information on the following bug: https://bugs.launchpad.net/keystone/+bug/1745905 We tried removing the project_id from cinder endpoints in order to test system scopes and see if more is needed than oslo policy configuration, but cannot get anything else than 404 responses. The bug description suggests it is just a documentation issue, but I could not get a working setup. I also don't this mentioned in ptg documentation regarding system scopes support. Any information or hint is welcome! Regards, -- Nicolas Parquet Gandi nicolas.parquet(a)gandi.net

2 2

nova novnc timeout
by Ankele zhang 01 Dec '20

01 Dec '20

Hello~ I have a OpenStack Rocky platform. My nova.cfg has configured "[consoleauth] token_ttl=360000 [workarounds] enable_consoleauth=true", I get the console url and access my VM console in web. the console url invalid after two or one minutes not 360000s. How can I resolve this? Look forward to hearing from you. Ankele!

2 2

[tripleo][ci] cm2 pip failures
by Wesley Hayutin 01 Dec '20

01 Dec '20

Greetings, FYI.. https://bugs.launchpad.net/tripleo/+bug/1906265 0/

2 2

[qa][all] pip 20.3 breaks jobs
by Balázs Gibizer 01 Dec '20

01 Dec '20

Hi, Today pip 20.3 is released[1] and jobs started failing during keystone install in devstack with: 2020-11-30 15:14:31.117 | + inc/python:pip_install:193 : sudo -H LC_ALL=en_US.UTF-8 SETUPTOOLS_USE_DISTUTILS=stdlib http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite python3.6 -m pip install -c /opt/stack/old/requirements/upper-constraints.txt -e /opt/stack/old/neutron 2020-11-30 15:14:32.271 | Looking in indexes: https://mirror.gra1.ovh.opendev.org/pypi/simple, https://mirror.gra1.ovh.opendev.org/wheel/ubuntu-18.04-x86_64 2020-11-30 15:14:32.272 | DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210. 2020-11-30 15:14:32.272 | ERROR: Links are not allowed as constraints Logstash signature: [2] It seems not all the jobs are effected but at least grenade on master is hit [3]. It seems that passing jobs are using pip from the base image instead of installing a fresh one from pypi. As a first glance the change "Flip the switch on new resolver"[4] seem to be related. Cheers, gibi [1] https://pip.pypa.io/en/stable/news/#id1 [2] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3… [3] https://zuul.opendev.org/t/openstack/builds?job_name=grenade&branch=master [4] https://github.com/pypa/pip/pull/9019

3 2

senlin SR-IOV instance support
by Satish Patel 30 Nov '20

30 Nov '20

Folks, Does senlin support instance type SRIOV, I am successfully able to deploy using standard vm using virtio but for sriov instance we need to create a neutron port and then attach that port to VM. How does that process work with senlin? Currently I have the following profile: (if i want sriov instance then what i need to do?) type: os.nova.server version: 1.0 properties: name: cirros_server flavor: m1.small image: "cirros" key_name: ssh-key networks: - network: net1 metadata: test_key: test_value user_data: | #!/bin/sh echo 'hello, world' > /tmp/test_file ~S

2 2

[devstack][ussuri][keystone] ERROR: Links are not allowed as constraints
by Neil Jerram 30 Nov '20

30 Nov '20

With devstack pinned to stable/ussuri, on Ubuntu Bionic, stack.sh fails for me when it gets to the step of installing keystone: ... + lib/keystone:install_keystone:500 : setup_develop /opt/stack/keystone + inc/python:setup_develop:351 : local bindep + inc/python:setup_develop:352 : [[ /opt/stack/keystone == -bindep* ]] + inc/python:setup_develop:356 : local project_dir=/opt/stack/keystone + inc/python:setup_develop:357 : local extras= + inc/python:setup_develop:358 : _setup_package_with_constraints_edit /opt/stack/keystone -e + inc/python:_setup_package_with_constraints_edit:377 : local bindep + inc/python:_setup_package_with_constraints_edit:378 : [[ /opt/stack/keystone == -bindep* ]] + inc/python:_setup_package_with_constraints_edit:382 : local project_dir=/opt/stack/keystone + inc/python:_setup_package_with_constraints_edit:383 : local flags=-e + inc/python:_setup_package_with_constraints_edit:384 : local extras= ++ inc/python:_setup_package_with_constraints_edit:391 : cd /opt/stack/keystone ++ inc/python:_setup_package_with_constraints_edit:391 : pwd + inc/python:_setup_package_with_constraints_edit:391 : project_dir=/opt/stack/keystone + inc/python:_setup_package_with_constraints_edit:393 : '[' -n /opt/stack/requirements ']' + inc/python:_setup_package_with_constraints_edit:395 : local name ++ inc/python:_setup_package_with_constraints_edit:396 : awk '/^name.*=/ {print $3}' /opt/stack/keystone/setup.cfg + inc/python:_setup_package_with_constraints_edit:396 : name=keystone + inc/python:_setup_package_with_constraints_edit:398 : /opt/stack/requirements/.venv/bin/edit-constraints /opt/stack/requirements/upper-constraints.txt -- keystone '-e file:///opt/stack/keystone#egg=keystone' + inc/python:_setup_package_with_constraints_edit:402 : setup_package /opt/stack/keystone -e + inc/python:setup_package:430 : local bindep=0 + inc/python:setup_package:431 : local bindep_flag= + inc/python:setup_package:432 : local bindep_profiles= + inc/python:setup_package:433 : [[ /opt/stack/keystone == -bindep* ]] + inc/python:setup_package:438 : local project_dir=/opt/stack/keystone + inc/python:setup_package:439 : local flags=-e + inc/python:setup_package:440 : local extras= + inc/python:setup_package:444 : [[ -n -e ]] + inc/python:setup_package:444 : [[ -z '' ]] + inc/python:setup_package:444 : [[ ! -e =~ ^-.* ]] + inc/python:setup_package:449 : [[ ! -z '' ]] + inc/python:setup_package:454 : [[ 0 == 1 ]] + inc/python:setup_package:458 : pip_install -e /opt/stack/keystone Using python 3.6 to install /opt/stack/keystone because python3_enabled=True + inc/python:pip_install:200 : sudo -H LC_ALL=en_US.UTF-8 SETUPTOOLS_USE_DISTUTILS=stdlib http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite /usr/local/bin/pip3.6 install -c /opt/stack/requirements/upper-constraints.txt -e /opt/stack/keystone WARNING: The directory '/home/semaphore/.pip_download_cache' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210. ERROR: Links are not allowed as constraints As far as I can work out, it's because install_keystone edits /opt/stack/requirements/upper-constraints.txt so that the line for keystone has "-e file:///opt/stack/keystone#egg=keystone", and pip then complains that file:// constraints are not supported. I'm sure there's a good reason but - as is often the case - I'm left wondering why this isn't broken for everyone! Any clues or suggestions gratefully received. Neil

1 0

OVS-DPDK poor performance with Intel 82599
by Satish Patel 30 Nov '20

30 Nov '20

Folks, I am playing with DPDK on my openstack with NIC model 82599 and seeing poor performance, i may be wrong with my numbers so want to see what the community thinks about these results. Compute node hardware: CPU: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz Memory: 64G NIC: Intel 82599 (dual 10G port) [root@compute-lxb-3 ~]# ovs-vswitchd --version ovs-vswitchd (Open vSwitch) 2.13.2 DPDK 19.11.3 VM dpdk (DUT): 8vCPU / 8GB memory I have configured my computer node for all best practice available on the internet to get more performance out. 1. Used isolcpus to islocate CPUs 2. 4 dedicated core for PMD 3. echo isolated_cores=1,9,25,33 >> /etc/tuned/cpu-partitioning-variables.conf 4. Huge pages 5. CPU pinning for VM 6. increase ( ovs-vsctl set interface dpdk-1 options:n_rxq=4 ) 7. VM virtio_ring = 1024 After doing all above I am getting the following result using the Trex packet generator using 64B UDP stream (Total-PPS : 391.93 Kpps) Do you think it's an acceptable result or should it be higher on these NIC models? On the internet folks say it should be a million packets per second so not sure what and how those people reached there or i am missing something in my load test profile. Notes: I am using 8vCPU core on VM do you think adding more cores will help? OR should i add more PMD? Cpu Utilization : 2.2 % 1.8 Gb/core Platform_factor : 1.0 Total-Tx : 200.67 Mbps Total-Rx : 200.67 Mbps Total-PPS : 391.93 Kpps Total-CPS : 391.89 Kcps Expected-PPS : 700.00 Kpps Expected-CPS : 700.00 Kcps Expected-BPS : 358.40 Mbps This is my all configuration: grub.conf: GRUB_CMDLINE_LINUX="vmalloc=384M crashkernel=auto rd.lvm.lv=rootvg01/lv01 console=ttyS1,118200 rhgb quiet intel_iommu=on iommu=pt spectre_v2=off nopti pti=off nospec_store_bypass_disable spec_store_bypass_disable=off l1tf=off default_hugepagesz=1GB hugepagesz=1G hugepages=60 transparent_hugepage=never selinux=0 isolcpus=2,3,4,5,6,7,10,11,12,13,14,15,26,27,28,29,30,31,34,35,36,37,38,39" [root@compute-lxb-3 ~]# ovs-appctl dpif/show netdev@ovs-netdev: hit:605860720 missed:2129 br-int: br-int 65534/3: (tap) int-br-vlan 1/none: (patch: peer=phy-br-vlan) patch-tun 2/none: (patch: peer=patch-int) vhu1d64ea7d-d9 5/6: (dpdkvhostuserclient: configured_rx_queues=8, configured_tx_queues=8, mtu=1500, requested_rx_queues=8, requested_tx_queues=8) vhu9c32faf6-ac 6/7: (dpdkvhostuserclient: configured_rx_queues=8, configured_tx_queues=8, mtu=1500, requested_rx_queues=8, requested_tx_queues=8) br-tun: br-tun 65534/4: (tap) patch-int 1/none: (patch: peer=patch-tun) vxlan-0a410071 2/5: (vxlan: egress_pkt_mark=0, key=flow, local_ip=10.65.0.114, remote_ip=10.65.0.113) br-vlan: br-vlan 65534/1: (tap) dpdk-1 2/2: (dpdk: configured_rx_queues=4, configured_rxq_descriptors=2048, configured_tx_queues=5, configured_txq_descriptors=2048, lsc_interrupt_mode=false, mtu=1500, requested_rx_queues=4, requested_rxq_descriptors=2048, requested_tx_queues=5, requested_txq_descriptors=2048, rx_csum_offload=true, tx_tso_offload=false) phy-br-vlan 1/none: (patch: peer=int-br-vlan) [root@compute-lxb-3 ~]# ovs-appctl dpif-netdev/pmd-rxq-show pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-1 queue-id: 0 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 3 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 4 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 3 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 4 (enabled) pmd usage: 0 % pmd thread numa_id 0 core_id 9: isolated : false port: dpdk-1 queue-id: 1 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 2 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 5 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 2 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 5 (enabled) pmd usage: 0 % pmd thread numa_id 0 core_id 25: isolated : false port: dpdk-1 queue-id: 3 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 0 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 7 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 0 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 7 (enabled) pmd usage: 0 % pmd thread numa_id 0 core_id 33: isolated : false port: dpdk-1 queue-id: 2 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 1 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 6 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 1 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 6 (enabled) pmd usage: 0 %

2 6

Re :[all][infra] CI test result table in the new gerrit review UI
by Bogdan Dobrelya 30 Nov '20

30 Nov '20

Hi stackers, Until we have that pretty formatter for CI results, here's [0] a quick script instead. It queries Zuul CI comments by a given Gerrit SSH login and a patch set number, and colorizes it in your console outputs. PS. I'm sorry for top posting. [0] https://gist.github.com/bogdando/ed345b1f974c8e97edd33b61e7bf8158 > On Thu, 2020-11-26 at 17:30 +0000, Tristan Cacqueray wrote: >> On Thu, Nov 26, 2020 at 15:59 Sean Mooney wrote: >> > On Thu, 2020-11-26 at 15:19 +0000, Jeremy Stanley wrote: >> > > On 2020-11-26 15:10:09 +0000 (+0000), Tristan Cacqueray wrote: >> > > [...] >> > > > Here is a CR to add the test result table back in the new gerrit UI: >> > > > >> > > > https://review.opendev.org/c/opendev/system-config/+/763891 >> > > > >> > > > This new implementation uses the javascript plugin extension API. >> > > >> > > Thanks--that was quick work! >> >> You're welcome :) >> >> > is that going to be moved to the zuul org in opendev as a part of the zuul projects >> > deliverables. >> > >> > i kind of think it makes sne to develop it there rather then on the rdo gerrit instance. >> > >> >> I'd be happy to move the project to the opendev gerrit. >> >> > looking at the impleation it does not look like this owrks teh ssame way by parsing the comment >> > https://softwarefactory-project.io/cgit/software-factory/zuul-results-gerri… >> > >> >> That is correct, this is using the Gerrit Js plugin api, which let you >> register a `showChange` callback, which is called with the change info >> https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#… >> >> Thus no need to hack through the dom to get the comments. >> >> Note that the gerrit related code is in another project here: >> https://softwarefactory-project.io/cgit/software-factory/re-gerrit/ >> >> > will this just show the results form zuul. it need to pars all the comments form the third party cis too. >> > basicaly you need to parse all coment form a author that end isn CI and comments form zuul >> >> It should do that already, though I haven't tested it with real data. I >> guess the next step would be to enable such plugin on review-dev. > so the author regular expression is only looking for zuul > https://softwarefactory-project.io/cgit/software-factory/re-gerrit/tree/src… > > so it lookes liek you would need to refactor that slightly to match comments form autors that end in ci too. > > each CI systems comemtn then need have there won tabel section with the job results for that ci to mimic the same behavior > we had in the old ui. > > >> >> Cheers, >> -Tristan -- Best regards, Bogdan Dobrelya, Irc #bogdando

1 0

XDG_SESSION_TYPE error on devstack installation
by Nitish Goel 30 Nov '20

30 Nov '20

Hi Team, I'm trying to install a devstack on a ubuntu 18.04 VM having python 3.6.9 but getting below error. Any suggestions? python - 3.6.9 stack user with sudo access. pip version - pip 20.2.4 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6) https://stackoverflow.com/questions/64989221/xdg-session-type-error-on-devs… NFO keystone.cmd.bootstrap [None req-f46e9c41-1b14-4e5b-82b2-ada81e8b0dcd None None] Created region RegionOne INFO keystone.cmd.bootstrap [None req-f46e9c41-1b14-4e5b-82b2-ada81e8b0dcd None None] Created public endpoint http://10.61.62.241/identity INFO keystone.cmd.bootstrap [None req-f46e9c41-1b14-4e5b-82b2-ada81e8b0dcd None None] Created admin endpoint http://10.61.62.241/identity +./stack.sh:main:1084 create_keystone_accounts +lib/keystone:create_keystone_accounts:314 local admin_project ++lib/keystone:create_keystone_accounts:315 oscwrap project show admin -f value -c id Traceback (most recent call last): File "/usr/local/bin/openstack", line 5, in <module> from openstackclient.shell import main File "/usr/local/lib/python3.6/dist-packages/openstackclient/shell.py", line 24, in <module> from osc_lib import shell File "/usr/local/lib/python3.6/dist-packages/osc_lib/shell.py", line 24, in <module> from cliff import app File "/usr/local/lib/python3.6/dist-packages/cliff/app.py", line 24, in <module> import cmd2 File "/usr/local/lib/python3.6/dist-packages/cmd2/__init__.py", line 30, in <module> from .cmd2 import Cmd File "/usr/local/lib/python3.6/dist-packages/cmd2/cmd2.py", line 48, in <module> from .clipboard import can_clip, get_paste_buffer, write_to_paste_buffer File "/usr/local/lib/python3.6/dist-packages/cmd2/clipboard.py", line 12, in <module> _ = pyperclip.paste() File "/usr/local/lib/python3.6/dist-packages/pyperclip/__init__.py", line 680, in lazy_load_stub_paste copy, paste = determine_clipboard() File "/usr/local/lib/python3.6/dist-packages/pyperclip/__init__.py", line 568, in determine_clipboard os.environ["XDG_SESSION_TYPE"] == "wayland" and File "/usr/lib/python3.6/os.py", line 669, in __getitem__ raise KeyError(key) from None KeyError: 'XDG_SESSION_TYPE' ++functions-common:oscwrap:2346 return 1 +lib/keystone:create_keystone_accounts:315 admin_project= +lib/keystone:create_keystone_accounts:1 exit_trap +./stack.sh:exit_trap:491 local r=1 ++./stack.sh:exit_trap:492 jobs -p +./stack.sh:exit_trap:492 jobs= +./stack.sh:exit_trap:495 [[ -n '' ]] +./stack.sh:exit_trap:501 '[' -f /tmp/tmp.LRWsRkTTkV ']' +./stack.sh:exit_trap:502 rm /tmp/tmp.LRWsRkTTkV +./stack.sh:exit_trap:506 kill_spinner +./stack.sh:kill_spinner:401 '[' '!' -z '' ']' +./stack.sh:exit_trap:508 [[ 1 -ne 0 ]] +./stack.sh:exit_trap:509 echo 'Error on exit' Error on exit +./stack.sh:exit_trap:511 type -p generate-subunit +./stack.sh:exit_trap:512 generate-subunit 1606228299 592 fail +./stack.sh:exit_trap:514 [[ -z /opt/stack/logs ]] +./stack.sh:exit_trap:517 /usr/bin/python3.6 /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs +./stack.sh:exit_trap:526 exit 1

3 6

Re: [designate] DKIM TXT record problem
by pawel＠dpnet.pl 30 Nov '20

30 Nov '20

I have the same problem, would you like to share a command line example, how did you manage to add such a long txt record in the designate (more than 255 characters in one record)? -- Pawel D

1 0

[ironic] [tripleo] RFC: lzma vs gzip for compressing IPA initramfs
by Dmitry Tantsur 30 Nov '20

30 Nov '20

Hi folks, I've been playing with ways to reduce the size of our IPA images. While package removals can only save us tens of megabytes, switching from gzip to lzma reduces the size by around a third (from 373M to 217M in my testing). What's the caveat? The unpacking time increases VERY substantially. On my nested virt lab the 217M image took around 5 minutes to unpack. I'm not sure how much it will impact real bare metal, please feel free to test https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/764371 and tell me. So, what do you think? Switching to lzma by default will likely affect CI run time (assuming we still have DIB jobs somewhere...) and development environments, but it will also provide a visible reduction in the image size (which benefit all environments). Large TripleO images may particularly benefit from this (but also particularly affected by the unpacking time). Feedback is very welcome. Dmitry -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

7 9

[openstack][neutron]How to establish existing Openstack in a single server to other multiple physical NIC port
by dangerzone ar 30 Nov '20

30 Nov '20

Hi, I have deployed packstack (All-In-One) openstack over single server and that openstack traffic is IN/Out via eth1 of the server and that's the network port configured during the packstack deployment. The server has 4 nic ports and only one (nic1) is used during openstack deployment. The openstack running good and my vm/vnf traffic can in/out via eth1. I would like to know how I can utilize other nic port (2,3,4) in my server so that openstack can leverage that port for my other VM/VNF that instantiate in my openstack. For example from the image attached, I want VM/VNF1 connect to nic2 and VM/VNF2 connect to nic3. Please advise how it can be done. I really hope someone could advise and help me further. Thank you. **This is standard openstack (deployed via packstack all-in-one) over single centos7 server that have 4 nic ports**

2 1

[telemetry] wallaby cycle planning session
by Matthias Runge 30 Nov '20

30 Nov '20

Hi there, one of the biggest challenges for the telemetry stack is currently the state of gnocchi, which is... undefined/unfortunate/under-contributed/...? Telemetry started long time ago as a simple component ceilometer, which was split into several components ceilometer, aodh, panko, and gnocchi. Julien wrote a story about this some time ago[1]. There has also been an attempt to fork gnocchi back to OpenStack[2]. To my knowledge, the original contributors are not paid anymore to work on gnocchi, and at the same time, moving on to do something else is totally fine. However, I am not sure if we (in OpenStack Telemetry) should or could maintain in addition to the rest of the telemetry stack a time-series database. I'd like to discuss this during a call. Please select time(s) that suit you best in this poll[3]. If you have questions or hints, don't hesitate to contact me. Thank you, Matthias [1] https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/ [2] https://review.opendev.org/#/c/744592/ [3] https://doodle.com/poll/uqq328x5shr43awy

5 15

[training-labs] OpenStack training-labs expired link
by Jeremy SNIDARO 28 Nov '20

28 Nov '20

Hello, The scripts to setup the labs (labs-master\labs\osbash\wbatch\create_base.bat) is trying to download an version of ubuntu that is no longer available on the URL : http://cdimage.ubuntu.com/releases/18.04/release/ubuntu-18.04.4-server-amd6… The new URL should be : http://cdimage.ubuntu.com/releases/18.04/release/ubuntu-18.04.*5* -server-amd64.iso On github it is on the file labs/stacktrain/distros/ubuntu_18_04_server_amd64.py <https://github.com/openstack/training-labs/blob/master/labs/stacktrain/dist…> Regards, ----------------------------------------------------------------- Jérémy SNIDARO GNU/Linux System Administrator - DevOps jeremysnidaro.com - France 86000 Poitiers [image: LinkedIn] <https://www.linkedin.com/in/jeremy-snidaro/>

1 0

[openstack] [InteropWG] Planning Approval by Nov 30th through email and later at Dec 8th Board meeting
by prakash RAMCHANDRAN 27 Nov '20

27 Nov '20

Hi all, Hope you all had a good Thanks Giving and like to proceed from draft guidelines on the VIctoria Interop to Approval at next Board meeting on Dec 8th. https://wiki.openstack.org/wiki/Governance/Foundation/8Dec2020BoardMeeting Please review the draft before Nov 30th Monday noon as we would like this 2020.11.* date maintained and will request an approval by Nov 30th, https://review.opendev.org/c/osf/interop/+/762719 The Interop WG meeting on Dec 3 , we will focus on maintenance of ReftStack and any additional discussions you like to bring for OIF InteropWG wrt add-on modules and or independent k8s conformance for Open Infrastructure Projects. https://etherpad.opendev.org/p/interop ThanksPrakash For InteropWG

1 0

[all][nova][placement] Merging 'placement' IRC channel into 'nova'
by Ghanshyam Mann 27 Nov '20

27 Nov '20

Hello Everyone, As the placement project is under Nova governance now[1], can we merge the IRC channel 'placement' into nova? I think that can be done via redirect way? [1] https://review.opendev.org/c/openstack/governance/+/760917 -gmann

3 2

Re: [neutron][openstack-ansible] Instances can only connect to provider-net via tenant-net but not directly
by Oliver Wenz 27 Nov '20

27 Nov '20

> So this looks like issue during sending notification to Nova. You should check > in Nova logs now why it returned You 422 The relevant nova-compute log on the node where I try to start the instance looks like this: Nov 27 13:33:24 bc1blade15 nova-compute[4240]: 2020-11-27 13:33:24.870 4240 INFO nova.compute.claims [req-4125d882-0753-4a0a-844c-5a948681ffa1 920e739127a14018a55fb4422b0885e7 0f14905dab5546e0adec2b56c0f6be88 - default default] [instance: 20a23a18-0d13-4aba-b0be-37e243b21336] Claim successful on node bc1blade15.openstack.local Nov 27 13:33:26 bc1blade15 nova-compute[4240]: 2020-11-27 13:33:26.151 4240 INFO nova.virt.libvirt.driver [req-4125d882-0753-4a0a-844c-5a948681ffa1 920e739127a14018a55fb4422b0885e7 0f14905dab5546e0adec2b56c0f6be88 - default default] [instance: 20a23a18-0d13-4aba-b0be-37e243b21336] Creating image Nov 27 13:33:31 bc1blade15 nova-compute[4240]: 2020-11-27 13:33:31.463 4240 INFO os_vif [req-4125d882-0753-4a0a-844c-5a948681ffa1 920e739127a14018a55fb4422b0885e7 0f14905dab5546e0adec2b56c0f6be88 - default default] Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:57:d5:f4,bridge_name='brq497a58ad-e5',has_traffic_filtering=True,id=c2e13a92-86bc-4c8e-ad74-1cad0a6bcffc,network=Network(497a58ad-e57c-4bf1-a1c7-dad58f4795ec),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tapc2e13a92-86') Nov 27 13:33:33 bc1blade15 nova-compute[4240]: 2020-11-27 13:33:33.961 4240 INFO nova.compute.manager [req-633ff74b-b16d-4063-9f54-5ee618ea2bb9 - - - - -] [instance: 20a23a18-0d13-4aba-b0be-37e243b21336] VM Started (Lifecycle Event) Nov 27 13:33:34 bc1blade15 nova-compute[4240]: 2020-11-27 13:33:34.104 4240 INFO nova.compute.manager [req-633ff74b-b16d-4063-9f54-5ee618ea2bb9 - - - - -] [instance: 20a23a18-0d13-4aba-b0be-37e243b21336] VM Paused (Lifecycle Event) Nov 27 13:33:34 bc1blade15 nova-compute[4240]: 2020-11-27 13:33:34.388 4240 INFO nova.compute.manager [req-633ff74b-b16d-4063-9f54-5ee618ea2bb9 - - - - -] [instance: 20a23a18-0d13-4aba-b0be-37e243b21336] During sync_power_state the instance has a pending task (spawning). Again, the corresponding neutron-server log: Nov 27 13:39:02 infra1-neutron-server-container-cbf4b105 neutron-server[87]: 2020-11-27 13:39:02.106 87 WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': '20a23a18-0d13-4aba-b0be-37e243b21336', 'name': 'network-vif-deleted', 'tag': 'c2e13a92-86bc-4c8e-ad74-1cad0a6bcffc', 'status': 'failed', 'code': 422} returned with failed status I also looked in all the logs of the nova related services on the managament node for the instance id but didn't find it. So the nova-compute log seems to indicate that the instance can connect to the bridge successfully?!

2 1

Re: [tripleo] Copying tripleo container images to quay.io
by Bogdan Dobrelya 27 Nov '20

27 Nov '20

So I explored how exactly skopeo copy works with the WIP zstd:chunked compression format. The main case here is when copying *new* layers of the *same* image. What is not the case in TripleO CI. We build images from scratch and tag it only once, like current-tripleo. Here ends that build's lifecycle. Had we decided to change that to update existing images, we could come back and try that new feature of skopeo, or may be even consume it directly via buildah bud, when it's there. Another caveat is that only OCI compliant registries could accept such zstd:chunked compressed layers. But anyway that was a good try... Bogdan Dobrelya bdobreli at redhat.com writes: > Note that skopeo will shortly have the layers diffs [0] based "copy" > operations [1],[2],[3]. It also deduplicates things and uses local cache > to not repeat redundant data transfers. Would be great if TripleO > container registry could start using that super cool feature as early > adoption. > > We could make a build with these patches and try that for the subject > tooling for TCIB distribution over quay.io. > > IIUC, the process is like: > > * as input we have full layers from the source registry, built with TCIB; > * skopeo uses its super powers to redo it on fly into layers diff > * either skopeo as well, or maybe even tripleoclient prepare image CLI > pushes those optimized layers of the images to the target quay.io registry. > * ??? > * profit! > > [0] https://github.com/containers/image/pull/902 > [1] https://github.com/containers/image/pull/1084 > [2] https://github.com/containers/storage/pull/775 > [3] https://github.com/containers/skopeo/pull/1111 -- Best regards, Bogdan Dobrelya, Irc #bogdando

1 0

[tripleo] Please avoid merge patches for releases before Victoria
by Arx Cruz 27 Nov '20

27 Nov '20

Hello, We need these patches [1] landed first in order to merge other patches, otherwise CI will not be able to properly test it. We have already had two known issues [2] related to this problem, so I ask you for patience while we are merging those patches. [1] https://review.opendev.org/c/openstack/tripleo-quickstart/+/763747 https://review.opendev.org/#/c/763712 [2] https://launchpad.net/bugs/1903033 https://bugs.launchpad.net/tripleo/+bug/1905536 Kind regards, -- Arx Cruz Software Engineer Red Hat EMEA <https://www.redhat.com> arxcruz(a)redhat.com @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 1

[release] Release countdown for week R-20, Nov 23 - Nov 27
by Herve Beraud 27 Nov '20

27 Nov '20

Greetings Everyone! Development Focus ----------------- The Wallaby-1 milestone is next week, on December 03! Project team plans for the Wallaby cycle should now be solidified. General Information ------------------- If you planned to change the release model for any of your deliverables this cycle, please remember to do so ASAP, before milestone-1. Libraries need to be released at least once per milestone period. Next week, the release team will propose releases for any library which had changes but has not been otherwise released since the Victoria release. PTL's or release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well, by posting a -1. If we do not hear anything at all by the end of the week, we will assume things are OK to proceed. NB: If one of your libraries is still releasing 0.x versions, start thinking about when it will be appropriate to do a 1.0 version. The version number does signal the state, real or perceived, of the library, so we strongly encourage going to a full major version once things are in a good and usable state. Upcoming Deadlines & Dates -------------------------- Wallaby-1 milestone: December 03 Hervé Beraud (hberaud) and the Release Management Team -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 0

[all][tc][goals] Migrate RBAC Policy Format from JSON to YAML: Week R-20 Update
by Ghanshyam Mann 27 Nov '20

27 Nov '20

Hello Everyone, As you all know, 'Migrate RBAC Policy Format from JSON to YAML' is one of the community-wide goals for Wallaby development cycle. I am starting this thread to report the progress of this work. Tracking: https://etherpad.opendev.org/p/migrate-policy-format-from-json-to-yaml Gerrit Topic: https://review.opendev.org/q/topic:%22policy-json-to-yaml%22+(status:open%2… Updates: ======== * Common code of upgrade checks and policy.json file fallback logic is moved from nova to oslo.upgradechecks (in version 1.2.0) and oslo.policy(in version 3.6.0) respectively. ** For upgrade checks, you just need to enable it from the project side. Example: https://review.opendev.org/c/openstack/keystone/+/764240/4/keystone/cmd/sta… ** For olso.policy fallback logic, you just need to bump the min version to 3.6.0. For project who already change the policy_file default in previous releases (like Cinder ) then you can disable this fallback logic via flag 'fallback_to_json_file' * I have started few projects to propose the changes and will continue to do that for other projects too. If you are interested to help, please write it in tracking etherpad [1]. * I am updating the goal documents also for oslo side common code and the latest example. - https://review.opendev.org/c/openstack/governance/+/764261 - This is an example you can refer: https://review.opendev.org/c/openstack/keystone/+/764240 [1] https://etherpad.opendev.org/p/migrate-policy-format-from-json-to-yaml -gmann

1 0

[neutron] Drivers meeting - 27.11.2020
by Slawek Kaplonski 26 Nov '20

26 Nov '20

Hi, As it's Thanksgiving week and Black Friday so our US folks will probably not be there and lets cancel the meeting this week. But we have couple of new RFEs which needs to be triaged: * https://bugs.launchpad.net/neutron/+bug/1905391 * https://bugs.launchpad.net/neutron/+bug/1905295 * https://bugs.launchpad.net/neutron/+bug/1905115 * https://bugs.launchpad.net/neutron/+bug/1900934 So please take some time and help with triaging them so we will be able to discuss them next week hopefully :) Have a great weekend! -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[ptl][release][stable][EM] Extended Maintenance - Stein
by Előd Illés 26 Nov '20

26 Nov '20

Hi, As Victoria was released yesterday and we are in a less busy period, it is a good opportunity to call your attention to the following: In less than a month Stein is planned to enter into Extended Maintenance phase [1] (planned date: 2020-11-11). I have generated the list of *open* and *unreleased* changes in *stable/stein* for the follows-policy tagged repositories [2]. These lists could help the teams, who are planning to do a *final* release on Stein before moving stable/stein branches to Extended Maintenance. Feel free to edit and extend these lists to track your progress! * At the transition date the Release Team will tag the *latest* (Stein) releases of repositories with *stein-em* tag. * After the transition stable/stein will be still open for bugfixes, but there won't be any official releases. NOTE: teams, please focus on wrapping up your libraries first if there is any concern about the changes, in order to avoid broken releases! Thanks, Előd [1] https://releases.openstack.org [2] https://etherpad.openstack.org/p/stein-final-release-before-em

2 2

[oslo] py-amqp TLS peer authentication
by Moises Guimaraes de Medeiros 26 Nov '20

26 Nov '20

Hi all, This email is about a potential MITM attack, I would like to get some input on my proposed solution. *Disclaimer: I'm not an oslo.messaging expert or even versed in its codebase but I have a good* * number of hours working with TLS already.* Back in September, there was a fix[1] in py-amqp to stop to use deprecated method ssl.wrap_socket. Although I didn't follow anything about oslo.messaging, the PR was brought to my attention by Hervé, another oslo core, who sees me as an SME on TLS among the oslo cores. I didn't follow the entire conversation back then, I was only able to give some quick input, but Hervé was able to finish the PR with the help of another developer. Then earlier this month, another PR[2] was reintroducing a couple of parameters that got dropped in the deprecation fix. At that point it was raised the possibility of a MITM attack in an issue[3] and by reintroducing the dropped parameters, the problem should be fixed. A couple of days went by and I started to have a hunch that there could still be more to fix. I went to py-amqp codebase instead of just looking to the diffs on GitHub and noticed a not straightforward logic on how verify_mode and check_hostname were being set. Today I had time to go back to the py-amqp codebase and rework the _wrap_socket_sni() method. I've put up this PR[4] and it is already failing in my travis[5] for integration tests which make me believe that the client sockets were actually not validating the server certs. There are a few more comments on the fix in the description of PR[4]. [1]: https://github.com/celery/py-amqp/pull/327 [2]: https://github.com/celery/py-amqp/pull/344 [3]: https://github.com/celery/py-amqp/issues/342 [4]: https://github.com/celery/py-amqp/pull/347 [5]: https://travis-ci.com/github/moisesguimaraes/py-amqp/builds/204747399 Thanks, -- Moisés Guimarães Software Engineer Red Hat <https://www.redhat.com> <https://red.ht/sig>

1 0

SFC for OVN
by Lazuardi Nasution 26 Nov '20

26 Nov '20

Hi, By https://docs.openstack.org/neutron/ussuri/admin/config-sfc.html, it is stated that SFC essentially is the SDN version of PBR. On the other side, PBR is supported by OVN. Is there any initiative of Neutron SFC driver for OVN which utilize native PBR support of OVN without having to alternate the OVN itself? For example, for having a port pair of port A to port B for flow classifier of HTTP traffic, we might make PBR which match HTTP traffic from port A address and set the next hop to port B address. The other way which might be used is destination NATing (redirection) on port A for HTTP traffic to port B address. OVN has native support for NAT too, but I think the PBR way is better. Best regards,

1 1

[tc] weekly update & agenda
by Mohammed Naser 26 Nov '20

26 Nov '20

Hi everyone, Here's an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. # Patches ## Open Reviews - Clarify the requirements for supports-api-interoperability https://review.opendev.org/c/openstack/governance/+/760562 - Add election schedule exceptions in charter https://review.opendev.org/c/openstack/governance/+/751941 - Add Resolution of TC stance on the OpenStackClient https://review.opendev.org/c/openstack/governance/+/759904 - Add assert:supports-standalone https://review.opendev.org/c/openstack/governance/+/722399 - Remove assert_supports-zero-downtime-upgrade tag https://review.opendev.org/c/openstack/governance/+/761975 - Add Magpie charm to OpenStack charms https://review.opendev.org/c/openstack/governance/+/762820 - Propose Kendall Nelson for vice chair https://review.opendev.org/c/openstack/governance/+/762014 - Clarify impact on releases for SIGs https://review.opendev.org/c/openstack/governance/+/752699 ## General Changes - Move Placement under Nova's governance https://review.opendev.org/c/openstack/governance/+/760917 ## Other Reminders - Our next [TC] Weekly meeting is scheduled for November 26. If you would like to add topics for discussion, please go to https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting and fill out your suggestions by Wednesday, November 26th, at 0000 UTC. Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 1

[Freezer][Keystone][Mistral][Monasca][Murano][Openstack-Chef][Openstack-Helm][Openstacksdk][Trove][Watcher][Zun] Action required: Gerrit code audit
by Mohammed Naser 26 Nov '20

26 Nov '20

Hello there, As per the `service-announce` on October 20 regarding Gerrit Outage Update email http://lists.opendev.org/pipermail/service-announce/2020-October/000011.html, all project teams are required to audit changes for projects from 2020-10-01 to 2020-10-21. I'm reaching out to those projects in particular who the TC believes have not completed their audit yet. Let us know if you need any type of assistance in completing the audit. In case you didn’t know you needed to do this, feel free to reach out for support. Regards, Mohammed -- Mohammed Naser VEXXHOST, Inc.

13 14

[all][healthcheck]
by Lajos Katona 26 Nov '20

26 Nov '20

Hi, I send this mail out to summarize the discussion around Healthcheck API on Neutron PTG, and start a discussion how we can make this most valuable to the operators. On the Neutron PTG etherpad this topic is from L114: https://etherpad.opendev.org/p/neutron-wallaby-ptg . Background: oslo_middleware provides /healthcheck API path(see [1]), which can be used to poll by services like haproxy, and gives a plugin mechanism to add some more complicated checks, which can be switched on/off from config. The main questions: - Some common guidance what to present to the operators (if you check [2] and [3] in the comments there are really good questions/concerns) - Perhaps the API SIG has something about healtcheck, just I can't find it. - What to present with and without authentication (after checking again, I am not sure that it is possible to use authentication for the healthcheck) - A way forward can be to make it configurable with default to authenticated, and give the decision to the admin. - During the discussion the agreement was to separate the frontend health from the backend health and use direct indicators (like working db connectivity, and mq connectivity) instead of indirect indicators (like agents' health). Thanks in advance for the feedback. [1] https://docs.openstack.org/oslo.middleware/latest/reference/healthcheck_plu… [2] https://review.opendev.org/731396 [3] https://review.opendev.org/731554 Regards Lajos Katona (lajoskatona)

3 4

[neutron] [nova] [ussuri] IP of the VM at the provider network is different than the one registered at the Openstack controller
by Pavlos Basaras 26 Nov '20

26 Nov '20

Hello, Setup: i follow the main instruction for deploying openstack for learning purposes. I can successfully deploy vms in an internal network, and also connect them to a router that gives internet access using floating ips etc. My setup is based on ubuntu 18. I have virtualbox for deploying controller (10.0.0.11) and compute (10.0.0.31) etc. I have host only adapters at virtual box (dhcp enabled) for the two networks: mgmt: 10.0.0.0/24 and provider 203.0.111.0/24 as per the default instructions, and i use iptables to NAT the vms. Problem: When following the instructions from https://docs.openstack.org/install-guide/launch-instance-provider.html the vm is deployed successfully but the ip that it is assigned is different from the one that the controller has. It seems that there are more than one dhcp instances running? How can i debug this? Any advice? all the best, Pavlos.

2 2

Re: [neutron][openstack-ansible] Instances can only connect to provider-net via tenant-net but not directly
by Oliver Wenz 26 Nov '20

26 Nov '20

> First of all I'd suggest checking `openstack network agent list` and ensure that lxb agent is present there up and healthy. > Next I'd check nova-scheduler log (journalctl eventually) as it might have more insights. linuxbridge agent is up and healthy and the nova-scheduler log doesn't show any errors either > Please check in neutron server logs for the problems with binding port Indeed, running journalctl and grepping for the instance id shows an error (last lines): journalctl -u neutron-server -n 300 | grep 33c662c2-99d6-40fb-a716-118ad8b3b193 Nov 25 15:06:58 infra1-neutron-server-container-cbf4b105 neutron-server[88]: 2020-11-25 15:06:58.738 88 INFO neutron.wsgi [req-e5b7df6c-bfde-438a-bbb1-f2713b63e16b 920e739127a14018a55fb4422b0885e7 0f14905dab5546e0adec2b56c0f6be88 - default default] 192.168.110.128,192.168.110.201 "GET /v2.0/ports?device_id=33c662c2-99d6-40fb-a716-118ad8b3b193 HTTP/1.1" status: 200 len: 205 time: 0.3313231 Nov 25 15:07:04 infra1-neutron-server-container-cbf4b105 neutron-server[89]: 2020-11-25 15:07:04.782 89 INFO neutron.wsgi [req-9bdadc77-27d9-4f9d-86b5-349b598b1ba4 eaf8235312004e2da519b54d196f5415 474b7aa9b7894d6782402135c6ef4c2a - default default] 192.168.110.205,192.168.110.201 "GET /v2.0/ports?tenant_id=0f14905dab5546e0adec2b56c0f6be88&device_id=33c662c2-99d6-40fb-a716-118ad8b3b193 HTTP/1.1" status: 200 len: 1084 time: 0.0956030 Nov 25 15:07:05 infra1-neutron-server-container-cbf4b105 neutron-server[89]: 2020-11-25 15:07:05.020 89 INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '33c662c2-99d6-40fb-a716-118ad8b3b193', 'tag': '1e24a151-bd29-47f8-914b-05c66587541b', 'status': 'completed', 'code': 200} Nov 25 15:07:05 infra1-neutron-server-container-cbf4b105 neutron-server[89]: 2020-11-25 15:07:05.781 89 INFO neutron.wsgi [req-9b729386-4847-4be3-80fa-dce244435ddd eaf8235312004e2da519b54d196f5415 474b7aa9b7894d6782402135c6ef4c2a - default default] 192.168.110.205,192.168.110.201 "GET /v2.0/ports?tenant_id=0f14905dab5546e0adec2b56c0f6be88&device_id=33c662c2-99d6-40fb-a716-118ad8b3b193 HTTP/1.1" status: 200 len: 1084 time: 0.1147285 Nov 25 15:12:12 infra1-neutron-server-container-cbf4b105 neutron-server[89]: 2020-11-25 15:12:12.445 89 INFO neutron.wsgi [req-15cbcb76-04cd-4813-b1f9-d7f6596e14bf 920e739127a14018a55fb4422b0885e7 0f14905dab5546e0adec2b56c0f6be88 - default default] 192.168.110.205,192.168.110.201 "GET /v2.0/ports?device_id=33c662c2-99d6-40fb-a716-118ad8b3b193 HTTP/1.1" status: 200 len: 1085 time: 0.4594653 Nov 25 15:12:14 infra1-neutron-server-container-cbf4b105 neutron-server[83]: 2020-11-25 15:12:14.453 83 WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': '33c662c2-99d6-40fb-a716-118ad8b3b193', 'name': 'network-vif-deleted', 'tag': '1e24a151-bd29-47f8-914b-05c66587541b', 'status': 'failed', 'code': 422} returned with failed status Kind regards, Oliver

2 1

[qa][cinder][glance][swift] Gate status for swift test failing Bug#1905432
by Ghanshyam Mann 26 Nov '20

26 Nov '20

Hello Everyone, As you have seen that test_create_object_with_transfer_encoding is failing consistently since 20th Nov which is failing integrated-gate-storage and integrated-gate-object-storage jobs. The issue is that Tempest is getting 500 from swift API, I could not find the root cause yet so decided to disable this test for now to unlock the gate. Avoid recheck for this failure until this is merge - https://review.opendev.org/c/openstack/tempest/+/764061 I have filled bug against Tempest and Swift to track it - https://bugs.launchpad.net/swift/+bug/1905432 -gmann

2 1

[ironic] Proposing Steve Baker (stevebaker) for ironic-core
by Julia Kreger 25 Nov '20

25 Nov '20

Greetings everyone, I would like to propose Steve Baker for ironic-core. Steve shifted into focusing on Ironic contribution this year and he has been actively reviewing changes during the Victoria and now Wallaby cycle. He has brought insight, drive, and a curiosity to ask questions. Plus he took on the challenge of removing wsme from Ironic's API which is no small feat. I've run this nomination past the existing cores and everyone responded in full support of doing so. So if there are no objections, I'll add Steve to ironic-core next week. Otherwise, Congratulations Steve! -Julia

3 2

[qa][all] New hacking 4.0.0 is available
by Ghanshyam Mann 25 Nov '20

25 Nov '20

Hello Everyone, Hacking 4.0.0 is available to use now which includes many common checks used in various projects. While upgrading to new hacking you can remove those checks definition from your repo hacking checks. Check release notes to find what all common checks are being moved from the project side to hacking: - https://docs.openstack.org/releasenotes/hacking/unreleased.html#relnotes-4-… -gmann

1 0

[cinder][kayobe][kolla][OSA][tripleo][release] Victoria cycle-trailing release deadline
by Herve Beraud 25 Nov '20

25 Nov '20

Hello teams with deliverables following the cycle-trailing release model! This is just a reminder about wrapping those Victoria trailing deliverables up. A few cycles ago we extended the deadline for cycle-trailing to give more time, so the actual deadline isn't until January 14, 2021: https://releases.openstack.org/wallaby/schedule.html#w-cycle-trail If things are ready sooner than that though, all the better for our downstream consumers. Just for awareness, the following cycle-trailing deliverables will need their final releases at some point in the next few months: cinderlib kayobe kolla-ansible kolla openstack-ansible os-collect-config os-refresh-config tripleo-ipsec Thanks! -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 0

[tripleo][ci] Issues w/ the quay container registry
by Wesley Hayutin 25 Nov '20

25 Nov '20

FYI.. This is impacting jobs atm. https://bugs.launchpad.net/tripleo/+bug/1905592 Will follow up when we're clear. 0/

1 0

[magnum] Can not deploy a cluster using Fedora CoreOS 32
by monadt 25 Nov '20

25 Nov '20

I am trying to use Magnum on Openstack that was deployed using kolla-ansible. The instance is completely private (so floating IPs are only accessible inside a VPN network) - not sure if this is a problem. The deployment looks as follows: Openstack Nodes: Ubuntu 20.04 Kolla-Ansible version: '10.1.0' kolla_base_distro: "ubuntu" openstack_release: "ussuri" I could successfully deploy the required magnum services and created a cluster template using the magnum cli as follows: openstack coe cluster template create kubernetes-cluster-template --image fedora-coreos-32 --external-network public-external --dns-nameserver 1.1.1.1 --master-flavor m1.small --flavor m1.small --docker-storage-driver overlay --coe kubernetes I am using the Fedora CoreOS 32 release (and uploaded the image like this): openstack image create --disk-format=qcow2 --container-format=bare --file=fedora-coreos-32.20201104.3.0-openstack.x86_64.qcow2 --property os_distro='fedora-coreos' fedora-coreos-32 I made sure to set the os_distro as stated in the repository when using fedora coreos. When I now try to create a cluster (openstack coe cluster create kubernetes-cluster --cluster-template kubernetes-cluster-template --master-count 1 --node-count 1 --keypair mykey) all the heat resources start spinning up, but the kube-masters resource will never come up. While debugging it seems to hang during the following step inside the heat-container-agent: + '[' ok = ok ']' + kubectl patch node kubernetes-cluster-flfrufngma6x-master-0 --patch '{"metadata": {"labels": {"node-role.kubernetes.io/master": ""}}}' error: no configuration has been provided, try setting KUBERNETES_MASTER environment variable + echo 'Trying to label master node with node-role.kubernetes.io/master=""' + sleep 5s Trying to label master node with node-role.kubernetes.io/master="" I also didn't find a way to set the KUBERNETES_MASTER environment variable from the outside. I assume there is no other supported distribution given that Fedora Atomic and non-Fedora CoreOS do no longer exist? Sent with [ProtonMail](https://protonmail.com) Secure Email.

3 2

[cinder] wallaby R-18 virtual mid-cycle on 9 december
by Brian Rosmaita 25 Nov '20

25 Nov '20

As voted upon at today's weekly meeting, the Cinder Wallaby R-18 virtual mid-cycle will be held: DATE: WEDNESDAY 9 DECEMBER 2020 TIME: 1400-1600 UTC LOCATION: https://bluejeans.com/3228528973 The meeting will be recorded. Please add topics to the mid-cycle etherpad: https://etherpad.opendev.org/p/cinder-wallaby-mid-cycles cheers, brian

1 0

[neutron][operators][all] Watch out for updates of stable/train and stable/stein releases in Neutron
by Slawek Kaplonski 25 Nov '20

25 Nov '20

Hi, I want to warn all of You about terrible mistake which we made in Neutron some time ago. We backported to stable releases patch [1] which broke update workflow. So if You are now updating Your Stein or Train Neutron to latest version and You will do it as it should be done, so first neutron-server and then agents, Your neutron-ovs-agents will not work properly with newer neutron-server. Details are in reported bug [2] Broken versions are: * for Train 15.2.0 and 15.3.0 * for Stein 14.3.1, 14.4.0 and 14.4.1 We proposed reverts of [1] and those reverts are now in gate. As soon as they will be merged we will release new, fixed versions for both Stein and Train. So if You didn't update to those broken versions yet, please don't do it now and wait for next version with fix. If You already updated and fixed that issue on Your clusters - You will have exactly same problem during next update again. I know it's very bad but unfortunately we don't have any other way to fix that issue. [1] https://review.opendev.org/#/c/744133/ [2] https://bugs.launchpad.net/neutron/+bug/1903531 -- Slawek Kaplonski Principal Software Engineer Red Hat

5 7

[neutron][openstack-ansible] Instances can only connect to provider-net via tenant-net but not directly
by Oliver Wenz 25 Nov '20

25 Nov '20

Hi, Starting instances which are connected directly to a provider network results in an error in my Ussuri cloud. However, I can associate floating IPs for this provider network when I connect instances to it via a tenant network and it works fine. Both nova-compute and neutron-linuxbridge-agent services don't show errors, I only get an error in the instance status: 'code': 500, 'created': '2020-11-25T12:05:42Z', 'message': 'Build of instance 274e0a7d-fb33-430a-986e-74fceae6a36d aborted: Failed to allocate the network(s), not rescheduling.', 'details': 'Traceback (most recent call last): File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6549, in _create_domain_and_network post_xml_callback=post_xml_callback) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/compute/manager.py", line 513, in wait_for_instance_event actual_event = event.wait() File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait result = hub.switch() File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 298, in switch return self.greenlet.switch() eventlet.timeout.Timeout: 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/compute/manager.py", line 2378, in _build_and_run_instance accel_info=accel_info) File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3683, in spawn cleanup_instance_disks=created_disks) File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6572, in _create_domain_and_network raise exception.VirtualInterfaceCreateException() nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/compute/manager.py", line 2200, in _do_build_and_run_instance filter_properties, request_spec, accel_uuids) File "/openstack/venvs/nova-21.1.0/lib/python3.6/site-packages/nova/compute/manager.py", line 2444, in _build_and_run_instance reason=msg) nova.exception.BuildAbortException: Build of instance 274e0a7d-fb33-430a-986e-74fceae6a36d aborted: Failed to allocate the network(s), not rescheduling. ' Any ideas what could cause this? Kind regards, Oliver On 2020-11-25 13:00, openstack-discuss-request(a)lists.openstack.org wrote: > Send openstack-discuss mailing list submissions to > openstack-discuss(a)lists.openstack.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss > or, via email, send a message with subject or body 'help' to > openstack-discuss-request(a)lists.openstack.org > > You can reach the person managing the list at > openstack-discuss-owner(a)lists.openstack.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openstack-discuss digest..." > > > Today's Topics: > > 1. Re: > [nova][tripleo][rpm-packaging][kolla][puppet][debian][osa] Nova > enforces that no DB credentials are allowed for the nova-compute > service (Balázs Gibizer) > 2. Re: [ironic] [infra] Making Glean work with IPA for static IP > assignment (Dmitry Tantsur) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 25 Nov 2020 11:13:23 +0100 > From: Balázs Gibizer <balazs.gibizer(a)est.tech> > To: Thomas Goirand <zigo(a)debian.org> > Cc: openstack maillist <openstack-discuss(a)lists.openstack.org> > Subject: Re: > [nova][tripleo][rpm-packaging][kolla][puppet][debian][osa] Nova > enforces that no DB credentials are allowed for the nova-compute > service > Message-ID: <BEKCKQ.PYGQ12VO6AF23(a)est.tech> > Content-Type: text/plain; charset=iso-8859-1; format=flowed > > > > On Mon, Nov 23, 2020 at 13:47, Thomas Goirand <zigo(a)debian.org> wrote: >> On 11/23/20 11:31 AM, Balázs Gibizer wrote: >>> It is still a security problem if nova-compute ignores the config >>> as the >>> config still exists on the hypervisor node (in some deployment >>> scenarios) >> >> Let's say we apply the patch you're proposing, and that nova-compute >> isn't loaded anymore with the db credentials, because it's on a >> separate >> file, and nova-compute doesn't load it. >> >> In such scenario, the /etc/nova/nova-db.conf could still be present >> with >> db credentials filled-in. So, the patch you're proposing is still not >> effective for wrong configuration of nova-compute hosts. > > Obviously we cannot prevent that the deployer stores the DB creds on a > compute host as we cannot detect it in general. But we can detect it if > it is stored in the config the nova-compute reads. I don't know why > should we not make sure to tell the deployer not to do that as it is > generally considered unsafe. > >> >>> From the nova-compute perspective we might be able to >>> replace the [api_database]connection dependency with some hack. E.g >>> to >>> put the service name to the global CONF object at the start of the >>> service binary and depend on that instead of other part of the >>> config. >>> But I feel pretty bad about this hack. >> >> Because of the above, I very much think it'd be the best way to go, >> but >> I understand your point of view. Going to the /etc/nova/nova-db.conf >> and >> nova-api-db.conf thing is probably good anyways. >> >> As for the nova-conductor thing, I very much would prefer if we had a >> clean and explicit "superconductor=true" directive, with possibly some >> checks to display big warnings in the nova-conductor.log file in case >> of >> a wrong configuration. If we don't have that, then at least things >> must >> be extensively documented, because that's really not obvious what's >> going on. > > I agree that superconductor=true would be a more explicit config option > than [api_database]connection. However this would also enforce that > deployers need a separate config file for nova-compute as there neither > superconductor=true nor superconductor=false (meaning it is a cell > conductor) make sense. > >> >> Cheers, >> >> Thomas Goirand (zigo) >> > > > > > > ------------------------------ > > Message: 2 > Date: Wed, 25 Nov 2020 11:54:13 +0100 > From: Dmitry Tantsur <dtantsur(a)redhat.com> > To: Ian Wienand <iwienand(a)redhat.com> > Cc: openstack-discuss <openstack-discuss(a)lists.openstack.org> > Subject: Re: [ironic] [infra] Making Glean work with IPA for static IP > assignment > Message-ID: > <CACNgkFwVhMMxVRK2PkFEkHORRwY4wY49g7G3CpzYwaFzC27Bjw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, > > Thank you for your input! > > On Wed, Nov 25, 2020 at 3:09 AM Ian Wienand <iwienand(a)redhat.com> wrote: > >> On Tue, Nov 24, 2020 at 11:54:55AM +0100, Dmitry Tantsur wrote: >>> The problem is, I cannot make Glean work with any ramdisk I build. The >> crux >>> of the problem seems to be that NetworkManager (used by default in RHEL, >>> CentOS, Fedora and Debian at least) starts very early, creates the >> default >>> connection and ignores whatever files Glean happens to write afterwards. >> On >>> Debian running `systemctl restart networking` actually helped to pick the >>> new configuration, but I'm not sure we want to do that in Glean. I >> haven't >>> been able to make NetworkManager pick up the changes on RH systems so >> far. >> >> So we do use NetworkManager in the OpenDev images, and we do not see >> NetworkManager starting before glean. >> > > Okay, thanks for confirming. Maybe it's related to how IPA is built? It's > not exactly a normal image after all, although it's pretty close to one. > > >> >> The way it should work is that simple-init in dib installs glean to >> the image. That runs the glean install script (use --use-nm argument >> if DIB_SIMPLE_INIT_NETWORKMANAGER, which is default on centos/fedora) >> which installs two things; udev rules and a systemd handler. >> > > I have checked that these are installed, but I don't know how to verify a > udev rule. > > >> >> The udev is pretty simple [1] and should add a "Wants" for each net >> device; e.g. eth1 would match and create a Wants glean(a)eth1.service, >> which then matches [2] which should write out the ifcfg config file. >> After this, NetworkManager should start, notice the config file for >> the interface and bring it up. >> > > Yeah, I definitely see logging from NetworkManager DHCP before this service > is run (i.e. before the output from Glean). > > >> >>> Do you maybe have any hints how to proceed? I'd be curious to know how >>> static IP assignment works in the infra setup. Do you have images with >>> NetworkManager there? Do you use the simple-init element? >> >> As noted yes we use this. Really only in two contexts; it's Rackspace >> that doesn't have DHCP so we have to setup the interface statically >> from the configdrive data. Other clouds all provide DHCP, which is >> used when there's no configdrive data. >> >> Here is a systemd-analyze from one of our Centos nodes if it helps: >> > >> graphical.target @18.403s >> └─multi-user.target @18.403s >> └─unbound.service @5.467s +12.934s >> └─network.target @5.454s >> └─NetworkManager.service @5.339s +112ms >> └─network-pre.target @5.334s >> └─glean(a)ens3.service @4.227s +1.102s >> └─basic.target @4.167s >> └─sockets.target @4.166s >> └─iscsiuio.socket @4.165s >> └─sysinit.target @4.153s >> └─systemd-udev-settle.service @1.905s +2.245s >> └─systemd-udev-trigger.service @1.242s +659ms >> └─systemd-udevd-control.socket @1.239s >> └─system.slice >> > > # systemd-analyze critical-chain > multi-user.target @2min 6.301s > └─tuned.service @1min 32.273s +34.024s > └─network.target @1min 31.590s > └─network-pre.target @1min 31.579s > └─glean(a)enp1s0.service @36.594s +54.952s > └─system-glean.slice @36.493s > └─system.slice @4.083s > └─-.slice @4.080s > > # systemd-analyze critical-chain NetworkManager.service > NetworkManager.service +9.287s > └─network-pre.target @1min 31.579s > └─glean(a)enp1s0.service @36.594s +54.952s > └─system-glean.slice @36.493s > └─system.slice @4.083s > └─-.slice @4.080s > > # cat /etc/sysconfig/network-scripts/ifcfg-enp1s0 > # Automatically generated, do not edit > DEVICE=enp1s0 > BOOTPROTO=static > HWADDR=52:54:00:1f:79:7e > IPADDR=192.168.122.42 > NETMASK=255.255.255.0 > ONBOOT=yes > NM_CONTROLLED=yes > > # ip addr > ... > 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state > UP group default qlen 1000 > link/ether 52:54:00:1f:79:7e brd ff:ff:ff:ff:ff:ff > inet 192.168.122.77/24 brd 192.168.122.255 scope global dynamic > noprefixroute enp1s0 > valid_lft 42957sec preferred_lft 42957sec > inet6 fe80::f182:7fb4:7a39:eb7b/64 scope link noprefixroute > valid_lft forever preferred_lft forever > > >> At a guess; I feel like the udev bits are probably not happening >> correctly in your case? That's important to get the glean@<interface> >> service in the chain to pre-create the config file >> > > It seems that the ordering is correct and the interface service is > executed, but the IP address is nonetheless wrong. > > Can it be related to how long glean takes to run in my case (54 seconds vs > 1 second in your case)? > > Dmitry > > >> >> -i >> >> [1] >> https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-udev.r… >> [2] >> https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-nm@.se… >> >> >

3 2

[nova][tripleo][rpm-packaging][kolla][puppet][debian][osa] Nova enforces that no DB credentials are allowed for the nova-compute service
by Balázs Gibizer 25 Nov '20

25 Nov '20

Dear packagers and deployment engine developers, Since Icehouse nova-compute service does not need any database configuration as it uses the message bus to access data in the database via the conductor service. Also, the nova configuration guide states that the nova-compute service should not have the [api_database]connection config set. Having any DB credentials configured for the nova-compute is a security risk as well since that service runs close to the hypervisor. Since Rocky[1] nova-compute service fails if you configure API DB credentials and set upgrade_level config to 'auto'. Now we are proposing a patch[2] that makes nova-compute fail at startup if the [database]connection or the [api_database]connection is configured. We know that this breaks at least the rpm packaging, debian packaging, and puppet-nova. The problem there is that in an all-in-on deployment scenario the nova.conf file generated by these tools is shared between all the nova services and therefore nova-compute sees DB credentials. As a counter-example, devstack generates a separate nova-cpu.conf and passes that to the nova-compute service even in an all-in-on setup. The nova team would like to merge [2] during Wallaby but we are OK to delay the patch until Wallaby Milestone 2 so that the packagers and deployment tools can catch up. Please let us know if you are impacted and provide a way to track when you are ready with the modification that allows [2] to be merged. There was a long discussion on #openstack-nova today[3] around this topic. So you can find more detailed reasoning there[3]. Cheers, gibi [1] https://github.com/openstack/nova/blob/dc93e3b510f53d5b2198c8edd22528f0c899… [2] https://review.opendev.org/#/c/762176 [3] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… -- http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.…

11 53

[release][infra] SSH key error - RedFlag stop approving
by Herve Beraud 25 Nov '20

25 Nov '20

Hello, We are facing SSH issues on our release jobs [1][2]. Indeed, our release account failed to connect to gerrit through SSH [2]. The ID seems to be recognized when we try it. So that would probably mean we have a stale host key on the machine the script is running on . We suppose that the issue could be a side effect of the gerrit upgrade, we updated the etherpad to inform the infra team [3] @release managers: We should stop approving, so, please hold all our unmerged patches while this incident isn't solved. Thanks for your reading, [1] http://lists.openstack.org/pipermail/release-job-failures/2020-November/001… Example of faced error: ``` ssh://release@review.opendev.org:29418/openstack/python-senlinclient.git did not work. Description: Host key verification failed. Could not read from remote repository ``` [3] https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 2

[all] X Release Naming
by Sean McGinnis 24 Nov '20

24 Nov '20

Hello everyone, This is to notify everyone that we are now kicking off the release naming process for the 'X' OpenStack release. Starting with the last W naming, we had altered our traditional way of choosing a release name to allow any name that meets the basic criteria - basically that the name begins with the chosen letter, it is made up if the ISO basic Latin alphabet, and that it is 10 characters or less in length. Community proposed names are collected on the wiki page for the X Release Naming. This page also includes more details about the process and timing: https://wiki.openstack.org/wiki/Release_Naming/X_Proposals We welcome any names from the community. This should be an intersting release to see what everyone comes up with! Please add any ideas to the wiki page. -- Sean McGinnis

1 1

[cinder] this week's meeting in video+IRC
by Brian Rosmaita 24 Nov '20

24 Nov '20

Quick reminder that this week's Cinder team meeting on Wednesday 25 November, being the final meeting of the month, will be held in both videoconference and IRC at the regularly scheduled time of 1400 UTC. Here's a quick reminder of the video meeting rules we've agreed to: * Everyone will keep IRC open during the meeting. * We'll take notes in IRC to leave a record similar to what we have for our regular IRC meetings. * Some people are more comfortable communicating in written English. So at any point, any attendee may request that the discussion of the current topic be conducted entirely in IRC. connection info: https://bluejeans.com/3228528973 cheers, brian

1 0

RDO Community Meeting Update
by Amy Marrich 23 Nov '20

23 Nov '20

During the virtual Get Together, we decided to have the first meeting of the month virtually to help with the lack of face to face time between contributors at events. Please join us during the virtual meetings using the following link: meet.google.com/uzo-tfkt-top We will also be keeping notes within the #rdo channel on Freenode for our normal post-meeting sharing so if you can not join us virtually please join us via IRC as usual. Thanks and hope to see everyone virtually on December 2nd at 14:00 UTC! Amy (spotz)

1 0

[neutron] Cannot deploy instance in provider network openstack Ussuri -- network fails
by Pavlos Basaras 23 Nov '20

23 Nov '20

Hello, i am new to Openstack. I have followed the instructions from https://docs.openstack.org/install-guide/ to install openstack Ussuri where i use a controller node and a compute node (i can ping each other and access the internet from both nodes through the mgmt iface). I use ubuntu 18.04 for all components. I have configured the network option 2, seems to work fine: openstack network agent list +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ | 2d8c3a89-32c4-4b97-aa4f-ca19db53b24f | L3 agent | controller | nova | :-) | UP | neutron-l3-agent | | 413cd13d-88d7-45ce-8b2e-26fdb265740f | Metadata agent | controller | None | :-) | UP | neutron-metadata-agent | | 42f57bee-63b3-44e6-9392-939ece98719d | Linux bridge agent | compute | None | :-) | UP | neutron-linuxbridge-agent | | 4a787a09-04aa-4350-bd32-0c0177ed06a1 | DHCP agent | controller | nova | :-) | UP | neutron-dhcp-agent | | fdafc337-7581-4ecd-b175-810713a25e1f | Linux bridge agent | controller | None | :-) | UP | neutron-linuxbridge-agent | +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ If i try to bring up instances connected by an internal network, they are successfully deployed. When i use the provider network to bring up an instance with the command: openstack server create --flavor m1.tiny --image cirros034 --nic net-id=4463e4a7-3b06-46d9-b24f-3e2e7867ee96 --security-group=5cf380e5-efb3-48b3-b8e9-095b1ebd817d provider-instance the instance fails to be instantiated because the network cannot be deployed. I noticed two points from the neutron-linuxbridge-agent.log file at the compute node (the logs at the controller seem fine): 1) ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied') 2) INFO neutron.plugins.ml2.drivers.agent._common_agent [req-816902d4-6581-49c1-939a-6ea69b437c99 - - - - -] Linux bridge agent Agent out of sync with plugin! -- not sure if this one is a problem. One part that i am not sure is the following configuration since i am on ubuntu 18 and not ubuntu 16: # The provider network interface auto INTERFACE_NAME iface INTERFACE_NAME inet manual up ip link set dev $IFACE up down ip link set dev $IFACE down I did not include this part in my file since its ubuntu 18, and netplan does not have an equivalent (i think) option. I have this netplan config enp0s8: --> mgmt dhcp4: false addresses: [10.0.0.31/24] gateway4: 10.0.0.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] enp0s9: --> provider dhcp4: false any advice? all the best, Pavlos Basaras

2 4

[manila] Proposing Maari Tamm for python-manilaclient core
by Victoria Martínez de la Cruz 23 Nov '20

23 Nov '20

Hi everyone, It's a great pleasure for me to propose Maari Tamm (maaritamm) for the python-manilaclient core group. Maari joined us as an Outreachy intern working on the manila support for openstackclient in December 2019. She excelled at delivering the main task, and she went above and beyond by continuing to contribute with the community even after her internship finished. She now continues submitting enhancements for the different manila components and taking on new challenges as they appear. Not only that, Maari joined us as mentor for the manila project in different efforts we have made: the Open Source Day in Grace Hopper Celebration 2020 and also mentoring for three students from the Boston University that are currently working with us in manila. Maari is a truly valuable member of our team and I believe she will be a great addition to the python-manilaclient core reviewers group. Looking forward to a positive response. All the best, Victoria

8 8

[oslo] following or not following the DST changes for our meeting time
by Herve Beraud 23 Nov '20

23 Nov '20

Hello Oslofolk, As discussed during our previous meeting, at each DST change, the agenda of some of us conflicting with our Oslo meeting time slot, this thread just wants to propose a solution to avoid that. We could just move our meeting time when DST changes, and then move back to this slot once DST is back. Right now our meeting time is UTC based, we suppose that UTC is required because of the OpenStack meeting tooling. By following that we will get the following time slots: - meeting time slot when DST start: 5:00pm UTC - meeting time slot when DST end: 4:00pm UTC Does it fit well for you? -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

4 6

[nova] heads up about the limitations of the new gerrit
by Balázs Gibizer 23 Nov '20

23 Nov '20

Hi, We have the new shiny gerrit. But please be aware the limitations[1]. I want to highlight one. The launchpad - gerrit integration does not work so the bluperints and bugs are not updated in launchpad based on the changes in gerrit. So please manually update launchpad. cheers, gibi [1] https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes

2 1

[neutron] Bug deputy report November 16 - 22
by Jakub Libosvar 23 Nov '20

23 Nov '20

Hi all, I was the bug deputy for the last week, here is the report: Critical * Failure in test_l2_agent_restart(OVS,Flat network) https://bugs.launchpad.net/neutron/+bug/1904897 This is failing on the gate occasionally Needs an assignee High * GRE tunnels over IPv6 are created in wrong way https://bugs.launchpad.net/neutron/+bug/1904564 Assigned to: Slawek Patch review pending: https://review.opendev.org/c/openstack/neutron/+/763204/ * [OVN] Metadata namespace checksum error with DPDK https://bugs.launchpad.net/neutron/+bug/1904871 Assigned to: Rodolfo Medium * [ovn] Don't include IP addresses for OVN ports if both port security and DHCP are disabled https://bugs.launchpad.net/neutron/+bug/1904412 Assigned to: Elvira Patch review pending: https://review.opendev.org/c/openstack/neutron/+/763567/ * Designate driver: allow_reverse_dns_lookup doesn't works if dns_domain zone wasn't created https://bugs.launchpad.net/neutron/+bug/1904559 Reporter offered a patch but hasn't sent yet * [OVN] Inconsistent "flooding to unregistered" IGMP configuration https://bugs.launchpad.net/neutron/+bug/1904399 Assigned to: Lucas Patch review pending: https://review.opendev.org/c/openstack/neutron/+/762818/ Wishlist * "gateway" option is not honored when creating a subnet with subnet pool https://bugs.launchpad.net/neutron/+bug/1904436 [RFE] Extend neutron-metadata-agent to support to proxy multiple external services https://bugs.launchpad.net/neutron/+bug/1905115 This is an RFE, so far marked as wishlist, should be discussed by the drivers team. Undecided * Enforce security group cleanup (Resources leaks) https://bugs.launchpad.net/neutron/+bug/1904694 It was not clear to me where does the error come from, I assumed tempest. * Neutron – it’s possible to delete router’s port related to external gateway https://bugs.launchpad.net/neutron/+bug/1904751 * neutron-dynamic-routing reported as not alive after adding BGP Peer to BGP Speaker https://bugs.launchpad.net/neutron/+bug/1904869 It seems to me like a configuration issue, agent missing rabbitmq settings. Invalid * Neutron – it’s possible to delete router’s port related to external gateway There is a patch: https://review.opendev.org/#/c/763198/ It's claimed this patch is reproducing the issue but the patch fails elsewhere. I don't think this report is valid.

1 0

[kolla][ptg] Kolla Wallaby priorities
by Mark Goddard 23 Nov '20

23 Nov '20

Hi, Thanks to everyone who attended the Kolla PTG sessions, we covered a lot of ground. The Etherpad is available at [1]. I wrote an email [2] summarising the discussions. I have moved candidate work items across to a second Etherpad [2] for voting on. Now is the time to vote! Please add your name against a maximum of 12 items across the 3 deliverables (kolla, kolla-ansible, kayobe) in the priorities [2] Etherpad. Anyone in the community is welcome to vote. [1] https://etherpad.opendev.org/p/kolla-wallaby-ptg [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018445.… [3] https://etherpad.opendev.org/p/kolla-wallaby-priorities Thanks, Mark

1 3

[nova] How best to provide virtio-based input devices?
by Stephen Finucane 20 Nov '20

20 Nov '20

Design discussion for interested parties. As discussed during the nova team meeting today, we're looking at providing support for virtio-based input devices in nova, given benefits w.r.t. performance and security (no emulated USB bus!). The current proposal I have for this is to introduce a new image metadata property, 'hw_input_bus', and extend the existing 'hw_pointer_model' image metadata property, which currently only accepts a 'usbtablet' value, to accept 'mouse' or 'tablet' values. This means you'd end up with a matrix something like so: +------------------+--------------+------------------------+ | hw_pointer_model | hw_input_bus | Result | +------------------|--------------+------------------------+ | - | - | PS2-based mouse [*] | | usbtablet | - | USB-based tablet | | usbtablet | usb | USB-based tablet | | usbtablet | virtio | **ERROR** | | mouse | - | USB-based mouse | | mouse | usb | USB-based mouse | | mouse | virtio | virtio-based mouse | | tablet | - | USB-based tablet | | tablet | usb | USB-based tablet | | tablet | virtio | virtio-based tablet | +------------------+--------------+------------------------+ [*] libvirt adds these by default on x86 hosts dansmith noted that the only reason to select the 'mouse' pointer model nowadays is for compatibility, and something like a virtio-based mouse didn't really make sense. That being the case, I agree that this change is likely more complex that it needs to be. We do, however, disagree on the remedy. dansmith's idea is to drop the 'hw_input_bus' image metadata property entirely and simply add a new 'virtiotablet' value to 'hw_pointer_model' instead. This still leaves the question of what bus we should use for keyboards, and his suggestion there is to extrapolate out and use virtio for keyboards if 'hw_pointer_model=virtiotablet' is specified and presumably USB if 'hw_pointer_model=usbtablet'. Needless to say, I don't like this idea and prefer we took another tack and kept 'hw_input_bus' but didn't build on 'hw_pointer_model' and instead "deprecated" it. We can't really ever remove the an image metadata property, since that would be a breaking upgrade, which means we'll eventually be left with effectively dead code to maintain forever. However, I don't think that's a big deal. 'hw_pointer_model=usbtablet' is already on a path to obsolescence as the Q35 machine type slowly becomes the default on x86 hosts and the use of non-x86 hosts grows since neither support PS2 and must use a USB-based input device. In addition, I think the extrapolation of 'virtiotablet' to mean also virtio-based keyboard is unclear and leaves a gaping hole w.r.t. requesting USB-based keyboards on non-AArch64 hosts (where it's currently added by default), since we don't currently do this extrapolation and introducing it would be breaking change on x86 hosts (instances would suddenly switch from PS2-based keyboards to USB-based ones). We need to decide what approach to go for before I rework this. If anyone has input, particularly operators that think they'd use this feature, I'd love to hear it so that I can t̶e̶l̶l̶ ̶d̶a̶n̶s̶m̶i̶t̶h̶ ̶t̶o̶ ̶s̶h̶o̶v̶e̶ ̶i̶t̶ come to the best possible solution ;-) Feel free to either reply here or on the review [1]. Cheers, Stephen [1] https://review.opendev.org/#/c/756552/

3 7

[openstack-helm] No Meeting Next Week
by Gage Hugo 20 Nov '20

20 Nov '20

Hello everyone, Due to Thanksgiving in the US next week, the openstack-helm meeting will be cancelled, we will meet again at our regular time on Dec 1st. Hope everyone has a good holiday and stays safe!

1 0

[Security-SIG] No meeting next week
by Gage Hugo 20 Nov '20

20 Nov '20

Hello everyone, Due to Thanksgiving in the US next week, the Security SIG meeting will be cancelled, we will meet again at our regular time on Dec 3rd. Hope everyone has a good holiday and stays safe!

1 0

[all][diversity] IRC Channel Renames
by Kendall Nelson 20 Nov '20

20 Nov '20

Hello! So now that the OpenStack Foundation is now the Open Infrastructure Foundation, I think it time to take a look at some of the IRC channels we use. There are a few that come to mind that are used by the larger Foundation community- not just OpenStack anymore- that should probably be renamed[0]. Below is a list of the channels that come to mind and proposed new channel names. Please add to the list or suggest alternative names if you hate what I've come up with. My goal is to process these renames by the end of the year. - #openstack-ptg -> #oif-ptg - #openstack-forum -> #oif-forum - #openstack-diversity -> #oif-diversity - #openstack-board -> #oif-board - #openstack-foundation -> #oif-foundation Look forward to everyone's thoughts! -Kendall (diablo_rojo) [0] https://docs.opendev.org/opendev/system-config/latest/irc.html#renaming-an-…

4 5

[heat][tripleo] upstream heat ci job dashboard
by Wesley Hayutin 20 Nov '20

20 Nov '20

Greetings, Just an FYI.. Adriano has put together a nice dashboard for getting a high level view of the upstream CI jobs for heat [1]. Thanks Adriano, well done! [1] http://dashboard-ci.tripleo.org/d/TsdDSjtMz/upstream-heat?orgId=1

1 0

[openstack] [Interop] Todays Friday weekly meeting cancelled - will meet next week same time same meetpad
by prakash RAMCHANDRAN 20 Nov '20

20 Nov '20

Hi all,We have gerrit access issues for any Interop WG activity to be possible for Victoria cycle. Will meet next Friday 10 AM PST instead refer: [service-announce] review.opendev.org Gerrit outage and upgrade 15:00UTC November 20 to 01:00UTC November 23, 2020 | | | | [service-announce] review.opendev.org Gerrit outage and upgrade 15:00UTC... | | | Please add your items if any for next week 27th NovOpenDev Etherpad | | | | OpenDev Etherpad | | | ThanksFor Interop WGPrakash

1 0

[neutron] Floating ips instances not appear in tcpdump
by Cristina Mayo 20 Nov '20

20 Nov '20

Hello, I have a multinode Openstack cloud installed on Ubuntu machines following the official guides, without extra settings. I have realised that all the income traffic on my instances with floating ips have the same source ip (controller's node ip address). Could anyone help to understand this behaviour? I would like source ip address remains because I am interested in filter traffic, and it's currently impossible. It seems that my controller node is changing the original ip to the packets. Thanks in advance, Regards

3 4

[horizon] Error when clicking instance detailed view after upgrading from ussuri to victoria
by Bujack, Stefan 20 Nov '20

20 Nov '20

Hello, I am a little lost here. Hopefully some of you nice people could help me with this issue please. We had an Openstack Ussuri deployment on Ubuntu 20.04 and upgraded to Victoria Release via cloud_archive repository Now when I click on any instance for a detailed view I get an error 500 with the following message: ==> /var/log/apache2/error.log <== [Wed Nov 18 19:42:37.883040 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] /usr/lib/python3/dist-packages/django/contrib/staticfiles/templatetags/staticfiles.py:24: RemovedInDjango30Warning: {% load staticfiles %} is deprecated in favor of {% load static %}. [Wed Nov 18 19:42:37.883091 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] warnings.warn( [Wed Nov 18 19:42:37.886358 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] ERROR django.request Internal Server Error: /project/instances/25d58259-9d9d-4215-8fb6-521ff9a5befe/ [Wed Nov 18 19:42:37.886400 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] Traceback (most recent call last): [Wed Nov 18 19:42:37.886407 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 829, in _resolve_lookup [Wed Nov 18 19:42:37.886412 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] current = current[bit] [Wed Nov 18 19:42:37.886416 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] TypeError: 'Server' object is not subscriptable [Wed Nov 18 19:42:37.886421 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] [Wed Nov 18 19:42:37.886425 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] During handling of the above exception, another exception occurred: [Wed Nov 18 19:42:37.886430 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] [Wed Nov 18 19:42:37.886456 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] Traceback (most recent call last): [Wed Nov 18 19:42:37.886460 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/openstack_dashboard/api/base.py", line 139, in __getattribute__ [Wed Nov 18 19:42:37.886465 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return object.__getattribute__(self, attr) [Wed Nov 18 19:42:37.886469 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] AttributeError: 'Server' object has no attribute 'OS-EXT-SRV-ATTR:instance_name' [Wed Nov 18 19:42:37.886473 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] [Wed Nov 18 19:42:37.886477 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] During handling of the above exception, another exception occurred: [Wed Nov 18 19:42:37.886482 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] [Wed Nov 18 19:42:37.886486 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] Traceback (most recent call last): [Wed Nov 18 19:42:37.886501 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner [Wed Nov 18 19:42:37.886647 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] response = get_response(request) [Wed Nov 18 19:42:37.886651 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 145, in _get_response [Wed Nov 18 19:42:37.886656 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] response = self.process_exception_by_middleware(e, request) [Wed Nov 18 19:42:37.886661 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 143, in _get_response [Wed Nov 18 19:42:37.886665 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] response = response.render() [Wed Nov 18 19:42:37.886668 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/response.py", line 106, in render [Wed Nov 18 19:42:37.886672 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] self.content = self.rendered_content [Wed Nov 18 19:42:37.886676 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/response.py", line 83, in rendered_content [Wed Nov 18 19:42:37.886680 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] content = template.render(context, self._request) [Wed Nov 18 19:42:37.886684 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 61, in render [Wed Nov 18 19:42:37.886688 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.template.render(context) [Wed Nov 18 19:42:37.886692 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 171, in render [Wed Nov 18 19:42:37.886696 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self._render(context) [Wed Nov 18 19:42:37.886700 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 163, in _render [Wed Nov 18 19:42:37.886704 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.nodelist.render(context) [Wed Nov 18 19:42:37.886708 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.886717 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.886721 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.886725 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.886730 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 150, in render [Wed Nov 18 19:42:37.886863 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return compiled_parent._render(context) [Wed Nov 18 19:42:37.886871 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 163, in _render [Wed Nov 18 19:42:37.886875 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.nodelist.render(context) [Wed Nov 18 19:42:37.886894 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.886898 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.886902 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.886906 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.886909 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 62, in render [Wed Nov 18 19:42:37.886913 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] result = block.nodelist.render(context) [Wed Nov 18 19:42:37.886917 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.886945 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.886973 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.886978 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.886982 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 62, in render [Wed Nov 18 19:42:37.886986 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] result = block.nodelist.render(context) [Wed Nov 18 19:42:37.886991 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.886996 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.887001 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887006 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887019 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 987, in render [Wed Nov 18 19:42:37.887093 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] output = self.filter_expression.resolve(context) [Wed Nov 18 19:42:37.887101 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 671, in resolve [Wed Nov 18 19:42:37.887105 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] obj = self.var.resolve(context) [Wed Nov 18 19:42:37.887109 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 796, in resolve [Wed Nov 18 19:42:37.887113 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] value = self._resolve_lookup(context) [Wed Nov 18 19:42:37.887117 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 858, in _resolve_lookup [Wed Nov 18 19:42:37.887121 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] current = current() [Wed Nov 18 19:42:37.887155 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/horizon/tabs/base.py", line 227, in render [Wed Nov 18 19:42:37.887221 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return render_to_string(self.template_name, {"tab_group": self}) [Wed Nov 18 19:42:37.887236 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/loader.py", line 62, in render_to_string [Wed Nov 18 19:42:37.887242 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return template.render(context, request) [Wed Nov 18 19:42:37.887246 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 61, in render [Wed Nov 18 19:42:37.887250 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.template.render(context) [Wed Nov 18 19:42:37.887254 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 171, in render [Wed Nov 18 19:42:37.887258 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self._render(context) [Wed Nov 18 19:42:37.887262 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 163, in _render [Wed Nov 18 19:42:37.887267 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.nodelist.render(context) [Wed Nov 18 19:42:37.887272 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.887276 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.887280 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887284 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887288 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 150, in render [Wed Nov 18 19:42:37.887293 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return compiled_parent._render(context) [Wed Nov 18 19:42:37.887297 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 163, in _render [Wed Nov 18 19:42:37.887382 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.nodelist.render(context) [Wed Nov 18 19:42:37.887395 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.887403 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.887407 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887411 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887415 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 513, in render [Wed Nov 18 19:42:37.887425 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.nodelist.render(context) [Wed Nov 18 19:42:37.887429 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.887434 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.887438 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887443 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887448 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 309, in render [Wed Nov 18 19:42:37.887453 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return nodelist.render(context) [Wed Nov 18 19:42:37.887458 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.887462 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.887467 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887472 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887478 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 209, in render [Wed Nov 18 19:42:37.887483 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] nodelist.append(node.render_annotated(context)) [Wed Nov 18 19:42:37.887488 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887493 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887498 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 987, in render [Wed Nov 18 19:42:37.887503 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] output = self.filter_expression.resolve(context) [Wed Nov 18 19:42:37.887587 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 671, in resolve [Wed Nov 18 19:42:37.887594 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] obj = self.var.resolve(context) [Wed Nov 18 19:42:37.887598 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 796, in resolve [Wed Nov 18 19:42:37.887602 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] value = self._resolve_lookup(context) [Wed Nov 18 19:42:37.887606 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 858, in _resolve_lookup [Wed Nov 18 19:42:37.887610 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] current = current() [Wed Nov 18 19:42:37.887614 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/horizon/tabs/base.py", line 372, in render [Wed Nov 18 19:42:37.887619 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return render_to_string(self.get_template_name(self.request), context) [Wed Nov 18 19:42:37.887634 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/loader.py", line 62, in render_to_string [Wed Nov 18 19:42:37.887638 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return template.render(context, request) [Wed Nov 18 19:42:37.887642 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 61, in render [Wed Nov 18 19:42:37.887652 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.template.render(context) [Wed Nov 18 19:42:37.887657 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 171, in render [Wed Nov 18 19:42:37.887661 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self._render(context) [Wed Nov 18 19:42:37.887665 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 163, in _render [Wed Nov 18 19:42:37.887669 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.nodelist.render(context) [Wed Nov 18 19:42:37.887673 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 937, in render [Wed Nov 18 19:42:37.887678 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] bit = node.render_annotated(context) [Wed Nov 18 19:42:37.887682 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 904, in render_annotated [Wed Nov 18 19:42:37.887686 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.render(context) [Wed Nov 18 19:42:37.887691 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 302, in render [Wed Nov 18 19:42:37.887696 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] match = condition.eval(context) [Wed Nov 18 19:42:37.887707 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 876, in eval [Wed Nov 18 19:42:37.887711 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return self.value.resolve(context, ignore_failures=True) [Wed Nov 18 19:42:37.887729 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 671, in resolve [Wed Nov 18 19:42:37.887734 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] obj = self.var.resolve(context) [Wed Nov 18 19:42:37.887739 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 796, in resolve [Wed Nov 18 19:42:37.887744 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] value = self._resolve_lookup(context) [Wed Nov 18 19:42:37.887748 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/django/template/base.py", line 837, in _resolve_lookup [Wed Nov 18 19:42:37.887753 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] current = getattr(current, bit) [Wed Nov 18 19:42:37.887758 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/openstack_dashboard/api/base.py", line 139, in __getattribute__ [Wed Nov 18 19:42:37.887763 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return object.__getattribute__(self, attr) [Wed Nov 18 19:42:37.887767 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/openstack_dashboard/api/_nova.py", line 88, in has_extended_attrs [Wed Nov 18 19:42:37.887772 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return any(getattr(self, attr) for attr in [ [Wed Nov 18 19:42:37.887777 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/openstack_dashboard/api/_nova.py", line 88, in <genexpr> [Wed Nov 18 19:42:37.887782 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return any(getattr(self, attr) for attr in [ [Wed Nov 18 19:42:37.887786 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/openstack_dashboard/api/base.py", line 144, in __getattribute__ [Wed Nov 18 19:42:37.887791 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] return getattr(self._apiresource, attr) [Wed Nov 18 19:42:37.887795 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] File "/usr/lib/python3/dist-packages/novaclient/base.py", line 176, in __getattr__ [Wed Nov 18 19:42:37.887800 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] raise AttributeError(k) [Wed Nov 18 19:42:37.887805 2020] [wsgi:error] [pid 846:tid 140680051717888] [remote 131.169.5.116:50070] AttributeError: OS-EXT-SRV-ATTR:instance_name Thanks in advance, Stefan Bujack

2 2

[ops][rdo-users][TripleO][RDO] novajoin project
by Belmiro Moreira 20 Nov '20

20 Nov '20

Hello, at CERN we are deploying FreeIPA and the project "novajoin" is very interesting to enroll OpenStack instances. Novajoin is "a nova vendordata plugin for the OpenStack nova metadata service to manage host instantiation in an IPA server." The project is packaged by RDO, so I believe there are some users around. It would be great to hear your experiences with "novajoin". In my prototype the project is working correctly, but now that I'm going deep I'm finding some issues that I would like to discuss. What is the preferred way to report/discuss issues? There is a launchpad page (https://launchpad.net/novajoin) but not much activity during the last months. Is there someone still working on it? thanks, Belmiro

2 2

[nova] Wallaby spec review day, round 2
by Balázs Gibizer 20 Nov '20

20 Nov '20

Hi, We had a successful spec review day on Tuesday. Most of the open specs has got reviewer attention. We will do another coordinated spec review round on 1st of December. Cheers, gibi

1 0

[neutron] Drivers meeting 20.11.2020 - cancelled
by Slawek Kaplonski 19 Nov '20

19 Nov '20

Hi, As there is no any new RFE to discuss, lets cancel this week's meeting. Please instead spent some time on reviewing some opened specs to approved RFEs [1]. Especially: * https://review.opendev.org/#/c/729532/ * https://review.opendev.org/#/c/739549/ * https://review.opendev.org/#/c/728628/ Have a great weekend and see You all online :) [1] https://review.opendev.org/#/q/project:openstack/neutron-specs++status:open -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[tripleo] Copying tripleo container images to quay.io
by Arx Cruz 19 Nov '20

19 Nov '20

Hello, I just wrote a tool in go that is right now copying every 2 hours all the containers from rdo registry to quay.io. Right now it's copying all branches: master, victoria, ussuri, train, stein, queens and rocky. You can see the containers in https://quay.io/tripleo{release} for example https://quay.io/tripleomaster The tool basically searches for container build jobs with success status, parse the containers that was built, copy these containers to quay.io and tag it with current-tripleo and the built hash. The code it's based on skopeo, and I did not use it, because there are some other stuff required on quay.io side that requires the use of quay.io api, like create an already public repository, parsing the tripleo container build job, tagging, etc. Also I wanted to play with go :) Right now the code is under review at https://review.rdoproject.org/r/#/c/31133/ and I got it running on our toolbox, and he's an log example (without the --debug flag) time="2020-11-19T12:06:17Z" level=info msg="Copying image centos-binary-horizon:4fad79713786f77292e59fa1c036f588 in tripleoussuri namespace" time="2020-11-19T12:06:18Z" level=info msg="Tagging current-tripleo to sha256:db39e7d43d4c8eec82f61acd4956d7f165b595d515f270f8dabe0c9b009c95f2" "Updated" time="2020-11-19T12:06:19Z" level=info msg="Copying image centos-binary-ceilometer-base:4fad79713786f77292e59fa1c036f588 in tripleoussuri namespace" time="2020-11-19T12:06:20Z" level=info msg="Tagging current-tripleo to sha256:5eee849a9f74d0107cbfd2f456f701be37256ea711079cffcc189d140e8f3176" "Updated" time="2020-11-19T12:06:21Z" level=info msg="Copying image centos-binary-gnocchi-base:4fad79713786f77292e59fa1c036f588 in tripleoussuri namespace" time="2020-11-19T12:06:22Z" level=info msg="Tagging current-tripleo to sha256:5b46ea43d41e56dd81005a234631905a8ad594ef8432ae78542392114956f85e" "Updated" If you want to play around with this, feel free to do so, and any feedback is welcome :) Kind regards, -- Arx Cruz Software Engineer Red Hat EMEA <https://www.redhat.com> arxcruz(a)redhat.com @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

4 4

[release] Meeting Time Poll
by Herve Beraud 19 Nov '20

19 Nov '20

Hello We have a few regular attendees of the Release Management meeting who have conflicts with the current meeting time. As a result, we would like to find a new time to hold the meeting. I've created a Doodle poll[1] for everyone to give their input on times. It's mostly limited to times that reasonably overlap the working day in the US and Europe since that's where most of our attendees are located. If you attend the Release Management meeting, please fill out the poll so we can hopefully find a time that works better for everyone. Thanks! [1] https://doodle.com/poll/7zgvfb8ne54bwhzw -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 1

[oslo][release] Oslo's Transition To Independent
by Herve Beraud 19 Nov '20

19 Nov '20

Greetings, Is it time for us to move some parts of Oslo to the independent release model [1]? I think we could consider Oslo world is mostly stable enough to answer yes to the previous question. However, the goal of this email is to trigger the debat and see which deliverables could be transitioned to the independent model. Do we need to expect that major changes will happen within Oslo and who could warrant to continue to follow cycle-with-intermediary model [2]? The following etherpad [3] is designed to help us to consider which deliverables could be moved. Concerning the roots of this topic, it was raised during the PTG of the Release Management Team [4]. [1] https://releases.openstack.org/reference/release_models.html#independent [2] https://releases.openstack.org/reference/release_models.html#cycle-with-int… [3] https://etherpad.opendev.org/p/oslo-transition-to-independent [4] https://etherpad.opendev.org/p/wallaby-ptg-os-relmgt /me takes off Oslo team hat and puts on release team hat. This kind of transition could be an option for all stable libraries even outside the Oslo scope, don't hesitate to consider this as an option for your projects. Thanks for reading, -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

3 7

NUMATopologyFilter and AMD Epyc Rome
by Eyle Brinkhuis 19 Nov '20

19 Nov '20

Hi all, We’re running into an issue with deploying our infrastructure to run high throughput, low latency workloads. Background: We run Lenovo SR635 systems with an AMD Epyc 7502P processor. In the BIOS of this system, we are able to define the amount of NUMA cells per socket (called NPS). We can set 1, 2 or 4. As we run a 2x 100Gbit/s Mellanox CX5 in this system as well, we use the preferred-io setting in the BIOS to give preferred io throughput to the Mellanox CX5. To make sure we get as high performance as possible, we set the NPS setting to 1, resulting in a single numa cell with 64 CPU threads available. Next, in Nova (train distribution), we demand huge pages. Hugepages however, demands a NUMAtopology, but as this is one large NUMA cell, even with cpu=dedicated or requesting a single numa domain, we fail: compute03, compute03 fails NUMA topology requirements. No host NUMA topology while the instance specified one. host_passes /usr/lib/python3/dist-packages/nova/scheduler/filters/numa_topology_filter.py:119 Any idea how to counter this? Setting NPS-2 will create two NUMA domains, but also cut our performance way down. Thanks! Regards, Eyle

3 5

[cinder] Ceph, active-active and no coordination
by Radosław Piliszek 19 Nov '20

19 Nov '20

Dear Cinder Masters, I have a question for you. (or two, or several; well, actually the whole Kolla team has :-) ) The background is that Kolla has been happily deploying multinode cinder-volume with Ceph RBD backend, with no coordination configured, cluster parameter unset, host properly set per host and backend_host normalised (as well as any other relevant config) between the cinder-volume hosts. The first question is: do we correctly understand that this was an active-active deployment? Or really something else? Now, there have been no reports that it misbehaved for anyone. It certainly has not for any Kolla core. The fact is it was brought to our attention because due to the drop of Kolla-deployed Ceph, the recommendation to set backend_host was not present and users tripped over non-uniform backend_host. And this is expected, of course. The second and final question is, building up on the first one, were we doing it wrong all the time? (plus extras: Why did it work? Were there any quirks? What should we do?) PS: Please let me know if this thought process is actually Ceph-independent as well. -yoctozepto

5 5

Ask a question about nova console
by Yingji Sun 19 Nov '20

19 Nov '20

Buddies, I hit an issue on nova console. In vm console, it only handles keyboard event, but not mouse event. That is I can only use keyboard in the vm, but not mouse. The backend is vsphere. I can use both keyboard and mouse through vsphere remote console. So where does the issue likely hide ? Is there any option in nova configure file that relates to this issue ? Any feedback would be appreciated. Yingji.

1 1

Clone Cinder-Backed Image (Ussuri)
by Lazuardi Nasution 18 Nov '20

18 Nov '20

Hi, I have made a cinder-backed image where my cinder is RBD-backed. The backing topology of that image is as follows. glance -> cinder -> RBD. Is it possible to clone that image like RBD-backed images? I think the following bluprint is related but I don't know if it has been implemented yet, at least on Ussuri. https://blueprints.launchpad.net/cinder/+spec/clone-image-in-glance-cinder-… Best regards,

2 2

[neutron] HA network issues on Rocky
by Jean-Philippe Méthot 18 Nov '20

18 Nov '20

Hi, We’ve been running an Openstack setup in production for several years and we upgraded it to Rocky last May. We are now creating a router for a new project for the first time since that upgrade and we’ve been running into issues with L3HA for that router. Essentially, the router for this project gets initialized with a faulty HA network. How do I know it’s faulty? -Both Keepalived instances think they are master. -Both HA network ports get the wrong MTU (9000 instead of 8958) -Even if I manually set the proper MTU, VRRP packets don’t get received on the other node -If I then disable and enable the router, MTU gets rolled back to 9000 -If I force the MTU for this port through the neutron database, it will also get rolled back to 9000 once the router gets recreated This setup is using openvswitch and GRE tunnels. I have 7 other projects with L3HA routers running and they do not have this issue, even if I create new routers in them. They are also older projects created before the upgrade to Rocky. This problem is strictly isolated to this new project. I’m not sure exactly what’s going on here and why Openstack isn’t taking encapsulation into account when automatically setting its HA network MTU for this. Any suggestion on how to figure out what’s happening behind the scene? Jean-Philippe Méthot Senior Openstack system administrator Administrateur système Openstack sénior PlanetHoster inc. 4414-4416 Louis B Mayer Laval, QC, H7P 0G1, Canada TEL : +1.514.802.1644 - Poste : 2644 FAX : +1.514.612.0678 CA/US : 1.855.774.4678 FR : 01 76 60 41 43 UK : 0808 189 0423

1 1

Fwd: [sdk] issue in update image method
by Roman Budnyk 18 Nov '20

18 Nov '20

Hello dear developers, I was trying to update image data by calling method *update_image_properties. *Each time I was facing the error: TypeError: existing() argument after ** must be a mapping, not str I did small research in the source code and found a strange solution. Could you please check this place in the code: class BaseImageProxy, method update_image_properties (path /openstack/_base_proxy.py): [image: image.png] When I changed it to the below - everything works (with name, id or image instance): [image: image.png] could you please check on your end, why the construction *if image is None *is using and how can we execute the code *self._connection.get_image(image) *if None is passing as the argument. Many thanks! Regards, Roman

2 4

[cinder] python-cinderclient: API v2 support removed in Wallaby
by Brian Rosmaita 18 Nov '20

18 Nov '20

As announced previously on this mailing list [0], at yesterday's Cinder meeting the Cinder team discussed the timeline for removing Block Storage API v2 support from the python-cinderclient. The Block Storage API v2, which has been deprecated since Pike, will not be present in the OpenStack Wallaby release. The team decided not to delay removing Block Storage API v2 support from the python-cinderclient. Thus, v2 support will *not* be available in the Wallaby python-cinderclient. Given the long deprecation period, we trust that this will not adversely impact too many users. We point out that microversion 3.0 of the Block Storage API v3 is functionally identical to v2; thus scripts that rely on v2 responses should continue to work with minimal changes. We also point out that v3 has been available since Mitaka and has reached microversion 3.62 with the Victoria release; thus it may be worth re-examining such scripts to see what functionality you are missing out on. This email serves both as an announcement and a final call for comments about this proposal. Please reply to this email thread with any comments before next week's Cinder meeting (Wednesday 18 November 2020). Thank you, brian [0] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018531…

1 1

[openstack][stein][cinder] iscsi multipath issues
by Ignazio Cassano 18 Nov '20

18 Nov '20

Hello Everyone, I am testing stein with cinder iscsi driver for unity. Since I became crazy on queens, I decided to try with stein. At this time I have only one virtual machine with only one iscsi volume. It was running on podiscsivc-kvm02 and I migrated it on podiscsivc-kvm01. Let me show what multipath -ll displays on podiscsivc-kvm02 after live migration: 36006016006e04400dce5b45f0ac77301 dm-3 DGC ,VRAID size=40G features='1 retain_attached_hw_handler' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=0 status=active | `- 17:0:0:60 sdm 8:192 failed faulty running `-+- policy='round-robin 0' prio=0 status=enabled |- 19:0:0:60 sdk 8:160 failed faulty running `- 21:0:0:60 sdj 8:144 failed faulty running And now on destination node podiscsivc-kvm01: 36006016006e04400dce5b45f0ac77301 dm-3 DGC ,VRAID size=40G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 17:0:0:14 sdm 8:192 active ready running | `- 15:0:0:14 sdl 8:176 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 21:0:0:14 sdj 8:144 active ready running `- 19:0:0:14 sdk 8:160 active ready running On source node in /var/log/messages I get: Nov 18 10:34:19 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: failed in domap for addition of new path sdl Nov 18 10:34:19 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: uev_add_path sleep Nov 18 10:34:20 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: failed in domap for addition of new path sdl Nov 18 10:34:20 podiscsivc-kvm02 kernel: device-mapper: table: 253:3: multipath: error getting device Nov 18 10:34:20 podiscsivc-kvm02 kernel: device-mapper: ioctl: error adding target to table Nov 18 10:34:20 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: uev_add_path sleep On storage node seems it wotks fine, because the host access migrated from source to destination node. Multipath.conf is the following: blacklist { # Skip the files uner /dev that are definitely not FC/iSCSI devices # Different system may need different customization devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z][0-9]*" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]" # Skip LUNZ device from VNX/Unity device { vendor "DGC" product "LUNZ" } } defaults { user_friendly_names no flush_on_last_del yes remove_retries 12 skip_kpartx yes } devices { # Device attributed for EMC CLARiiON and VNX/Unity series ALUA device { vendor "DGC" product ".*" product_blacklist "LUNZ" path_grouping_policy group_by_prio path_selector "round-robin 0" path_checker emc_clariion features "0" no_path_retry 12 hardware_handler "1 alua" prio alua failback immediate } } Why it leaves failed faulty running devices ? Is it a correct behaviour ? Every time I migrate my single instance from a node to another the number of files under /dev/disk/by-path increase of 4 (the number of path used for unity storage) on the destination node. I contacted Dell support and they said it is not an a problem of their cinder driver but ti could be related to nova. Please help me !!! I must decide if acquire nfs storage or iscsi storage. On the cinder compatibility drive matrix most vendors are iscsi.

1 2

[octavia] Octavia weekly IRC meeting is cancelled November 25th
by Michael Johnson 18 Nov '20

18 Nov '20

A number of the Octavia community members will be taking vacation next week, so we will be cancelling the meeting next week on the 25th. Enjoy your break! Michael

1 0

[largescale-sig] Next meeting: November 18, 14utc
by Thierry Carrez 18 Nov '20

18 Nov '20

Hi everyone, After a well-deserved break after the Summit and the PTG, the Large Scale SIG is back with regular meetings this Wednesday in #openstack-meeting-3 on IRC, at our new time: 14UTC. See how it translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20201118T14 Feel free to add topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting Talk to you all later, -- Thierry Carrez

1 1

[tripleo] next meeting Tuesday Nov 24 @ 1400 UTC in #tripleo
by Marios Andreou 18 Nov '20

18 Nov '20

Hello TripleOs reminder our next scheduled irc meeting is ** Tuesday 24th November at 1400 UTC in #tripleo. ** ** https://etherpad.opendev.org/p/tripleo-meeting-items ** Please add anything you want to raise at https://etherpad.opendev.org/p/tripleo-meeting-items This could be anything including review requests, any blocking issues or major ongoing work. Our last meeting was held on the 10th - you can find the logs there http://eavesdrop.openstack.org/meetings/tripleo/2020/tripleo.2020-11-10-14.… thanks, marios

1 0

Re: Where to send mascot graphic adjustment proposals
by James Cole 18 Nov '20

18 Nov '20

Hi Michael, I’m James Cole, a graphic designer at the Open Infrastructure Foundation. Hopefully I can address some of your questions. Regarding the diagram, I’m told it is managed by Designate in their documentation. We can provide suggestions but ultimately it is up to them how they want to create those docs. You may have to get in touch with an active contributor to that project. Regarding mascot design changes, we’d need broader community input to make edits like that. The mascot designs undergo a community review process with each project when they’re created, so we’re unable to make design changes at this time. If it helps, the mark is vertically aligned with the text, not base aligned. Enlarging the crocodile will probably make it seem disproportionately large compared with other project mascots, which all follow the same design rules. Thanks for flagging the inactive email address for Anne. Would you mind providing the URL of the page you found that address? We will replace it with community(a)openinfra.dev <mailto:community@openinfra.dev> as the contact email for mascots. Thanks for reaching out! James Cole Graphic Designer Open Infrastructure Foundation

2 3

[horizon] Weekly Meeting time poll
by Ivan Kolodyazhny 18 Nov '20

18 Nov '20

Hi team, As discussed during the last meeting, let's vote for a new meeting time for Horizon. You can vote for the current time if it's OK for you: https://doodle.com/poll/a5eeh6h3v356w8ss Feel free to reach me by e-mail or in IRC if you have any questions. Regards, Ivan Kolodyazhny

1 0

Failed to Create Virtual Instances for manila Service Due to neutron and nova Services
by ninad jadhav 18 Nov '20

18 Nov '20

Hi, I am trying to deploy OpenStack, using Openstack installation guide https://docs.openstack.org/install-guide/openstack-services.html#minimal-de… . I am able to launch self-service instance, but during manila(shared-file System) creation I am getting share creation error as per manila-share.log "SSH to default NFS server (10.254.0.0/24) failed. I did all the necessary steps required as per document provided but still not able to create Instances. For Neutron Service I have used Linuxbridge (Networking Option 2). Following are the error logs and configuration file for nova and neutron services. As per my discussion with manila developer manila.conf file looks fine but there is an issue with minimal deploy which allows to create virtual machine but for further manila instance creation its failing due to 1. manila-share.log https://gist.github.com/Ninad-cloud/8c77ccb53df26e6c32505a30c988004c 2. nov-log for instance https://gist.github.com/Ninad-cloud/9129f1c25147e5ee7b874b77e591c9ad 2. controller - neutron all conf files https://gist.github.com/Ninad-cloud/bb25e70007a2bd9ea0848edb081f6525 https://gist.github.com/Ninad-cloud/61b3d357953ddaacc16a926770fbdb72 https://gist.github.com/Ninad-cloud/624a4846157fc294bdfa59093e49fa55 https://gist.github.com/Ninad-cloud/5bb3ad0cb9a38582a7c82816de653693 https://gist.github.com/Ninad-cloud/7366ca93b03f4718a424352fe8ec63e8 https://gist.github.com/Ninad-cloud/56ea190a333cbe0102b676b7de48b7ae 3.compute- neutron all .conf files https://gist.github.com/Ninad-cloud/8b25f38bfcd001177b8f5857f6aacd96 https://gist.github.com/Ninad-cloud/0fba595e3f3cce56695a3fc3c4795d3d https://gist.github.com/Ninad-cloud/c8538c5b00ed2760163c3c73d1247145 https://gist.github.com/Ninad-cloud/73f8bddebf032ea5a5e50023cc7808df 4.controller-nova .conf file https://gist.github.com/Ninad-cloud/4ed5b009a62e9fbc5f9e32ddfb3f1e67 5. compute-nova .conf file https://gist.github.com/Ninad-cloud/f21cd484dd9ad6c0befc66ba3c54457e regards, Ninad

2 3

[kolla] How to recover MariaDB if there's a controller failed to boot?
by Eddie Yen 18 Nov '20

18 Nov '20

Hi everyone, We met an issue few days ago. When we launch Openstack cluster, we found that there's a controller couldn't boot up caused by hardware failure, And MariaDB lost sync after left controllers booted into system. In this case, Can I just mark down the failed controller hostname in inventory file and do mariadb_recovery? Any risk after do this after failure one back online? Many thanks, Eddie.

3 6

[stein][cinder][iscsi] multipath does not seem to detach during live migration
by Ignazio Cassano 18 Nov '20

18 Nov '20

Hello Everyone, I am testing stein with cinder iscsi driver for unity. Since I became crazy on queens, I decided to try with stein. At this time I have only one virtual machine with only one iscsi volume. It was running on podiscsivc-kvm02 and I migrated it on podiscsivc-kvm01. Let me show what multipath -ll displays on podiscsivc-kvm02 after live migration: 36006016006e04400dce5b45f0ac77301 dm-3 DGC ,VRAID size=40G features='1 retain_attached_hw_handler' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=0 status=active | `- 17:0:0:60 sdm 8:192 failed faulty running `-+- policy='round-robin 0' prio=0 status=enabled |- 19:0:0:60 sdk 8:160 failed faulty running `- 21:0:0:60 sdj 8:144 failed faulty running And now on destination node podiscsivc-kvm01: 36006016006e04400dce5b45f0ac77301 dm-3 DGC ,VRAID size=40G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 17:0:0:14 sdm 8:192 active ready running | `- 15:0:0:14 sdl 8:176 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 21:0:0:14 sdj 8:144 active ready running `- 19:0:0:14 sdk 8:160 active ready running On source node in /var/log/messages I get: Nov 18 10:34:19 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: failed in domap for addition of new path sdl Nov 18 10:34:19 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: uev_add_path sleep Nov 18 10:34:20 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: failed in domap for addition of new path sdl Nov 18 10:34:20 podiscsivc-kvm02 kernel: device-mapper: table: 253:3: multipath: error getting device Nov 18 10:34:20 podiscsivc-kvm02 kernel: device-mapper: ioctl: error adding target to table Nov 18 10:34:20 podiscsivc-kvm02 multipathd: 36006016006e04400dce5b45f0ac77301: uev_add_path sleep On storage node seems it wotks fine, because the host access migrated from source to destination node. Multipath.conf is the following: blacklist { # Skip the files uner /dev that are definitely not FC/iSCSI devices # Different system may need different customization devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z][0-9]*" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]" # Skip LUNZ device from VNX/Unity device { vendor "DGC" product "LUNZ" } } defaults { user_friendly_names no flush_on_last_del yes remove_retries 12 skip_kpartx yes } devices { # Device attributed for EMC CLARiiON and VNX/Unity series ALUA device { vendor "DGC" product ".*" product_blacklist "LUNZ" path_grouping_policy group_by_prio path_selector "round-robin 0" path_checker emc_clariion features "0" no_path_retry 12 hardware_handler "1 alua" prio alua failback immediate } } Why it leaves failed faulty running devices ? Is it a correct behaviour ? I contacted Dell support and they said it is not an a problem of their cinder driver but ti could be related to nova. Please help me !!! I must decide if acquire nfs storage or iscsi storage. On the cinder compatibility drive matrix most vendors are iscsi. Ignazio

1 1