September 2021 - openstack-discuss

Horizon connection errors from object store
by Michel Niyoyita 09 Sep '21

09 Sep '21

Hello team , I am facing an issue when I am trying to connect to the object store containers on the horizon dashboad . Once click on containers it automatically disconnect. please find below logs I am getting and help for further analysis. [Thu Sep 09 06:35:22.185771 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55130] Attempted scope to domain Default failed, will attempt to scope to another domain. [Thu Sep 09 06:35:22.572522 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55130] Login successful for user "admin" using domain "Default", remote address 10.10.29.150. [Thu Sep 09 06:35:51.494815 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] REQ: curl -i http://ceph-mon2:8080/swift/v1?format=json&limit=1001 -X GET -H "X-Auth-Token: gAAAAABhOasqHFyB..." -H "Accept-Encoding: gzip" [Thu Sep 09 06:35:51.495140 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP STATUS: 401 Unauthorized [Thu Sep 09 06:35:51.495541 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx00000000000000000000f-006139ab44-9fc1a-default', 'X-Openstack-Request-Id': 'tx00000000000000000000f-006139ab44-9fc1a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 09 Sep 2021 06:35:51 GMT', 'Connection': 'Keep-Alive'} [Thu Sep 09 06:35:51.495792 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx00000000000000000000f-006139ab44-9fc1a-default","HostId":"9fc1a-default-default"}' [Thu Sep 09 06:35:51.498743 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Unauthorized: /api/swift/containers/ [Thu Sep 09 06:35:52.924169 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] REQ: curl -i http://ceph-mon2:8080/swift/v1?format=json&limit=1001 -X GET -H "X-Auth-Token: gAAAAABhOasqHFyB..." -H "Accept-Encoding: gzip" [Thu Sep 09 06:35:52.924520 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP STATUS: 401 Unauthorized [Thu Sep 09 06:35:52.924789 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx000000000000000000010-006139ab48-9fc1a-default', 'X-Openstack-Request-Id': 'tx000000000000000000010-006139ab48-9fc1a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 09 Sep 2021 06:35:52 GMT', 'Connection': 'Keep-Alive'} [Thu Sep 09 06:35:52.925034 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx000000000000000000010-006139ab48-9fc1a-default","HostId":"9fc1a-default-default"}' [Thu Sep 09 06:35:52.929398 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Unauthorized: /api/swift/containers/ [Thu Sep 09 06:35:52.935799 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:56016] Logging out user "admin". [Thu Sep 09 06:35:53.061489 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Logging out user "". [Thu Sep 09 06:35:54.541593 2021] [wsgi:error] [pid 165:tid 139887608641280] [remote 10.10.29.150:55852] The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example. [Thu Sep 09 06:35:54.542896 2021] [wsgi:error] [pid 165:tid 139887608641280] [remote 10.10.29.150:55852] Bad Request: /api/swift/policies/ [Thu Sep 09 06:35:54.566055 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55860] The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example. [Thu Sep 09 06:35:54.567130 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55860] Bad Request: /api/swift/info/ (kolla-open1) stack@kolla-open1 :/var/lib/docker/volumes/kolla_logs/_data/horizon$ [image: image.png] [image: image.png] Michel

1 0

[neutron][neutron-vpnaas] Propose to EOL stable/queens, stable/rocky and stable/stein branches or neutron-vpnaas
by Slawek Kaplonski 09 Sep '21

09 Sep '21

Hi, On our last meeting of the CI team we discussed the problem with broken stable branches (stein and older) in the neutron-vpnaas project [1]. I checked that last patches which were merged in those branches are more than year ago: * Queens - last merged patch in April 2019 * Rocky - last merged patch in February 2020 * Stein - last merged patch in September 2019 Giving the above and the current status of the CI in those branches, I propose to make them End Of Life. I will wait until end of next week for anyone who would like to maybe step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in neutron- vpnaas. [1] https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci. 2021-09-07-15.00.log.html#l-64 -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Question about vDPA support for live migration
by Jeffrey Zhang 09 Sep '21

09 Sep '21

Hey everyone, I am testing vDPA on OpenStack Wallaby. I have set it up successfully. But I found there are several features that are not implemented, including live/code migration[0], which I wanted. Anyone know what's the latest progress about this? I found no talk about this. [0] https://review.opendev.org/c/openstack/nova/+/780866/3/releasenotes/notes/v… <https://signaldomn.online/click?redirect=https%3A%2F%2Freview.opendev.org%2…> --- Regards, Jeffrey Zhang Blog: http://xcodest.me <https://signaldomn.online/click?redirect=http%3A%2F%2Fxcodest.me%2F&dID=163…>

1 0

[neutron] Slow router provisioning during full resync of L3 Agent
by Patryk Jakuszew 09 Sep '21

09 Sep '21

Hello, I have a long-standing issue with L3 Agent which I would like to finally solve - *very* slow router provisioning in L3 Agent. We are operating a Rocky-based OpenStack deployment with three bare-metal L3 Agent nodes running in legacy mode. After restarting the L3 node, it takes a really long time for the L3 agent to become fully operational. There are two parts of resync which take much time: getting a list of routers from neutron-server and actually recreate them in the L3 node. While the long running time of router list retrieval is somewhat understandable, the router provisioning process itself proves to be very troublesome in our operations tasks. In our production deployment with around 250 routers, it takes around 2 hours (!) to recreate the router namespaces and have the L3 node fully functional again. Two hours of router re-provisioning is actually an optimistic scenario, this proved to be much longer during the outages we encountered (sometimes the sync took nearly 6-8 hours). This effectively prolongs any maintenance upgrades, configuration changes and OpenStack release upgrades. Another thing is, on that same production environment the first 100 routers usually get provisioned fast (around 30 minutes), after that it slows down with each router - this kind of non deterministic behavior makes it hard to communicate the maintenance finish ETA for our users. We also have a test environment with Stein already installed, where this problem is also present - full resync of 150 routers, with only one external gateway ports, takes around an hour to complete. Are there any operators here who also encountered that issue? Does anyone have any experience with similar situation and are willing to share their observations and optimizations? -- Regards, Patryk Jakuszew

3 3

[all][tc] Technical Committee next weekly meeting on Sept 9th at 1500 UTC
by Ghanshyam Mann 08 Sep '21

08 Sep '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Sept 9th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Sept 8th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[devstack][openstack-qa] DevStack on VirtualBox with Host-only Adapter
by os-user157 08 Sep '21

08 Sep '21

Hi all, I'm trying to deploy DevStack inside an Ubuntu Server 20.04.3 LTS virtual machine inside VirtualBox, but I'm struggling a lot, I got a lot of different errors. My use case: I want to test Zuul and DevStack, so I'll create a virtual machine for DevStack and one for Zuul, both on VirtualBox. I'm using a Host-only adapter with CIDR 192.168.200.0/24, and I need DevStack to be accessible on this network, and to use this network as the floating range for giving floating IP addresses to instances inside DevStack, so that Zuul can also access them. My local.conf: https://paste.openstack.org/show/809182/ Screenshots of the network configuration of my VirtualBox VM: First adapter, NAT (used for internet connectivity): https://i.imgur.com/C7XTiLJ.png Second adapter, host-only, used for communication with my Windows host and other VMs in the same network: https://i.imgur.com/Qeciuqx.png DevStack is erroring with the arping command on lib/neutron-legacy, arping gets 0 responses and exits the script with exit code 1 Does anyone have experience in dpeloying DevStack inside VirtualBox in such manner? I really need some help with this, if anyone could give me some ideas of waht could be wrong, it'd very much appreciated! Thanks.

1 0

Tenant vlan Network Issue
by Karera Tony 08 Sep '21

08 Sep '21

Hello Team, Whenever I create an internal network with type vlan. The Instances don't get IPs but for external Network it's fine. Below is the etc/kolla/config/neutron/ml2_conf.ini file [ml2] type_drivers = flat,vlan,vxlan tenant_network_types = vlan,flat mechanism_drivers = openvswitch,l2population [ml2_type_vlan] network_vlan_ranges = physnet1 [ml2_type_flat] flat_networks = physnet1 Is there something I need to change or I need to configure the interface differently Regards Tony Karera

4 12

[TripleO] Yoga PTG topics etherpad
by James Slagle 08 Sep '21

08 Sep '21

I've started an etherpad where we can collect topics for discussion at the upcoming PTG: https://etherpad.opendev.org/p/tripleo-yoga-topics Please add topics over the next couple of weeks, and then we'll work on a schedule. Thanks. -- -- James Slagle --

1 0

Re: [all][tc] Minium python version for yoga and centos 9 stream.
by Sean Mooney 08 Sep '21

08 Sep '21

resending to correct list On Tue, 2021-09-07 at 10:51 +0100, Sean Mooney wrote: > hi i was jsut looking at the yoga runtimes an notice that we have not really updated it > with regards to minium python versions. > > currently > > the runtime defintion is as follows https://github.com/openstack/governance/blob/master/reference/runtimes/yoga… > > It is the :doc:`policy <../../resolutions/20181024-python-update-process>` that each OpenStack release cycle will target the latest available version of Python; default Python runtimes on the > distributions listed above; and versions used in integration tests at the start of the cycle, at least until the point when all projects have migrated to a later version. > > Based on the criteria above, all Python-based projects must target and test against, at a minimum: > > Python 3.6 (default in CentOS 8 Stream) > Python 3.8 (latest available; default in Ubuntu 20.04) > > looking at to look at https://repology.org/project/python/versions what python version are shpping by default now in many distro > more and more of them are moving to python 3.9 which is currently not in the test list and wer are starting to fall quite far behind > > debian 11/12/stable are currently using 3.9.2 > centos 8 defaults to 3.6.8 but ships 3.8.6 and 3.9.2 as appstreams > centos 9 stream will default to python3-3.9.6-6.el9.x86_64.rpm https://composes.stream.centos.org/test/latest-CentOS-Stream/compose/BaseOS… > ubuntu 20.04 default to 3.8.2 but also ships 3.9.5 in universe > ubuntu 21.04 default to 3.9.5 so we can expect that when yoga ships on ubuntu 22.04 it will be based on at least python 3.9 and proably 3.10 > fedora well they basically ship everythin but i belive 3.9 is not the default based on complaints some of my fellow contiutors had with > runnign unit and functional tests. > > openSUSE Leap 15.3 main/oss python39 3.9.4 ship 3.9 but default to 3.6 > they aslo nolonger have an openstack product but it would still be nice to keep them in mind. > > > over all i think we shoudl consider raising or min tested/supported python version to python 3.8 for yoga and consider updating from centos 8 to centos 9. > i also think it worth including python 3.9 in the required testing versions for python projects. > > several distros will be releaseing openstack xena/wallaby with python 3.9 and also yoga likely with 3.10 in some cases. > if we only offically test with 3.6 and 3.8 we could have some hard probalems to solve going forward. the optional testing with unit and functional tests > that some project have for python 3.9 certenly helps but actully having an integrated tempest gate job that runs on 3.9 i think is going to be increasingly important. > > readring centos 9 stream, it was orginally planned of release at the end of q2. now that date has obviosly passed but i know that they are still hopeful to have > a release ready in the next month or so. cloud images are avaibel at https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/… although the repos are still only > partly complete https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/… and there are currently no active mirros so its still officaly > not released https://www.centos.org/centos-stream/ and i don not have any ETA beyond soonTM. as such im aware we cant offically declare centos 9 stream as part of the tested > runtimes at this point but it might be at least worht listing it as an experimtal runtime as i suspec there will be a desire to swtich to it during the yoga cycle for some proejct. > ooo/rdo come to mind as they discussed this for xena but obviously that was not possibel since it was not released in time. > > > regards > sean. > > ps removing python 3.6 and replacing it with 3.9 instead of just adding 3.9 as an addtional runtime is more to manage our testing > there isnt nessarly any python feature that i explcitly would like to be able to use that are added in 3.8 that would be enabled > by making that the new min python verion that is motivating this suggestion. there are some nice thingin 3.9/3.10 such as pattern > mataching that are commign but it will be some time (2-4 years)before we can actully use those as our min version so i was just being practial > and suggestion we raise to 3.8 and add 3.9 as required with 3.10 as optional/experimal for those that want it. >

5 7

[cinder] Bug deputy report for week of 2021-08-09
by Sofia Enriquez 08 Sep '21

08 Sep '21

This is a bug report from 2021-01-09 to 2021-08-09. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- High - https://bugs.launchpad.net/cinder/+bug/1942696 "RateLimitingMiddleware still belongs to v2 API code" Medium - https://bugs.launchpad.net/cinder/+bug/1942953 "Upload volume to image will terminate the vm's volume attachment". Assigned to xuanyandong - https://bugs.launchpad.net/cinder/+bug/1942342 "Huawei FC driver fails to create a volume from a snapshot with Dorado V6 - Bad or unexpected response from the storage volume backend API: Create luncopy error". Unassigned. - https://bugs.launchpad.net/cinder/+bug/1934036 "Undefined CONF.quota_metadata_items is used in unit tests". Assigned to Takashi Kajinami. Cheers, -- L. Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0