- openstack-discuss - lists.openstack.org

[oslo] oslo-rootwrap daemon test failures
by Chuck Tuffli 26 Oct '23

26 Oct '23

There are a handful of failures when running the oslo-rootwrap test suite on FreeBSD. Most I have been able to fix, but I'm looking for some help understanding what is going wrong with the daemon tests. One example of a failing test is test_daemon_cleanup_client. The test reports: AssertionError: -9 == -9 : Server haven't stopped in one second which I _think_ occurs because of an exception while closing the connection (see below). Any suggestions as to what I might check (and where) to understand why this is happening? Thanks! Running the latest from git on FreeBSD 13.2 with Python 3.9 {2} oslo_rootwrap.tests.test_functional.RootwrapDaemonTest.test_daemon_cleanup_client [1.455829s] ... FAILED Captured traceback: ~~~~~~~~~~~~~~~~~~~ Traceback (most recent call last): File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/oslo_rootwrap/tests/test_functional.py", line 299, in test_daemon_cleanup_client self.client._finalize() File "/usr/local/lib/python3.9/contextlib.py", line 126, in __exit__ next(self.gen) File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/oslo_rootwrap/tests/test_functional.py", line 289, in _test_daemon_cleanup self.assertNotEqual(-signal.SIGKILL, process.returncode, File "/usr/local/lib/python3.9/unittest/case.py", line 846, in assertNotEqual raise self.failureException(msg) AssertionError: -9 == -9 : Server haven't stopped in one second Captured daemon_log: ~~~~~~~~~~~~~~~~~~~~ 2023-10-19 18:33:10,718 | [ 1664]+DEBUG | Created temporary directory /tmp/rootwrap-snf7oiau 2023-10-19 18:33:10,718 | [ 1664]+DEBUG | Will listen on socket /tmp/rootwrap-snf7oiau/rootwrap.sock 2023-10-19 18:33:10,719 | [ 1664]+ INFO | Starting rootwrap daemon main loop 2023-10-19 18:33:11,719 | [ 1664]+ INFO | Got signal 2. Shutting down server 2023-10-19 18:33:11,719 | [ 1664]+DEBUG | Removing temporary directory /tmp/rootwrap-snf7oiau Traceback (most recent call last): File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/.tox/py3/lib/python3.9/site-packages/oslo_rootwrap/daemon.py", line 166, in daemon_start server.serve_forever() File "/usr/local/lib/python3.9/multiprocessing/managers.py", line 184, in serve_forever sys.exit(0) SystemExit: 0 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/oslo_rootwrap/tests/run_daemon.py", line 58, in <module> cmd.daemon() File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/.tox/py3/lib/python3.9/site-packages/oslo_rootwrap/cmd.py", line 62, in daemon return main(run_daemon=True) File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/.tox/py3/lib/python3.9/site-packages/oslo_rootwrap/cmd.py", line 133, in main daemon_mod.daemon_start(config, filters) File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/.tox/py3/lib/python3.9/site-packages/oslo_rootwrap/daemon.py", line 171, in daemon_start conn.close() File "/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/.tox/py3/lib/python3.9/site-packages/oslo_rootwrap/jsonrpc.py", line 92, in close self._socket.shutdown(socket.SHUT_RDWR) OSError: [Errno 57] Socket is not connected Captured client_log: ~~~~~~~~~~~~~~~~~~~~ 2023-10-19 18:33:10,290 | [ 1610] DEBUG | Popen for ['/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/.tox/py3/bin/python', '/usr/home/tuffli/dev/openstack/oslo.rootwrap.git/oslo_rootwrap/tests/run_daemon.py', '/tmp/tmpjri8_7we/rootwrap.conf'] command has been instantiated 2023-10-19 18:33:10,719 | [ 1610] INFO | Spawned new rootwrap daemon process with pid=1664 2023-10-19 18:33:10,728 | [ 1610] INFO | Stopping rootwrap daemon process with pid=1664 2023-10-19 18:33:11,733 | [ 1610] INFO | Rootwrap daemon process exit with status: -9

3 4

[CloudKitty] Cancelling weekly meeting 30 of October
by Rafael Weingärtner 25 Oct '23

25 Oct '23

Hello guys, As we discussed in the vPTG, we will be cancelling the next CloudKitty bi-weekly meeting that would take place on October the 30. If you need any assistance until the next meeting, you can reach me via e-mail. Best regards, -- Rafael Weingärtner

1 0

[manila] Oct 26th upstream meeting is cancelled
by Carlos Silva 25 Oct '23

25 Oct '23

Hello everyone! Cancelling this week's upstream meeting as we will be focused on the OpenStack Manila PTG [1]. Next meeting will take place on Nov 2nd. [1] https://etherpad.opendev.org/p/caracal-ptg-manila Thanks, carloss

1 0

[cinder] Cancelling this week's meeting (25 Oct)
by Rajat Dhasmana 25 Oct '23

25 Oct '23

Hello Argonauts, We have the 2024.1 Caracal PTG this week so cancelling the upstream meeting. See you at the PTG! Dates: 24 October (Tuesday) - 27 October (Friday), 2023 Time: 1300 - 1700 UTC Room: Cactus Meeting Link: https://www.openstack.org/ptg/rooms/cactus Thanks Rajat Dhasmana

1 0

[neutron][octavia] New deployment / suggestions for network
by Eric K. Miller 25 Oct '23

25 Oct '23

Hi, We have been deploying OpenStack for quite some time, using Kolla-Ansible, and typically choose DVR and OVS with Amphorae deployed by Octavia for load balancing. With the issues that DVR has with Octavia's Amphorae and Virtual IPs, with essentially non-functional automated fail-over, we have always wanted to move to OVN since it appears to be the popular approach now. I have also read that OVN appears to work properly with allowed-address-pairs correctly, whereas with DVR, OVS does not, and thus some of the issues with Amphorae Virtual IPs. However, OVN, from what I understand, has issues with, or doesn't support, VPNaaS, which we use extensively. Plus, it only supports Layer 4 load balancing, whereas with Amphorae, we get Layer 7 load balancing - also used extensively. I'm not sure, though - maybe OVN with Octavia still supports Amphorae if we need Layer 7 load balancing? Am I wrong regarding any of the comments above? What is the best back-end networking architecture that provides scalability (so, not VLANs), Layer 7 load balancing with Octavia, along with VPNaaS, in a brand new install with the latest version of OpenStack? Note that we used Midonet long long ago, and it seemed to have everything we wanted, but shortly after purchasing it, Midokura immediately decided to abandon support for OpenStack and went the Kubernetes route. Not sure if they still do this, but needless to say, Midonet isn't a valid solution unfortunately. Tungsten Fabric appears like an alternate solution to Midonet, but that project is sunsetting in 2024, so that's dead too. :( Thank you for any suggestions! Eric

4 6

[kolla] Cancelling weekly meeting tomorrow (25th Oct 2023)
by Michał Nasiadka 25 Oct '23

25 Oct '23

Hello, Cancelling tomorrows weekly meeting since we’re already meeting at vPTG sessions. Thanks, Michal

1 0

RDO Bobcat 2023.2 Released
by Amy Marrich 24 Oct '23

24 Oct '23

RDO Bobcat Released The RDO community is pleased to announce the general availability of the RDO build for OpenStack 2023.2 Bobcat for RPM-based distributions, CentOS Stream and Red Hat Enterprise Linux. RDO is suitable for building private, public, and hybrid clouds. Bobcat is the 28th release from the OpenStack project, which is the work of more than 1,000 contributors from around the world. The release is already available for CentOS Stream 9 on the CentOS mirror network in: http://mirror.stream.centos.org/SIGs/9-stream/cloud/x86_64/openstack-bobcat/ The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Stream and is a member of the CentOS Cloud Infrastructure SIG. The Cloud Infrastructure SIG focuses on delivering a great user experience for CentOS users looking to build and maintain their own on-premise, public or hybrid clouds. All work on RDO and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first. The highlights of the broader upstream OpenStack project may be read via https://releases.openstack.org/bobcat/highlights.html but here are some highlights: - New Cinder driver features were added, notably, QoS support for Fujitsu ETERNUS DX driver, replication-enabled consistency groups support for Pure Storage driver, and Active/Active support for NetApp NFS driver. - Glance added support for RBD driver to move images to the trash if they cannot be deleted immediately due to having snapshots. - The Neutron service has enabled the new API policies (RBAC) with system scope and default roles by default. - The Nova legacy quota driver is now deprecated and a nova-manage limits command is provided in order to migrate the orginal limits into Keystone. We plan to change the default quota driver to the unified limits driver in an upcoming release. It is recommended that you begin planning and executing a migration to unified limits as soon as possible. OpenStack Bobcat is not marked as Skip Level Upgrade Release Process or SLURP. According to this model ( https://governance.openstack.org/tc/resolutions/20220210-release-cadence-ad…) this means that upgrades will only be supported from the Antelope 2023.1 release. RDO Bobcat 2023.2 has been built and tested with the recently released Ceph 18.2.0 Reef version (https://docs.ceph.com/en/latest/releases/reef/) which has been published by the CentOS Storage SIG in the official CentOS repositories. *Note:* Follow the instructions in [RDO documentation]( https://www.rdoproject.org/install/install-with-ceph/) to install OpenStack and Ceph services in the same host. During the Bobcat 2023.2 development cycle, the RDO community has implemented automatic dependency detection at run and build time. We expect that these changes will lead to more accurate dependency chains in OpenStack packages and less manual maintenance tasks for community maintainers. Following upstream retirement, some packages are not present in RDO Bobcat 2023.2 release: - python-networking-odl - python-networking-omnipath - python-networking-vmware-nsx - python-oswin-tests-tempest - python-os-xenapi - python-patrole - python-stackviz - python-vmware-nsxlib - python-vmware-nsx-tests-tempest Contributors - During the Bobcat cycle, we saw the following new RDO contributors: - Arkady Shtempler - Dariusz Smigiel - Dave Wilde - Fabricio Aguiar - Jakub Skunda - Joan Francesc Gilabert - Maor Blaustein - Mohammad Abunemeh - Szymon Datko - Yadnesh Kulkarni Welcome to all of you and Thank You So Much for participating! But we wouldn’t want to overlook anyone. A super massive Thank You to all 47 contributors who participated in producing this release. This list includes commits to rdo-packages, rdo-infra, and rdo-website repositories: - Alfredo Moralejo Alonso - Amy Marrich - Ananya Banerjee - Arkady Shtempler - Artom Lifshitz - Arx Cruz - Bhagyashri Shewale - Bohdan Dobrelia - Chandan Kumar - Daniel Pawlik - Dariusz Smigiel - Dave Wilde - Douglas Viroel - Enrique Vallespi Gil - Fabricio Aguiar - Giulio Fidente - Goutham Pacha Ravi - Gregory Thiemonge - Grzegorz Grasza - Ihar Hrachyshka - Jakub Skunda - Jiří Podivín - Jiří Stránský - Joan Francesc Gilabert - Joel Capitao - Karolina Kula - Karthik Sundaravel - Luca Miccini - Lucas Alvares Gomes - Luigi Toscano - Luis Tomas Bolivar - Maor Blaustein - Marios Andreou - Mathieu Bultel - Matthias Runge - Mohammad Abunemeh - Rodolfo Alonso Hernandez - Ronelle Landy - Sandeep Yadav - Slawomir Kaplonski - Soniya29 vyas - Szymon Datko - Takashi Kajinami - Tobias Urdin - Tom Weininger - Yadnesh Kulkarni - Yatin Karel The Next Release Cycle At the end of one release, focus shifts immediately to the next release i.e Caracal. Get Started To spin up a proof of concept cloud, quickly, and on limited hardware, try an All-In-One Packstack installation. You can run RDO on a single node to get a feel for how it works. Finally, for those that don’t have any hardware or physical resources, there’s the OpenStack Global Passport Program. This is a collaborative effort between OpenStack public cloud providers to let you experience the freedom, performance and interoperability of open source infrastructure. You can quickly and easily gain access to OpenStack infrastructure via trial programs from participating OpenStack public cloud providers around the world. Get Help The RDO Project has our users(a)lists.rdoproject.org for RDO-specific users and operators. For more developer-oriented content we recommend joining the dev(a)lists.rdoproject.org mailing list. Remember to post a brief introduction about yourself and your RDO story. The mailing lists archives are all available at https://mail.rdoproject.org. You can also find extensive documentation on RDOproject.org. The #rdo channel on OFTC IRC is also an excellent place to find and give help. We also welcome comments and requests on the CentOS devel mailing list and the CentOS IRC channels (#centos, #centos-cloud, #centos-devel in Libera.Chat network), however we have a more focused audience within the RDO venues. Get Involved To get involved in the OpenStack RPM packaging effort, check out the RDO contribute pages, peruse the CentOS Cloud SIG page, and inhale the RDO packaging documentation. Join us in #rdo and on the OFTC IRC network and follow us on Twitter @RDOCommunity. You can also find us on Facebook and YouTube.

1 0

Re: [kolla-ansible][yoga] Cannot authenticate to openstack after deploying self-signed cert
by wodel youchi 24 Oct '23

24 Oct '23

Hi, From the beginning I had kolla_verify_tls_backend: "no" in globals.yml The weird thing is that the openstack cli works fine. I even created a new user with admin role, I get the same behavior. Horizon does not connect, the cli works. I activated the debug mode on keystone [root@rscdeployer ~]# cat /etc/yogakolla/config/keystone.conf [DEFAULT] debug = True insecure_debug= True But nothing in the log file, when I try to login via horizon, I don't get anything on keystone.log. I tested with a wrong password to see the behavior of the platform, and this is what I got on keystone.log : - Openstack CLI with wrong pass, I got : 2023-10-22 13:51:02.344 43 WARNING keystone.server.flask.application [req-73100d88-8357-42ce-8865-e36c34a9bfa9 - - - - -] Authorization failed. The request you have made requires authentication. from 10.10.3.16: keystone.exception.Unauthorized: The request you have made requires authentication. - Openstack Horizon with wrong pass, I got : Nothing How can I follow up this, how can I be sure that it's not a horizon problem or something else? Regards. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free.www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Le mer. 18 oct. 2023 à 13:17, Kaster, Jörn <Joern.Kaster(a)epg.com> a écrit : > Hi wodel, > nice to hear that the patch helps you. > > With the keystone problem i can't help, but i think the certs are correct. > Two thoughts about that. > > - If you have in the respective configurations of the OpenStack > Services the IP address instead of any DNS Name configured (here is this > the case) then the certificates don't need the dns name in it. > - Could you please look for any error Messages in the keystone logs > and also check if you can establish a connection to the keystone on the > mentioned port with openssl s_client. If so, it could be possible that you > have to disable the certificate verification in the deployment. > > > ------------------------------ > *Von:* wodel youchi <wodel.youchi(a)gmail.com> > *Gesendet:* Mittwoch, 18. Oktober 2023 11:55 > *An:* Kaster, Jörn <Joern.Kaster(a)epg.com> > *Cc:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> > *Betreff:* Re: [kolla-ansible][yoga] Cannot authenticate to openstack > after deploying self-signed cert > > > OUTSIDE-EPG! > > Thanks Jörn, it worked for cloudkitty, after applying the patch the > deployment went well. But : > > - I still can't access the web console : An error occurred authenticating. > Please try again later. > > - in cloudkitty-processor.log I am still having : > 2023-10-18 10:46:25.271 8106 WARNING keystoneauth.identity.generic.base > [-] Failed to discover available identity versions when contacting > https://dinternal.cloud.domain.tld:35357. Attempting to parse version > from URL.: keystoneauth1.exceptions.connection.SSLError: SSL exception > connecting to https:// dinternal.cloud.domain.tld :35357: > HTTPSConnectionPool(host=' dinternal.cloud.domain.tld ', port=35357): Max > retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: > CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),)) > > > When generating the self-signed certificate, I noticed that the process > had generated : > - two haproxy certificates, one for the internet with the external FQDN > and the second for internal communication with the local internal FQDN. > > - It also generated a backend certificate, that contains only the IP > addresses of the 03 controllers as Subject Alternate Names without any > mention of the domain I am using, is this correct? > > [root@rscdeployer ~]# openssl x509 -noout -text -in > /etc/yogakolla/certificates/backend-cert.pem > Certificate: > Data: > Version: 3 (0x2) > Serial Number: > 1c:66:7e:37:85:cf:ca:1c:da:42:f6:f1:1f:dc:1e:97..... > Signature Algorithm: sha256WithRSAEncryption > Issuer: CN = KollaTestCA > Validity > Not Before: Oct 17 15:04:26 2023 GMT > Not After : Oct 15 15:04:26 2025 GMT > Subject: C = US, ST = NC, L = RTP, OU = kolla > Subject Public Key Info: > Public Key Algorithm: rsaEncryption > RSA Public-Key: (2048 bit) > Modulus: > ..... > ..... > e6:23:a4:7f:30:74:ac:0c:2d:22:00:95:b6:ab:20: > 98:6b > Exponent: 65537 (0x10001) > X509v3 extensions: > > *X509v3 Subject Alternative Name: IP Address:10.10.3.5, IP > Address:10.10.3.9, IP Address:10.10.3.13* > Signature Algorithm: sha256WithRSAEncryption > 36:86:cb:b4:9a:fe:33:0d:ff:af:87:5e:00:9d:69:4e:32:21: > > > > Regards. > > > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> > Virus-free.www.avast.com > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> > <#m_-6993449584919071302_x_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > Le mer. 18 oct. 2023 à 07:12, Kaster, Jörn <Joern.Kaster(a)epg.com> a > écrit : > > Hello wodel, > the problem with cloudkitty deployment with self signed certs could > resolve to the following bugreport [1]. > > [1] https://bugs.launchpad.net/kolla-ansible/+bug/1998831 > Bug #1998831 “CloudKitty bootstrap fails when using internal TLS...” : > Bugs : kolla-ansible > <https://bugs.launchpad.net/kolla-ansible/+bug/1998831> > When InfluxDB is behind HAProxy's internal TLS, CloudKitty fails to > bootstrap its InfluxDB database with the following error: TASK [cloudkitty > : Creating Cloudkitty influxdb database] > *************************************************************************************************************************** > fatal: [controller01 -> controller01]: FAILED! => changed=false action: > influxdb_database msg: ('Connection aborted.', RemoteDisconnected('Remote > end closed connection without response... > bugs.launchpad.net > > > ------------------------------ > *Von:* wodel youchi <wodel.youchi(a)gmail.com> > *Gesendet:* Mittwoch, 18. Oktober 2023 01:33 > *An:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> > *Betreff:* [kolla-ansible][yoga] Cannot authenticate to openstack after > deploying self-signed cert > > > OUTSIDE-EPG! > > Hi, > > Our ssl certificate expired a couple of days ago, and we started > experiencing failed login, to workaround the problem rapidly we decided to > deploy the self-signed certificates generated by kolla. > > We generated the certificates then we did a reconfigure, but still the > problem remains : An error occurred authenticating. Please try again later. > > on horizon.log we have : > [Wed Oct 18 00:25:55.379383 2023] [wsgi:error] [pid 103:tid > 140182314505984] [remote 10.10.3.5:40848] Login failed for user "admin" > using domain "default", remote address 10.10.3.5 > > The openstack command line works fine. > > How can we debug this? > > The second problem we have is with cloudkitty that refuses to reconfigure > with the generated self-signed certificate, we had to ignore it from the > reconfiguration process by putting the cloudkitty variable to no before > restarting the reconfigure process. > > How can we debug this? > > > > > Regards. > > > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> > Virus-free.www.avast.com > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> > <#m_-6993449584919071302_x_m_9147058937014271290_x_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > >

2 4

[ironic] [baremetal-sig] Feedback on ironic "ilo" or "ilo5" driver features
by Nisha Agarwal 24 Oct '23

24 Oct '23

Hello, We had a discussion with baremetal-sig in PTG yesterday(23rd Oct 2023). Below are some of the ironic ilo driver features discussed and the summary of our understanding: - In-band and OOB RAID configuration and disk erase : · Cleaning steps are invoked via operator-written scripts for RAID, disk erase etc. The OS commands are directly invoked in the scripts for these tasks. · We are not clear if iLO driver inband RAID feature is used or not. · OOB RAID configuration and disk-erase is not used. - Out-of-Band Inspection: · We are not clear if OOB inspection is used or not. · LLDP data based inventory is most desired. Not all BMCs provide this data today. · Inspection is also used for getting the DHCP information especially, hostname. - UEFI HTTPS boot: · UEFI HTTPS boot feature of ilo driver is not used. · vMedia is the preferred way of deploy in the field. · UEFI HTTP deploy is proposed this cycle in redfish driver. · Some customers use PXE boot. - Smart Update Manager (SUM)-based firmware update: · Firmware update feature of ilo drivers is not used, rather redfish driver is used. Above is our understanding from the discussion in the baremetal-sig operator in PTG. Please correct/clarify our understanding. Basically, we are trying to figure out if any of the following ilo driver features are used : In-band and out-of-band RAID configuration In-band and out-of-band diskerase UEFI-HTTPS boot One button secure erase Smart Update Manager (SUM)-based firmware update Updating security parameters as manual cleaning step GPU hardware inspection We like to also know if proliantutils is used separately as a hardware manager or independently for any of the OOB/Inband features. Regards Nisha

1 0

[Octavia] cancelling meeting Oct 25
by Gregory Thiemonge 24 Oct '23

24 Oct '23

Hi Folks, As discussed in the previous meeting, tomorrow's meeting is cancelled. See you at the vPTG! Gregory

1 0